2024-03-23T07:58:31.039 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-23T07:58:31.039 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372 --verbose --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372/orig.config.yaml 2024-03-23T07:58:31.090 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T07:58:31.177 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372 branch: wip-yuri11-testing-2024-03-21-0851-reef description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: yweinste@redhat.com first_in_suite: false job_id: '7617372' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri11-testing-2024-03-21-0851-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 82316b04821a6c2c1d458a881920454bbadb8837 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 82316b04821a6c2c1d458a881920454bbadb8837 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri11-testing-2024-03-21-0851-reef sha1: 82316b04821a6c2c1d458a881920454bbadb8837 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 1919 sha1: 82316b04821a6c2c1d458a881920454bbadb8837 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri11-testing-2024-03-21-0851-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 82316b04821a6c2c1d458a881920454bbadb8837 targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL4i+W5DRwmOgZYbJcLOoOi0qrYObgZlV3JzOakKuREzK1auMEGXJcspeAP0YUg1yvK5fw5bJnrt0W1FhFAuwvQ= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTbhBHKlOBrLX4ErpDkLCeYOQu5jeSrdCwEznFfBbS3/vWxLCqB5gBuhEh2zoQdrvPy5mnUUpPe8Qn7GueCE5g= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_13:15:24 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-23T07:58:31.177 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa; will attempt to use it 2024-03-23T07:58:31.178 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks 2024-03-23T07:58:31.178 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-23T07:58:31.180 INFO:teuthology.task.internal:Checking packages... 2024-03-23T07:58:31.201 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '82316b04821a6c2c1d458a881920454bbadb8837' 2024-03-23T07:58:31.201 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T07:58:31.201 INFO:teuthology.packaging:ref: None 2024-03-23T07:58:31.201 INFO:teuthology.packaging:tag: None 2024-03-23T07:58:31.201 INFO:teuthology.packaging:branch: wip-yuri11-testing-2024-03-21-0851-reef 2024-03-23T07:58:31.201 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T07:58:31.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri11-testing-2024-03-21-0851-reef 2024-03-23T07:58:31.346 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-954.g82316b04 2024-03-23T07:58:31.348 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-23T07:58:31.354 INFO:teuthology.task.internal:no buildpackages task found 2024-03-23T07:58:31.354 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-23T07:58:31.396 INFO:teuthology.task.internal:Saving configuration 2024-03-23T07:58:31.416 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-23T07:58:31.422 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T07:58:31.447 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 07:53:25.992070', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL4i+W5DRwmOgZYbJcLOoOi0qrYObgZlV3JzOakKuREzK1auMEGXJcspeAP0YUg1yvK5fw5bJnrt0W1FhFAuwvQ='} 2024-03-23T07:58:31.467 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 07:53:25.994032', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTbhBHKlOBrLX4ErpDkLCeYOQu5jeSrdCwEznFfBbS3/vWxLCqB5gBuhEh2zoQdrvPy5mnUUpPe8Qn7GueCE5g='} 2024-03-23T07:58:31.467 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-23T07:58:31.509 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-23T07:58:31.509 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-23T07:58:31.509 INFO:teuthology.run_tasks:Running task console_log... 2024-03-23T07:58:31.627 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9956213160>, signals=[15]) 2024-03-23T07:58:31.627 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-23T07:58:31.633 INFO:teuthology.task.internal:Opening connections... 2024-03-23T07:58:31.633 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T07:58:31.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:58:31.710 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-03-23T07:58:31.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T07:58:31.783 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-23T07:58:31.790 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-03-23T07:58:31.812 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-03-23T07:58:31.812 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-03-23T07:58:31.874 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T07:58:31.875 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-23T07:58:31.875 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T07:58:31.875 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-03-23T07:58:31.896 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-03-23T07:58:31.918 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-03-23T07:58:31.918 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-03-23T07:58:31.979 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-03-23T07:58:31.979 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="8" 2024-03-23T07:58:31.979 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-03-23T07:58:31.979 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-03-23T07:58:31.979 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="8" 2024-03-23T07:58:31.980 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el8" 2024-03-23T07:58:31.980 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-23T07:58:31.980 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-03-23T07:58:31.980 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-23T07:58:31.980 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-03-23T07:58:31.980 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-23T07:58:31.980 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-23T07:58:31.980 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-23T07:58:31.980 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-03-23T07:58:31.996 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-23T07:58:32.005 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-23T07:58:32.035 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-23T07:58:32.035 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-03-23T07:58:32.038 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-03-23T07:58:32.057 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-23T07:58:32.065 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-23T07:58:32.066 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-03-23T07:58:32.099 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-03-23T07:58:32.135 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-23T07:58:32.186 INFO:teuthology.run_tasks:Running task kernel... 2024-03-23T07:58:32.237 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-23T07:58:32.237 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-23T07:58:32.238 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T07:58:32.238 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T07:58:32.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:58:32.258 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-03-23T07:58:32.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T07:58:32.259 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-03-23T07:58:32.319 INFO:teuthology.orchestra.run.smithi193.stdout:4.18.0-489.el8.x86_64 2024-03-23T07:58:32.319 INFO:teuthology.task.kernel:Running kernel on smithi193: 4.18.0-489.el8.x86_64 2024-03-23T07:58:32.319 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2024-03-23T07:58:32.320 INFO:teuthology.orchestra.run.smithi070.stdout:4.18.0-489.el8.x86_64 2024-03-23T07:58:32.320 INFO:teuthology.task.kernel:Running kernel on smithi070: 4.18.0-489.el8.x86_64 2024-03-23T07:58:32.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-03-23T07:58:38.062 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 8 - AppStream 8.8 MB/s | 28 MB 00:03 2024-03-23T07:58:38.342 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 8 - AppStream 8.1 MB/s | 28 MB 00:03 2024-03-23T07:58:44.295 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 8 - BaseOS 8.5 MB/s | 10 MB 00:01 2024-03-23T07:58:44.509 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 8 - BaseOS 8.8 MB/s | 10 MB 00:01 2024-03-23T07:58:46.320 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 8 - Extras 43 kB/s | 18 kB 00:00 2024-03-23T07:58:46.526 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 8 - Extras 47 kB/s | 18 kB 00:00 2024-03-23T07:58:46.796 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 8 - Extras common packages 20 kB/s | 7.5 kB 00:00 2024-03-23T07:58:47.010 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 8 - Extras common packages 20 kB/s | 7.5 kB 00:00 2024-03-23T07:58:47.030 INFO:teuthology.orchestra.run.smithi070.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-03-23T07:58:47.229 INFO:teuthology.orchestra.run.smithi193.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-03-23T07:58:48.254 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2024-03-23T07:58:49.564 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 16 MB 00:02 2024-03-23T07:58:51.979 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 422 kB/s | 24 kB 00:00 2024-03-23T07:58:53.297 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 418 kB/s | 24 kB 00:00 2024-03-23T07:58:55.185 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-03-23T07:58:55.185 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-03-23T07:58:55.298 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T07:58:55.298 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T07:58:55.298 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 4.18.0-546.el8 baseos 10 M 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-core x86_64 4.18.0-546.el8 baseos 44 M 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules x86_64 4.18.0-546.el8 baseos 36 M 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout:Install 3 Packages 2024-03-23T07:58:55.299 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T07:58:55.300 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 90 M 2024-03-23T07:58:55.300 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 97 M 2024-03-23T07:58:55.300 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T07:58:56.294 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-03-23T07:58:56.294 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-03-23T07:58:56.411 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T07:58:56.412 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T07:58:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-03-23T07:58:56.412 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T07:58:56.412 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T07:58:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 4.18.0-546.el8 baseos 10 M 2024-03-23T07:58:56.412 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T07:58:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core x86_64 4.18.0-546.el8 baseos 44 M 2024-03-23T07:58:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules x86_64 4.18.0-546.el8 baseos 36 M 2024-03-23T07:58:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T07:58:56.413 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T07:58:56.413 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T07:58:56.413 INFO:teuthology.orchestra.run.smithi070.stdout:Install 3 Packages 2024-03-23T07:58:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T07:58:56.413 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 90 M 2024-03-23T07:58:56.413 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 97 M 2024-03-23T07:58:56.414 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T07:58:56.506 INFO:teuthology.orchestra.run.smithi193.stdout:(1/3): kernel-4.18.0-546.el8.x86_64.rpm 10 MB/s | 10 MB 00:01 2024-03-23T07:58:57.473 INFO:teuthology.orchestra.run.smithi070.stdout:(1/3): kernel-4.18.0-546.el8.x86_64.rpm 10 MB/s | 10 MB 00:01 2024-03-23T07:58:57.497 INFO:teuthology.orchestra.run.smithi193.stdout:(2/3): kernel-modules-4.18.0-546.el8.x86_64.rpm 18 MB/s | 36 MB 00:02 2024-03-23T07:58:58.056 INFO:teuthology.orchestra.run.smithi193.stdout:(3/3): kernel-core-4.18.0-546.el8.x86_64.rpm 17 MB/s | 44 MB 00:02 2024-03-23T07:58:58.056 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T07:58:58.056 INFO:teuthology.orchestra.run.smithi193.stdout:Total 33 MB/s | 90 MB 00:02 2024-03-23T07:58:58.498 INFO:teuthology.orchestra.run.smithi070.stdout:(2/3): kernel-modules-4.18.0-546.el8.x86_64.rpm 18 MB/s | 36 MB 00:02 2024-03-23T07:58:58.739 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T07:58:59.057 INFO:teuthology.orchestra.run.smithi070.stdout:(3/3): kernel-core-4.18.0-546.el8.x86_64.rpm 17 MB/s | 44 MB 00:02 2024-03-23T07:58:59.057 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T07:58:59.058 INFO:teuthology.orchestra.run.smithi070.stdout:Total 34 MB/s | 90 MB 00:02 2024-03-23T07:58:59.193 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T07:58:59.193 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T07:58:59.743 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T07:59:00.155 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T07:59:00.155 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T07:59:00.431 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T07:59:00.454 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T07:59:01.385 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T07:59:01.409 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T07:59:03.227 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T07:59:03.767 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-core-4.18.0-546.el8.x86_64 1/3 2024-03-23T07:59:04.177 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T07:59:04.690 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-core-4.18.0-546.el8.x86_64 1/3 2024-03-23T07:59:07.315 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-4.18.0-546.el8.x86_64 1/3 2024-03-23T07:59:07.699 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-modules-4.18.0-546.el8.x86_64 2/3 2024-03-23T07:59:08.275 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-4.18.0-546.el8.x86_64 1/3 2024-03-23T07:59:08.681 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-4.18.0-546.el8.x86_64 2/3 2024-03-23T07:59:11.138 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-4.18.0-546.el8.x86_64 2/3 2024-03-23T07:59:11.414 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:12.110 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-4.18.0-546.el8.x86_64 2/3 2024-03-23T07:59:12.397 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:33.476 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:33.498 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:33.848 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:33.848 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-4.18.0-546.el8.x86_64 1/3 2024-03-23T07:59:33.848 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-core-4.18.0-546.el8.x86_64 2/3 2024-03-23T07:59:34.210 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:34.233 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:34.359 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-modules-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:34.359 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T07:59:34.359 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T07:59:34.359 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-4.18.0-546.el8.x86_64 kernel-core-4.18.0-546.el8.x86_64 2024-03-23T07:59:34.359 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-4.18.0-546.el8.x86_64 2024-03-23T07:59:34.359 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T07:59:34.360 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T07:59:34.498 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T07:59:34.664 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:34.664 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-4.18.0-546.el8.x86_64 1/3 2024-03-23T07:59:34.665 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-core-4.18.0-546.el8.x86_64 2/3 2024-03-23T07:59:34.754 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-4.18.0-546.el8.x86_64 2024-03-23T07:59:34.754 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-546.el8.x86_64 2024-03-23T07:59:34.754 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-546.el8.x86_64 2024-03-23T07:59:34.755 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-23T07:59:34.755 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-23T07:59:34.755 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-23T07:59:34.755 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi193.front.sepia.ceph.com, path=None, version=distro) 2024-03-23T07:59:34.755 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T07:59:34.967 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-4.18.0-546.el8.x86_64 2024-03-23T07:59:34.967 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-546.el8.x86_64 2024-03-23T07:59:34.967 DEBUG:teuthology.orchestra.run.smithi193:> sudo rpm -qi grub2-tools 2024-03-23T07:59:35.000 INFO:teuthology.orchestra.run.smithi193.stdout:Name : grub2-tools 2024-03-23T07:59:35.000 INFO:teuthology.orchestra.run.smithi193.stdout:Epoch : 1 2024-03-23T07:59:35.000 INFO:teuthology.orchestra.run.smithi193.stdout:Version : 2.02 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Release : 129.el8 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Architecture: x86_64 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Group : System Environment/Base 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Size : 9265586 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:License : GPLv3+ 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Build Host : kbuilder.bsys.centos.org 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Relocations : (not relocatable) 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Packager : CentOS BuildSystem 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Vendor : CentOS 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T07:59:35.001 INFO:teuthology.orchestra.run.smithi193.stdout:Summary : Support tools for GRUB. 2024-03-23T07:59:35.002 INFO:teuthology.orchestra.run.smithi193.stdout:Description : 2024-03-23T07:59:35.002 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T07:59:35.002 INFO:teuthology.orchestra.run.smithi193.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T07:59:35.002 INFO:teuthology.orchestra.run.smithi193.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T07:59:35.002 INFO:teuthology.orchestra.run.smithi193.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T07:59:35.002 INFO:teuthology.orchestra.run.smithi193.stdout:hardware devices. 2024-03-23T07:59:35.002 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T07:59:35.002 INFO:teuthology.orchestra.run.smithi193.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T07:59:35.003 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T07:59:35.003 INFO:teuthology.task.kernel:Updating grub on smithi193 to boot 4.18.0-546.el8.x86_64 2024-03-23T07:59:35.003 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T07:59:35.491 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-4.18.0-546.el8.x86_64 3/3 2024-03-23T07:59:35.492 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T07:59:35.492 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T07:59:35.492 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-4.18.0-546.el8.x86_64 kernel-core-4.18.0-546.el8.x86_64 2024-03-23T07:59:35.492 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-4.18.0-546.el8.x86_64 2024-03-23T07:59:35.492 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T07:59:35.492 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T07:59:35.662 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T07:59:35.879 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-4.18.0-546.el8.x86_64 2024-03-23T07:59:35.879 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-546.el8.x86_64 2024-03-23T07:59:35.879 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-546.el8.x86_64 2024-03-23T07:59:35.879 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-23T07:59:35.880 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-23T07:59:35.880 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-23T07:59:35.880 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2024-03-23T07:59:35.880 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-03-23T07:59:36.090 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-4.18.0-546.el8.x86_64 2024-03-23T07:59:36.090 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-546.el8.x86_64 2024-03-23T07:59:36.090 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-03-23T07:59:36.123 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.02 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 129.el8 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Group : System Environment/Base 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 9265586 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-03-23T07:59:36.124 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : kbuilder.bsys.centos.org 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:Relocations : (not relocatable) 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : CentOS BuildSystem 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T07:59:36.125 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-03-23T07:59:36.126 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-23T07:59:36.126 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 4.18.0-546.el8.x86_64 2024-03-23T07:59:36.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-23T07:59:36.658 INFO:teuthology.orchestra.run.smithi193.stderr:Generating grub configuration file ... 2024-03-23T07:59:37.399 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-03-23T07:59:38.540 INFO:teuthology.orchestra.run.smithi193.stderr:done 2024-03-23T07:59:38.541 DEBUG:teuthology.orchestra.run.smithi193:> mktemp 2024-03-23T07:59:38.572 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.omB4tUZwei 2024-03-23T07:59:38.572 DEBUG:teuthology.orchestra.run.smithi193:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.omB4tUZwei 2024-03-23T07:59:38.644 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.omB4tUZwei 2024-03-23T07:59:38.785 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.omB4tUZwei is 6KB 2024-03-23T07:59:38.833 DEBUG:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.omB4tUZwei 2024-03-23T07:59:38.851 DEBUG:teuthology.orchestra.run.smithi193:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T07:59:38.915 INFO:teuthology.orchestra.run.smithi193.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-03-23T07:59:38.915 INFO:teuthology.orchestra.run.smithi193.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-03-23T07:59:38.915 INFO:teuthology.orchestra.run.smithi193.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-03-23T07:59:38.915 INFO:teuthology.orchestra.run.smithi193.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-546.el8.x86_64.conf 2024-03-23T07:59:38.916 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-546.el8.x86_64 2024-03-23T07:59:39.088 DEBUG:teuthology.orchestra.run.smithi193:> sudo shutdown -r now 2024-03-23T07:59:39.335 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-03-23T07:59:39.337 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-03-23T07:59:39.360 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.7lwed20veA 2024-03-23T07:59:39.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7lwed20veA 2024-03-23T07:59:39.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.7lwed20veA 2024-03-23T07:59:39.578 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.7lwed20veA is 6KB 2024-03-23T07:59:39.626 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.7lwed20veA 2024-03-23T07:59:39.644 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-03-23T07:59:39.729 INFO:teuthology.orchestra.run.smithi070.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-03-23T07:59:39.729 INFO:teuthology.orchestra.run.smithi070.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-03-23T07:59:39.729 INFO:teuthology.orchestra.run.smithi070.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-03-23T07:59:39.730 INFO:teuthology.orchestra.run.smithi070.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-546.el8.x86_64.conf 2024-03-23T07:59:39.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-546.el8.x86_64 2024-03-23T07:59:39.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-03-23T08:00:09.120 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-23T08:00:09.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-03-23T08:00:09.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T08:00:09.859 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-23T08:00:09.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-03-23T08:00:09.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T08:00:27.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-03-23T08:00:28.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-03-23T08:00:36.684 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-03-23T08:00:36.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T08:00:37.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-03-23T08:00:37.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T08:00:39.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-03-23T08:00:51.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-03-23T08:00:51.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T08:01:10.175 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-03-23T08:01:25.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-03-23T08:01:25.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T08:01:26.041 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-03-23T08:01:26.861 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-03-23T08:01:26.861 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-546.el8.x86_64"... 2024-03-23T08:01:26.861 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-03-23T08:01:26.918 INFO:teuthology.orchestra.run.smithi193.stdout:4.18.0-546.el8.x86_64 2024-03-23T08:01:26.919 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-546.el8.x86_64 vs 4.18.0-546.el8.x86_64 2024-03-23T08:01:26.919 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T08:01:26.919 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-23T08:01:27.920 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-23T08:01:27.921 DEBUG:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-23T08:01:27.996 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2024-03-23T08:01:28.010 DEBUG:teuthology.parallel:result is None 2024-03-23T08:01:37.194 DEBUG:teuthology.orchestra.remote:timed out 2024-03-23T08:01:49.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-03-23T08:01:49.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T08:01:49.561 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-03-23T08:01:49.831 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-03-23T08:01:49.831 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-546.el8.x86_64"... 2024-03-23T08:01:49.831 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-03-23T08:01:49.850 INFO:teuthology.orchestra.run.smithi070.stdout:4.18.0-546.el8.x86_64 2024-03-23T08:01:49.850 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-546.el8.x86_64 vs 4.18.0-546.el8.x86_64 2024-03-23T08:01:49.850 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T08:01:49.850 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-23T08:01:50.851 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-23T08:01:50.851 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-23T08:01:50.939 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-03-23T08:01:50.952 DEBUG:teuthology.parallel:result is None 2024-03-23T08:01:50.953 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-23T08:01:50.961 INFO:teuthology.task.internal:Creating test directory... 2024-03-23T08:01:50.961 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T08:01:50.963 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T08:01:50.985 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-23T08:01:51.072 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-23T08:01:51.118 INFO:teuthology.task.internal:Creating archive directory... 2024-03-23T08:01:51.118 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T08:01:51.120 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T08:01:51.175 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-23T08:01:51.181 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-23T08:01:51.181 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T08:01:51.184 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T08:01:51.221 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T08:01:51.231 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T08:01:51.237 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T08:01:51.248 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T08:01:51.250 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-23T08:01:51.256 INFO:teuthology.task.internal:Configuring sudo... 2024-03-23T08:01:51.256 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T08:01:51.275 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T08:01:51.305 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-23T08:01:51.314 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-23T08:01:51.315 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T08:01:51.345 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T08:01:51.364 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T08:01:51.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T08:01:51.499 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T08:01:51.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T08:01:51.628 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:01:51.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T08:01:51.696 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T08:01:51.745 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T08:01:51.801 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T08:01:51.888 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T08:01:51.932 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:01:51.933 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T08:01:52.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-03-23T08:01:52.003 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-03-23T08:01:52.049 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T08:01:52.116 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T08:01:52.498 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-23T08:01:52.506 INFO:teuthology.task.internal:Starting timer... 2024-03-23T08:01:52.506 INFO:teuthology.run_tasks:Running task pcp... 2024-03-23T08:01:52.625 INFO:teuthology.run_tasks:Running task selinux... 2024-03-23T08:01:52.656 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-23T08:01:52.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-03-23T08:01:52.732 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-03-23T08:01:52.733 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-03-23T08:01:52.797 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-03-23T08:01:52.799 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-23T08:01:52.799 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-03-23T08:01:52.848 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-03-23T08:01:52.848 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-03-23T08:01:52.902 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-03-23T08:01:52.902 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-03-23T08:01:52.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T08:01:52.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T08:01:52.931 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T08:01:52.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T08:01:52.960 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-23T08:01:52.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-03-23T08:01:52.998 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-03-23T08:01:53.030 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-23T08:01:53.039 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-23T08:01:53.055 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-23T08:01:53.056 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-23T08:08:28.468 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-03-23T08:08:28.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-03-23T08:08:28.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T08:08:28.553 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-03-23T08:08:28.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-03-23T08:08:28.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-03-23T08:08:28.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T08:08:28.698 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-03-23T08:08:28.768 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-03-23T08:08:28.769 INFO:teuthology.run_tasks:Running task clock... 2024-03-23T08:08:28.780 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-23T08:08:28.780 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T08:08:28.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T08:08:28.782 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T08:08:28.783 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T08:08:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T08:08:28.824 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T08:08:28.840 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-23T08:08:28.853 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-23T08:08:28.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-03-23T08:08:28.898 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-03-23T08:08:28.911 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T08:08:28.916 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-03-23T08:08:28.924 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T08:08:28.927 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-03-23T08:08:28.940 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-23T08:08:28.953 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-23T08:08:28.996 INFO:teuthology.orchestra.run.smithi070.stderr:bash: ntpq: command not found 2024-03-23T08:08:29.000 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T08:08:29.000 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-03-23T08:08:29.000 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:08:29.000 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:08:29.000 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:08:29.000 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:08:29.021 INFO:teuthology.orchestra.run.smithi193.stderr:bash: ntpq: command not found 2024-03-23T08:08:29.025 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T08:08:29.025 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-03-23T08:08:29.026 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:08:29.026 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:08:29.026 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:08:29.026 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:08:29.027 INFO:teuthology.run_tasks:Running task pexec... 2024-03-23T08:08:29.036 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-23T08:08:29.036 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:08:29.036 DEBUG:teuthology.orchestra.run.smithi070:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-23T08:08:29.037 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi193.front.sepia.ceph.com 2024-03-23T08:08:29.037 DEBUG:teuthology.orchestra.run.smithi193:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-23T08:08:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:20 ago on Sat 23 Mar 2024 08:07:09 AM UTC. 2024-03-23T08:08:29.823 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:19 ago on Sat 23 Mar 2024 08:07:10 AM UTC. 2024-03-23T08:08:31.887 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:08:31.887 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:08:31.888 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-03-23T08:08:31.888 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:08:31.888 INFO:teuthology.orchestra.run.smithi070.stdout:Resetting modules: 2024-03-23T08:08:31.888 INFO:teuthology.orchestra.run.smithi070.stdout: container-tools 2024-03-23T08:08:31.888 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:08:31.888 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:08:31.888 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:08:31.888 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:08:31.931 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:08:31.932 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:08:31.932 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-03-23T08:08:31.932 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:08:31.932 INFO:teuthology.orchestra.run.smithi193.stdout:Resetting modules: 2024-03-23T08:08:31.932 INFO:teuthology.orchestra.run.smithi193.stdout: container-tools 2024-03-23T08:08:31.932 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:08:31.932 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:08:31.933 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:08:31.933 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:08:32.139 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:08:32.182 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:08:32.962 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:23 ago on Sat 23 Mar 2024 08:07:09 AM UTC. 2024-03-23T08:08:32.999 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:22 ago on Sat 23 Mar 2024 08:07:10 AM UTC. 2024-03-23T08:08:35.056 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout:Installing group/module packages: 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-03-23T08:08:35.093 INFO:teuthology.orchestra.run.smithi070.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout:Installing module profiles: 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout: container-tools/common 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling module streams: 2024-03-23T08:08:35.094 INFO:teuthology.orchestra.run.smithi070.stdout: container-tools rhel8 2024-03-23T08:08:35.095 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:08:35.095 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:08:35.095 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:08:35.095 INFO:teuthology.orchestra.run.smithi070.stdout:Install 13 Packages 2024-03-23T08:08:35.095 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:08:35.095 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 27 M 2024-03-23T08:08:35.095 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 91 M 2024-03-23T08:08:35.095 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:08:35.112 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout:Installing group/module packages: 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-03-23T08:08:35.149 INFO:teuthology.orchestra.run.smithi193.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout:Installing module profiles: 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: container-tools/common 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout:Enabling module streams: 2024-03-23T08:08:35.150 INFO:teuthology.orchestra.run.smithi193.stdout: container-tools rhel8 2024-03-23T08:08:35.151 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:08:35.151 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:08:35.151 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:08:35.151 INFO:teuthology.orchestra.run.smithi193.stdout:Install 13 Packages 2024-03-23T08:08:35.151 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:08:35.151 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 27 M 2024-03-23T08:08:35.151 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 91 M 2024-03-23T08:08:35.151 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:08:35.409 INFO:teuthology.orchestra.run.smithi070.stdout:(1/13): cockpit-podman-84.1-1.module_el8+977+8a 2.6 MB/s | 689 kB 00:00 2024-03-23T08:08:35.560 INFO:teuthology.orchestra.run.smithi193.stdout:(1/13): cockpit-podman-84.1-1.module_el8+977+8a 3.2 MB/s | 689 kB 00:00 2024-03-23T08:08:35.668 INFO:teuthology.orchestra.run.smithi193.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 3.1 MB/s | 1.0 MB 00:00 2024-03-23T08:08:35.693 INFO:teuthology.orchestra.run.smithi070.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 1.9 MB/s | 1.0 MB 00:00 2024-03-23T08:08:35.860 INFO:teuthology.orchestra.run.smithi070.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 84 kB/s | 38 kB 00:00 2024-03-23T08:08:35.868 INFO:teuthology.orchestra.run.smithi193.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 123 kB/s | 38 kB 00:00 2024-03-23T08:08:35.968 INFO:teuthology.orchestra.run.smithi070.stdout:(4/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 936 kB/s | 258 kB 00:00 2024-03-23T08:08:35.969 INFO:teuthology.orchestra.run.smithi193.stdout:(4/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 858 kB/s | 258 kB 00:00 2024-03-23T08:08:36.094 INFO:teuthology.orchestra.run.smithi070.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.4 MB/s | 180 kB 00:00 2024-03-23T08:08:36.102 INFO:teuthology.orchestra.run.smithi193.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.3 MB/s | 180 kB 00:00 2024-03-23T08:08:36.302 INFO:teuthology.orchestra.run.smithi070.stdout:(6/13): buildah-1.33.6-2.module_el8+976+1136073 8.4 MB/s | 9.6 MB 00:01 2024-03-23T08:08:36.327 INFO:teuthology.orchestra.run.smithi070.stdout:(7/13): python3-pytoml-0.1.14-5.git7dea353.el8. 106 kB/s | 25 kB 00:00 2024-03-23T08:08:36.344 INFO:teuthology.orchestra.run.smithi193.stdout:(6/13): buildah-1.33.6-2.module_el8+976+1136073 9.7 MB/s | 9.6 MB 00:00 2024-03-23T08:08:36.370 INFO:teuthology.orchestra.run.smithi193.stdout:(7/13): python3-pytoml-0.1.14-5.git7dea353.el8. 93 kB/s | 25 kB 00:00 2024-03-23T08:08:36.620 INFO:teuthology.orchestra.run.smithi193.stdout:(8/13): python3-pyxdg-0.25-16.el8.noarch.rpm 341 kB/s | 94 kB 00:00 2024-03-23T08:08:36.628 INFO:teuthology.orchestra.run.smithi070.stdout:(8/13): python3-pyxdg-0.25-16.el8.noarch.rpm 289 kB/s | 94 kB 00:00 2024-03-23T08:08:36.711 INFO:teuthology.orchestra.run.smithi070.stdout:(9/13): netavark-1.10.3-1.module_el8+951+32019c 4.8 MB/s | 4.1 MB 00:00 2024-03-23T08:08:36.853 INFO:teuthology.orchestra.run.smithi070.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 371 kB/s | 53 kB 00:00 2024-03-23T08:08:36.887 INFO:teuthology.orchestra.run.smithi193.stdout:(9/13): netavark-1.10.3-1.module_el8+951+32019c 4.0 MB/s | 4.1 MB 00:01 2024-03-23T08:08:36.937 INFO:teuthology.orchestra.run.smithi070.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 491 kB/s | 41 kB 00:00 2024-03-23T08:08:36.945 INFO:teuthology.orchestra.run.smithi193.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 906 kB/s | 53 kB 00:00 2024-03-23T08:08:37.004 INFO:teuthology.orchestra.run.smithi193.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 700 kB/s | 41 kB 00:00 2024-03-23T08:08:37.095 INFO:teuthology.orchestra.run.smithi070.stdout:(12/13): toolbox-0.0.99.5-2.module_el8+971+3d3d 5.4 MB/s | 2.5 MB 00:00 2024-03-23T08:08:37.154 INFO:teuthology.orchestra.run.smithi193.stdout:(12/13): toolbox-0.0.99.5-2.module_el8+971+3d3d 4.7 MB/s | 2.5 MB 00:00 2024-03-23T08:08:37.472 INFO:teuthology.orchestra.run.smithi193.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 8.0 MB/s | 8.8 MB 00:01 2024-03-23T08:08:37.472 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:08:37.472 INFO:teuthology.orchestra.run.smithi193.stdout:Total 12 MB/s | 27 MB 00:02 2024-03-23T08:08:37.512 INFO:teuthology.orchestra.run.smithi070.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 7.4 MB/s | 8.8 MB 00:01 2024-03-23T08:08:37.513 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:08:37.513 INFO:teuthology.orchestra.run.smithi070.stdout:Total 11 MB/s | 27 MB 00:02 2024-03-23T08:08:37.741 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:08:37.762 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:08:37.762 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:08:37.782 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:08:37.803 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:08:37.803 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:08:38.018 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:08:38.022 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:08:38.074 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:08:38.077 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:08:38.656 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:08:38.767 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:08:39.579 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-03-23T08:08:39.745 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-03-23T08:08:39.753 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-03-23T08:08:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-03-23T08:08:39.932 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-03-23T08:08:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-03-23T08:08:40.127 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-03-23T08:08:40.271 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-03-23T08:08:40.343 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-03-23T08:08:40.494 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-03-23T08:08:40.926 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-03-23T08:08:41.072 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-03-23T08:08:41.896 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-03-23T08:08:42.043 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-03-23T08:08:42.122 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-03-23T08:08:42.246 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-03-23T08:08:42.342 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-03-23T08:08:42.450 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-03-23T08:08:42.814 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-03-23T08:08:42.929 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-03-23T08:08:43.013 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-03-23T08:08:43.154 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-03-23T08:08:43.216 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-03-23T08:08:43.376 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-03-23T08:08:43.423 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-03-23T08:08:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-03-23T08:08:43.844 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-03-23T08:08:43.844 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-03-23T08:08:43.844 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-03-23T08:08:43.845 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-03-23T08:08:43.989 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-03-23T08:08:44.387 INFO:teuthology.orchestra.run.smithi193.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-03-23T08:08:44.388 INFO:teuthology.orchestra.run.smithi193.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-03-23T08:08:44.388 INFO:teuthology.orchestra.run.smithi193.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-23T08:08:44.388 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:08:44.388 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-03-23T08:08:44.621 INFO:teuthology.orchestra.run.smithi070.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-03-23T08:08:44.622 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-23T08:08:44.622 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-23T08:08:44.622 INFO:teuthology.orchestra.run.smithi070.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-03-23T08:08:44.622 INFO:teuthology.orchestra.run.smithi070.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-03-23T08:08:44.622 INFO:teuthology.orchestra.run.smithi070.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-03-23T08:08:44.622 INFO:teuthology.orchestra.run.smithi070.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-23T08:08:44.622 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:08:44.622 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:08:44.738 DEBUG:teuthology.parallel:result is None 2024-03-23T08:08:44.907 DEBUG:teuthology.parallel:result is None 2024-03-23T08:08:44.907 INFO:teuthology.run_tasks:Running task install... 2024-03-23T08:08:44.916 DEBUG:teuthology.task.install:project ceph 2024-03-23T08:08:44.916 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'}} 2024-03-23T08:08:44.916 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'} 2024-03-23T08:08:44.916 INFO:teuthology.task.install:Using flavor: default 2024-03-23T08:08:44.925 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-23T08:08:44.925 INFO:teuthology.task.install:extra packages: [] 2024-03-23T08:08:44.925 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'tag': None, 'wait_for_package': False} 2024-03-23T08:08:44.925 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T08:08:44.925 INFO:teuthology.packaging:ref: None 2024-03-23T08:08:44.926 INFO:teuthology.packaging:tag: None 2024-03-23T08:08:44.926 INFO:teuthology.packaging:branch: quincy 2024-03-23T08:08:44.926 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:08:44.926 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-03-23T08:08:44.928 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837', 'tag': None, 'wait_for_package': False} 2024-03-23T08:08:44.928 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T08:08:44.928 INFO:teuthology.packaging:ref: None 2024-03-23T08:08:44.928 INFO:teuthology.packaging:tag: None 2024-03-23T08:08:44.928 INFO:teuthology.packaging:branch: quincy 2024-03-23T08:08:44.928 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:08:44.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-03-23T08:08:45.219 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/8/flavors/default/ 2024-03-23T08:08:45.220 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-23T08:08:45.317 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/8/flavors/default/ 2024-03-23T08:08:45.318 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-23T08:08:45.416 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T08:08:45.417 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:08:45.417 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T08:08:45.445 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-23T08:08:45.445 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T08:08:45.445 INFO:teuthology.packaging:ref: None 2024-03-23T08:08:45.445 INFO:teuthology.packaging:tag: None 2024-03-23T08:08:45.445 INFO:teuthology.packaging:branch: quincy 2024-03-23T08:08:45.445 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:08:45.445 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T08:08:45.513 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T08:08:45.539 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-23T08:08:45.539 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:08:45.539 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-23T08:08:45.567 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-23T08:08:45.567 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T08:08:45.568 INFO:teuthology.packaging:ref: None 2024-03-23T08:08:45.568 INFO:teuthology.packaging:tag: None 2024-03-23T08:08:45.568 INFO:teuthology.packaging:branch: quincy 2024-03-23T08:08:45.568 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:08:45.568 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-23T08:08:45.589 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T08:08:45.635 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-23T08:08:45.655 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-03-23T08:08:45.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-03-23T08:08:45.712 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-23T08:08:45.779 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-03-23T08:08:45.780 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-03-23T08:08:45.927 INFO:teuthology.orchestra.run.smithi070.stdout:76 files removed 2024-03-23T08:08:45.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw 2024-03-23T08:08:46.044 INFO:teuthology.orchestra.run.smithi193.stdout:76 files removed 2024-03-23T08:08:46.071 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw 2024-03-23T08:08:46.628 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 298 kB/s | 78 kB 00:00 2024-03-23T08:08:46.917 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 180 kB/s | 78 kB 00:00 2024-03-23T08:08:46.962 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 31 kB/s | 9.1 kB 00:00 2024-03-23T08:08:47.309 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 26 kB/s | 9.1 kB 00:00 2024-03-23T08:08:47.370 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2024-03-23T08:08:47.651 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 4.6 kB/s | 1.5 kB 00:00 2024-03-23T08:08:50.744 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 8 - AppStream 9.0 MB/s | 28 MB 00:03 2024-03-23T08:08:51.479 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 8 - AppStream 6.8 MB/s | 28 MB 00:04 2024-03-23T08:08:56.886 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 8 - BaseOS 9.4 MB/s | 10 MB 00:01 2024-03-23T08:08:57.537 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 8 - BaseOS 10 MB/s | 10 MB 00:01 2024-03-23T08:08:58.770 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 8 - Extras 68 kB/s | 18 kB 00:00 2024-03-23T08:08:59.236 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 8 - Extras common packages 21 kB/s | 7.5 kB 00:00 2024-03-23T08:08:59.563 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 8 - Extras 44 kB/s | 18 kB 00:00 2024-03-23T08:09:00.047 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 8 - Extras common packages 20 kB/s | 7.5 kB 00:00 2024-03-23T08:09:57.191 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 8 - PowerTools 87 kB/s | 4.9 MB 00:57 2024-03-23T08:09:57.964 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 8 - PowerTools 87 kB/s | 4.9 MB 00:57 2024-03-23T08:09:58.241 INFO:teuthology.orchestra.run.smithi193.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-03-23T08:09:59.039 INFO:teuthology.orchestra.run.smithi070.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-03-23T08:10:00.565 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-03-23T08:10:01.458 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 5.2 MB/s | 16 MB 00:03 2024-03-23T08:10:04.348 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 428 kB/s | 24 kB 00:00 2024-03-23T08:10:05.250 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 395 kB/s | 24 kB 00:00 2024-03-23T08:10:07.922 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 17 M 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 3.5 M 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 3.6 M 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 5.2 M 2024-03-23T08:10:07.925 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 23 M 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 24 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 798 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 506 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 5.6 M 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 45 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 117 k 2024-03-23T08:10:07.926 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 190 k 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 370 k 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 358 k 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 106 k 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout:Install 21 Packages 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-03-23T08:10:07.927 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:10:07.928 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 64 M 2024-03-23T08:10:07.928 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:10:08.855 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:10:08.857 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:10:08.857 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 17 M 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 3.5 M 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 3.6 M 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 5.2 M 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 23 M 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 24 k 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 798 k 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-23T08:10:08.858 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 506 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 5.6 M 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 45 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 117 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 190 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 370 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 358 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 106 k 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-23T08:10:08.859 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:10:08.860 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:10:08.860 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:10:08.860 INFO:teuthology.orchestra.run.smithi193.stdout:Install 21 Packages 2024-03-23T08:10:08.860 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 2 Packages 2024-03-23T08:10:08.860 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:10:08.860 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 64 M 2024-03-23T08:10:08.860 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:10:08.999 INFO:teuthology.orchestra.run.smithi070.stdout:(1/23): ceph-base-17.2.7-726.g9c4e0c34.el8.x86_ 5.5 MB/s | 5.2 MB 00:00 2024-03-23T08:10:09.074 INFO:teuthology.orchestra.run.smithi070.stdout:(2/23): ceph-selinux-17.2.7-726.g9c4e0c34.el8.x 326 kB/s | 24 kB 00:00 2024-03-23T08:10:09.216 INFO:teuthology.orchestra.run.smithi070.stdout:(3/23): libcephfs2-17.2.7-726.g9c4e0c34.el8.x86 5.5 MB/s | 798 kB 00:00 2024-03-23T08:10:09.366 INFO:teuthology.orchestra.run.smithi070.stdout:(4/23): libradosstriper1-17.2.7-726.g9c4e0c34.e 3.3 MB/s | 506 kB 00:00 2024-03-23T08:10:10.291 INFO:teuthology.orchestra.run.smithi070.stdout:(5/23): ceph-radosgw-17.2.7-726.g9c4e0c34.el8.x 7.5 MB/s | 17 MB 00:02 2024-03-23T08:10:10.392 INFO:teuthology.orchestra.run.smithi070.stdout:(6/23): librgw2-17.2.7-726.g9c4e0c34.el8.x86_64 5.4 MB/s | 5.6 MB 00:01 2024-03-23T08:10:10.417 INFO:teuthology.orchestra.run.smithi070.stdout:(7/23): python3-ceph-argparse-17.2.7-726.g9c4e0 357 kB/s | 45 kB 00:00 2024-03-23T08:10:10.467 INFO:teuthology.orchestra.run.smithi070.stdout:(8/23): python3-ceph-common-17.2.7-726.g9c4e0c3 1.5 MB/s | 117 kB 00:00 2024-03-23T08:10:10.492 INFO:teuthology.orchestra.run.smithi070.stdout:(9/23): python3-cephfs-17.2.7-726.g9c4e0c34.el8 2.5 MB/s | 190 kB 00:00 2024-03-23T08:10:10.743 INFO:teuthology.orchestra.run.smithi070.stdout:(10/23): ceph-common-17.2.7-726.g9c4e0c34.el8.x 8.8 MB/s | 23 MB 00:02 2024-03-23T08:10:10.768 INFO:teuthology.orchestra.run.smithi070.stdout:(11/23): python3-rados-17.2.7-726.g9c4e0c34.el8 1.2 MB/s | 370 kB 00:00 2024-03-23T08:10:10.793 INFO:teuthology.orchestra.run.smithi070.stdout:(12/23): python3-rbd-17.2.7-726.g9c4e0c34.el8.x 1.2 MB/s | 358 kB 00:00 2024-03-23T08:10:10.818 INFO:teuthology.orchestra.run.smithi070.stdout:(13/23): python3-rgw-17.2.7-726.g9c4e0c34.el8.x 1.4 MB/s | 106 kB 00:00 2024-03-23T08:10:10.952 INFO:teuthology.orchestra.run.smithi070.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 329 kB/s | 44 kB 00:00 2024-03-23T08:10:10.977 INFO:teuthology.orchestra.run.smithi070.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2024-03-23T08:10:11.003 INFO:teuthology.orchestra.run.smithi070.stdout:(16/23): librdkafka-1.6.1-1.el8.x86_64.rpm 2.8 MB/s | 677 kB 00:00 2024-03-23T08:10:11.117 INFO:teuthology.orchestra.run.smithi193.stdout:(1/23): ceph-base-17.2.7-726.g9c4e0c34.el8.x86_ 2.6 MB/s | 5.2 MB 00:01 2024-03-23T08:10:11.129 INFO:teuthology.orchestra.run.smithi070.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 466 kB/s | 59 kB 00:00 2024-03-23T08:10:11.178 INFO:teuthology.orchestra.run.smithi070.stdout:(18/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.5 MB/s | 306 kB 00:00 2024-03-23T08:10:11.198 INFO:teuthology.orchestra.run.smithi193.stdout:(2/23): ceph-selinux-17.2.7-726.g9c4e0c34.el8.x 303 kB/s | 24 kB 00:00 2024-03-23T08:10:11.203 INFO:teuthology.orchestra.run.smithi070.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2024-03-23T08:10:11.254 INFO:teuthology.orchestra.run.smithi070.stdout:(20/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 156 kB/s | 47 kB 00:00 2024-03-23T08:10:11.362 INFO:teuthology.orchestra.run.smithi070.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 9.4 MB/s | 1.7 MB 00:00 2024-03-23T08:10:11.376 INFO:teuthology.orchestra.run.smithi193.stdout:(3/23): libcephfs2-17.2.7-726.g9c4e0c34.el8.x86 4.4 MB/s | 798 kB 00:00 2024-03-23T08:10:11.568 INFO:teuthology.orchestra.run.smithi193.stdout:(4/23): libradosstriper1-17.2.7-726.g9c4e0c34.e 2.6 MB/s | 506 kB 00:00 2024-03-23T08:10:11.788 INFO:teuthology.orchestra.run.smithi070.stdout:(22/23): librbd1-17.2.7-726.g9c4e0c34.el8.x86_6 6.7 MB/s | 3.6 MB 00:00 2024-03-23T08:10:11.938 INFO:teuthology.orchestra.run.smithi070.stdout:(23/23): librados2-17.2.7-726.g9c4e0c34.el8.x86 4.7 MB/s | 3.5 MB 00:00 2024-03-23T08:10:11.940 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:10:11.940 INFO:teuthology.orchestra.run.smithi070.stdout:Total 16 MB/s | 64 MB 00:04 2024-03-23T08:10:11.981 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:10:12.027 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:10:12.027 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:10:12.389 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:10:12.392 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:10:12.436 INFO:teuthology.orchestra.run.smithi193.stdout:(5/23): librgw2-17.2.7-726.g9c4e0c34.el8.x86_64 6.4 MB/s | 5.6 MB 00:00 2024-03-23T08:10:12.661 INFO:teuthology.orchestra.run.smithi193.stdout:(6/23): ceph-radosgw-17.2.7-726.g9c4e0c34.el8.x 4.8 MB/s | 17 MB 00:03 2024-03-23T08:10:12.677 INFO:teuthology.orchestra.run.smithi193.stdout:(7/23): python3-ceph-argparse-17.2.7-726.g9c4e0 185 kB/s | 45 kB 00:00 2024-03-23T08:10:12.737 INFO:teuthology.orchestra.run.smithi193.stdout:(8/23): python3-ceph-common-17.2.7-726.g9c4e0c3 1.5 MB/s | 117 kB 00:00 2024-03-23T08:10:12.761 INFO:teuthology.orchestra.run.smithi193.stdout:(9/23): python3-cephfs-17.2.7-726.g9c4e0c34.el8 2.2 MB/s | 190 kB 00:00 2024-03-23T08:10:12.820 INFO:teuthology.orchestra.run.smithi193.stdout:(10/23): python3-rados-17.2.7-726.g9c4e0c34.el8 4.3 MB/s | 370 kB 00:00 2024-03-23T08:10:12.854 INFO:teuthology.orchestra.run.smithi193.stdout:(11/23): python3-rbd-17.2.7-726.g9c4e0c34.el8.x 3.8 MB/s | 358 kB 00:00 2024-03-23T08:10:13.112 INFO:teuthology.orchestra.run.smithi193.stdout:(12/23): ceph-common-17.2.7-726.g9c4e0c34.el8.x 5.9 MB/s | 23 MB 00:03 2024-03-23T08:10:13.129 INFO:teuthology.orchestra.run.smithi193.stdout:(13/23): python3-rgw-17.2.7-726.g9c4e0c34.el8.x 345 kB/s | 106 kB 00:00 2024-03-23T08:10:13.263 INFO:teuthology.orchestra.run.smithi193.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 329 kB/s | 44 kB 00:00 2024-03-23T08:10:13.338 INFO:teuthology.orchestra.run.smithi193.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.1 MB/s | 259 kB 00:00 2024-03-23T08:10:13.439 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:10:13.572 INFO:teuthology.orchestra.run.smithi193.stdout:(16/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 152 kB/s | 47 kB 00:00 2024-03-23T08:10:13.656 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-03-23T08:10:13.764 INFO:teuthology.orchestra.run.smithi193.stdout:(17/23): librdkafka-1.6.1-1.el8.x86_64.rpm 744 kB/s | 677 kB 00:00 2024-03-23T08:10:13.841 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-03-23T08:10:13.873 INFO:teuthology.orchestra.run.smithi193.stdout:(18/23): gperftools-libs-2.7-9.el8.x86_64.rpm 573 kB/s | 306 kB 00:00 2024-03-23T08:10:13.951 INFO:teuthology.orchestra.run.smithi193.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 155 kB/s | 59 kB 00:00 2024-03-23T08:10:14.123 INFO:teuthology.orchestra.run.smithi193.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 209 kB/s | 75 kB 00:00 2024-03-23T08:10:14.224 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-03-23T08:10:14.390 INFO:teuthology.orchestra.run.smithi193.stdout:(21/23): librados2-17.2.7-726.g9c4e0c34.el8.x86 7.9 MB/s | 3.5 MB 00:00 2024-03-23T08:10:14.424 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 3/25 2024-03-23T08:10:14.520 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 3/25 2024-03-23T08:10:14.523 INFO:teuthology.orchestra.run.smithi193.stdout:(22/23): librbd1-17.2.7-726.g9c4e0c34.el8.x86_6 9.0 MB/s | 3.6 MB 00:00 2024-03-23T08:10:14.732 INFO:teuthology.orchestra.run.smithi193.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 2.0 MB/s | 1.7 MB 00:00 2024-03-23T08:10:14.735 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:10:14.735 INFO:teuthology.orchestra.run.smithi193.stdout:Total 11 MB/s | 64 MB 00:05 2024-03-23T08:10:14.766 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el8.x86_64 4/25 2024-03-23T08:10:14.778 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:10:14.825 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:10:14.825 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:10:14.933 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/25 2024-03-23T08:10:15.187 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:10:15.191 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:10:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/25 2024-03-23T08:10:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/25 2024-03-23T08:10:15.749 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/25 2024-03-23T08:10:15.912 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-03-23T08:10:16.100 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-03-23T08:10:16.225 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:10:16.417 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-03-23T08:10:16.591 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-03-23T08:10:16.754 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-03-23T08:10:16.891 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el8.x86_64 10/25 2024-03-23T08:10:16.957 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el8.x86_64 10/25 2024-03-23T08:10:16.985 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-03-23T08:10:17.126 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el8.x 11/25 2024-03-23T08:10:17.175 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 3/25 2024-03-23T08:10:17.256 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 3/25 2024-03-23T08:10:17.322 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el8.x86_64 12/25 2024-03-23T08:10:17.516 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el8.x86_64 4/25 2024-03-23T08:10:17.535 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 13/25 2024-03-23T08:10:17.700 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/25 2024-03-23T08:10:17.735 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 14/25 2024-03-23T08:10:17.857 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el8.x86_64 15/25 2024-03-23T08:10:17.930 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el8.x86_64 15/25 2024-03-23T08:10:18.091 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/25 2024-03-23T08:10:18.140 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-03-23T08:10:18.277 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/25 2024-03-23T08:10:18.328 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-03-23T08:10:18.485 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/25 2024-03-23T08:10:18.506 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-03-23T08:10:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-03-23T08:10:18.699 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86 19/25 2024-03-23T08:10:18.833 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-03-23T08:10:19.497 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-03-23T08:10:19.649 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el8.x86_64 10/25 2024-03-23T08:10:19.717 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el8.x86_64 10/25 2024-03-23T08:10:19.876 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el8.x 11/25 2024-03-23T08:10:20.071 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el8.x86_64 12/25 2024-03-23T08:10:20.329 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 13/25 2024-03-23T08:10:20.552 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 14/25 2024-03-23T08:10:20.699 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el8.x86_64 15/25 2024-03-23T08:10:20.765 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el8.x86_64 15/25 2024-03-23T08:10:20.958 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-03-23T08:10:21.197 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-03-23T08:10:21.374 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-03-23T08:10:21.410 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 20/25 2024-03-23T08:10:21.575 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86 19/25 2024-03-23T08:10:21.657 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 20/25 2024-03-23T08:10:22.258 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 20/25 2024-03-23T08:10:22.503 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 21/25 2024-03-23T08:10:22.608 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 21/25 2024-03-23T08:10:22.814 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el8.x86_64 22/25 2024-03-23T08:10:24.248 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 20/25 2024-03-23T08:10:24.474 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 20/25 2024-03-23T08:10:25.073 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 20/25 2024-03-23T08:10:25.326 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 21/25 2024-03-23T08:10:25.429 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 21/25 2024-03-23T08:10:25.632 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el8.x86_64 22/25 2024-03-23T08:10:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el8.x86_64 22/25 2024-03-23T08:10:41.832 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-03-23T08:10:41.832 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-03-23T08:10:41.832 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /dev 2024-03-23T08:10:41.832 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /run 2024-03-23T08:10:41.832 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-03-23T08:10:41.833 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-03-23T08:10:41.833 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-03-23T08:10:41.833 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-03-23T08:10:41.833 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /dev 2024-03-23T08:10:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:10:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 23/25 2024-03-23T08:10:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 23/25 2024-03-23T08:10:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-03-23T08:10:43.812 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-03-23T08:10:43.812 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-03-23T08:10:44.485 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-03-23T08:10:44.485 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/25 2024-03-23T08:10:44.485 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 3/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el8.x86_64 4/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el8.x86_64 7/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el8.x 8/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86 9/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el8.x86_64 10/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el8.x86_64 11/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 12/25 2024-03-23T08:10:44.486 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 13/25 2024-03-23T08:10:44.487 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-03-23T08:10:44.487 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-03-23T08:10:44.487 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-03-23T08:10:44.487 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-03-23T08:10:44.487 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-03-23T08:10:44.487 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-03-23T08:10:44.487 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-03-23T08:10:44.487 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-03-23T08:10:44.488 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 22/25 2024-03-23T08:10:44.488 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-03-23T08:10:44.488 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 24/25 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el8.x86_64 22/25 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /dev 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /run 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /dev 2024-03-23T08:10:44.742 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:10:45.410 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-03-23T08:10:45.410 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:10:45.410 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-23T08:10:45.411 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.412 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.413 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:45.413 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-23T08:10:45.413 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:10:45.413 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:10:45.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-test 2024-03-23T08:10:46.456 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 23/25 2024-03-23T08:10:46.469 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:42 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:10:46.513 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 23/25 2024-03-23T08:10:46.513 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-03-23T08:10:46.713 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-03-23T08:10:46.714 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-03-23T08:10:47.372 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-03-23T08:10:47.372 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/25 2024-03-23T08:10:47.372 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 3/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el8.x86_64 4/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el8.x86_64 7/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el8.x 8/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86 9/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el8.x86_64 10/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el8.x86_64 11/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 12/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 13/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-03-23T08:10:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-03-23T08:10:47.374 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-03-23T08:10:47.374 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-03-23T08:10:47.374 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-03-23T08:10:47.374 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-03-23T08:10:47.374 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-03-23T08:10:47.374 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-03-23T08:10:47.374 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 22/25 2024-03-23T08:10:47.374 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-03-23T08:10:47.374 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 24/25 2024-03-23T08:10:48.088 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:10:48.089 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:10:48.089 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 52 M 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 179 k 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:10:48.090 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-03-23T08:10:48.091 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:10:48.091 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 53 M 2024-03-23T08:10:48.091 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 234 M 2024-03-23T08:10:48.092 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:10:48.167 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-03-23T08:10:48.167 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:10:48.167 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-03-23T08:10:48.167 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.167 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.167 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:10:48.167 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.168 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.169 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.169 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-23T08:10:48.169 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.169 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.169 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:48.169 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-23T08:10:48.169 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:10:48.169 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:10:48.329 INFO:teuthology.orchestra.run.smithi070.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 1.1 MB/s | 203 kB 00:00 2024-03-23T08:10:48.354 INFO:teuthology.orchestra.run.smithi070.stdout:(2/6): libcephsqlite-17.2.7-726.g9c4e0c34.el8.x 851 kB/s | 179 kB 00:00 2024-03-23T08:10:48.380 INFO:teuthology.orchestra.run.smithi070.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-03-23T08:10:48.430 INFO:teuthology.orchestra.run.smithi070.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-23T08:10:48.493 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-test 2024-03-23T08:10:48.539 INFO:teuthology.orchestra.run.smithi070.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2024-03-23T08:10:49.250 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:44 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:10:50.647 INFO:teuthology.orchestra.run.smithi070.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el8.x86_6 21 MB/s | 52 MB 00:02 2024-03-23T08:10:50.649 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:10:50.649 INFO:teuthology.orchestra.run.smithi070.stdout:Total 21 MB/s | 53 MB 00:02 2024-03-23T08:10:50.691 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:10:50.711 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:10:50.711 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:10:50.873 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:10:50.874 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:10:50.874 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:10:50.874 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:10:50.874 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:10:50.874 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 52 M 2024-03-23T08:10:50.874 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T08:10:50.874 INFO:teuthology.orchestra.run.smithi193.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-03-23T08:10:50.874 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 179 k 2024-03-23T08:10:50.875 INFO:teuthology.orchestra.run.smithi193.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-03-23T08:10:50.875 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-03-23T08:10:50.875 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-03-23T08:10:50.875 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:10:50.875 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:10:50.875 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:10:50.875 INFO:teuthology.orchestra.run.smithi193.stdout:Install 6 Packages 2024-03-23T08:10:50.875 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:10:50.876 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 53 M 2024-03-23T08:10:50.876 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 234 M 2024-03-23T08:10:50.876 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:10:50.998 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:10:51.001 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:10:51.247 INFO:teuthology.orchestra.run.smithi193.stdout:(1/6): libcephsqlite-17.2.7-726.g9c4e0c34.el8.x 992 kB/s | 179 kB 00:00 2024-03-23T08:10:51.272 INFO:teuthology.orchestra.run.smithi193.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 986 kB/s | 203 kB 00:00 2024-03-23T08:10:51.306 INFO:teuthology.orchestra.run.smithi193.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-03-23T08:10:51.356 INFO:teuthology.orchestra.run.smithi193.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-23T08:10:51.465 INFO:teuthology.orchestra.run.smithi193.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2024-03-23T08:10:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:10:51.694 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-03-23T08:10:51.877 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-03-23T08:10:52.008 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-03-23T08:10:52.068 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-03-23T08:10:52.282 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-03-23T08:10:52.466 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/6 2024-03-23T08:10:53.573 INFO:teuthology.orchestra.run.smithi193.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el8.x86_6 21 MB/s | 52 MB 00:02 2024-03-23T08:10:53.575 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:10:53.575 INFO:teuthology.orchestra.run.smithi193.stdout:Total 20 MB/s | 53 MB 00:02 2024-03-23T08:10:53.617 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:10:53.638 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:10:53.638 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:10:53.927 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:10:53.931 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:10:54.400 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:10:54.624 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-03-23T08:10:54.804 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-03-23T08:10:54.960 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-03-23T08:10:55.044 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-03-23T08:10:55.234 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-03-23T08:10:55.418 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/6 2024-03-23T08:10:57.470 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/6 2024-03-23T08:10:57.722 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/6 2024-03-23T08:10:58.013 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/6 2024-03-23T08:10:58.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/6 2024-03-23T08:10:58.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/6 2024-03-23T08:10:58.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-03-23T08:10:58.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-03-23T08:10:58.014 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-03-23T08:10:58.437 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-03-23T08:10:58.437 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:10:58.437 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:10:58.437 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:58.438 INFO:teuthology.orchestra.run.smithi070.stdout: jq-1.6-8.el8.x86_64 2024-03-23T08:10:58.438 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:10:58.438 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-03-23T08:10:58.438 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-23T08:10:58.438 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-23T08:10:58.438 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:10:58.438 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:10:58.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph 2024-03-23T08:10:59.418 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:55 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:11:00.390 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/6 2024-03-23T08:11:00.607 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/6 2024-03-23T08:11:00.898 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el8.x86_64 6/6 2024-03-23T08:11:00.898 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/6 2024-03-23T08:11:00.898 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/6 2024-03-23T08:11:00.898 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-03-23T08:11:00.898 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-03-23T08:11:00.898 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-03-23T08:11:01.010 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:11:01.013 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:01.013 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 6.4 k 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 2.3 M 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 1.6 M 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 240 k 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 3.9 M 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 17 M 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-23T08:11:01.014 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-03-23T08:11:01.015 INFO:teuthology.orchestra.run.smithi070.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout:Install 32 Packages 2024-03-23T08:11:01.016 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:01.017 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 28 M 2024-03-23T08:11:01.017 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 110 M 2024-03-23T08:11:01.017 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:11:01.349 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-03-23T08:11:01.350 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:01.350 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:11:01.350 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:01.350 INFO:teuthology.orchestra.run.smithi193.stdout: jq-1.6-8.el8.x86_64 2024-03-23T08:11:01.350 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:01.350 INFO:teuthology.orchestra.run.smithi193.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-03-23T08:11:01.350 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-23T08:11:01.350 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-23T08:11:01.350 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:01.351 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:11:01.549 INFO:teuthology.orchestra.run.smithi070.stdout:(1/32): ceph-17.2.7-726.g9c4e0c34.el8.x86_64.rp 35 kB/s | 6.4 kB 00:00 2024-03-23T08:11:01.564 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph 2024-03-23T08:11:01.665 INFO:teuthology.orchestra.run.smithi070.stdout:(2/32): ceph-mgr-17.2.7-726.g9c4e0c34.el8.x86_6 5.2 MB/s | 1.6 MB 00:00 2024-03-23T08:11:02.323 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:57 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:11:02.366 INFO:teuthology.orchestra.run.smithi070.stdout:(3/32): ceph-mds-17.2.7-726.g9c4e0c34.el8.x86_6 2.3 MB/s | 2.3 MB 00:01 2024-03-23T08:11:02.583 INFO:teuthology.orchestra.run.smithi070.stdout:(4/32): ceph-osd-17.2.7-726.g9c4e0c34.el8.x86_6 18 MB/s | 17 MB 00:00 2024-03-23T08:11:02.608 INFO:teuthology.orchestra.run.smithi070.stdout:(5/32): ceph-mgr-modules-core-17.2.7-726.g9c4e0 992 kB/s | 240 kB 00:00 2024-03-23T08:11:02.667 INFO:teuthology.orchestra.run.smithi070.stdout:(6/32): ceph-mon-17.2.7-726.g9c4e0c34.el8.x86_6 3.5 MB/s | 3.9 MB 00:01 2024-03-23T08:11:02.750 INFO:teuthology.orchestra.run.smithi070.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 273 kB/s | 39 kB 00:00 2024-03-23T08:11:02.801 INFO:teuthology.orchestra.run.smithi070.stdout:(8/32): python3-mako-1.0.6-14.el8.noarch.rpm 723 kB/s | 157 kB 00:00 2024-03-23T08:11:02.826 INFO:teuthology.orchestra.run.smithi070.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 651 kB/s | 103 kB 00:00 2024-03-23T08:11:02.851 INFO:teuthology.orchestra.run.smithi070.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 567 kB/s | 57 kB 00:00 2024-03-23T08:11:02.927 INFO:teuthology.orchestra.run.smithi070.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.6 MB/s | 457 kB 00:00 2024-03-23T08:11:03.302 INFO:teuthology.orchestra.run.smithi070.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 291 kB/s | 109 kB 00:00 2024-03-23T08:11:03.327 INFO:teuthology.orchestra.run.smithi070.stdout:(13/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 497 kB/s | 249 kB 00:00 2024-03-23T08:11:03.378 INFO:teuthology.orchestra.run.smithi070.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 1.2 MB/s | 641 kB 00:00 2024-03-23T08:11:03.645 INFO:teuthology.orchestra.run.smithi070.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 127 kB/s | 44 kB 00:00 2024-03-23T08:11:03.745 INFO:teuthology.orchestra.run.smithi070.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 444 kB/s | 185 kB 00:00 2024-03-23T08:11:03.770 INFO:teuthology.orchestra.run.smithi070.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 440 kB/s | 173 kB 00:00 2024-03-23T08:11:03.845 INFO:teuthology.orchestra.run.smithi070.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-03-23T08:11:03.871 INFO:teuthology.orchestra.run.smithi070.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 175 kB/s | 18 kB 00:00 2024-03-23T08:11:03.913 INFO:teuthology.orchestra.run.smithi070.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.4 MB/s | 384 kB 00:00 2024-03-23T08:11:03.938 INFO:teuthology.orchestra.run.smithi070.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 529 kB/s | 49 kB 00:00 2024-03-23T08:11:03.939 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 6.4 k 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 2.3 M 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 1.6 M 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 240 k 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 3.9 M 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 17 M 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-23T08:11:03.943 INFO:teuthology.orchestra.run.smithi193.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-03-23T08:11:03.944 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout:Install 32 Packages 2024-03-23T08:11:03.945 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:03.946 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 28 M 2024-03-23T08:11:03.946 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 110 M 2024-03-23T08:11:03.947 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:11:03.963 INFO:teuthology.orchestra.run.smithi070.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 637 kB/s | 59 kB 00:00 2024-03-23T08:11:04.014 INFO:teuthology.orchestra.run.smithi070.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2024-03-23T08:11:04.039 INFO:teuthology.orchestra.run.smithi070.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 154 kB/s | 16 kB 00:00 2024-03-23T08:11:04.064 INFO:teuthology.orchestra.run.smithi070.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 186 kB/s | 19 kB 00:00 2024-03-23T08:11:04.123 INFO:teuthology.orchestra.run.smithi070.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 219 kB/s | 24 kB 00:00 2024-03-23T08:11:04.148 INFO:teuthology.orchestra.run.smithi070.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 268 kB/s | 29 kB 00:00 2024-03-23T08:11:04.173 INFO:teuthology.orchestra.run.smithi070.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 252 kB/s | 27 kB 00:00 2024-03-23T08:11:04.224 INFO:teuthology.orchestra.run.smithi070.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.5 MB/s | 255 kB 00:00 2024-03-23T08:11:04.291 INFO:teuthology.orchestra.run.smithi070.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 729 kB/s | 85 kB 00:00 2024-03-23T08:11:04.324 INFO:teuthology.orchestra.run.smithi070.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 228 kB/s | 23 kB 00:00 2024-03-23T08:11:04.350 INFO:teuthology.orchestra.run.smithi070.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.2 MB/s | 251 kB 00:00 2024-03-23T08:11:04.351 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:04.351 INFO:teuthology.orchestra.run.smithi070.stdout:Total 8.5 MB/s | 28 MB 00:03 2024-03-23T08:11:04.431 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:11:04.468 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:11:04.469 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:11:04.642 INFO:teuthology.orchestra.run.smithi193.stdout:(1/32): ceph-17.2.7-726.g9c4e0c34.el8.x86_64.rp 33 kB/s | 6.4 kB 00:00 2024-03-23T08:11:04.788 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:11:04.792 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:11:04.842 INFO:teuthology.orchestra.run.smithi193.stdout:(2/32): ceph-mds-17.2.7-726.g9c4e0c34.el8.x86_6 5.9 MB/s | 2.3 MB 00:00 2024-03-23T08:11:04.984 INFO:teuthology.orchestra.run.smithi193.stdout:(3/32): ceph-mgr-17.2.7-726.g9c4e0c34.el8.x86_6 2.9 MB/s | 1.6 MB 00:00 2024-03-23T08:11:05.051 INFO:teuthology.orchestra.run.smithi193.stdout:(4/32): ceph-mgr-modules-core-17.2.7-726.g9c4e0 3.5 MB/s | 240 kB 00:00 2024-03-23T08:11:05.226 INFO:teuthology.orchestra.run.smithi193.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 898 kB/s | 157 kB 00:00 2024-03-23T08:11:05.285 INFO:teuthology.orchestra.run.smithi193.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 665 kB/s | 39 kB 00:00 2024-03-23T08:11:05.344 INFO:teuthology.orchestra.run.smithi193.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.7 MB/s | 103 kB 00:00 2024-03-23T08:11:05.394 INFO:teuthology.orchestra.run.smithi193.stdout:(8/32): ceph-mon-17.2.7-726.g9c4e0c34.el8.x86_6 5.2 MB/s | 3.9 MB 00:00 2024-03-23T08:11:05.420 INFO:teuthology.orchestra.run.smithi193.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 758 kB/s | 57 kB 00:00 2024-03-23T08:11:05.686 INFO:teuthology.orchestra.run.smithi193.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.5 MB/s | 457 kB 00:00 2024-03-23T08:11:05.871 INFO:teuthology.orchestra.run.smithi193.stdout:(11/32): ceph-osd-17.2.7-726.g9c4e0c34.el8.x86_ 16 MB/s | 17 MB 00:01 2024-03-23T08:11:05.888 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:11:05.896 INFO:teuthology.orchestra.run.smithi193.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 523 kB/s | 249 kB 00:00 2024-03-23T08:11:06.004 INFO:teuthology.orchestra.run.smithi193.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 403 kB/s | 44 kB 00:00 2024-03-23T08:11:06.101 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-03-23T08:11:06.113 INFO:teuthology.orchestra.run.smithi193.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.7 MB/s | 185 kB 00:00 2024-03-23T08:11:06.180 INFO:teuthology.orchestra.run.smithi193.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.5 MB/s | 173 kB 00:00 2024-03-23T08:11:06.238 INFO:teuthology.orchestra.run.smithi193.stdout:(16/32): python3-pycparser-2.14-14.el8.noarch.r 297 kB/s | 109 kB 00:00 2024-03-23T08:11:06.280 INFO:teuthology.orchestra.run.smithi193.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 3.7 MB/s | 384 kB 00:00 2024-03-23T08:11:06.312 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-03-23T08:11:06.314 INFO:teuthology.orchestra.run.smithi193.stdout:(18/32): python3-cryptography-3.2.1-7.el8.x86_6 1.0 MB/s | 641 kB 00:00 2024-03-23T08:11:06.339 INFO:teuthology.orchestra.run.smithi193.stdout:(19/32): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2024-03-23T08:11:06.364 INFO:teuthology.orchestra.run.smithi193.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 209 kB/s | 18 kB 00:00 2024-03-23T08:11:06.390 INFO:teuthology.orchestra.run.smithi193.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 646 kB/s | 49 kB 00:00 2024-03-23T08:11:06.415 INFO:teuthology.orchestra.run.smithi193.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 777 kB/s | 59 kB 00:00 2024-03-23T08:11:06.440 INFO:teuthology.orchestra.run.smithi193.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2024-03-23T08:11:06.466 INFO:teuthology.orchestra.run.smithi193.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 374 kB/s | 19 kB 00:00 2024-03-23T08:11:06.491 INFO:teuthology.orchestra.run.smithi193.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 475 kB/s | 24 kB 00:00 2024-03-23T08:11:06.516 INFO:teuthology.orchestra.run.smithi193.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 581 kB/s | 29 kB 00:00 2024-03-23T08:11:06.537 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-03-23T08:11:06.541 INFO:teuthology.orchestra.run.smithi193.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 548 kB/s | 27 kB 00:00 2024-03-23T08:11:06.567 INFO:teuthology.orchestra.run.smithi193.stdout:(28/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2024-03-23T08:11:06.609 INFO:teuthology.orchestra.run.smithi193.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.7 MB/s | 255 kB 00:00 2024-03-23T08:11:06.634 INFO:teuthology.orchestra.run.smithi193.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.2 MB/s | 85 kB 00:00 2024-03-23T08:11:06.659 INFO:teuthology.orchestra.run.smithi193.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 459 kB/s | 23 kB 00:00 2024-03-23T08:11:06.684 INFO:teuthology.orchestra.run.smithi193.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.7 MB/s | 251 kB 00:00 2024-03-23T08:11:06.686 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:06.686 INFO:teuthology.orchestra.run.smithi193.stdout:Total 10 MB/s | 28 MB 00:02 2024-03-23T08:11:06.767 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:11:06.777 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-03-23T08:11:06.804 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:11:06.804 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:11:06.975 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-03-23T08:11:07.118 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:11:07.121 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:11:07.138 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-03-23T08:11:07.316 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-03-23T08:11:07.468 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-03-23T08:11:07.678 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-03-23T08:11:07.898 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-03-23T08:11:08.102 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-03-23T08:11:08.223 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:11:08.335 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-03-23T08:11:08.454 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-03-23T08:11:08.666 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-03-23T08:11:08.725 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-03-23T08:11:08.932 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-03-23T08:11:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-03-23T08:11:09.153 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-03-23T08:11:09.155 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-03-23T08:11:09.337 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-03-23T08:11:09.411 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-03-23T08:11:09.508 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-03-23T08:11:09.596 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-23T08:11:09.702 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-03-23T08:11:09.863 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-03-23T08:11:09.886 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-03-23T08:11:10.073 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-03-23T08:11:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-03-23T08:11:10.293 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-03-23T08:11:10.297 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-03-23T08:11:10.467 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-03-23T08:11:10.531 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-03-23T08:11:10.700 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-03-23T08:11:10.780 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-03-23T08:11:10.972 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-03-23T08:11:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-03-23T08:11:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-03-23T08:11:11.393 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-03-23T08:11:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-03-23T08:11:11.598 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-03-23T08:11:11.766 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-03-23T08:11:11.841 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-03-23T08:11:12.050 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-23T08:11:12.113 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el8.n 27/32 2024-03-23T08:11:12.333 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 28/32 2024-03-23T08:11:12.379 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-03-23T08:11:12.605 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-03-23T08:11:12.760 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-03-23T08:11:12.946 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-03-23T08:11:13.192 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-03-23T08:11:13.445 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-03-23T08:11:13.662 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-03-23T08:11:13.809 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 28/32 2024-03-23T08:11:13.979 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-03-23T08:11:14.049 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el8.x86_64 29/32 2024-03-23T08:11:14.278 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-03-23T08:11:14.450 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el8.x86_64 29/32 2024-03-23T08:11:14.614 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el8.n 27/32 2024-03-23T08:11:14.636 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el8.x86_64 30/32 2024-03-23T08:11:14.826 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 28/32 2024-03-23T08:11:14.859 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el8.x86_64 30/32 2024-03-23T08:11:15.028 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el8.x86_64 31/32 2024-03-23T08:11:15.045 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el8.x86_64 31/32 2024-03-23T08:11:15.152 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el8.x86_64 32/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el8.x86_64 32/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 3/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el8.x86_64 4/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el8.n 6/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-03-23T08:11:15.555 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-03-23T08:11:15.556 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-03-23T08:11:15.557 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-03-23T08:11:15.557 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-03-23T08:11:15.557 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-03-23T08:11:15.558 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-03-23T08:11:15.558 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-03-23T08:11:15.558 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-03-23T08:11:16.297 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 28/32 2024-03-23T08:11:16.421 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-03-23T08:11:16.421 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:16.421 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:11:16.421 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:16.421 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:16.421 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:16.421 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-23T08:11:16.422 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-2.6-1.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-23T08:11:16.423 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:16.424 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:11:16.500 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el8.x86_64 29/32 2024-03-23T08:11:16.718 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-base 2024-03-23T08:11:16.905 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el8.x86_64 29/32 2024-03-23T08:11:17.082 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el8.x86_64 30/32 2024-03-23T08:11:17.308 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el8.x86_64 30/32 2024-03-23T08:11:17.490 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el8.x86_64 31/32 2024-03-23T08:11:17.506 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:13 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:11:17.507 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el8.x86_64 31/32 2024-03-23T08:11:17.639 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el8.x86_64 32/32 2024-03-23T08:11:18.046 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el8.x86_64 32/32 2024-03-23T08:11:18.046 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/32 2024-03-23T08:11:18.046 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/32 2024-03-23T08:11:18.046 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 3/32 2024-03-23T08:11:18.046 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el8.x86_64 4/32 2024-03-23T08:11:18.046 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el8.x86_64 5/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el8.n 6/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-03-23T08:11:18.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-03-23T08:11:18.048 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:18.853 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-23T08:11:18.854 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-2.6-1.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-23T08:11:18.855 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-23T08:11:18.856 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:18.856 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:11:19.077 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:11:19.099 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-base 2024-03-23T08:11:19.125 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:11:19.129 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-03-23T08:11:19.130 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:11:19.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install cephadm 2024-03-23T08:11:19.865 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:14 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:11:19.995 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:15 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:11:21.431 INFO:teuthology.orchestra.run.smithi193.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:11:21.479 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:11:21.480 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-03-23T08:11:21.480 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:11:21.572 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install cephadm 2024-03-23T08:11:21.600 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:11:21.601 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:21.601 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:11:21.601 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:21.601 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:11:21.601 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 82 k 2024-03-23T08:11:21.602 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:21.602 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:11:21.602 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:21.602 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-03-23T08:11:21.602 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:21.602 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 82 k 2024-03-23T08:11:21.602 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 372 k 2024-03-23T08:11:21.602 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:11:21.796 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm-17.2.7-726.g9c4e0c34.el8.noarch.rpm 423 kB/s | 82 kB 00:00 2024-03-23T08:11:21.796 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:21.796 INFO:teuthology.orchestra.run.smithi070.stdout:Total 420 kB/s | 82 kB 00:00 2024-03-23T08:11:21.797 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:11:21.804 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:11:21.804 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:11:21.806 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:11:21.809 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:11:21.852 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:11:22.324 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:17 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:11:22.366 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/1 2024-03-23T08:11:22.510 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/1 2024-03-23T08:11:22.733 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/1 2024-03-23T08:11:23.417 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/1 2024-03-23T08:11:23.417 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:23.418 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:11:23.418 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:23.418 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:23.418 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:11:23.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-immutable-object-cache 2024-03-23T08:11:23.951 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 82 k 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 82 k 2024-03-23T08:11:23.952 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 372 k 2024-03-23T08:11:23.953 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:11:24.234 INFO:teuthology.orchestra.run.smithi193.stdout:cephadm-17.2.7-726.g9c4e0c34.el8.noarch.rpm 291 kB/s | 82 kB 00:00 2024-03-23T08:11:24.235 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:24.235 INFO:teuthology.orchestra.run.smithi193.stdout:Total 290 kB/s | 82 kB 00:00 2024-03-23T08:11:24.235 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:11:24.243 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:11:24.243 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:11:24.244 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:11:24.248 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:11:24.288 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:11:24.421 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:20 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:11:24.757 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/1 2024-03-23T08:11:24.945 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/1 2024-03-23T08:11:25.172 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/1 2024-03-23T08:11:25.534 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/1 2024-03-23T08:11:25.534 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:25.534 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:11:25.534 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:25.534 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:25.535 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:11:25.748 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-immutable-object-cache 2024-03-23T08:11:26.027 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 162 k 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-03-23T08:11:26.028 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:26.029 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 162 k 2024-03-23T08:11:26.029 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 494 k 2024-03-23T08:11:26.029 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:11:26.215 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 871 kB/s | 162 kB 00:00 2024-03-23T08:11:26.215 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:26.215 INFO:teuthology.orchestra.run.smithi070.stdout:Total 865 kB/s | 162 kB 00:00 2024-03-23T08:11:26.216 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:11:26.226 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:11:26.226 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:11:26.272 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:11:26.275 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:11:26.399 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:11:26.508 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:21 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:11:26.625 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-23T08:11:27.019 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-23T08:11:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-23T08:11:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:27.377 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:11:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:27.377 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:11:27.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr 2024-03-23T08:11:28.123 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:11:28.124 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:28.124 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-03-23T08:11:28.124 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:28.124 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:11:28.124 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 162 k 2024-03-23T08:11:28.124 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:28.124 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:11:28.124 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:28.124 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-03-23T08:11:28.125 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:28.125 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 162 k 2024-03-23T08:11:28.125 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 494 k 2024-03-23T08:11:28.125 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:11:28.366 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:24 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:11:28.465 INFO:teuthology.orchestra.run.smithi193.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 476 kB/s | 162 kB 00:00 2024-03-23T08:11:28.465 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:28.466 INFO:teuthology.orchestra.run.smithi193.stdout:Total 475 kB/s | 162 kB 00:00 2024-03-23T08:11:28.466 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:11:28.477 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:11:28.477 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:11:28.525 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:11:28.528 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:11:28.647 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:11:28.866 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-23T08:11:29.283 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-23T08:11:29.691 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-23T08:11:29.691 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:29.691 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:11:29.691 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:11:29.691 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:29.691 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:11:29.907 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr 2024-03-23T08:11:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:11:29.986 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:11:29.988 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-03-23T08:11:29.988 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:11:30.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-dashboard 2024-03-23T08:11:30.671 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:25 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:11:30.909 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:26 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:11:32.249 INFO:teuthology.orchestra.run.smithi193.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:11:32.297 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:11:32.298 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-03-23T08:11:32.299 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:11:32.400 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-dashboard 2024-03-23T08:11:32.520 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 1.7 M 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 23 k 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 14 k 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-23T08:11:32.521 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-23T08:11:32.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:32.522 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:11:32.522 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:32.522 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-03-23T08:11:32.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:32.522 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 2.0 M 2024-03-23T08:11:32.523 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 8.0 M 2024-03-23T08:11:32.523 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:11:32.977 INFO:teuthology.orchestra.run.smithi070.stdout:(1/6): ceph-prometheus-alerts-17.2.7-726.g9c4e0 106 kB/s | 14 kB 00:00 2024-03-23T08:11:33.094 INFO:teuthology.orchestra.run.smithi070.stdout:(2/6): ceph-grafana-dashboards-17.2.7-726.g9c4e 92 kB/s | 23 kB 00:00 2024-03-23T08:11:33.150 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:28 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:11:33.261 INFO:teuthology.orchestra.run.smithi070.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 150 kB/s | 43 kB 00:00 2024-03-23T08:11:33.353 INFO:teuthology.orchestra.run.smithi070.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 130 kB/s | 34 kB 00:00 2024-03-23T08:11:33.612 INFO:teuthology.orchestra.run.smithi070.stdout:(5/6): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 2.2 MB/s | 1.7 MB 00:00 2024-03-23T08:11:33.695 INFO:teuthology.orchestra.run.smithi070.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 453 kB/s | 196 kB 00:00 2024-03-23T08:11:33.697 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:33.697 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.7 MB/s | 2.0 MB 00:01 2024-03-23T08:11:33.707 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:11:33.715 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:11:33.716 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:11:33.795 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:11:33.798 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:11:34.048 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:11:34.238 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-03-23T08:11:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-03-23T08:11:34.718 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-03-23T08:11:34.753 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:11:34.754 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:34.754 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 1.7 M 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 23 k 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 14 k 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:34.755 INFO:teuthology.orchestra.run.smithi193.stdout:Install 6 Packages 2024-03-23T08:11:34.756 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:34.756 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 2.0 M 2024-03-23T08:11:34.756 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 8.0 M 2024-03-23T08:11:34.756 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:11:34.821 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el8.no 4/6 2024-03-23T08:11:35.160 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el8.n 5/6 2024-03-23T08:11:35.186 INFO:teuthology.orchestra.run.smithi193.stdout:(1/6): ceph-grafana-dashboards-17.2.7-726.g9c4e 165 kB/s | 23 kB 00:00 2024-03-23T08:11:35.211 INFO:teuthology.orchestra.run.smithi193.stdout:(2/6): ceph-prometheus-alerts-17.2.7-726.g9c4e0 87 kB/s | 14 kB 00:00 2024-03-23T08:11:35.303 INFO:teuthology.orchestra.run.smithi193.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 366 kB/s | 34 kB 00:00 2024-03-23T08:11:35.395 INFO:teuthology.orchestra.run.smithi193.stdout:(4/6): python3-routes-2.4.1-12.el8.noarch.rpm 2.1 MB/s | 196 kB 00:00 2024-03-23T08:11:35.537 INFO:teuthology.orchestra.run.smithi193.stdout:(5/6): python3-jwt-1.6.1-2.el8.noarch.rpm 121 kB/s | 43 kB 00:00 2024-03-23T08:11:35.644 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el8.noarch 6/6 2024-03-23T08:11:35.796 INFO:teuthology.orchestra.run.smithi193.stdout:(6/6): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 2.3 MB/s | 1.7 MB 00:00 2024-03-23T08:11:35.797 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:35.798 INFO:teuthology.orchestra.run.smithi193.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2024-03-23T08:11:35.808 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:11:35.818 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:11:35.818 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:11:35.897 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:11:35.900 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:11:35.932 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el8.noarch 6/6 2024-03-23T08:11:35.933 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el8.n 1/6 2024-03-23T08:11:35.933 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el8.noarch 2/6 2024-03-23T08:11:35.933 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el8.no 3/6 2024-03-23T08:11:35.933 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-03-23T08:11:35.933 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-03-23T08:11:36.159 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:11:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-03-23T08:11:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:36.320 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:11:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-23T08:11:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-23T08:11:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-23T08:11:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:36.321 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:11:36.367 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-03-23T08:11:36.462 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-23T08:11:36.654 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-03-23T08:11:36.856 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-03-23T08:11:36.967 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el8.no 4/6 2024-03-23T08:11:37.208 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:33 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:11:37.297 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el8.n 5/6 2024-03-23T08:11:37.546 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el8.noarch 6/6 2024-03-23T08:11:37.821 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el8.noarch 6/6 2024-03-23T08:11:37.821 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el8.n 1/6 2024-03-23T08:11:37.822 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el8.noarch 2/6 2024-03-23T08:11:37.822 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el8.no 3/6 2024-03-23T08:11:37.822 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-03-23T08:11:37.822 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-03-23T08:11:38.216 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-03-23T08:11:38.216 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:38.216 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:11:38.216 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:38.216 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:38.216 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:38.217 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-23T08:11:38.217 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-23T08:11:38.217 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-23T08:11:38.217 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:38.217 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:11:38.333 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-23T08:11:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:11:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2024-03-23T08:11:38.839 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:11:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2024-03-23T08:11:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 7.4 M 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:Install 8 Packages 2024-03-23T08:11:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:38.841 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 36 M 2024-03-23T08:11:38.841 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 210 M 2024-03-23T08:11:38.841 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:11:39.102 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:34 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:11:39.393 INFO:teuthology.orchestra.run.smithi070.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2024-03-23T08:11:39.593 INFO:teuthology.orchestra.run.smithi070.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 19 MB/s | 3.7 MB 00:00 2024-03-23T08:11:39.661 INFO:teuthology.orchestra.run.smithi070.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 3.3 MB/s | 225 kB 00:00 2024-03-23T08:11:39.944 INFO:teuthology.orchestra.run.smithi070.stdout:(4/8): ceph-mgr-diskprediction-local-17.2.7-726 7.4 MB/s | 7.4 MB 00:00 2024-03-23T08:11:40.111 INFO:teuthology.orchestra.run.smithi070.stdout:(5/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 4.2 MB/s | 4.9 MB 00:01 2024-03-23T08:11:40.461 INFO:teuthology.orchestra.run.smithi070.stdout:(6/8): libgfortran-8.5.0-21.el8.x86_64.rpm 1.2 MB/s | 645 kB 00:00 2024-03-23T08:11:40.486 INFO:teuthology.orchestra.run.smithi070.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 460 kB/s | 172 kB 00:00 2024-03-23T08:11:40.720 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:11:40.721 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================================ 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================================ 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 7.4 M 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-03-23T08:11:40.722 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-03-23T08:11:40.723 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:40.723 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:11:40.723 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================================ 2024-03-23T08:11:40.723 INFO:teuthology.orchestra.run.smithi193.stdout:Install 8 Packages 2024-03-23T08:11:40.723 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:40.723 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 36 M 2024-03-23T08:11:40.723 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 210 M 2024-03-23T08:11:40.723 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:11:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 7.0 MB/s | 14 MB 00:02 2024-03-23T08:11:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:Total 13 MB/s | 36 MB 00:02 2024-03-23T08:11:41.905 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:11:41.922 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:11:41.923 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:11:42.035 INFO:teuthology.orchestra.run.smithi193.stdout:(1/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 4.6 MB/s | 4.9 MB 00:01 2024-03-23T08:11:42.311 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:11:42.314 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:11:42.462 INFO:teuthology.orchestra.run.smithi193.stdout:(2/8): ceph-mgr-diskprediction-local-17.2.7-726 5.0 MB/s | 7.4 MB 00:01 2024-03-23T08:11:42.704 INFO:teuthology.orchestra.run.smithi193.stdout:(3/8): openblas-0.3.15-4.el8.x86_64.rpm 2.7 MB/s | 4.7 MB 00:01 2024-03-23T08:11:42.729 INFO:teuthology.orchestra.run.smithi193.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 843 kB/s | 225 kB 00:00 2024-03-23T08:11:42.813 INFO:teuthology.orchestra.run.smithi193.stdout:(5/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 4.8 MB/s | 3.7 MB 00:00 2024-03-23T08:11:42.866 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:11:42.988 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-03-23T08:11:43.117 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-03-23T08:11:43.180 INFO:teuthology.orchestra.run.smithi193.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 470 kB/s | 172 kB 00:00 2024-03-23T08:11:43.246 INFO:teuthology.orchestra.run.smithi193.stdout:(7/8): libgfortran-8.5.0-21.el8.x86_64.rpm 1.2 MB/s | 645 kB 00:00 2024-03-23T08:11:43.263 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-03-23T08:11:43.859 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-03-23T08:11:44.013 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-03-23T08:11:44.629 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-03-23T08:11:44.779 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-03-23T08:11:45.355 INFO:teuthology.orchestra.run.smithi193.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 5.3 MB/s | 14 MB 00:02 2024-03-23T08:11:45.355 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:45.356 INFO:teuthology.orchestra.run.smithi193.stdout:Total 7.7 MB/s | 36 MB 00:04 2024-03-23T08:11:45.479 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-03-23T08:11:45.589 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:11:45.607 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:11:45.607 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:11:45.849 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-03-23T08:11:46.007 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:11:46.010 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:11:46.559 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:11:46.710 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-03-23T08:11:46.823 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-03-23T08:11:46.959 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-03-23T08:11:47.079 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-03-23T08:11:47.539 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-03-23T08:11:48.518 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-03-23T08:11:48.840 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-03-23T08:11:49.134 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-03-23T08:11:49.301 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-03-23T08:11:49.992 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-03-23T08:11:50.118 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-03-23T08:11:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34 8/8 2024-03-23T08:11:50.311 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-03-23T08:11:50.509 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-03-23T08:11:50.595 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34 8/8 2024-03-23T08:11:50.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34 1/8 2024-03-23T08:11:50.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-03-23T08:11:50.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-03-23T08:11:50.596 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-03-23T08:11:50.596 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-03-23T08:11:50.596 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-03-23T08:11:50.596 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-03-23T08:11:50.999 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.15-4.el8.x86_64 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:51.000 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:11:51.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-rook 2024-03-23T08:11:51.888 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:47 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:11:52.254 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-03-23T08:11:53.467 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-03-23T08:11:53.509 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:11:53.511 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:53.511 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 47 k 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-03-23T08:11:53.512 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-03-23T08:11:53.513 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-23T08:11:53.513 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-23T08:11:53.513 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:53.513 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:11:53.513 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:11:53.513 INFO:teuthology.orchestra.run.smithi070.stdout:Install 13 Packages 2024-03-23T08:11:53.513 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:53.514 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 1.9 M 2024-03-23T08:11:53.514 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 25 M 2024-03-23T08:11:53.514 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:11:53.708 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34 8/8 2024-03-23T08:11:54.028 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34 8/8 2024-03-23T08:11:54.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34 1/8 2024-03-23T08:11:54.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-03-23T08:11:54.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-03-23T08:11:54.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-03-23T08:11:54.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-03-23T08:11:54.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-03-23T08:11:54.029 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-03-23T08:11:54.130 INFO:teuthology.orchestra.run.smithi070.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 173 kB/s | 20 kB 00:00 2024-03-23T08:11:54.155 INFO:teuthology.orchestra.run.smithi070.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 189 kB/s | 27 kB 00:00 2024-03-23T08:11:54.181 INFO:teuthology.orchestra.run.smithi070.stdout:(3/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el8. 285 kB/s | 47 kB 00:00 2024-03-23T08:11:54.239 INFO:teuthology.orchestra.run.smithi070.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-03-23T08:11:54.265 INFO:teuthology.orchestra.run.smithi070.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.0 MB/s | 110 kB 00:00 2024-03-23T08:11:54.390 INFO:teuthology.orchestra.run.smithi070.stdout:(6/13): python3-cachetools-3.1.1-4.el8.noarch.r 263 kB/s | 33 kB 00:00 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.15-4.el8.x86_64 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-23T08:11:54.438 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-03-23T08:11:54.439 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:54.439 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:11:54.440 INFO:teuthology.orchestra.run.smithi070.stdout:(7/13): python3-certifi-2018.10.15-7.el8.noarch 327 kB/s | 16 kB 00:00 2024-03-23T08:11:54.516 INFO:teuthology.orchestra.run.smithi070.stdout:(8/13): python3-google-auth-1.1.1-10.el8.noarch 1.1 MB/s | 81 kB 00:00 2024-03-23T08:11:54.541 INFO:teuthology.orchestra.run.smithi070.stdout:(9/13): python3-requests-oauthlib-1.0.0-1.el8.n 141 kB/s | 43 kB 00:00 2024-03-23T08:11:54.561 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-rook 2024-03-23T08:11:54.566 INFO:teuthology.orchestra.run.smithi070.stdout:(10/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 402 kB/s | 155 kB 00:00 2024-03-23T08:11:54.667 INFO:teuthology.orchestra.run.smithi070.stdout:(11/13): python3-kubernetes-11.0.0-6.el8.noarch 7.7 MB/s | 1.2 MB 00:00 2024-03-23T08:11:54.692 INFO:teuthology.orchestra.run.smithi070.stdout:(12/13): python3-rsa-4.9-2.el8.noarch.rpm 426 kB/s | 64 kB 00:00 2024-03-23T08:11:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:(13/13): python3-websocket-client-0.56.0-5.el8. 431 kB/s | 61 kB 00:00 2024-03-23T08:11:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.6 MB/s | 1.9 MB 00:01 2024-03-23T08:11:54.766 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:11:54.782 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:11:54.782 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:11:54.869 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:11:54.872 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:11:55.317 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:50 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:11:55.439 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:11:55.671 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-03-23T08:11:55.851 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-03-23T08:11:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-03-23T08:11:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-03-23T08:11:56.474 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-03-23T08:11:56.714 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-03-23T08:11:56.942 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:11:56.944 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:56.944 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:11:56.944 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:56.944 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:11:56.944 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 47 k 2024-03-23T08:11:56.944 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:56.945 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:11:56.946 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:11:56.946 INFO:teuthology.orchestra.run.smithi193.stdout:Install 13 Packages 2024-03-23T08:11:56.946 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:11:56.946 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 1.9 M 2024-03-23T08:11:56.946 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 25 M 2024-03-23T08:11:56.946 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:11:56.980 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-03-23T08:11:57.239 INFO:teuthology.orchestra.run.smithi193.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 146 kB/s | 20 kB 00:00 2024-03-23T08:11:57.239 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-03-23T08:11:57.263 INFO:teuthology.orchestra.run.smithi193.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 164 kB/s | 27 kB 00:00 2024-03-23T08:11:57.347 INFO:teuthology.orchestra.run.smithi193.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-03-23T08:11:57.372 INFO:teuthology.orchestra.run.smithi193.stdout:(4/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el8. 175 kB/s | 47 kB 00:00 2024-03-23T08:11:57.398 INFO:teuthology.orchestra.run.smithi193.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 820 kB/s | 110 kB 00:00 2024-03-23T08:11:57.639 INFO:teuthology.orchestra.run.smithi193.stdout:(6/13): python3-cachetools-3.1.1-4.el8.noarch.r 136 kB/s | 33 kB 00:00 2024-03-23T08:11:57.665 INFO:teuthology.orchestra.run.smithi193.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 146 kB/s | 43 kB 00:00 2024-03-23T08:11:57.732 INFO:teuthology.orchestra.run.smithi193.stdout:(8/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 402 kB/s | 155 kB 00:00 2024-03-23T08:11:57.749 INFO:teuthology.orchestra.run.smithi193.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 149 kB/s | 16 kB 00:00 2024-03-23T08:11:57.916 INFO:teuthology.orchestra.run.smithi193.stdout:(10/13): python3-rsa-4.9-2.el8.noarch.rpm 385 kB/s | 64 kB 00:00 2024-03-23T08:11:57.975 INFO:teuthology.orchestra.run.smithi193.stdout:(11/13): python3-google-auth-1.1.1-10.el8.noarc 263 kB/s | 81 kB 00:00 2024-03-23T08:11:58.008 INFO:teuthology.orchestra.run.smithi193.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 667 kB/s | 61 kB 00:00 2024-03-23T08:11:58.246 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-03-23T08:11:58.342 INFO:teuthology.orchestra.run.smithi193.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.9 MB/s | 1.2 MB 00:00 2024-03-23T08:11:58.342 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:11:58.342 INFO:teuthology.orchestra.run.smithi193.stdout:Total 1.4 MB/s | 1.9 MB 00:01 2024-03-23T08:11:58.392 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:11:58.407 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:11:58.407 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:11:58.491 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:11:58.494 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:11:58.507 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-03-23T08:11:58.691 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-03-23T08:11:58.868 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-03-23T08:11:59.017 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:11:59.051 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el8.noarch 13/13 2024-03-23T08:11:59.251 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-03-23T08:11:59.339 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el8.noarch 13/13 2024-03-23T08:11:59.339 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el8.noarch 1/13 2024-03-23T08:11:59.339 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-03-23T08:11:59.339 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-03-23T08:11:59.339 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-03-23T08:11:59.339 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-03-23T08:11:59.339 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-03-23T08:11:59.340 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-03-23T08:11:59.340 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-03-23T08:11:59.340 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-03-23T08:11:59.340 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-03-23T08:11:59.340 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-03-23T08:11:59.340 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-03-23T08:11:59.441 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-03-23T08:11:59.668 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-03-23T08:11:59.772 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-23T08:11:59.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-23T08:11:59.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-23T08:11:59.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-23T08:11:59.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-23T08:11:59.774 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:11:59.774 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:11:59.872 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-03-23T08:11:59.893 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-cephadm 2024-03-23T08:12:00.080 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-03-23T08:12:00.302 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-03-23T08:12:00.592 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-03-23T08:12:00.651 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:56 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:00.869 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-03-23T08:12:01.908 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-03-23T08:12:02.172 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-03-23T08:12:02.270 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:02.271 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:12:02.271 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:02.271 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:12:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 123 k 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout:Install 5 Packages 2024-03-23T08:12:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:02.273 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 6.0 M 2024-03-23T08:12:02.273 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 26 M 2024-03-23T08:12:02.273 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:12:02.347 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-03-23T08:12:02.515 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-03-23T08:12:02.658 INFO:teuthology.orchestra.run.smithi070.stdout:(1/5): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 408 kB/s | 123 kB 00:00 2024-03-23T08:12:02.723 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el8.noarch 13/13 2024-03-23T08:12:02.767 INFO:teuthology.orchestra.run.smithi070.stdout:(2/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.4 MB/s | 578 kB 00:00 2024-03-23T08:12:02.884 INFO:teuthology.orchestra.run.smithi070.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 516 kB/s | 60 kB 00:00 2024-03-23T08:12:02.976 INFO:teuthology.orchestra.run.smithi070.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.5 MB/s | 476 kB 00:00 2024-03-23T08:12:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el8.noarch 13/13 2024-03-23T08:12:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el8.noarch 1/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-03-23T08:12:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-23T08:12:03.473 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-23T08:12:03.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-23T08:12:03.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-23T08:12:03.474 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-23T08:12:03.474 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:03.474 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:03.621 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-cephadm 2024-03-23T08:12:03.751 INFO:teuthology.orchestra.run.smithi070.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 3.4 MB/s | 4.8 MB 00:01 2024-03-23T08:12:03.751 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:03.751 INFO:teuthology.orchestra.run.smithi070.stdout:Total 4.1 MB/s | 6.0 MB 00:01 2024-03-23T08:12:03.802 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:12:03.813 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:12:03.813 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:12:03.924 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:12:03.927 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:12:04.190 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:12:04.389 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:59 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:04.505 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-03-23T08:12:05.445 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-03-23T08:12:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-03-23T08:12:06.005 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:06.006 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:06.006 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:12:06.006 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:06.006 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:12:06.006 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el8 ceph-noarch 123 k 2024-03-23T08:12:06.006 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T08:12:06.006 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-23T08:12:06.006 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-03-23T08:12:06.007 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-03-23T08:12:06.007 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-23T08:12:06.007 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:06.007 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:12:06.007 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:06.007 INFO:teuthology.orchestra.run.smithi193.stdout:Install 5 Packages 2024-03-23T08:12:06.007 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:06.007 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 6.0 M 2024-03-23T08:12:06.008 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 26 M 2024-03-23T08:12:06.008 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:12:06.093 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-03-23T08:12:06.250 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 5/5 2024-03-23T08:12:06.335 INFO:teuthology.orchestra.run.smithi193.stdout:(1/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 2.4 MB/s | 578 kB 00:00 2024-03-23T08:12:06.377 INFO:teuthology.orchestra.run.smithi193.stdout:(2/5): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 441 kB/s | 123 kB 00:00 2024-03-23T08:12:06.428 INFO:teuthology.orchestra.run.smithi193.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 5.1 MB/s | 476 kB 00:00 2024-03-23T08:12:06.453 INFO:teuthology.orchestra.run.smithi193.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 799 kB/s | 60 kB 00:00 2024-03-23T08:12:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 5/5 2024-03-23T08:12:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/5 2024-03-23T08:12:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-03-23T08:12:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-03-23T08:12:06.521 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-03-23T08:12:06.828 INFO:teuthology.orchestra.run.smithi193.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 6.6 MB/s | 4.8 MB 00:00 2024-03-23T08:12:06.829 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:06.829 INFO:teuthology.orchestra.run.smithi193.stdout:Total 7.3 MB/s | 6.0 MB 00:00 2024-03-23T08:12:06.880 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:12:06.891 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:12:06.891 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:12:06.895 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-03-23T08:12:06.895 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:06.895 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:12:06.895 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:12:06.895 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-23T08:12:06.895 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-23T08:12:06.896 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-03-23T08:12:06.896 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-23T08:12:06.896 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:06.896 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:06.995 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:12:06.998 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:12:07.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-fuse 2024-03-23T08:12:07.302 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:12:07.612 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-03-23T08:12:07.766 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:03 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:08.566 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-03-23T08:12:09.043 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-03-23T08:12:09.291 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-03-23T08:12:09.371 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 876 k 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:12:09.372 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:09.373 INFO:teuthology.orchestra.run.smithi070.stdout:Install 2 Packages 2024-03-23T08:12:09.373 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:09.373 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 959 k 2024-03-23T08:12:09.373 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 3.1 M 2024-03-23T08:12:09.373 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:12:09.481 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 5/5 2024-03-23T08:12:09.726 INFO:teuthology.orchestra.run.smithi070.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 277 kB/s | 83 kB 00:00 2024-03-23T08:12:09.749 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 5/5 2024-03-23T08:12:09.749 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 1/5 2024-03-23T08:12:09.749 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-03-23T08:12:09.750 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-03-23T08:12:09.750 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-03-23T08:12:09.854 INFO:teuthology.orchestra.run.smithi070.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el8.x86_6 2.0 MB/s | 876 kB 00:00 2024-03-23T08:12:09.854 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:09.855 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.9 MB/s | 959 kB 00:00 2024-03-23T08:12:09.865 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:12:09.885 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:12:09.885 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:12:09.970 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:12:09.974 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:12:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:12:10.162 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-03-23T08:12:10.162 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:10.162 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:12:10.162 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el8.noarch 2024-03-23T08:12:10.163 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-23T08:12:10.163 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-23T08:12:10.163 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-03-23T08:12:10.163 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-23T08:12:10.163 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:10.163 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:10.284 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-fuse 2024-03-23T08:12:10.416 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-03-23T08:12:10.648 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/2 2024-03-23T08:12:11.036 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/2 2024-03-23T08:12:11.036 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/2 2024-03-23T08:12:11.040 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:06 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-03-23T08:12:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:11.430 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:12:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-03-23T08:12:11.430 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:11.431 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:11.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados-devel 2024-03-23T08:12:12.392 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:08 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:12.655 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:12.656 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:12.656 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-23T08:12:12.656 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:12.656 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:12:12.656 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 876 k 2024-03-23T08:12:12.656 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-23T08:12:12.656 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-03-23T08:12:12.657 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:12.657 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:12:12.657 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:12.657 INFO:teuthology.orchestra.run.smithi193.stdout:Install 2 Packages 2024-03-23T08:12:12.657 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:12.657 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 959 k 2024-03-23T08:12:12.657 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 3.1 M 2024-03-23T08:12:12.657 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:12:13.028 INFO:teuthology.orchestra.run.smithi193.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 262 kB/s | 83 kB 00:00 2024-03-23T08:12:13.411 INFO:teuthology.orchestra.run.smithi193.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el8.x86_6 1.2 MB/s | 876 kB 00:00 2024-03-23T08:12:13.412 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:13.412 INFO:teuthology.orchestra.run.smithi193.stdout:Total 1.2 MB/s | 959 kB 00:00 2024-03-23T08:12:13.422 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:12:13.444 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:12:13.444 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:12:13.530 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:12:13.533 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:12:13.715 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:12:13.930 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 130 k 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:13.931 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 130 k 2024-03-23T08:12:13.932 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 464 k 2024-03-23T08:12:13.932 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:12:13.989 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-03-23T08:12:14.180 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/2 2024-03-23T08:12:14.231 INFO:teuthology.orchestra.run.smithi070.stdout:librados-devel-17.2.7-726.g9c4e0c34.el8.x86_64. 434 kB/s | 130 kB 00:00 2024-03-23T08:12:14.231 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:14.231 INFO:teuthology.orchestra.run.smithi070.stdout:Total 432 kB/s | 130 kB 00:00 2024-03-23T08:12:14.232 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:12:14.237 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:12:14.237 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:12:14.287 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:12:14.290 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:12:14.409 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:12:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 2/2 2024-03-23T08:12:14.615 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/2 2024-03-23T08:12:14.655 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:14.943 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:15.004 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-03-23T08:12:15.005 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:15.005 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:12:15.005 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-03-23T08:12:15.005 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:15.005 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:15.202 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librados-devel 2024-03-23T08:12:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:15.337 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:12:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:15.338 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:15.552 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs2 2024-03-23T08:12:15.964 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:10 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:16.305 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:12 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:17.568 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:17.569 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:17.569 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-03-23T08:12:17.569 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:17.569 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:12:17.569 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 130 k 2024-03-23T08:12:17.569 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:17.569 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:12:17.570 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:17.570 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-03-23T08:12:17.570 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:17.570 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 130 k 2024-03-23T08:12:17.570 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 464 k 2024-03-23T08:12:17.570 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:12:17.866 INFO:teuthology.orchestra.run.smithi193.stdout:librados-devel-17.2.7-726.g9c4e0c34.el8.x86_64. 439 kB/s | 130 kB 00:00 2024-03-23T08:12:17.866 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:17.866 INFO:teuthology.orchestra.run.smithi193.stdout:Total 437 kB/s | 130 kB 00:00 2024-03-23T08:12:17.867 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:12:17.872 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:12:17.872 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:12:17.880 INFO:teuthology.orchestra.run.smithi070.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:17.922 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:12:17.925 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:12:17.928 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:17.930 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-03-23T08:12:17.930 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:18.037 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:12:18.253 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:18.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs-devel 2024-03-23T08:12:18.535 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:18.910 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:18.911 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:18.911 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:12:18.911 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:18.911 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:18.911 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:19.169 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:15 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:20.223 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs2 2024-03-23T08:12:20.782 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:20.783 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:20.783 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-03-23T08:12:20.783 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:20.783 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:12:20.783 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 25 k 2024-03-23T08:12:20.783 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:20.783 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:12:20.784 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:20.784 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-03-23T08:12:20.784 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:20.784 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 25 k 2024-03-23T08:12:20.784 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 109 k 2024-03-23T08:12:20.784 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:12:21.005 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:16 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:21.023 INFO:teuthology.orchestra.run.smithi070.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el8.x86_64 104 kB/s | 25 kB 00:00 2024-03-23T08:12:21.023 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:21.023 INFO:teuthology.orchestra.run.smithi070.stdout:Total 104 kB/s | 25 kB 00:00 2024-03-23T08:12:21.024 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:12:21.027 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:12:21.027 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:12:21.029 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:12:21.032 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:12:21.082 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:12:21.261 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:21.513 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:21.862 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:21.862 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:21.862 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:12:21.863 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:21.863 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:21.863 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:21.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados2 2024-03-23T08:12:22.567 INFO:teuthology.orchestra.run.smithi193.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:22.616 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:22.617 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-03-23T08:12:22.617 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:22.718 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs-devel 2024-03-23T08:12:22.728 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:18 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:23.473 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:18 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:24.293 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:24.341 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:24.342 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-03-23T08:12:24.342 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:24.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librbd1 2024-03-23T08:12:25.087 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 25 k 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-03-23T08:12:25.088 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:25.089 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 25 k 2024-03-23T08:12:25.089 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 109 k 2024-03-23T08:12:25.089 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:12:25.312 INFO:teuthology.orchestra.run.smithi193.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el8.x86_64 111 kB/s | 25 kB 00:00 2024-03-23T08:12:25.313 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:25.313 INFO:teuthology.orchestra.run.smithi193.stdout:Total 111 kB/s | 25 kB 00:00 2024-03-23T08:12:25.313 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:12:25.316 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:12:25.317 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:12:25.318 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:12:25.322 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:12:25.370 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:12:25.424 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:21 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:25.544 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:26.868 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:26.868 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:26.868 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:12:26.868 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:26.868 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:26.868 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:26.977 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:26.986 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librados2 2024-03-23T08:12:27.025 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:27.026 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-03-23T08:12:27.026 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:27.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rados 2024-03-23T08:12:27.735 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:22 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:27.891 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:23 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:29.297 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:29.345 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:29.347 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-03-23T08:12:29.347 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:29.446 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librbd1 2024-03-23T08:12:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:29.497 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:29.499 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-03-23T08:12:29.499 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:29.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rgw 2024-03-23T08:12:30.192 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:25 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:30.363 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:26 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:31.842 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:31.890 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:31.891 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-03-23T08:12:31.891 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:31.924 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:31.972 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:31.974 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-03-23T08:12:31.974 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:32.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-cephfs 2024-03-23T08:12:32.095 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rados 2024-03-23T08:12:32.837 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:28 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:32.855 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:27 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:34.414 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:34.461 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:34.462 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-03-23T08:12:34.462 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:34.492 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:34.540 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:34.542 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-03-23T08:12:34.542 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:34.567 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rgw 2024-03-23T08:12:34.648 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rbd 2024-03-23T08:12:35.318 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:30 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:35.394 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:31 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:36.885 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:36.933 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:36.934 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-03-23T08:12:36.934 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:36.946 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:36.994 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:36.995 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-03-23T08:12:36.995 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:37.042 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-cephfs 2024-03-23T08:12:37.105 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-fuse 2024-03-23T08:12:37.800 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:32 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:33 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:39.361 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:39.409 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:39.410 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-03-23T08:12:39.410 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:39.465 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:39.465 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:39.465 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-03-23T08:12:39.465 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:39.465 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:12:39.465 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 90 k 2024-03-23T08:12:39.466 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:39.466 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:12:39.466 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:39.466 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-03-23T08:12:39.466 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:39.466 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 90 k 2024-03-23T08:12:39.466 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 268 k 2024-03-23T08:12:39.467 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:12:39.511 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rbd 2024-03-23T08:12:39.628 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el8.x86_64.rpm 560 kB/s | 90 kB 00:00 2024-03-23T08:12:39.629 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:39.629 INFO:teuthology.orchestra.run.smithi070.stdout:Total 556 kB/s | 90 kB 00:00 2024-03-23T08:12:39.629 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:12:39.638 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:12:39.638 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:12:39.687 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:12:39.690 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:12:39.797 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:12:40.005 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:40.264 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:35 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:40.312 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:40.679 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:12:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:40.680 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:41.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-mirror 2024-03-23T08:12:41.822 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 is already installed. 2024-03-23T08:12:41.870 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:41.871 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-03-23T08:12:41.871 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:41.969 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-fuse 2024-03-23T08:12:41.998 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:37 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:42.720 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:37 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:43.589 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:43.590 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-03-23T08:12:43.590 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:43.590 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:12:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 3.5 M 2024-03-23T08:12:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:43.591 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:12:43.591 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:43.591 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-03-23T08:12:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:43.591 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 3.5 M 2024-03-23T08:12:43.591 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 16 M 2024-03-23T08:12:43.591 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:12:44.248 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el8.x86_64.rpm 5.3 MB/s | 3.5 MB 00:00 2024-03-23T08:12:44.249 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:44.249 INFO:teuthology.orchestra.run.smithi070.stdout:Total 5.3 MB/s | 3.5 MB 00:00 2024-03-23T08:12:44.249 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:12:44.260 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:12:44.260 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:12:44.323 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:12:44.326 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:12:44.344 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:44.344 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:44.345 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-03-23T08:12:44.345 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:44.345 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:12:44.345 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 90 k 2024-03-23T08:12:44.345 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:44.345 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:12:44.345 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:44.345 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-03-23T08:12:44.345 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:44.346 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 90 k 2024-03-23T08:12:44.346 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 268 k 2024-03-23T08:12:44.346 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:12:44.604 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el8.x86_64.rpm 350 kB/s | 90 kB 00:00 2024-03-23T08:12:44.605 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:44.605 INFO:teuthology.orchestra.run.smithi193.stdout:Total 349 kB/s | 90 kB 00:00 2024-03-23T08:12:44.605 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:12:44.615 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:12:44.615 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:12:44.664 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:12:44.667 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:12:44.761 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:12:44.769 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:12:44.933 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:44.999 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:45.279 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:45.348 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:45.656 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:45.657 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:12:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:45.657 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:45.657 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:45.732 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:12:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:45.732 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:45.903 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-mirror 2024-03-23T08:12:45.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-nbd 2024-03-23T08:12:46.667 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:41 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:46.744 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:42 ago on Sat 23 Mar 2024 08:10:04 AM UTC. 2024-03-23T08:12:48.285 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:48.285 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:48.285 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-03-23T08:12:48.285 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:48.286 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:12:48.286 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 3.5 M 2024-03-23T08:12:48.286 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:48.286 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:12:48.286 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:48.286 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-03-23T08:12:48.286 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:48.287 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 3.5 M 2024-03-23T08:12:48.287 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 16 M 2024-03-23T08:12:48.287 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:12:48.356 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-03-23T08:12:48.357 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:48.357 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-03-23T08:12:48.357 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:48.357 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-03-23T08:12:48.357 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 176 k 2024-03-23T08:12:48.357 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:48.357 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-03-23T08:12:48.357 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-03-23T08:12:48.358 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-03-23T08:12:48.358 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:48.358 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 176 k 2024-03-23T08:12:48.358 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 556 k 2024-03-23T08:12:48.358 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-03-23T08:12:48.668 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el8.x86_64.rpm 565 kB/s | 176 kB 00:00 2024-03-23T08:12:48.669 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:48.669 INFO:teuthology.orchestra.run.smithi070.stdout:Total 563 kB/s | 176 kB 00:00 2024-03-23T08:12:48.669 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-03-23T08:12:48.680 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-03-23T08:12:48.680 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-03-23T08:12:48.730 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-03-23T08:12:48.733 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-03-23T08:12:48.846 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-03-23T08:12:48.925 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el8.x86_64.rpm 5.4 MB/s | 3.5 MB 00:00 2024-03-23T08:12:48.925 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:48.926 INFO:teuthology.orchestra.run.smithi193.stdout:Total 5.4 MB/s | 3.5 MB 00:00 2024-03-23T08:12:48.926 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:12:48.937 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:12:48.938 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:12:48.999 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:12:49.002 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:12:49.031 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:49.325 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:49.434 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:12:49.608 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:49.711 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-03-23T08:12:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:12:49.711 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-03-23T08:12:49.919 DEBUG:teuthology.parallel:result is None 2024-03-23T08:12:50.026 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:50.420 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:50.420 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:50.420 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:12:50.420 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:50.420 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:50.420 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:50.598 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-nbd 2024-03-23T08:12:51.373 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:02:46 ago on Sat 23 Mar 2024 08:10:05 AM UTC. 2024-03-23T08:12:53.048 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-23T08:12:53.048 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:53.048 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-03-23T08:12:53.048 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:53.048 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-23T08:12:53.049 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el8 ceph 176 k 2024-03-23T08:12:53.049 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:53.049 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-23T08:12:53.049 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-03-23T08:12:53.049 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-03-23T08:12:53.049 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:53.049 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 176 k 2024-03-23T08:12:53.050 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 556 k 2024-03-23T08:12:53.050 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-23T08:12:53.245 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el8.x86_64.rpm 899 kB/s | 176 kB 00:00 2024-03-23T08:12:53.245 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:12:53.246 INFO:teuthology.orchestra.run.smithi193.stdout:Total 894 kB/s | 176 kB 00:00 2024-03-23T08:12:53.246 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-23T08:12:53.256 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-23T08:12:53.256 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-23T08:12:53.305 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-23T08:12:53.309 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-23T08:12:53.561 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-23T08:12:53.731 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:54.008 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:54.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 1/1 2024-03-23T08:12:54.387 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:54.387 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-23T08:12:54.387 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el8.x86_64 2024-03-23T08:12:54.387 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:12:54.387 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-23T08:12:54.590 DEBUG:teuthology.parallel:result is None 2024-03-23T08:12:54.591 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T08:12:54.591 INFO:teuthology.packaging:ref: None 2024-03-23T08:12:54.591 INFO:teuthology.packaging:tag: None 2024-03-23T08:12:54.591 INFO:teuthology.packaging:branch: quincy 2024-03-23T08:12:54.591 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:12:54.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-03-23T08:12:54.983 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T08:12:55.010 INFO:teuthology.orchestra.run.smithi070.stdout:17.2.7-726.g9c4e0c34.el8 2024-03-23T08:12:55.011 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el8 2024-03-23T08:12:55.011 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-23T08:12:55.012 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T08:12:55.013 INFO:teuthology.packaging:ref: None 2024-03-23T08:12:55.013 INFO:teuthology.packaging:tag: None 2024-03-23T08:12:55.013 INFO:teuthology.packaging:branch: quincy 2024-03-23T08:12:55.013 INFO:teuthology.packaging:sha1: 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:12:55.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-03-23T08:12:55.288 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-23T08:12:55.311 INFO:teuthology.orchestra.run.smithi193.stdout:17.2.7-726.g9c4e0c34.el8 2024-03-23T08:12:55.312 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el8 2024-03-23T08:12:55.312 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-23T08:12:55.314 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-23T08:12:55.314 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:12:55.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T08:12:55.348 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:12:55.348 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T08:12:55.379 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-23T08:12:55.380 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:12:55.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T08:12:55.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T08:12:55.489 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:12:55.489 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T08:12:55.516 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T08:12:55.582 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-23T08:12:55.582 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:12:55.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T08:12:55.611 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T08:12:55.679 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:12:55.679 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T08:12:55.705 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T08:12:55.772 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-23T08:12:55.772 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:12:55.772 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T08:12:55.800 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T08:12:55.867 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:12:55.868 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T08:12:55.894 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T08:12:55.961 INFO:teuthology.run_tasks:Running task print... 2024-03-23T08:12:55.971 INFO:teuthology.task.print:**** done install task... 2024-03-23T08:12:55.971 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-23T08:12:56.074 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '82316b04821a6c2c1d458a881920454bbadb8837'} 2024-03-23T08:12:56.074 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-03-23T08:12:56.074 INFO:tasks.cephadm:Cluster fsid is 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:12:56.075 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-23T08:12:56.075 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-23T08:12:56.075 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi070': '172.21.15.70', 'mon.smithi193': '172.21.15.193'} 2024-03-23T08:12:56.075 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-23T08:12:56.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname $(hostname -s) 2024-03-23T08:12:56.103 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-03-23T08:12:56.130 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-03-23T08:12:56.130 DEBUG:teuthology.orchestra.run.smithi070:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T08:12:56.343 INFO:teuthology.orchestra.run.smithi070.stdout:-rw-rw-r--. 1 ubuntu ubuntu 375153 Mar 23 08:12 /home/ubuntu/cephtest/cephadm 2024-03-23T08:12:56.343 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-23T08:12:56.567 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-rw-r--. 1 ubuntu ubuntu 375153 Mar 23 08:12 /home/ubuntu/cephtest/cephadm 2024-03-23T08:12:56.567 DEBUG:teuthology.orchestra.run.smithi070:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T08:12:56.588 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-23T08:12:56.616 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-03-23T08:12:56.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-23T08:12:56.631 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-23T08:12:57.829 INFO:teuthology.orchestra.run.smithi070.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-23T08:12:57.931 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-23T08:13:26.968 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-03-23T08:13:26.968 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-23T08:13:26.968 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-23T08:13:26.968 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-03-23T08:13:26.968 INFO:teuthology.orchestra.run.smithi193.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-23T08:13:26.969 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-03-23T08:13:26.969 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-03-23T08:13:27.309 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:13:27.309 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-23T08:13:27.309 INFO:teuthology.orchestra.run.smithi070.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-23T08:13:27.309 INFO:teuthology.orchestra.run.smithi070.stdout: "repo_digests": [ 2024-03-23T08:13:27.309 INFO:teuthology.orchestra.run.smithi070.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-23T08:13:27.309 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-03-23T08:13:27.309 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:13:27.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph 2024-03-23T08:13:27.360 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-03-23T08:13:27.392 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /etc/ceph 2024-03-23T08:13:27.428 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-03-23T08:13:27.459 INFO:tasks.cephadm:Writing seed config... 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-23T08:13:27.460 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-23T08:13:27.461 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-23T08:13:27.462 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-23T08:13:27.463 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:13:27.463 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-23T08:13:27.485 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2762dd54-e8ed-11ee-95cd-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-23T08:13:27.486 DEBUG:teuthology.orchestra.run.smithi070:mon.smithi070> sudo journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi070.service 2024-03-23T08:13:27.528 INFO:tasks.cephadm:Bootstrapping... 2024-03-23T08:13:27.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.70 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-23T08:13:27.554 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:-- Logs begin at Sat 2024-03-23 08:00:59 UTC. -- 2024-03-23T08:13:27.727 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-03-23T08:13:27.728 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '2762dd54-e8ed-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.70', '--skip-admin-label'] 2024-03-23T08:13:27.760 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-23T08:13:27.761 INFO:teuthology.orchestra.run.smithi070.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-23T08:13:27.761 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying podman|docker is present... 2024-03-23T08:13:27.791 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-23T08:13:27.791 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying lvm2 is present... 2024-03-23T08:13:27.791 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying time synchronization is in place... 2024-03-23T08:13:27.798 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T08:13:27.798 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T08:13:27.804 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T08:13:27.804 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-03-23T08:13:27.810 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2024-03-23T08:13:27.816 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2024-03-23T08:13:27.816 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2024-03-23T08:13:27.816 INFO:teuthology.orchestra.run.smithi070.stdout:Repeating the final host check... 2024-03-23T08:13:27.850 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-23T08:13:27.850 INFO:teuthology.orchestra.run.smithi070.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-23T08:13:27.850 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl is present 2024-03-23T08:13:27.850 INFO:teuthology.orchestra.run.smithi070.stdout:lvcreate is present 2024-03-23T08:13:27.856 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-23T08:13:27.856 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-23T08:13:27.861 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-23T08:13:27.861 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-03-23T08:13:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2024-03-23T08:13:27.873 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2024-03-23T08:13:27.873 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2024-03-23T08:13:27.873 INFO:teuthology.orchestra.run.smithi070.stdout:Host looks OK 2024-03-23T08:13:27.873 INFO:teuthology.orchestra.run.smithi070.stdout:Cluster fsid: 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:13:27.873 INFO:teuthology.orchestra.run.smithi070.stdout:Acquiring lock 140514671850384 on /run/cephadm/2762dd54-e8ed-11ee-95cd-87774f69a715.lock 2024-03-23T08:13:27.874 INFO:teuthology.orchestra.run.smithi070.stdout:Lock 140514671850384 acquired on /run/cephadm/2762dd54-e8ed-11ee-95cd-87774f69a715.lock 2024-03-23T08:13:27.874 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying IP 172.21.15.70 port 3300 ... 2024-03-23T08:13:27.874 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying IP 172.21.15.70 port 6789 ... 2024-03-23T08:13:27.874 INFO:teuthology.orchestra.run.smithi070.stdout:Base mon IP(s) is [172.21.15.70:3300, 172.21.15.70:6789], mon addrv is [v2:172.21.15.70:3300,v1:172.21.15.70:6789] 2024-03-23T08:13:27.878 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.70 metric 100 2024-03-23T08:13:27.878 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.70 metric 100 2024-03-23T08:13:27.880 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-23T08:13:27.880 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-23T08:13:27.881 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-03-23T08:13:27.883 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-23T08:13:27.883 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-23T08:13:27.883 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T08:13:27.883 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-23T08:13:27.883 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2024-03-23T08:13:27.884 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-23T08:13:27.884 INFO:teuthology.orchestra.run.smithi070.stdout:Mon IP `172.21.15.70` is in CIDR network `172.21.0.0/20` 2024-03-23T08:13:27.884 INFO:teuthology.orchestra.run.smithi070.stdout:Mon IP `172.21.15.70` is in CIDR network `172.21.0.0/20` 2024-03-23T08:13:27.884 INFO:teuthology.orchestra.run.smithi070.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-23T08:13:27.885 INFO:teuthology.orchestra.run.smithi070.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-23T08:13:27.885 INFO:teuthology.orchestra.run.smithi070.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-23T08:13:28.295 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-23T08:13:28.296 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-23T08:13:28.296 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Getting image source signatures 2024-03-23T08:13:28.296 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:601ec453fe24458667695dda21742a4f3ea719c31ad8aa8ec48bf585cc42a7d7 2024-03-23T08:13:28.296 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-23T08:13:28.296 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying config sha256:837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-23T08:13:28.296 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-23T08:13:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:ceph: stdout ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-23T08:13:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph version: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-23T08:13:29.441 INFO:teuthology.orchestra.run.smithi070.stdout:Extracting ceph user uid/gid from container image... 2024-03-23T08:13:30.412 INFO:teuthology.orchestra.run.smithi070.stdout:stat: stdout 167 167 2024-03-23T08:13:30.412 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial keys... 2024-03-23T08:13:31.516 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQAqj/5lTN1HMxAAn6Wz6BvDu9vjY44VNrtRow== 2024-03-23T08:13:32.651 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQArj/5lUVt5ORAAepa2tt+tIkgHg+MhMk2znw== 2024-03-23T08:13:33.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQAtj/5lwOT9BBAAy1KF8l/LXa7Th+lVMCU6+g== 2024-03-23T08:13:33.674 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial monmap... 2024-03-23T08:13:34.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T08:13:34.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-03-23T08:13:34.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:13:34.804 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T08:13:34.804 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool for smithi070 [v2:172.21.15.70:3300,v1:172.21.15.70:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-23T08:13:34.804 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = octopus 2024-03-23T08:13:34.804 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: set fsid to 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:13:34.804 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-23T08:13:34.804 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:13:34.804 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mon... 2024-03-23T08:13:35.884 INFO:teuthology.orchestra.run.smithi070.stdout:create mon.smithi070 on 2024-03-23T08:13:35.981 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-03-23T08:13:36.068 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-23T08:13:36.158 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715.target. 2024-03-23T08:13:36.158 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715.target. 2024-03-23T08:13:36.278 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi070 2024-03-23T08:13:36.279 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi070.service: Unit ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi070.service not loaded. 2024-03-23T08:13:36.368 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715.target.wants/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi070.service → /etc/systemd/system/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@.service. 2024-03-23T08:13:36.984 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T08:13:36.984 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-03-23T08:13:36.991 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T08:13:36.991 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-03-23T08:13:36.991 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-03-23T08:13:36.991 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T08:13:36.992 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon to start... 2024-03-23T08:13:36.992 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon... 2024-03-23T08:13:37.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:36 smithi070 bash[39175]: ef2aa9ef23decedac758c76d88b964a7d32051d1adc476f6882718867cb0dc71 2024-03-23T08:13:37.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:36 smithi070 systemd[1]: Started Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:13:38.176 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout cluster: 2024-03-23T08:13:38.176 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout id: 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:13:38.176 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-23T08:13:38.176 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:13:38.176 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout services: 2024-03-23T08:13:38.176 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi070 (age 0.716721s) 2024-03-23T08:13:38.176 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-23T08:13:38.176 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-23T08:13:38.176 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout data: 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pgs: 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.555+0000 7ff162602700 1 Processor -- start 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.555+0000 7ff162602700 1 -- start start 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.556+0000 7ff162602700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c1018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.556+0000 7ff162602700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c101e30 con 0x7ff15c1014e0 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.556+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c1018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.556+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c1018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36452/0 (socket says 172.21.15.70:36452) 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.556+0000 7ff15bfff700 1 -- 172.21.15.70:0/1389095371 learned_addr learned my addr 172.21.15.70:0/1389095371 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.556+0000 7ff15bfff700 1 -- 172.21.15.70:0/1389095371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c101f70 con 0x7ff15c1014e0 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.556+0000 7ff15bfff700 1 --2- 172.21.15.70:0/1389095371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c1018f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00bb80 tx=0x7ff14c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca85f97f198d62c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:38.177 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff15affd700 1 -- 172.21.15.70:0/1389095371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff14c00dd00 con 0x7ff15c1014e0 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff15affd700 1 -- 172.21.15.70:0/1389095371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c00de60 con 0x7ff15c1014e0 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff15affd700 1 -- 172.21.15.70:0/1389095371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff14c0105f0 con 0x7ff15c1014e0 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff162602700 1 -- 172.21.15.70:0/1389095371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 msgr2=0x7ff15c1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff162602700 1 --2- 172.21.15.70:0/1389095371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c1018f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00bb80 tx=0x7ff14c00b4f0 comp rx=0 tx=0).stop 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff162602700 1 -- 172.21.15.70:0/1389095371 shutdown_connections 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff162602700 1 --2- 172.21.15.70:0/1389095371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c1018f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff162602700 1 -- 172.21.15.70:0/1389095371 >> 172.21.15.70:0/1389095371 conn(0x7ff15c0fcc60 msgr2=0x7ff15c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff162602700 1 -- 172.21.15.70:0/1389095371 shutdown_connections 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.557+0000 7ff162602700 1 -- 172.21.15.70:0/1389095371 wait complete. 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.558+0000 7ff162602700 1 Processor -- start 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.558+0000 7ff162602700 1 -- start start 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.558+0000 7ff162602700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c112e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.558+0000 7ff162602700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c1133c0 con 0x7ff15c1014e0 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.558+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c112e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:38.178 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.558+0000 7ff15bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c112e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36454/0 (socket says 172.21.15.70:36454) 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.558+0000 7ff15bfff700 1 -- 172.21.15.70:0/2260066925 learned_addr learned my addr 172.21.15.70:0/2260066925 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.558+0000 7ff15bfff700 1 -- 172.21.15.70:0/2260066925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c00b790 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.559+0000 7ff15bfff700 1 --2- 172.21.15.70:0/2260066925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c112e80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00b530 tx=0x7ff14c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.559+0000 7ff158ff9700 1 -- 172.21.15.70:0/2260066925 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff14c0189b0 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.559+0000 7ff158ff9700 1 -- 172.21.15.70:0/2260066925 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c018b10 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.559+0000 7ff162602700 1 -- 172.21.15.70:0/2260066925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c1135c0 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.559+0000 7ff158ff9700 1 -- 172.21.15.70:0/2260066925 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff14c018e20 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.559+0000 7ff162602700 1 -- 172.21.15.70:0/2260066925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c113ff0 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.559+0000 7ff158ff9700 1 -- 172.21.15.70:0/2260066925 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff14c022450 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.559+0000 7ff158ff9700 1 -- 172.21.15.70:0/2260066925 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff14c014070 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.560+0000 7ff162602700 1 -- 172.21.15.70:0/2260066925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15c05f5f0 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.562+0000 7ff158ff9700 1 -- 172.21.15.70:0/2260066925 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff14c027070 con 0x7ff15c1014e0 2024-03-23T08:13:38.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.630+0000 7ff162602700 1 -- 172.21.15.70:0/2260066925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff15c1142d0 con 0x7ff15c1014e0 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.631+0000 7ff158ff9700 1 -- 172.21.15.70:0/2260066925 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7ff14c022930 con 0x7ff15c1014e0 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.631+0000 7ff162602700 1 -- 172.21.15.70:0/2260066925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 msgr2=0x7ff15c112e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.631+0000 7ff162602700 1 --2- 172.21.15.70:0/2260066925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c112e80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00b530 tx=0x7ff14c01a000 comp rx=0 tx=0).stop 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.631+0000 7ff162602700 1 -- 172.21.15.70:0/2260066925 shutdown_connections 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.631+0000 7ff162602700 1 --2- 172.21.15.70:0/2260066925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff15c1014e0 0x7ff15c112e80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.631+0000 7ff162602700 1 -- 172.21.15.70:0/2260066925 >> 172.21.15.70:0/2260066925 conn(0x7ff15c0fcc60 msgr2=0x7ff15c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.631+0000 7ff162602700 1 -- 172.21.15.70:0/2260066925 shutdown_connections 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:37.631+0000 7ff162602700 1 -- 172.21.15.70:0/2260066925 wait complete. 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:mon is available 2024-03-23T08:13:38.180 INFO:teuthology.orchestra.run.smithi070.stdout:Assimilating anything we can from ceph.conf... 2024-03-23T08:13:38.180 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:37 smithi070 ceph-mon[39213]: mon.smithi070 is new leader, mons smithi070 in quorum (ranks 0) 2024-03-23T08:13:38.180 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:37 smithi070 ceph-mon[39213]: monmap e1: 1 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T08:13:38.180 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:37 smithi070 ceph-mon[39213]: fsmap 2024-03-23T08:13:38.180 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:37 smithi070 ceph-mon[39213]: osdmap e1: 0 total, 0 up, 0 in 2024-03-23T08:13:38.181 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:37 smithi070 ceph-mon[39213]: mgrmap e1: no daemons active 2024-03-23T08:13:38.181 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:37 smithi070 ceph-mon[39213]: from='client.? 172.21.15.70:0/2260066925' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-23T08:13:39.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:39 smithi070 ceph-mon[39213]: from='client.? 172.21.15.70:0/4086818134' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-23T08:13:39.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:39 smithi070 ceph-mon[39213]: from='client.? 172.21.15.70:0/4086818134' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-23T08:13:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:13:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T08:13:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:13:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.70:3300,v1:172.21.15.70:6789] 2024-03-23T08:13:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T08:13:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T08:13:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T08:13:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T08:13:41.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.105+0000 7fdb2dd63700 1 Processor -- start 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.105+0000 7fdb2dd63700 1 -- start start 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.105+0000 7fdb2dd63700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb281018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.105+0000 7fdb2dd63700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb28101e30 con 0x7fdb281014e0 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.106+0000 7fdb277fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb281018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.106+0000 7fdb277fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb281018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55704/0 (socket says 172.21.15.70:55704) 2024-03-23T08:13:41.672 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.106+0000 7fdb277fe700 1 -- 172.21.15.70:0/636714198 learned_addr learned my addr 172.21.15.70:0/636714198 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.106+0000 7fdb277fe700 1 -- 172.21.15.70:0/636714198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28101f70 con 0x7fdb281014e0 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb277fe700 1 --2- 172.21.15.70:0/636714198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb281018f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdb180098e0 tx=0x7fdb18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103de3ebdd85f888 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb267fc700 1 -- 172.21.15.70:0/636714198 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb1800b860 con 0x7fdb281014e0 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb267fc700 1 -- 172.21.15.70:0/636714198 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1800b9c0 con 0x7fdb281014e0 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb267fc700 1 -- 172.21.15.70:0/636714198 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb1800bc90 con 0x7fdb281014e0 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb2dd63700 1 -- 172.21.15.70:0/636714198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 msgr2=0x7fdb281018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb2dd63700 1 --2- 172.21.15.70:0/636714198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb281018f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdb180098e0 tx=0x7fdb18009250 comp rx=0 tx=0).stop 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb2dd63700 1 -- 172.21.15.70:0/636714198 shutdown_connections 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb2dd63700 1 --2- 172.21.15.70:0/636714198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb281018f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb2dd63700 1 -- 172.21.15.70:0/636714198 >> 172.21.15.70:0/636714198 conn(0x7fdb280fcc60 msgr2=0x7fdb280ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb2dd63700 1 -- 172.21.15.70:0/636714198 shutdown_connections 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.107+0000 7fdb2dd63700 1 -- 172.21.15.70:0/636714198 wait complete. 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.108+0000 7fdb2dd63700 1 Processor -- start 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.108+0000 7fdb2dd63700 1 -- start start 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.108+0000 7fdb2dd63700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb28076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:41.673 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.108+0000 7fdb2dd63700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb281138c0 con 0x7fdb281014e0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.108+0000 7fdb277fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb28076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.108+0000 7fdb277fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb28076b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55712/0 (socket says 172.21.15.70:55712) 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.108+0000 7fdb277fe700 1 -- 172.21.15.70:0/4086818134 learned_addr learned my addr 172.21.15.70:0/4086818134 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.109+0000 7fdb277fe700 1 -- 172.21.15.70:0/4086818134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb180094f0 con 0x7fdb281014e0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.109+0000 7fdb277fe700 1 --2- 172.21.15.70:0/4086818134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb28076b30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdb18003080 tx=0x7fdb18016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.109+0000 7fdb0ffff700 1 -- 172.21.15.70:0/4086818134 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb1801c5c0 con 0x7fdb281014e0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.109+0000 7fdb0ffff700 1 -- 172.21.15.70:0/4086818134 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1801c720 con 0x7fdb281014e0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.109+0000 7fdb0ffff700 1 -- 172.21.15.70:0/4086818134 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb1801ca30 con 0x7fdb281014e0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.109+0000 7fdb2dd63700 1 -- 172.21.15.70:0/4086818134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb28113a00 con 0x7fdb281014e0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.109+0000 7fdb2dd63700 1 -- 172.21.15.70:0/4086818134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb28113e90 con 0x7fdb281014e0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.109+0000 7fdb0ffff700 1 -- 172.21.15.70:0/4086818134 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fdb1801a450 con 0x7fdb281014e0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.109+0000 7fdb0ffff700 1 -- 172.21.15.70:0/4086818134 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdb1802e790 con 0x7fdb281014e0 2024-03-23T08:13:41.674 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.110+0000 7fdb2dd63700 1 -- 172.21.15.70:0/4086818134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb28109530 con 0x7fdb281014e0 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.112+0000 7fdb0ffff700 1 -- 172.21.15.70:0/4086818134 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fdb1801a890 con 0x7fdb281014e0 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.178+0000 7fdb2dd63700 1 -- 172.21.15.70:0/4086818134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fdb2804bed0 con 0x7fdb281014e0 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.182+0000 7fdb0ffff700 1 -- 172.21.15.70:0/4086818134 <== mon.0 v2:172.21.15.70:3300/0 7 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fdb1802ee60 con 0x7fdb281014e0 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.182+0000 7fdb0ffff700 1 -- 172.21.15.70:0/4086818134 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fdb18024790 con 0x7fdb281014e0 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.183+0000 7fdb2dd63700 1 -- 172.21.15.70:0/4086818134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 msgr2=0x7fdb28076b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.183+0000 7fdb2dd63700 1 --2- 172.21.15.70:0/4086818134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb28076b30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdb18003080 tx=0x7fdb18016000 comp rx=0 tx=0).stop 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.183+0000 7fdb2dd63700 1 -- 172.21.15.70:0/4086818134 shutdown_connections 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.183+0000 7fdb2dd63700 1 --2- 172.21.15.70:0/4086818134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb281014e0 0x7fdb28076b30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.183+0000 7fdb2dd63700 1 -- 172.21.15.70:0/4086818134 >> 172.21.15.70:0/4086818134 conn(0x7fdb280fcc60 msgr2=0x7fdb28103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.183+0000 7fdb2dd63700 1 -- 172.21.15.70:0/4086818134 shutdown_connections 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:39.183+0000 7fdb2dd63700 1 -- 172.21.15.70:0/4086818134 wait complete. 2024-03-23T08:13:41.675 INFO:teuthology.orchestra.run.smithi070.stdout:Generating new minimal ceph.conf... 2024-03-23T08:13:42.550 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:42 smithi070 ceph-mon[39213]: from='client.? 172.21.15.70:0/861320721' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:13:42.800 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.162+0000 7f9d521a9700 1 Processor -- start 2024-03-23T08:13:42.800 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.162+0000 7f9d521a9700 1 -- start start 2024-03-23T08:13:42.800 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.162+0000 7f9d521a9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:42.800 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.162+0000 7f9d521a9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c104530 con 0x7f9d4c103be0 2024-03-23T08:13:42.800 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.163+0000 7f9d4b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:42.800 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.163+0000 7f9d4b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c103ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55728/0 (socket says 172.21.15.70:55728) 2024-03-23T08:13:42.800 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.163+0000 7f9d4b7fe700 1 -- 172.21.15.70:0/2159274710 learned_addr learned my addr 172.21.15.70:0/2159274710 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:42.800 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.163+0000 7f9d4b7fe700 1 -- 172.21.15.70:0/2159274710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c104670 con 0x7f9d4c103be0 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.163+0000 7f9d4b7fe700 1 --2- 172.21.15.70:0/2159274710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c103ff0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9d340098e0 tx=0x7f9d34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a975dd4e4c539d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.163+0000 7f9d4a7fc700 1 -- 172.21.15.70:0/2159274710 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d3400b860 con 0x7f9d4c103be0 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.163+0000 7f9d4a7fc700 1 -- 172.21.15.70:0/2159274710 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f9d3400be30 con 0x7f9d4c103be0 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.163+0000 7f9d4a7fc700 1 -- 172.21.15.70:0/2159274710 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d34010380 con 0x7f9d4c103be0 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.164+0000 7f9d521a9700 1 -- 172.21.15.70:0/2159274710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 msgr2=0x7f9d4c103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.164+0000 7f9d521a9700 1 --2- 172.21.15.70:0/2159274710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c103ff0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9d340098e0 tx=0x7f9d34009250 comp rx=0 tx=0).stop 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.164+0000 7f9d521a9700 1 -- 172.21.15.70:0/2159274710 shutdown_connections 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.164+0000 7f9d521a9700 1 --2- 172.21.15.70:0/2159274710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c103ff0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.164+0000 7f9d521a9700 1 -- 172.21.15.70:0/2159274710 >> 172.21.15.70:0/2159274710 conn(0x7f9d4c0fc400 msgr2=0x7f9d4c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.164+0000 7f9d521a9700 1 -- 172.21.15.70:0/2159274710 shutdown_connections 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.164+0000 7f9d521a9700 1 -- 172.21.15.70:0/2159274710 wait complete. 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.164+0000 7f9d521a9700 1 Processor -- start 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d521a9700 1 -- start start 2024-03-23T08:13:42.801 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d521a9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c1032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d521a9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c103830 con 0x7f9d4c103be0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d4b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c1032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d4b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c1032f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55736/0 (socket says 172.21.15.70:55736) 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d4b7fe700 1 -- 172.21.15.70:0/861320721 learned_addr learned my addr 172.21.15.70:0/861320721 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d4b7fe700 1 -- 172.21.15.70:0/861320721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d340094f0 con 0x7f9d4c103be0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d4b7fe700 1 --2- 172.21.15.70:0/861320721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c1032f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d34006b00 tx=0x7f9d34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d43fff700 1 -- 172.21.15.70:0/861320721 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d340109b0 con 0x7f9d4c103be0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d521a9700 1 -- 172.21.15.70:0/861320721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4c1018a0 con 0x7f9d4c103be0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d43fff700 1 -- 172.21.15.70:0/861320721 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f9d34024440 con 0x7f9d4c103be0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d43fff700 1 -- 172.21.15.70:0/861320721 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d3401a5f0 con 0x7f9d4c103be0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.165+0000 7f9d521a9700 1 -- 172.21.15.70:0/861320721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c101d90 con 0x7f9d4c103be0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.166+0000 7f9d43fff700 1 -- 172.21.15.70:0/861320721 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9d34010d90 con 0x7f9d4c103be0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.166+0000 7f9d43fff700 1 -- 172.21.15.70:0/861320721 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9d3402d740 con 0x7f9d4c103be0 2024-03-23T08:13:42.802 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.167+0000 7f9d521a9700 1 -- 172.21.15.70:0/861320721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4c04bed0 con 0x7f9d4c103be0 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.169+0000 7f9d43fff700 1 -- 172.21.15.70:0/861320721 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9d3401a8b0 con 0x7f9d4c103be0 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.234+0000 7f9d521a9700 1 -- 172.21.15.70:0/861320721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9d4c101fd0 con 0x7f9d4c103be0 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.235+0000 7f9d43fff700 1 -- 172.21.15.70:0/861320721 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f9d34032030 con 0x7f9d4c103be0 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.235+0000 7f9d521a9700 1 -- 172.21.15.70:0/861320721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 msgr2=0x7f9d4c1032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.235+0000 7f9d521a9700 1 --2- 172.21.15.70:0/861320721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c1032f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d34006b00 tx=0x7f9d34009250 comp rx=0 tx=0).stop 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.236+0000 7f9d521a9700 1 -- 172.21.15.70:0/861320721 shutdown_connections 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.236+0000 7f9d521a9700 1 --2- 172.21.15.70:0/861320721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c103be0 0x7f9d4c1032f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.236+0000 7f9d521a9700 1 -- 172.21.15.70:0/861320721 >> 172.21.15.70:0/861320721 conn(0x7f9d4c0fc400 msgr2=0x7f9d4c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.236+0000 7f9d521a9700 1 -- 172.21.15.70:0/861320721 shutdown_connections 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:42.236+0000 7f9d521a9700 1 -- 172.21.15.70:0/861320721 wait complete. 2024-03-23T08:13:42.803 INFO:teuthology.orchestra.run.smithi070.stdout:Restarting the monitor... 2024-03-23T08:13:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:42 smithi070 systemd[1]: Stopping Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:13:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:43 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070[39190]: 2024-03-23T08:13:43.051+0000 7fd10b820700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi070 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:13:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:43 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070[39190]: 2024-03-23T08:13:43.051+0000 7fd10b820700 -1 mon.smithi070@0(leader) e1 *** Got Signal Terminated *** 2024-03-23T08:13:43.629 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:43 smithi070 bash[39539]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070 2024-03-23T08:13:43.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:43 smithi070 systemd[1]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi070.service: Succeeded. 2024-03-23T08:13:43.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:43 smithi070 systemd[1]: Stopped Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:13:43.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:43 smithi070 systemd[1]: Starting Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:13:44.370 INFO:teuthology.orchestra.run.smithi070.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable), process ceph-mon, pid 7 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: pidfile_write: ignore empty --pid-file 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: load: jerasure load: lrc 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: RocksDB version: 6.15.5 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Compile date Mar 21 2024 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: DB SUMMARY 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: DB Session ID: BUXIK7GJ1O69P1O914AA 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: CURRENT file: CURRENT 2024-03-23T08:13:44.636 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi070/store.db dir, Total Num: 1, files: 000008.sst 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi070/store.db: 000010.log size: 96728 ; 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.error_if_exists: 0 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.create_if_missing: 0 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.paranoid_checks: 1 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.env: 0x5609a1bc4a80 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.fs: Posix File System 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.info_log: 0x5609a4235c00 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.statistics: (nil) 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.use_fsync: 0 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_log_file_size: 0 2024-03-23T08:13:44.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.allow_fallocate: 1 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.use_direct_reads: 0 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.db_log_dir: 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi070/store.db 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T08:13:44.638 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.write_buffer_manager: 0x5609a4faaab0 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T08:13:44.639 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.unordered_write: 0 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.row_cache: None 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.wal_filter: None 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.preserve_deletes: 0 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.two_write_queues: 0 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.atomic_flush: 0 2024-03-23T08:13:44.640 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.log_readahead_size: 0 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_background_jobs: 2 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_background_compactions: -1 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_subcompactions: 1 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T08:13:44.641 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_open_files: -1 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_background_flushes: -1 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Compression algorithms supported: 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: kZSTD supported: 0 2024-03-23T08:13:44.642 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: kXpressCompression supported: 0 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: kZlibCompression supported: 1 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: kSnappyCompression supported: 1 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: kLZ4Compression supported: 1 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: kBZip2Compression supported: 0 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000009 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.merge_operator: 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_filter: None 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T08:13:44.643 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5609a428af58) 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks: 1 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_top_level_index_and_filter: 1 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_type: 0 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_index_type: 0 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_shortening: 1 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: hash_index_allow_collision: 1 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: checksum: 1 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: no_block_cache: 0 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache: 0x5609a42bd090 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_name: BinnedLRUCache 2024-03-23T08:13:44.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_options: 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: capacity : 536870912 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_shard_bits : 4 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: strict_capacity_limit : 0 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: high_pri_pool_ratio: 0.000 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_compressed: (nil) 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: persistent_cache: (nil) 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size: 4096 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size_deviation: 10 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_restart_interval: 16 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_block_restart_interval: 1 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: metadata_block_size: 4096 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: partition_filters: 0 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: use_delta_encoding: 1 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-23T08:13:44.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: whole_key_filtering: 1 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: verify_compression: 0 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: read_amp_bytes_per_bit: 0 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: format_version: 4 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_index_compression: 1 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_align: 0 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compression: NoCompression 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.num_levels: 7 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T08:13:44.646 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T08:13:44.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T08:13:44.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T08:13:44.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T08:13:44.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T08:13:44.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T08:13:44.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T08:13:44.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T08:13:44.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T08:13:44.647 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T08:13:44.648 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T08:13:44.648 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T08:13:44.648 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T08:13:44.648 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T08:13:44.648 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T08:13:44.648 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T08:13:44.648 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T08:13:44.649 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.arena_block_size: 4194304 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T08:13:44.650 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.table_properties_collectors: 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.inplace_update_support: 0 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.bloom_locality: 0 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.max_successive_merges: 0 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T08:13:44.651 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.ttl: 2592000 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.enable_blob_files: false 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.min_blob_size: 0 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711181624334552, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-03-23T08:13:44.652 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711181624335281, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92097, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90192, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13225, "raw_average_key_size": 49, "raw_value_size": 82931, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 267, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711181624, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "e145644b-fd3d-41f4-945b-9a9ae6535b27", "db_session_id": "BUXIK7GJ1O69P1O914AA"}} 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711181624335597, "job": 1, "event": "recovery_finished"} 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi070/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5609a4251180 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: DB pointer 0x5609a4326000 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** DB Stats ** 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T08:13:44.653 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L0 2/0 91.74 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.8 0.00 0.00 1 0.000 0 0 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Sum 2/0 91.74 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.8 0.00 0.00 1 0.000 0 0 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.8 0.00 0.00 1 0.000 0 0 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 181.8 0.00 0.00 1 0.000 0 0 2024-03-23T08:13:44.654 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative compaction: 0.00 GB write, 29.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval compaction: 0.00 GB write, 29.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:13:44.655 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L0 2/0 91.74 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.8 0.00 0.00 1 0.000 0 0 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Sum 2/0 91.74 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 181.8 0.00 0.00 1 0.000 0 0 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 181.8 0.00 0.00 1 0.000 0 0 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative compaction: 0.00 GB write, 29.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:13:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: starting mon.smithi070 rank 0 at public addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] at bind addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi070 fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mon.smithi070@-1(???) e1 preinit fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mon.smithi070@-1(???).mds e1 new map 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mon.smithi070@-1(???).mds e1 print_map 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: e1 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: legacy client fscid: -1 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: No filesystems configured 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mon.smithi070@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T08:13:44.657 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mon.smithi070@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mon.smithi070@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mon.smithi070@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mon.smithi070@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 bash[39660]: dfce3cd82b237382530c3100cde8bbb618811654b34e0505bb81cb979a49628a 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 systemd[1]: Started Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mon.smithi070 is new leader, mons smithi070 in quorum (ranks 0) 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: monmap e1: 1 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: fsmap 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: osdmap e1: 0 total, 0 up, 0 in 2024-03-23T08:13:44.658 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:44 smithi070 ceph-mon[39699]: mgrmap e1: no daemons active 2024-03-23T08:13:45.619 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.998+0000 7f893668a700 1 Processor -- start 2024-03-23T08:13:45.619 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.998+0000 7f893668a700 1 -- start start 2024-03-23T08:13:45.619 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.998+0000 7f893668a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930102a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:45.619 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.998+0000 7f893668a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8930102fc0 con 0x7f8930102670 2024-03-23T08:13:45.619 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.999+0000 7f892ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:45.619 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.999+0000 7f892ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55740/0 (socket says 172.21.15.70:55740) 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.999+0000 7f892ffff700 1 -- 172.21.15.70:0/370033936 learned_addr learned my addr 172.21.15.70:0/370033936 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.999+0000 7f892ffff700 1 -- 172.21.15.70:0/370033936 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8930103100 con 0x7f8930102670 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.999+0000 7f892ffff700 1 --2- 172.21.15.70:0/370033936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930102a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f89200098e0 tx=0x7f8920009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4efbf753c49e360 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.999+0000 7f892effd700 1 -- 172.21.15.70:0/370033936 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f892000b860 con 0x7f8930102670 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.999+0000 7f892effd700 1 -- 172.21.15.70:0/370033936 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f892000be30 con 0x7f8930102670 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:44.999+0000 7f892effd700 1 -- 172.21.15.70:0/370033936 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8920010380 con 0x7f8930102670 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.000+0000 7f893668a700 1 -- 172.21.15.70:0/370033936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 msgr2=0x7f8930102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.000+0000 7f893668a700 1 --2- 172.21.15.70:0/370033936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930102a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f89200098e0 tx=0x7f8920009250 comp rx=0 tx=0).stop 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.000+0000 7f893668a700 1 -- 172.21.15.70:0/370033936 shutdown_connections 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.000+0000 7f893668a700 1 --2- 172.21.15.70:0/370033936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930102a80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.000+0000 7f893668a700 1 -- 172.21.15.70:0/370033936 >> 172.21.15.70:0/370033936 conn(0x7f89300fdd40 msgr2=0x7f8930100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.000+0000 7f893668a700 1 -- 172.21.15.70:0/370033936 shutdown_connections 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.000+0000 7f893668a700 1 -- 172.21.15.70:0/370033936 wait complete. 2024-03-23T08:13:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.000+0000 7f893668a700 1 Processor -- start 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.001+0000 7f893668a700 1 -- start start 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.001+0000 7f893668a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930111370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.001+0000 7f893668a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89301118b0 con 0x7f8930102670 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.001+0000 7f892ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930111370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.001+0000 7f892ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930111370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55752/0 (socket says 172.21.15.70:55752) 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.001+0000 7f892ffff700 1 -- 172.21.15.70:0/4033668119 learned_addr learned my addr 172.21.15.70:0/4033668119 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.001+0000 7f892ffff700 1 -- 172.21.15.70:0/4033668119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89200094f0 con 0x7f8930102670 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.001+0000 7f892ffff700 1 --2- 172.21.15.70:0/4033668119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930111370 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8920003080 tx=0x7f8920009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.001+0000 7f892cff9700 1 -- 172.21.15.70:0/4033668119 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89200109b0 con 0x7f8930102670 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.002+0000 7f892cff9700 1 -- 172.21.15.70:0/4033668119 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f892001a430 con 0x7f8930102670 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.002+0000 7f892cff9700 1 -- 172.21.15.70:0/4033668119 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89200245b0 con 0x7f8930102670 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.002+0000 7f893668a700 1 -- 172.21.15.70:0/4033668119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8930111ab0 con 0x7f8930102670 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.002+0000 7f893668a700 1 -- 172.21.15.70:0/4033668119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8930112620 con 0x7f8930102670 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.002+0000 7f892cff9700 1 -- 172.21.15.70:0/4033668119 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8920024a70 con 0x7f8930102670 2024-03-23T08:13:45.621 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.002+0000 7f892cff9700 1 -- 172.21.15.70:0/4033668119 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f892002eab0 con 0x7f8930102670 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.002+0000 7f893668a700 1 -- 172.21.15.70:0/4033668119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893005f5f0 con 0x7f8930102670 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.005+0000 7f892cff9700 1 -- 172.21.15.70:0/4033668119 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8920029070 con 0x7f8930102670 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.070+0000 7f893668a700 1 -- 172.21.15.70:0/4033668119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f8930104430 con 0x7f8930102670 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.072+0000 7f892cff9700 1 -- 172.21.15.70:0/4033668119 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f892001a9a0 con 0x7f8930102670 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.073+0000 7f893668a700 1 -- 172.21.15.70:0/4033668119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 msgr2=0x7f8930111370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.073+0000 7f893668a700 1 --2- 172.21.15.70:0/4033668119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930111370 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8920003080 tx=0x7f8920009250 comp rx=0 tx=0).stop 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.073+0000 7f893668a700 1 -- 172.21.15.70:0/4033668119 shutdown_connections 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.073+0000 7f893668a700 1 --2- 172.21.15.70:0/4033668119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8930102670 0x7f8930111370 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.073+0000 7f893668a700 1 -- 172.21.15.70:0/4033668119 >> 172.21.15.70:0/4033668119 conn(0x7f89300fdd40 msgr2=0x7f89300fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.073+0000 7f893668a700 1 -- 172.21.15.70:0/4033668119 shutdown_connections 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:45.073+0000 7f893668a700 1 -- 172.21.15.70:0/4033668119 wait complete. 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-23T08:13:45.622 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mgr... 2024-03-23T08:13:45.623 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 9283 ... 2024-03-23T08:13:45.740 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mgr.smithi070.jhsbch 2024-03-23T08:13:45.741 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mgr.smithi070.jhsbch.service: Unit ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mgr.smithi070.jhsbch.service not loaded. 2024-03-23T08:13:45.831 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715.target.wants/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mgr.smithi070.jhsbch.service → /etc/systemd/system/ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@.service. 2024-03-23T08:13:46.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:46 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/4033668119' entity='client.admin' 2024-03-23T08:13:46.845 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T08:13:46.845 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-03-23T08:13:46.854 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T08:13:46.854 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-03-23T08:13:46.854 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-03-23T08:13:46.854 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2024-03-23T08:13:46.862 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T08:13:46.863 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-03-23T08:13:46.870 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T08:13:46.870 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-03-23T08:13:46.870 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-03-23T08:13:46.870 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-23T08:13:46.870 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr to start... 2024-03-23T08:13:46.871 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr... 2024-03-23T08:13:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:47 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/722085247' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T08:13:48.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:13:48.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-03-23T08:13:48.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "2762dd54-e8ed-11ee-95cd-87774f69a715", 2024-03-23T08:13:48.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T08:13:48.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T08:13:48.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T08:13:48.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T08:13:48.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "smithi070" 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T08:13:48.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T08:13:48.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T08:13:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T08:13:36.916633+0000", 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.399+0000 7f6b54656700 1 Processor -- start 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.399+0000 7f6b54656700 1 -- start start 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.400+0000 7f6b54656700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c105270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.400+0000 7f6b54656700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1057b0 con 0x7f6b4c076bc0 2024-03-23T08:13:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.400+0000 7f6b523f2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.400+0000 7f6b523f2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c105270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55782/0 (socket says 172.21.15.70:55782) 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.400+0000 7f6b523f2700 1 -- 172.21.15.70:0/13612935 learned_addr learned my addr 172.21.15.70:0/13612935 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.400+0000 7f6b523f2700 1 -- 172.21.15.70:0/13612935 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c1058f0 con 0x7f6b4c076bc0 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.401+0000 7f6b523f2700 1 --2- 172.21.15.70:0/13612935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c105270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6b480098e0 tx=0x7f6b48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43216723ff7087dd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.401+0000 7f6b513f0700 1 -- 172.21.15.70:0/13612935 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b4800b860 con 0x7f6b4c076bc0 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.401+0000 7f6b513f0700 1 -- 172.21.15.70:0/13612935 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f6b4800be30 con 0x7f6b4c076bc0 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.401+0000 7f6b513f0700 1 -- 172.21.15.70:0/13612935 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b48010380 con 0x7f6b4c076bc0 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.401+0000 7f6b54656700 1 -- 172.21.15.70:0/13612935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 msgr2=0x7f6b4c105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.401+0000 7f6b54656700 1 --2- 172.21.15.70:0/13612935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c105270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6b480098e0 tx=0x7f6b48009250 comp rx=0 tx=0).stop 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.401+0000 7f6b54656700 1 -- 172.21.15.70:0/13612935 shutdown_connections 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.401+0000 7f6b54656700 1 --2- 172.21.15.70:0/13612935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c105270 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.401+0000 7f6b54656700 1 -- 172.21.15.70:0/13612935 >> 172.21.15.70:0/13612935 conn(0x7f6b4c0fcc60 msgr2=0x7f6b4c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.402+0000 7f6b54656700 1 -- 172.21.15.70:0/13612935 shutdown_connections 2024-03-23T08:13:48.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.402+0000 7f6b54656700 1 -- 172.21.15.70:0/13612935 wait complete. 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.402+0000 7f6b54656700 1 Processor -- start 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.402+0000 7f6b54656700 1 -- start start 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.402+0000 7f6b54656700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c111350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.402+0000 7f6b54656700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c111890 con 0x7f6b4c076bc0 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.402+0000 7f6b523f2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c111350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.402+0000 7f6b523f2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c111350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55794/0 (socket says 172.21.15.70:55794) 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.402+0000 7f6b523f2700 1 -- 172.21.15.70:0/722085247 learned_addr learned my addr 172.21.15.70:0/722085247 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.403+0000 7f6b523f2700 1 -- 172.21.15.70:0/722085247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b480094f0 con 0x7f6b4c076bc0 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.403+0000 7f6b523f2700 1 --2- 172.21.15.70:0/722085247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c111350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006b00 tx=0x7f6b48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.403+0000 7f6b42ffd700 1 -- 172.21.15.70:0/722085247 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b48010a80 con 0x7f6b4c076bc0 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.403+0000 7f6b42ffd700 1 -- 172.21.15.70:0/722085247 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f6b4801a430 con 0x7f6b4c076bc0 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.403+0000 7f6b42ffd700 1 -- 172.21.15.70:0/722085247 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b480245b0 con 0x7f6b4c076bc0 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.403+0000 7f6b54656700 1 -- 172.21.15.70:0/722085247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c111a90 con 0x7f6b4c076bc0 2024-03-23T08:13:48.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.403+0000 7f6b54656700 1 -- 172.21.15.70:0/722085247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c112600 con 0x7f6b4c076bc0 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.403+0000 7f6b42ffd700 1 -- 172.21.15.70:0/722085247 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6b48024a60 con 0x7f6b4c076bc0 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.403+0000 7f6b42ffd700 1 -- 172.21.15.70:0/722085247 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6b4802eaa0 con 0x7f6b4c076bc0 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.404+0000 7f6b54656700 1 -- 172.21.15.70:0/722085247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c05f5f0 con 0x7f6b4c076bc0 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.406+0000 7f6b42ffd700 1 -- 172.21.15.70:0/722085247 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f6b4802ec60 con 0x7f6b4c076bc0 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.478+0000 7f6b54656700 1 -- 172.21.15.70:0/722085247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6b4c1065c0 con 0x7f6b4c076bc0 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.479+0000 7f6b42ffd700 1 -- 172.21.15.70:0/722085247 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f6b4802ee00 con 0x7f6b4c076bc0 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.479+0000 7f6b54656700 1 -- 172.21.15.70:0/722085247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 msgr2=0x7f6b4c111350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.479+0000 7f6b54656700 1 --2- 172.21.15.70:0/722085247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c111350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b48006b00 tx=0x7f6b48009250 comp rx=0 tx=0).stop 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.479+0000 7f6b54656700 1 -- 172.21.15.70:0/722085247 shutdown_connections 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.479+0000 7f6b54656700 1 --2- 172.21.15.70:0/722085247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b4c076bc0 0x7f6b4c111350 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.479+0000 7f6b54656700 1 -- 172.21.15.70:0/722085247 >> 172.21.15.70:0/722085247 conn(0x7f6b4c0fcc60 msgr2=0x7f6b4c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.479+0000 7f6b54656700 1 -- 172.21.15.70:0/722085247 shutdown_connections 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:47.479+0000 7f6b54656700 1 -- 172.21.15.70:0/722085247 wait complete. 2024-03-23T08:13:48.084 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (1/15)... 2024-03-23T08:13:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:50 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1492882572' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T08:13:51.281 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "2762dd54-e8ed-11ee-95cd-87774f69a715", 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "smithi070" 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-03-23T08:13:51.282 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T08:13:51.283 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T08:13:51.284 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T08:13:36.916633+0000", 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.584+0000 7f348d075700 1 Processor -- start 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.584+0000 7f348d075700 1 -- start start 2024-03-23T08:13:51.285 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.584+0000 7f348d075700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488105210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.584+0000 7f348d075700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488105750 con 0x7f3488076c20 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.584+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.584+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488105210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48108/0 (socket says 172.21.15.70:48108) 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.584+0000 7f3486d9d700 1 -- 172.21.15.70:0/1565108819 learned_addr learned my addr 172.21.15.70:0/1565108819 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.585+0000 7f3486d9d700 1 -- 172.21.15.70:0/1565108819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488105890 con 0x7f3488076c20 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.585+0000 7f3486d9d700 1 --2- 172.21.15.70:0/1565108819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488105210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f34780098e0 tx=0x7f3478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d03606c1d53d177 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.585+0000 7f3485d9b700 1 -- 172.21.15.70:0/1565108819 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f347800b860 con 0x7f3488076c20 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.585+0000 7f3485d9b700 1 -- 172.21.15.70:0/1565108819 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f347800be30 con 0x7f3488076c20 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.585+0000 7f3485d9b700 1 -- 172.21.15.70:0/1565108819 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3478010380 con 0x7f3488076c20 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 -- 172.21.15.70:0/1565108819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 msgr2=0x7f3488105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 --2- 172.21.15.70:0/1565108819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488105210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f34780098e0 tx=0x7f3478009250 comp rx=0 tx=0).stop 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 -- 172.21.15.70:0/1565108819 shutdown_connections 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 --2- 172.21.15.70:0/1565108819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488105210 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 -- 172.21.15.70:0/1565108819 >> 172.21.15.70:0/1565108819 conn(0x7f34880fcc60 msgr2=0x7f34880ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 -- 172.21.15.70:0/1565108819 shutdown_connections 2024-03-23T08:13:51.286 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 -- 172.21.15.70:0/1565108819 wait complete. 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 Processor -- start 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 -- start start 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.586+0000 7f348d075700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488111380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f348d075700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34881118c0 con 0x7f3488076c20 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488111380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f3486d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488111380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48118/0 (socket says 172.21.15.70:48118) 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f3486d9d700 1 -- 172.21.15.70:0/1492882572 learned_addr learned my addr 172.21.15.70:0/1492882572 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f3486d9d700 1 -- 172.21.15.70:0/1492882572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780094f0 con 0x7f3488076c20 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f3486d9d700 1 --2- 172.21.15.70:0/1492882572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488111380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3478006b00 tx=0x7f3478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f346f7fe700 1 -- 172.21.15.70:0/1492882572 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3478010a80 con 0x7f3488076c20 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f346f7fe700 1 -- 172.21.15.70:0/1492882572 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f347801a430 con 0x7f3488076c20 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f348d075700 1 -- 172.21.15.70:0/1492882572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3488111ac0 con 0x7f3488076c20 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f346f7fe700 1 -- 172.21.15.70:0/1492882572 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34780245b0 con 0x7f3488076c20 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f348d075700 1 -- 172.21.15.70:0/1492882572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3488112630 con 0x7f3488076c20 2024-03-23T08:13:51.287 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f346f7fe700 1 -- 172.21.15.70:0/1492882572 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3478024ab0 con 0x7f3488076c20 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.587+0000 7f346f7fe700 1 -- 172.21.15.70:0/1492882572 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f347802eb00 con 0x7f3488076c20 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.588+0000 7f348d075700 1 -- 172.21.15.70:0/1492882572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348805f5f0 con 0x7f3488076c20 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.590+0000 7f346f7fe700 1 -- 172.21.15.70:0/1492882572 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f347802ecc0 con 0x7f3488076c20 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.661+0000 7f348d075700 1 -- 172.21.15.70:0/1492882572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f348804bed0 con 0x7f3488076c20 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.661+0000 7f346f7fe700 1 -- 172.21.15.70:0/1492882572 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f348804bed0 con 0x7f3488076c20 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.662+0000 7f348d075700 1 -- 172.21.15.70:0/1492882572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 msgr2=0x7f3488111380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.662+0000 7f348d075700 1 --2- 172.21.15.70:0/1492882572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488111380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3478006b00 tx=0x7f3478009250 comp rx=0 tx=0).stop 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.662+0000 7f348d075700 1 -- 172.21.15.70:0/1492882572 shutdown_connections 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.662+0000 7f348d075700 1 --2- 172.21.15.70:0/1492882572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3488076c20 0x7f3488111380 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.662+0000 7f348d075700 1 -- 172.21.15.70:0/1492882572 >> 172.21.15.70:0/1492882572 conn(0x7f34880fcc60 msgr2=0x7f34880fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.663+0000 7f348d075700 1 -- 172.21.15.70:0/1492882572 shutdown_connections 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:50.663+0000 7f348d075700 1 -- 172.21.15.70:0/1492882572 wait complete. 2024-03-23T08:13:51.288 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (2/15)... 2024-03-23T08:13:53.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: Activating manager daemon smithi070.jhsbch 2024-03-23T08:13:53.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: mgrmap e2: smithi070.jhsbch(active, starting, since 0.00286001s) 2024-03-23T08:13:53.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:13:53.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:13:53.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:13:53.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:13:53.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:13:53.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: Manager daemon smithi070.jhsbch is now available 2024-03-23T08:13:53.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:13:53.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/trash_purge_schedule"}]: dispatch 2024-03-23T08:13:53.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' 2024-03-23T08:13:53.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' 2024-03-23T08:13:53.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:53 smithi070 ceph-mon[39699]: from='mgr.14100 172.21.15.70:0/1601939895' entity='mgr.smithi070.jhsbch' 2024-03-23T08:13:55.345 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:55 smithi070 ceph-mon[39699]: mgrmap e3: smithi070.jhsbch(active, since 1.00637s) 2024-03-23T08:13:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:57 smithi070 ceph-mon[39699]: mgrmap e4: smithi070.jhsbch(active, since 3s) 2024-03-23T08:13:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:13:57 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3654775144' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "2762dd54-e8ed-11ee-95cd-87774f69a715", 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-23T08:14:01.240 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "smithi070" 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-23T08:14:01.241 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-23T08:14:01.242 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-03-23T08:14:01.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-03-23T08:13:36.916633+0000", 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.617+0000 7f01b0f0a700 1 Processor -- start 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.617+0000 7f01b0f0a700 1 -- start start 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.617+0000 7f01b0f0a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.617+0000 7f01b0f0a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac104530 con 0x7f01ac103be0 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.618+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.618+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac103ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48204/0 (socket says 172.21.15.70:48204) 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.618+0000 7f01aa59c700 1 -- 172.21.15.70:0/2130469047 learned_addr learned my addr 172.21.15.70:0/2130469047 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:01.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.618+0000 7f01aa59c700 1 -- 172.21.15.70:0/2130469047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac104670 con 0x7f01ac103be0 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.618+0000 7f01aa59c700 1 --2- 172.21.15.70:0/2130469047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac103ff0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01a00098e0 tx=0x7f01a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e7dd9acf6e230b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.618+0000 7f01a959a700 1 -- 172.21.15.70:0/2130469047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f01a000b860 con 0x7f01ac103be0 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.618+0000 7f01a959a700 1 -- 172.21.15.70:0/2130469047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f01a000be30 con 0x7f01ac103be0 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.618+0000 7f01a959a700 1 -- 172.21.15.70:0/2130469047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f01a0010380 con 0x7f01ac103be0 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.619+0000 7f01b0f0a700 1 -- 172.21.15.70:0/2130469047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 msgr2=0x7f01ac103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.619+0000 7f01b0f0a700 1 --2- 172.21.15.70:0/2130469047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac103ff0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f01a00098e0 tx=0x7f01a0009250 comp rx=0 tx=0).stop 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.619+0000 7f01b0f0a700 1 -- 172.21.15.70:0/2130469047 shutdown_connections 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.619+0000 7f01b0f0a700 1 --2- 172.21.15.70:0/2130469047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac103ff0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.619+0000 7f01b0f0a700 1 -- 172.21.15.70:0/2130469047 >> 172.21.15.70:0/2130469047 conn(0x7f01ac0fc400 msgr2=0x7f01ac0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.619+0000 7f01b0f0a700 1 -- 172.21.15.70:0/2130469047 shutdown_connections 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.619+0000 7f01b0f0a700 1 -- 172.21.15.70:0/2130469047 wait complete. 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.620+0000 7f01b0f0a700 1 Processor -- start 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.620+0000 7f01b0f0a700 1 -- start start 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.620+0000 7f01b0f0a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac111310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.620+0000 7f01b0f0a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac111850 con 0x7f01ac103be0 2024-03-23T08:14:01.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.620+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac111310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.620+0000 7f01aa59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac111310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48206/0 (socket says 172.21.15.70:48206) 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.620+0000 7f01aa59c700 1 -- 172.21.15.70:0/3654775144 learned_addr learned my addr 172.21.15.70:0/3654775144 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.620+0000 7f01aa59c700 1 -- 172.21.15.70:0/3654775144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a00094f0 con 0x7f01ac103be0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.621+0000 7f01aa59c700 1 --2- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac111310 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f01a0006b00 tx=0x7f01a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.621+0000 7f019affd700 1 -- 172.21.15.70:0/3654775144 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f01a00109b0 con 0x7f01ac103be0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.621+0000 7f019affd700 1 -- 172.21.15.70:0/3654775144 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f01a0024440 con 0x7f01ac103be0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.621+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac111a50 con 0x7f01ac103be0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.621+0000 7f019affd700 1 -- 172.21.15.70:0/3654775144 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f01a001a5f0 con 0x7f01ac103be0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.621+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac1125c0 con 0x7f01ac103be0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.621+0000 7f019affd700 1 -- 172.21.15.70:0/3654775144 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f01a0010b10 con 0x7f01ac103be0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.622+0000 7f019affd700 1 --2- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f0194036740 0x7f0194038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.622+0000 7f019affd700 1 -- 172.21.15.70:0/3654775144 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f01a004a600 con 0x7f01ac103be0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.622+0000 7f01a9d9b700 1 --2- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f0194036740 0x7f0194038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:01.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.622+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac05f5f0 con 0x7f01ac103be0 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.622+0000 7f01a9d9b700 1 --2- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f0194036740 0x7f0194038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f019c009990 tx=0x7f019c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.627+0000 7f019affd700 1 -- 172.21.15.70:0/3654775144 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f01a00239f0 con 0x7f01ac103be0 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.884+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f01ac112800 con 0x7f01ac103be0 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.884+0000 7f019affd700 1 -- 172.21.15.70:0/3654775144 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f01a00314f0 con 0x7f01ac103be0 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f0194036740 msgr2=0x7f0194038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 --2- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f0194036740 0x7f0194038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f019c009990 tx=0x7f019c006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 msgr2=0x7f01ac111310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 --2- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac111310 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f01a0006b00 tx=0x7f01a0009250 comp rx=0 tx=0).stop 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 shutdown_connections 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 --2- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f0194036740 0x7f0194038bf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 --2- 172.21.15.70:0/3654775144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01ac103be0 0x7f01ac111310 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 >> 172.21.15.70:0/3654775144 conn(0x7f01ac0fc400 msgr2=0x7f01ac0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 shutdown_connections 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:13:56.886+0000 7f01b0f0a700 1 -- 172.21.15.70:0/3654775144 wait complete. 2024-03-23T08:14:01.247 INFO:teuthology.orchestra.run.smithi070.stdout:mgr is available 2024-03-23T08:14:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:01 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1795683266' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-23T08:14:02.592 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.731+0000 7f76e0992700 1 Processor -- start 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.732+0000 7f76e0992700 1 -- start start 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.732+0000 7f76e0992700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc105210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.732+0000 7f76e0992700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc105750 con 0x7f76dc076c20 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.732+0000 7f76da59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.732+0000 7f76da59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc105210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38620/0 (socket says 172.21.15.70:38620) 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.732+0000 7f76da59c700 1 -- 172.21.15.70:0/1031369459 learned_addr learned my addr 172.21.15.70:0/1031369459 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.733+0000 7f76da59c700 1 -- 172.21.15.70:0/1031369459 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc105890 con 0x7f76dc076c20 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.733+0000 7f76da59c700 1 --2- 172.21.15.70:0/1031369459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc105210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f76c40098e0 tx=0x7f76c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c6d12a5c850614 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:02.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.733+0000 7f76d959a700 1 -- 172.21.15.70:0/1031369459 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76c400b860 con 0x7f76dc076c20 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.733+0000 7f76d959a700 1 -- 172.21.15.70:0/1031369459 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f76c400be30 con 0x7f76dc076c20 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.733+0000 7f76d959a700 1 -- 172.21.15.70:0/1031369459 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76c4010380 con 0x7f76dc076c20 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.733+0000 7f76e0992700 1 -- 172.21.15.70:0/1031369459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 msgr2=0x7f76dc105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.733+0000 7f76e0992700 1 --2- 172.21.15.70:0/1031369459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc105210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f76c40098e0 tx=0x7f76c4009250 comp rx=0 tx=0).stop 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76e0992700 1 -- 172.21.15.70:0/1031369459 shutdown_connections 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76e0992700 1 --2- 172.21.15.70:0/1031369459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc105210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76e0992700 1 -- 172.21.15.70:0/1031369459 >> 172.21.15.70:0/1031369459 conn(0x7f76dc0fcc60 msgr2=0x7f76dc0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76e0992700 1 -- 172.21.15.70:0/1031369459 shutdown_connections 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76e0992700 1 -- 172.21.15.70:0/1031369459 wait complete. 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76e0992700 1 Processor -- start 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76e0992700 1 -- start start 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76e0992700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc111380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76e0992700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc1118c0 con 0x7f76dc076c20 2024-03-23T08:14:02.594 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.734+0000 7f76da59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc111380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.735+0000 7f76da59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc111380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38622/0 (socket says 172.21.15.70:38622) 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.735+0000 7f76da59c700 1 -- 172.21.15.70:0/1795683266 learned_addr learned my addr 172.21.15.70:0/1795683266 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.735+0000 7f76da59c700 1 -- 172.21.15.70:0/1795683266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40094f0 con 0x7f76dc076c20 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.735+0000 7f76da59c700 1 --2- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc111380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f76c4006d40 tx=0x7f76c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.735+0000 7f76d2ffd700 1 -- 172.21.15.70:0/1795683266 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76c40109b0 con 0x7f76dc076c20 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.735+0000 7f76d2ffd700 1 -- 172.21.15.70:0/1795683266 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f76c4024440 con 0x7f76dc076c20 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.735+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc111ac0 con 0x7f76dc076c20 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.735+0000 7f76d2ffd700 1 -- 172.21.15.70:0/1795683266 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76c401a5f0 con 0x7f76dc076c20 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.735+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc112630 con 0x7f76dc076c20 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.736+0000 7f76d2ffd700 1 -- 172.21.15.70:0/1795683266 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f76c4024970 con 0x7f76dc076c20 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.736+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc05f5f0 con 0x7f76dc076c20 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.736+0000 7f76d2ffd700 1 --2- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f76c8036790 0x7f76c8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.736+0000 7f76d2ffd700 1 -- 172.21.15.70:0/1795683266 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f76c404a5d0 con 0x7f76dc076c20 2024-03-23T08:14:02.595 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.736+0000 7f76d9d9b700 1 --2- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f76c8036790 0x7f76c8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.737+0000 7f76d9d9b700 1 --2- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f76c8036790 0x7f76c8038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009990 tx=0x7f76cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.741+0000 7f76d2ffd700 1 -- 172.21.15.70:0/1795683266 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f76c40245b0 con 0x7f76dc076c20 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.924+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f76dc04bed0 con 0x7f76dc076c20 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.924+0000 7f76d2ffd700 1 -- 172.21.15.70:0/1795683266 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f76c40233a0 con 0x7f76dc076c20 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.926+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f76c8036790 msgr2=0x7f76c8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.926+0000 7f76e0992700 1 --2- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f76c8036790 0x7f76c8038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009990 tx=0x7f76cc006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.926+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 msgr2=0x7f76dc111380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.926+0000 7f76e0992700 1 --2- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc111380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f76c4006d40 tx=0x7f76c4009250 comp rx=0 tx=0).stop 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.926+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 shutdown_connections 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.926+0000 7f76e0992700 1 --2- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7f76c8036790 0x7f76c8038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.926+0000 7f76e0992700 1 --2- 172.21.15.70:0/1795683266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76dc076c20 0x7f76dc111380 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.926+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 >> 172.21.15.70:0/1795683266 conn(0x7f76dc0fcc60 msgr2=0x7f76dc0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.927+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 shutdown_connections 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:01.927+0000 7f76e0992700 1 -- 172.21.15.70:0/1795683266 wait complete. 2024-03-23T08:14:02.596 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling cephadm module... 2024-03-23T08:14:03.731 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:03 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2650858645' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-23T08:14:04.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.114+0000 7fe00c8c2700 1 Processor -- start 2024-03-23T08:14:04.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.114+0000 7fe00c8c2700 1 -- start start 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.114+0000 7fe00c8c2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008102970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.114+0000 7fe00c8c2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008102eb0 con 0x7fe008102560 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.114+0000 7fe00659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.114+0000 7fe00659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008102970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38624/0 (socket says 172.21.15.70:38624) 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.114+0000 7fe00659c700 1 -- 172.21.15.70:0/1755499266 learned_addr learned my addr 172.21.15.70:0/1755499266 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.114+0000 7fe00659c700 1 -- 172.21.15.70:0/1755499266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008102ff0 con 0x7fe008102560 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00659c700 1 --2- 172.21.15.70:0/1755499266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008102970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdff80098e0 tx=0x7fdff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c2eb5ef2cc5eb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00559a700 1 -- 172.21.15.70:0/1755499266 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdff800b860 con 0x7fe008102560 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00559a700 1 -- 172.21.15.70:0/1755499266 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fdff800be30 con 0x7fe008102560 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00559a700 1 -- 172.21.15.70:0/1755499266 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdff8010380 con 0x7fe008102560 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00c8c2700 1 -- 172.21.15.70:0/1755499266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 msgr2=0x7fe008102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00c8c2700 1 --2- 172.21.15.70:0/1755499266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008102970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdff80098e0 tx=0x7fdff8009250 comp rx=0 tx=0).stop 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00c8c2700 1 -- 172.21.15.70:0/1755499266 shutdown_connections 2024-03-23T08:14:04.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00c8c2700 1 --2- 172.21.15.70:0/1755499266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008102970 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00c8c2700 1 -- 172.21.15.70:0/1755499266 >> 172.21.15.70:0/1755499266 conn(0x7fe0080fdcc0 msgr2=0x7fe0081000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00c8c2700 1 -- 172.21.15.70:0/1755499266 shutdown_connections 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.115+0000 7fe00c8c2700 1 -- 172.21.15.70:0/1755499266 wait complete. 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.116+0000 7fe00c8c2700 1 Processor -- start 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.116+0000 7fe00c8c2700 1 -- start start 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.116+0000 7fe00c8c2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008111360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.116+0000 7fe00c8c2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081118a0 con 0x7fe008102560 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.116+0000 7fe00659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008111360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.116+0000 7fe00659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008111360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38628/0 (socket says 172.21.15.70:38628) 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.116+0000 7fe00659c700 1 -- 172.21.15.70:0/2650858645 learned_addr learned my addr 172.21.15.70:0/2650858645 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.116+0000 7fe00659c700 1 -- 172.21.15.70:0/2650858645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80094f0 con 0x7fe008102560 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.117+0000 7fe00659c700 1 --2- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008111360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdff8006b00 tx=0x7fdff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.117+0000 7fdff6ffd700 1 -- 172.21.15.70:0/2650858645 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdff80109b0 con 0x7fe008102560 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.117+0000 7fdff6ffd700 1 -- 172.21.15.70:0/2650858645 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fdff8024440 con 0x7fe008102560 2024-03-23T08:14:04.083 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.117+0000 7fdff6ffd700 1 -- 172.21.15.70:0/2650858645 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdff801a5f0 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.117+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe008111aa0 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.117+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe008112610 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.118+0000 7fdff6ffd700 1 -- 172.21.15.70:0/2650858645 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fdff801a860 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.118+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00805f5f0 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.118+0000 7fdff6ffd700 1 --2- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fdff0036790 0x7fdff0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.118+0000 7fdff6ffd700 1 -- 172.21.15.70:0/2650858645 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdff804a830 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.118+0000 7fe005d9b700 1 --2- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fdff0036790 0x7fdff0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.118+0000 7fe005d9b700 1 --2- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fdff0036790 0x7fdff0038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009990 tx=0x7fdffc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.122+0000 7fdff6ffd700 1 -- 172.21.15.70:0/2650858645 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdff801ab10 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.357+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fe00804bed0 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.410+0000 7fdff6ffd700 1 -- 172.21.15.70:0/2650858645 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fdff8024990 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.411+0000 7fdff6ffd700 1 -- 172.21.15.70:0/2650858645 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fdff802c970 con 0x7fe008102560 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fdff0036790 msgr2=0x7fdff0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 --2- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fdff0036790 0x7fdff0038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009990 tx=0x7fdffc006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:04.084 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 msgr2=0x7fe008111360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:04.085 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 --2- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008111360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdff8006b00 tx=0x7fdff8009250 comp rx=0 tx=0).stop 2024-03-23T08:14:04.085 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 shutdown_connections 2024-03-23T08:14:04.085 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 --2- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fdff0036790 0x7fdff0038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:04.085 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 --2- 172.21.15.70:0/2650858645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe008102560 0x7fe008111360 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:04.085 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 >> 172.21.15.70:0/2650858645 conn(0x7fe0080fdcc0 msgr2=0x7fe0080fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:04.085 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 shutdown_connections 2024-03-23T08:14:04.085 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:03.414+0000 7fe00c8c2700 1 -- 172.21.15.70:0/2650858645 wait complete. 2024-03-23T08:14:04.549 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:04 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2650858645' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-23T08:14:04.550 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:04 smithi070 ceph-mon[39699]: mgrmap e5: smithi070.jhsbch(active, since 10s) 2024-03-23T08:14:05.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-03-23T08:14:05.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-23T08:14:05.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T08:14:05.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "smithi070.jhsbch", 2024-03-23T08:14:05.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T08:14:05.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-03-23T08:14:05.446 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.614+0000 7fe27408f700 1 Processor -- start 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.614+0000 7fe27408f700 1 -- start start 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe27408f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c104040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe27408f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c104580 con 0x7fe26c103c30 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe271e2b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe271e2b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c104040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38664/0 (socket says 172.21.15.70:38664) 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe271e2b700 1 -- 172.21.15.70:0/3619326702 learned_addr learned my addr 172.21.15.70:0/3619326702 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe271e2b700 1 -- 172.21.15.70:0/3619326702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c1046c0 con 0x7fe26c103c30 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe271e2b700 1 --2- 172.21.15.70:0/3619326702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c104040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0098e0 tx=0x7fe25c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462ce3f0419f01f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe270e29700 1 -- 172.21.15.70:0/3619326702 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25c00b860 con 0x7fe26c103c30 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe270e29700 1 -- 172.21.15.70:0/3619326702 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fe25c00be30 con 0x7fe26c103c30 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.615+0000 7fe270e29700 1 -- 172.21.15.70:0/3619326702 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25c010380 con 0x7fe26c103c30 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.616+0000 7fe27408f700 1 -- 172.21.15.70:0/3619326702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 msgr2=0x7fe26c104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.616+0000 7fe27408f700 1 --2- 172.21.15.70:0/3619326702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c104040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0098e0 tx=0x7fe25c009250 comp rx=0 tx=0).stop 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.616+0000 7fe27408f700 1 -- 172.21.15.70:0/3619326702 shutdown_connections 2024-03-23T08:14:05.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.616+0000 7fe27408f700 1 --2- 172.21.15.70:0/3619326702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c104040 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.616+0000 7fe27408f700 1 -- 172.21.15.70:0/3619326702 >> 172.21.15.70:0/3619326702 conn(0x7fe26c0fc350 msgr2=0x7fe26c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.616+0000 7fe27408f700 1 -- 172.21.15.70:0/3619326702 shutdown_connections 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.616+0000 7fe27408f700 1 -- 172.21.15.70:0/3619326702 wait complete. 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe27408f700 1 Processor -- start 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe27408f700 1 -- start start 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe27408f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c1113d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe27408f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c111910 con 0x7fe26c103c30 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe271e2b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c1113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe271e2b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c1113d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38680/0 (socket says 172.21.15.70:38680) 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe271e2b700 1 -- 172.21.15.70:0/979677378 learned_addr learned my addr 172.21.15.70:0/979677378 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe271e2b700 1 -- 172.21.15.70:0/979677378 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c0094f0 con 0x7fe26c103c30 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe271e2b700 1 --2- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c1113d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe25c000c00 tx=0x7fe25c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe2627fc700 1 -- 172.21.15.70:0/979677378 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25c010960 con 0x7fe26c103c30 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe2627fc700 1 -- 172.21.15.70:0/979677378 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fe25c024440 con 0x7fe26c103c30 2024-03-23T08:14:05.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26c111b10 con 0x7fe26c103c30 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe2627fc700 1 -- 172.21.15.70:0/979677378 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25c01a5f0 con 0x7fe26c103c30 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.617+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26c112680 con 0x7fe26c103c30 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.618+0000 7fe2627fc700 1 -- 172.21.15.70:0/979677378 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fe25c01a870 con 0x7fe26c103c30 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.618+0000 7fe2627fc700 1 --2- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fe258036790 0x7fe258038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.618+0000 7fe2627fc700 1 -- 172.21.15.70:0/979677378 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe25c04a790 con 0x7fe26c103c30 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.618+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26c05f5f0 con 0x7fe26c103c30 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.618+0000 7fe27162a700 1 -- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fe258036790 msgr2=0x7fe258038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2736698870 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.618+0000 7fe27162a700 1 --2- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fe258036790 0x7fe258038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.623+0000 7fe2627fc700 1 -- 172.21.15.70:0/979677378 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe25c018af0 con 0x7fe26c103c30 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.819+0000 7fe27162a700 1 -- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fe258036790 msgr2=0x7fe258038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2736698870 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.819+0000 7fe27162a700 1 --2- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fe258036790 0x7fe258038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.863+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe26c1129c0 con 0x7fe26c103c30 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.864+0000 7fe2627fc700 1 -- 172.21.15.70:0/979677378 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fe25c018af0 con 0x7fe26c103c30 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fe258036790 msgr2=0x7fe258038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 --2- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fe258036790 0x7fe258038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 msgr2=0x7fe26c1113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:05.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 --2- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c1113d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe25c000c00 tx=0x7fe25c009250 comp rx=0 tx=0).stop 2024-03-23T08:14:05.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 shutdown_connections 2024-03-23T08:14:05.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 --2- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fe258036790 0x7fe258038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:05.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 --2- 172.21.15.70:0/979677378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe26c103c30 0x7fe26c1113d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:05.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 >> 172.21.15.70:0/979677378 conn(0x7fe26c0fc350 msgr2=0x7fe26c0fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:05.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 shutdown_connections 2024-03-23T08:14:05.450 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:04.866+0000 7fe27408f700 1 -- 172.21.15.70:0/979677378 wait complete. 2024-03-23T08:14:05.450 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2024-03-23T08:14:05.450 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 5... 2024-03-23T08:14:05.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:05 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/979677378' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T08:14:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: Active manager daemon smithi070.jhsbch restarted 2024-03-23T08:14:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: Activating manager daemon smithi070.jhsbch 2024-03-23T08:14:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: osdmap e2: 0 total, 0 up, 0 in 2024-03-23T08:14:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: mgrmap e6: smithi070.jhsbch(active, starting, since 0.00320551s) 2024-03-23T08:14:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:14:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:14:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:14:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:14:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:14:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: Manager daemon smithi070.jhsbch is now available 2024-03-23T08:14:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:14:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:14:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:09 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:14:11.217 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:10 smithi070 ceph-mon[39699]: Found migration_current of "None". Setting to last migration. 2024-03-23T08:14:11.217 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:10 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:14:11.218 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:10 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/trash_purge_schedule"}]: dispatch 2024-03-23T08:14:11.218 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:10 smithi070 ceph-mon[39699]: mgrmap e7: smithi070.jhsbch(active, since 1.0052s) 2024-03-23T08:14:11.218 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:10 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.005+0000 7fd60bbb0700 1 Processor -- start 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.006+0000 7fd60bbb0700 1 -- start start 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.006+0000 7fd60bbb0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd6041018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.006+0000 7fd60bbb0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604101e30 con 0x7fd6041014e0 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.006+0000 7fd60994c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd6041018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.006+0000 7fd60994c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd6041018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38690/0 (socket says 172.21.15.70:38690) 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.006+0000 7fd60994c700 1 -- 172.21.15.70:0/2280740932 learned_addr learned my addr 172.21.15.70:0/2280740932 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:11.544 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.006+0000 7fd60994c700 1 -- 172.21.15.70:0/2280740932 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd604101f70 con 0x7fd6041014e0 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.006+0000 7fd60994c700 1 --2- 172.21.15.70:0/2280740932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd6041018f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40098e0 tx=0x7fd5f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c72544e7a325216 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.007+0000 7fd60894a700 1 -- 172.21.15.70:0/2280740932 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5f400b860 con 0x7fd6041014e0 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.007+0000 7fd60894a700 1 -- 172.21.15.70:0/2280740932 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd5f400be30 con 0x7fd6041014e0 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.007+0000 7fd60894a700 1 -- 172.21.15.70:0/2280740932 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5f4010380 con 0x7fd6041014e0 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2280740932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 msgr2=0x7fd6041018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 --2- 172.21.15.70:0/2280740932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd6041018f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40098e0 tx=0x7fd5f4009250 comp rx=0 tx=0).stop 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2280740932 shutdown_connections 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 --2- 172.21.15.70:0/2280740932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd6041018f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2280740932 >> 172.21.15.70:0/2280740932 conn(0x7fd6040fcc60 msgr2=0x7fd6040ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2280740932 shutdown_connections 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2280740932 wait complete. 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 Processor -- start 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 -- start start 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd604076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.008+0000 7fd60bbb0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604111e90 con 0x7fd6041014e0 2024-03-23T08:14:11.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd60994c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd604076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd60994c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd604076b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38698/0 (socket says 172.21.15.70:38698) 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd60994c700 1 -- 172.21.15.70:0/2649196902 learned_addr learned my addr 172.21.15.70:0/2649196902 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd60994c700 1 -- 172.21.15.70:0/2649196902 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f40094f0 con 0x7fd6041014e0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd60994c700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd604076b30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4006b00 tx=0x7fd5f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5f40109b0 con 0x7fd6041014e0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fd5f4024440 con 0x7fd6041014e0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5f401a5f0 con 0x7fd6041014e0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2649196902 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6041164a0 con 0x7fd6041014e0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.009+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2649196902 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd604116870 con 0x7fd6041014e0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.010+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd5f401a860 con 0x7fd6041014e0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.010+0000 7fd5fa7fc700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 0x7fd5f0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.010+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 --> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5f00393a0 con 0x7fd5f00367e0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.010+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd5f404a960 con 0x7fd6041014e0 2024-03-23T08:14:11.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.010+0000 7fd60914b700 1 -- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 msgr2=0x7fd5f0038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2736698870 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.010+0000 7fd60914b700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 0x7fd5f0038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.211+0000 7fd60914b700 1 -- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 msgr2=0x7fd5f0038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2736698870 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.211+0000 7fd60914b700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 0x7fd5f0038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.612+0000 7fd60914b700 1 -- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 msgr2=0x7fd5f0038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2736698870 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:06.612+0000 7fd60914b700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 0x7fd5f0038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:07.413+0000 7fd60914b700 1 -- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 msgr2=0x7fd5f0038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2736698870 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:07.413+0000 7fd60914b700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 0x7fd5f0038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:09.014+0000 7fd60914b700 1 -- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 msgr2=0x7fd5f0038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2736698870 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:09.014+0000 7fd60914b700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 0x7fd5f0038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:09.815+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7fd5f400fdc0 con 0x7fd6041014e0 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:09.815+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 msgr2=0x7fd5f0038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:09.815+0000 7fd5fa7fc700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 0x7fd5f0038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.816+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7fd5f4024990 con 0x7fd6041014e0 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.816+0000 7fd5fa7fc700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd5f0039fc0 0x7fd5f003c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.816+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5f00393a0 con 0x7fd5f0039fc0 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.817+0000 7fd60914b700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd5f0039fc0 0x7fd5f003c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:11.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.817+0000 7fd60914b700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd5f0039fc0 0x7fd5f003c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd6000039d0 tx=0x7fd600009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.817+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fd5f00393a0 con 0x7fd5f0039fc0 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.823+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2649196902 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd60404bed0 con 0x7fd5f0039fc0 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.823+0000 7fd5fa7fc700 1 -- 172.21.15.70:0/2649196902 <== mgr.14118 v2:172.21.15.70:6800/2149030412 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd60404bed0 con 0x7fd5f0039fc0 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd5f0039fc0 msgr2=0x7fd5f003c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd5f0039fc0 0x7fd5f003c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd6000039d0 tx=0x7fd600009350 comp rx=0 tx=0).stop 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 msgr2=0x7fd604076b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd604076b30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4006b00 tx=0x7fd5f4009250 comp rx=0 tx=0).stop 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2649196902 shutdown_connections 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd5f0039fc0 0x7fd5f003c3a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6041014e0 0x7fd604076b30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 --2- 172.21.15.70:0/2649196902 >> [v2:172.21.15.70:6800/2736698870,v1:172.21.15.70:6801/2736698870] conn(0x7fd5f00367e0 0x7fd5f0038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2649196902 >> 172.21.15.70:0/2649196902 conn(0x7fd6040fcc60 msgr2=0x7fd6040fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2649196902 shutdown_connections 2024-03-23T08:14:11.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:10.824+0000 7fd60bbb0700 1 -- 172.21.15.70:0/2649196902 wait complete. 2024-03-23T08:14:11.549 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 5 is available 2024-03-23T08:14:11.549 INFO:teuthology.orchestra.run.smithi070.stdout:Setting orchestrator backend to cephadm... 2024-03-23T08:14:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:11 smithi070 ceph-mon[39699]: [23/Mar/2024:08:14:10] ENGINE Bus STARTING 2024-03-23T08:14:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:11 smithi070 ceph-mon[39699]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-23T08:14:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:11 smithi070 ceph-mon[39699]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-23T08:14:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:11 smithi070 ceph-mon[39699]: [23/Mar/2024:08:14:10] ENGINE Serving on https://172.21.15.70:7150 2024-03-23T08:14:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:11 smithi070 ceph-mon[39699]: [23/Mar/2024:08:14:10] ENGINE Bus STARTED 2024-03-23T08:14:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:11 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.104+0000 7f55c6275700 1 Processor -- start 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.104+0000 7f55c6275700 1 -- start start 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.104+0000 7f55c6275700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c01018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.104+0000 7f55c6275700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0101e00 con 0x7f55c01014b0 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.105+0000 7f55bffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c01018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.105+0000 7f55bffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c01018c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59230/0 (socket says 172.21.15.70:59230) 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.105+0000 7f55bffff700 1 -- 172.21.15.70:0/1634458193 learned_addr learned my addr 172.21.15.70:0/1634458193 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.105+0000 7f55bffff700 1 -- 172.21.15.70:0/1634458193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c0101f40 con 0x7f55c01014b0 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.105+0000 7f55bffff700 1 --2- 172.21.15.70:0/1634458193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c01018c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f55a80098e0 tx=0x7f55a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d0f273bc7a8835 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:12.964 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.105+0000 7f55beffd700 1 -- 172.21.15.70:0/1634458193 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55a800b860 con 0x7f55c01014b0 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.105+0000 7f55beffd700 1 -- 172.21.15.70:0/1634458193 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f55a800be30 con 0x7f55c01014b0 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.105+0000 7f55beffd700 1 -- 172.21.15.70:0/1634458193 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55a8010380 con 0x7f55c01014b0 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.106+0000 7f55c6275700 1 -- 172.21.15.70:0/1634458193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 msgr2=0x7f55c01018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.106+0000 7f55c6275700 1 --2- 172.21.15.70:0/1634458193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c01018c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f55a80098e0 tx=0x7f55a8009250 comp rx=0 tx=0).stop 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.106+0000 7f55c6275700 1 -- 172.21.15.70:0/1634458193 shutdown_connections 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.106+0000 7f55c6275700 1 --2- 172.21.15.70:0/1634458193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c01018c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.106+0000 7f55c6275700 1 -- 172.21.15.70:0/1634458193 >> 172.21.15.70:0/1634458193 conn(0x7f55c00fcc50 msgr2=0x7f55c00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.106+0000 7f55c6275700 1 -- 172.21.15.70:0/1634458193 shutdown_connections 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.106+0000 7f55c6275700 1 -- 172.21.15.70:0/1634458193 wait complete. 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.106+0000 7f55c6275700 1 Processor -- start 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55c6275700 1 -- start start 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55c6275700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c0076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55c6275700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01140a0 con 0x7f55c01014b0 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55bffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c0076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:12.965 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55bffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c0076b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59232/0 (socket says 172.21.15.70:59232) 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55bffff700 1 -- 172.21.15.70:0/530083892 learned_addr learned my addr 172.21.15.70:0/530083892 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55bffff700 1 -- 172.21.15.70:0/530083892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80094f0 con 0x7f55c01014b0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55bffff700 1 --2- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c0076b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55a8003080 tx=0x7f55a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55bcff9700 1 -- 172.21.15.70:0/530083892 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55a80109b0 con 0x7f55c01014b0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55bcff9700 1 -- 172.21.15.70:0/530083892 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f55a801a430 con 0x7f55c01014b0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c01141e0 con 0x7f55c01014b0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55bcff9700 1 -- 172.21.15.70:0/530083892 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55a80245b0 con 0x7f55c01014b0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.107+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c0114670 con 0x7f55c01014b0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.108+0000 7f55bcff9700 1 -- 172.21.15.70:0/530083892 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f55a80247a0 con 0x7f55c01014b0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.108+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c004bed0 con 0x7f55c01014b0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.108+0000 7f55bcff9700 1 --2- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f55ac0367e0 0x7f55ac038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.108+0000 7f55bcff9700 1 -- 172.21.15.70:0/530083892 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f55a804a7c0 con 0x7f55c01014b0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.108+0000 7f55bf7fe700 1 --2- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f55ac0367e0 0x7f55ac038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:12.966 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.109+0000 7f55bf7fe700 1 --2- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f55ac0367e0 0x7f55ac038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009990 tx=0x7f55b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.113+0000 7f55bcff9700 1 -- 172.21.15.70:0/530083892 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f55a802cd80 con 0x7f55c01014b0 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.307+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f55c00fde90 con 0x7f55ac0367e0 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.315+0000 7f55bcff9700 1 -- 172.21.15.70:0/530083892 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f55c00fde90 con 0x7f55ac0367e0 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.317+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f55ac0367e0 msgr2=0x7f55ac038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.317+0000 7f55c6275700 1 --2- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f55ac0367e0 0x7f55ac038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009990 tx=0x7f55b0006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.317+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 msgr2=0x7f55c0076b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.317+0000 7f55c6275700 1 --2- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c0076b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55a8003080 tx=0x7f55a8009250 comp rx=0 tx=0).stop 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.317+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 shutdown_connections 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.317+0000 7f55c6275700 1 --2- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f55ac0367e0 0x7f55ac038c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.317+0000 7f55c6275700 1 --2- 172.21.15.70:0/530083892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c01014b0 0x7f55c0076b30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.318+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 >> 172.21.15.70:0/530083892 conn(0x7f55c00fcc50 msgr2=0x7f55c00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.318+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 shutdown_connections 2024-03-23T08:14:12.967 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:12.318+0000 7f55c6275700 1 -- 172.21.15.70:0/530083892 wait complete. 2024-03-23T08:14:12.967 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:12 smithi070 ceph-mon[39699]: mgrmap e8: smithi070.jhsbch(active, since 2s) 2024-03-23T08:14:12.968 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:12 smithi070 ceph-mon[39699]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:12.968 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:12 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:12.968 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:12 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:14:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-23T08:14:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.509+0000 7f9d9a90e700 1 Processor -- start 2024-03-23T08:14:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d9a90e700 1 -- start start 2024-03-23T08:14:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d9a90e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d941051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d9a90e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d941056f0 con 0x7f9d94076c80 2024-03-23T08:14:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d941051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d941051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59246/0 (socket says 172.21.15.70:59246) 2024-03-23T08:14:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d93fff700 1 -- 172.21.15.70:0/1838816893 learned_addr learned my addr 172.21.15.70:0/1838816893 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d93fff700 1 -- 172.21.15.70:0/1838816893 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94105830 con 0x7f9d94076c80 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d93fff700 1 --2- 172.21.15.70:0/1838816893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d941051b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098e0 tx=0x7f9d88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9e69bb0be7ac39 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d92ffd700 1 -- 172.21.15.70:0/1838816893 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d8800b860 con 0x7f9d94076c80 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d92ffd700 1 -- 172.21.15.70:0/1838816893 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f9d8800be30 con 0x7f9d94076c80 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.510+0000 7f9d92ffd700 1 -- 172.21.15.70:0/1838816893 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d88010380 con 0x7f9d94076c80 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.511+0000 7f9d9a90e700 1 -- 172.21.15.70:0/1838816893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 msgr2=0x7f9d941051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.511+0000 7f9d9a90e700 1 --2- 172.21.15.70:0/1838816893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d941051b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098e0 tx=0x7f9d88009250 comp rx=0 tx=0).stop 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.511+0000 7f9d9a90e700 1 -- 172.21.15.70:0/1838816893 shutdown_connections 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.511+0000 7f9d9a90e700 1 --2- 172.21.15.70:0/1838816893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d941051b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.511+0000 7f9d9a90e700 1 -- 172.21.15.70:0/1838816893 >> 172.21.15.70:0/1838816893 conn(0x7f9d940fcc60 msgr2=0x7f9d940ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.511+0000 7f9d9a90e700 1 -- 172.21.15.70:0/1838816893 shutdown_connections 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.511+0000 7f9d9a90e700 1 -- 172.21.15.70:0/1838816893 wait complete. 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.511+0000 7f9d9a90e700 1 Processor -- start 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d9a90e700 1 -- start start 2024-03-23T08:14:14.363 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d9a90e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d94111370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d9a90e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d941118b0 con 0x7f9d94076c80 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d94111370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d94111370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59250/0 (socket says 172.21.15.70:59250) 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d93fff700 1 -- 172.21.15.70:0/3492063915 learned_addr learned my addr 172.21.15.70:0/3492063915 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d93fff700 1 -- 172.21.15.70:0/3492063915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880094f0 con 0x7f9d94076c80 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d93fff700 1 --2- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d94111370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d88006b00 tx=0x7f9d88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d90ff9700 1 -- 172.21.15.70:0/3492063915 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d88010a80 con 0x7f9d94076c80 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d90ff9700 1 -- 172.21.15.70:0/3492063915 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f9d8801a430 con 0x7f9d94076c80 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.512+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d94111ab0 con 0x7f9d94076c80 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.513+0000 7f9d90ff9700 1 -- 172.21.15.70:0/3492063915 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d880245b0 con 0x7f9d94076c80 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.513+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d94112620 con 0x7f9d94076c80 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.513+0000 7f9d90ff9700 1 -- 172.21.15.70:0/3492063915 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f9d880247e0 con 0x7f9d94076c80 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.513+0000 7f9d90ff9700 1 --2- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f9d7c0367e0 0x7f9d7c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:14.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.514+0000 7f9d90ff9700 1 -- 172.21.15.70:0/3492063915 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9d8804aee0 con 0x7f9d94076c80 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.514+0000 7f9d937fe700 1 --2- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f9d7c0367e0 0x7f9d7c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.514+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9405f5f0 con 0x7f9d94076c80 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.514+0000 7f9d937fe700 1 --2- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f9d7c0367e0 0x7f9d7c038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009990 tx=0x7f9d84006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.518+0000 7f9d90ff9700 1 -- 172.21.15.70:0/3492063915 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9d88024b30 con 0x7f9d94076c80 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.703+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d940fde50 con 0x7f9d7c0367e0 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.704+0000 7f9d90ff9700 1 -- 172.21.15.70:0/3492063915 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f9d940fde50 con 0x7f9d7c0367e0 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f9d7c0367e0 msgr2=0x7f9d7c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 --2- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f9d7c0367e0 0x7f9d7c038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009990 tx=0x7f9d84006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 msgr2=0x7f9d94111370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 --2- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d94111370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d88006b00 tx=0x7f9d88009250 comp rx=0 tx=0).stop 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 shutdown_connections 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 --2- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f9d7c0367e0 0x7f9d7c038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 --2- 172.21.15.70:0/3492063915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d94076c80 0x7f9d94111370 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 >> 172.21.15.70:0/3492063915 conn(0x7f9d940fcc60 msgr2=0x7f9d940fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:14.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 shutdown_connections 2024-03-23T08:14:14.366 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:13.706+0000 7f9d9a90e700 1 -- 172.21.15.70:0/3492063915 wait complete. 2024-03-23T08:14:14.366 INFO:teuthology.orchestra.run.smithi070.stdout:Generating ssh key... 2024-03-23T08:14:14.630 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:14 smithi070 ceph-mon[39699]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:15.893 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.971+0000 7f0fd00b5700 1 Processor -- start 2024-03-23T08:14:15.893 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.971+0000 7f0fd00b5700 1 -- start start 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.971+0000 7f0fd00b5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8102990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.971+0000 7f0fd00b5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc8102ed0 con 0x7f0fc8102580 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.971+0000 7f0fcde51700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.972+0000 7f0fcde51700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59252/0 (socket says 172.21.15.70:59252) 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.972+0000 7f0fcde51700 1 -- 172.21.15.70:0/786925153 learned_addr learned my addr 172.21.15.70:0/786925153 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.972+0000 7f0fcde51700 1 -- 172.21.15.70:0/786925153 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8103010 con 0x7f0fc8102580 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.972+0000 7f0fcde51700 1 --2- 172.21.15.70:0/786925153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8102990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80098e0 tx=0x7f0fb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b6d1c0ec3021ad server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.972+0000 7f0fcce4f700 1 -- 172.21.15.70:0/786925153 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fb800b860 con 0x7f0fc8102580 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.972+0000 7f0fcce4f700 1 -- 172.21.15.70:0/786925153 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f0fb800be30 con 0x7f0fc8102580 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.972+0000 7f0fcce4f700 1 -- 172.21.15.70:0/786925153 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fb8010380 con 0x7f0fc8102580 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.973+0000 7f0fd00b5700 1 -- 172.21.15.70:0/786925153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 msgr2=0x7f0fc8102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.973+0000 7f0fd00b5700 1 --2- 172.21.15.70:0/786925153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8102990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80098e0 tx=0x7f0fb8009250 comp rx=0 tx=0).stop 2024-03-23T08:14:15.894 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.973+0000 7f0fd00b5700 1 -- 172.21.15.70:0/786925153 shutdown_connections 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.973+0000 7f0fd00b5700 1 --2- 172.21.15.70:0/786925153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8102990 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.973+0000 7f0fd00b5700 1 -- 172.21.15.70:0/786925153 >> 172.21.15.70:0/786925153 conn(0x7f0fc80fdd00 msgr2=0x7f0fc8100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.973+0000 7f0fd00b5700 1 -- 172.21.15.70:0/786925153 shutdown_connections 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.973+0000 7f0fd00b5700 1 -- 172.21.15.70:0/786925153 wait complete. 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.973+0000 7f0fd00b5700 1 Processor -- start 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.973+0000 7f0fd00b5700 1 -- start start 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fd00b5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8111340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fd00b5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc8111880 con 0x7f0fc8102580 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fcde51700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8111340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fcde51700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8111340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59264/0 (socket says 172.21.15.70:59264) 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fcde51700 1 -- 172.21.15.70:0/2046705754 learned_addr learned my addr 172.21.15.70:0/2046705754 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fcde51700 1 -- 172.21.15.70:0/2046705754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb80094f0 con 0x7f0fc8102580 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fcde51700 1 --2- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8111340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8000c00 tx=0x7f0fb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fbe7fc700 1 -- 172.21.15.70:0/2046705754 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fb8010a80 con 0x7f0fc8102580 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc8111a80 con 0x7f0fc8102580 2024-03-23T08:14:15.895 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fbe7fc700 1 -- 172.21.15.70:0/2046705754 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f0fb801a430 con 0x7f0fc8102580 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fbe7fc700 1 -- 172.21.15.70:0/2046705754 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fb80245b0 con 0x7f0fc8102580 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.974+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc81125f0 con 0x7f0fc8102580 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.975+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc805f5f0 con 0x7f0fc8102580 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.976+0000 7f0fbe7fc700 1 -- 172.21.15.70:0/2046705754 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f0fb801a5a0 con 0x7f0fc8102580 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.976+0000 7f0fbe7fc700 1 --2- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0fb4036790 0x7f0fb4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.976+0000 7f0fbe7fc700 1 -- 172.21.15.70:0/2046705754 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0fb804b0b0 con 0x7f0fc8102580 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.976+0000 7f0fcd650700 1 --2- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0fb4036790 0x7f0fb4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.976+0000 7f0fcd650700 1 --2- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0fb4036790 0x7f0fb4038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009990 tx=0x7f0fc4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:14.980+0000 7f0fbe7fc700 1 -- 172.21.15.70:0/2046705754 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0fb80237a0 con 0x7f0fc8102580 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.164+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fc80fef40 con 0x7f0fb4036790 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.320+0000 7f0fbe7fc700 1 -- 172.21.15.70:0/2046705754 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0fc80fef40 con 0x7f0fb4036790 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0fb4036790 msgr2=0x7f0fb4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 --2- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0fb4036790 0x7f0fb4038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009990 tx=0x7f0fc4006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 msgr2=0x7f0fc8111340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:15.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 --2- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8111340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8000c00 tx=0x7f0fb8009250 comp rx=0 tx=0).stop 2024-03-23T08:14:15.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 shutdown_connections 2024-03-23T08:14:15.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 --2- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0fb4036790 0x7f0fb4038c40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:15.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 --2- 172.21.15.70:0/2046705754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fc8102580 0x7f0fc8111340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:15.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 >> 172.21.15.70:0/2046705754 conn(0x7f0fc80fdd00 msgr2=0x7f0fc80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:15.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 shutdown_connections 2024-03-23T08:14:15.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:15.322+0000 7f0fd00b5700 1 -- 172.21.15.70:0/2046705754 wait complete. 2024-03-23T08:14:16.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:16 smithi070 ceph-mon[39699]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:16.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:16 smithi070 ceph-mon[39699]: Generating ssh key... 2024-03-23T08:14:16.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:16 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:16.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:16 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:17.316 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ5BcJEIdUPCuMj1sWw+F0JrjnuDJpkaU3bRcgrOFpSpziJCJCh1ZKYfFil6zXvCwZNVGBE0VlK5zQIgYP7gomKbgD7OpokajMdkh5J6o7CWSZf4Ehk/d6wDFv8Q4fdmN83NJdJ07xDEHaT+yKmB/Pvs8hUBSDedfBBUpvwpnK/Kd5KsP0sBn3FRTZPal3z/5IjQB93iaZb2vWjSh3RHuN6J5RvelP2VCgnlRUZ8o55tXc2VP9O4zNNmxt9PFPzYp2AZXNfjwd8v65sn42a3gWyygZrLdESi/RCBT3RcSYQDKQmReEcrP+2kA7xU5vF6rC01qApgz7Te/WV/y5uPkflHo0SzUmfiO2aiMqOdwl/5fCZZjsKfDw0zRMkhOwTADjtrqtfj0UDjChKSh135OzYEKl6QHcrWD+RopmqlsE6rOPScgEUCD+BPT5zGPWdSOXRvn+lPGcacQMgiaOJkIvZO7YwaDOy5LmBKuCBuvEWnWxsYl14fRI3QqdWecbmCk= ceph-2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:14:17.316 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.379+0000 7f44680c1700 1 Processor -- start 2024-03-23T08:14:17.316 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.379+0000 7f44680c1700 1 -- start start 2024-03-23T08:14:17.316 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.379+0000 7f44680c1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f44601051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:17.316 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.379+0000 7f44680c1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4460105710 con 0x7f4460076c60 2024-03-23T08:14:17.316 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.380+0000 7f4465e5d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f44601051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:17.316 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.380+0000 7f4465e5d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f44601051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59274/0 (socket says 172.21.15.70:59274) 2024-03-23T08:14:17.316 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.380+0000 7f4465e5d700 1 -- 172.21.15.70:0/4195568633 learned_addr learned my addr 172.21.15.70:0/4195568633 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.380+0000 7f4465e5d700 1 -- 172.21.15.70:0/4195568633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4460105850 con 0x7f4460076c60 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.380+0000 7f4465e5d700 1 --2- 172.21.15.70:0/4195568633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f44601051d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f44540098e0 tx=0x7f4454009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b93ddcf435b6ce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.380+0000 7f4464e5b700 1 -- 172.21.15.70:0/4195568633 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f445400b860 con 0x7f4460076c60 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.380+0000 7f4464e5b700 1 -- 172.21.15.70:0/4195568633 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f445400be30 con 0x7f4460076c60 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.380+0000 7f4464e5b700 1 -- 172.21.15.70:0/4195568633 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4454010380 con 0x7f4460076c60 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.381+0000 7f44680c1700 1 -- 172.21.15.70:0/4195568633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 msgr2=0x7f44601051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.381+0000 7f44680c1700 1 --2- 172.21.15.70:0/4195568633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f44601051d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f44540098e0 tx=0x7f4454009250 comp rx=0 tx=0).stop 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.381+0000 7f44680c1700 1 -- 172.21.15.70:0/4195568633 shutdown_connections 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.381+0000 7f44680c1700 1 --2- 172.21.15.70:0/4195568633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f44601051d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.381+0000 7f44680c1700 1 -- 172.21.15.70:0/4195568633 >> 172.21.15.70:0/4195568633 conn(0x7f44600fcc60 msgr2=0x7f44600ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.381+0000 7f44680c1700 1 -- 172.21.15.70:0/4195568633 shutdown_connections 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.381+0000 7f44680c1700 1 -- 172.21.15.70:0/4195568633 wait complete. 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.381+0000 7f44680c1700 1 Processor -- start 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.381+0000 7f44680c1700 1 -- start start 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f44680c1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f4460102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:17.317 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f44680c1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4460103060 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f4465e5d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f4460102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f4465e5d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f4460102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59290/0 (socket says 172.21.15.70:59290) 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f4465e5d700 1 -- 172.21.15.70:0/2115384595 learned_addr learned my addr 172.21.15.70:0/2115384595 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f4465e5d700 1 -- 172.21.15.70:0/2115384595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540094f0 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f4465e5d700 1 --2- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f4460102b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4454006b00 tx=0x7f4454009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f44527fc700 1 -- 172.21.15.70:0/2115384595 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4454010a80 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f44527fc700 1 -- 172.21.15.70:0/2115384595 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f445401a430 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44601010d0 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f44527fc700 1 -- 172.21.15.70:0/2115384595 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44540245b0 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.382+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44601015c0 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.383+0000 7f44527fc700 1 -- 172.21.15.70:0/2115384595 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f44540247e0 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.383+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446004bed0 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.383+0000 7f44527fc700 1 --2- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f444c0367e0 0x7f444c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.383+0000 7f44527fc700 1 -- 172.21.15.70:0/2115384595 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f445404aef0 con 0x7f4460076c60 2024-03-23T08:14:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.383+0000 7f446565c700 1 --2- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f444c0367e0 0x7f444c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.384+0000 7f446565c700 1 --2- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f444c0367e0 0x7f444c038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f445c009990 tx=0x7f445c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.388+0000 7f44527fc700 1 -- 172.21.15.70:0/2115384595 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f445402c570 con 0x7f4460076c60 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.572+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4460101cd0 con 0x7f444c0367e0 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.573+0000 7f44527fc700 1 -- 172.21.15.70:0/2115384595 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f4460101cd0 con 0x7f444c0367e0 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.574+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f444c0367e0 msgr2=0x7f444c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.574+0000 7f44680c1700 1 --2- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f444c0367e0 0x7f444c038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f445c009990 tx=0x7f445c006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.574+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 msgr2=0x7f4460102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.574+0000 7f44680c1700 1 --2- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f4460102b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4454006b00 tx=0x7f4454009250 comp rx=0 tx=0).stop 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.575+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 shutdown_connections 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.575+0000 7f44680c1700 1 --2- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f444c0367e0 0x7f444c038c90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.575+0000 7f44680c1700 1 --2- 172.21.15.70:0/2115384595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4460076c60 0x7f4460102b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.575+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 >> 172.21.15.70:0/2115384595 conn(0x7f44600fcc60 msgr2=0x7f44600fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.575+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 shutdown_connections 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:16.575+0000 7f44680c1700 1 -- 172.21.15.70:0/2115384595 wait complete. 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-23T08:14:17.319 INFO:teuthology.orchestra.run.smithi070.stdout:Adding key to root@localhost authorized_keys... 2024-03-23T08:14:17.320 INFO:teuthology.orchestra.run.smithi070.stdout:Adding host smithi070... 2024-03-23T08:14:17.574 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:17 smithi070 ceph-mon[39699]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:18.670 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:18 smithi070 ceph-mon[39699]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi070", "addr": "172.21.15.70", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:19.762 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:19 smithi070 ceph-mon[39699]: Deploying cephadm binary to smithi070 2024-03-23T08:14:20.374 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Added host 'smithi070' with addr '172.21.15.70' 2024-03-23T08:14:20.374 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.855+0000 7f992b244700 1 Processor -- start 2024-03-23T08:14:20.374 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.855+0000 7f992b244700 1 -- start start 2024-03-23T08:14:20.374 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.855+0000 7f992b244700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924107600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.855+0000 7f992b244700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9924102ff0 con 0x7f9924076cd0 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.856+0000 7f9928fe0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924107600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.856+0000 7f9928fe0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924107600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59296/0 (socket says 172.21.15.70:59296) 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.856+0000 7f9928fe0700 1 -- 172.21.15.70:0/3036891624 learned_addr learned my addr 172.21.15.70:0/3036891624 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.856+0000 7f9928fe0700 1 -- 172.21.15.70:0/3036891624 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924107b40 con 0x7f9924076cd0 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.856+0000 7f9928fe0700 1 --2- 172.21.15.70:0/3036891624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924107600 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99180098e0 tx=0x7f9918009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4221d094eb8392db server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.856+0000 7f99237fe700 1 -- 172.21.15.70:0/3036891624 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f991800b860 con 0x7f9924076cd0 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.856+0000 7f99237fe700 1 -- 172.21.15.70:0/3036891624 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f991800be30 con 0x7f9924076cd0 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.856+0000 7f99237fe700 1 -- 172.21.15.70:0/3036891624 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9918010380 con 0x7f9924076cd0 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.857+0000 7f992b244700 1 -- 172.21.15.70:0/3036891624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 msgr2=0x7f9924107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.857+0000 7f992b244700 1 --2- 172.21.15.70:0/3036891624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924107600 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99180098e0 tx=0x7f9918009250 comp rx=0 tx=0).stop 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.857+0000 7f992b244700 1 -- 172.21.15.70:0/3036891624 shutdown_connections 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.857+0000 7f992b244700 1 --2- 172.21.15.70:0/3036891624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924107600 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.857+0000 7f992b244700 1 -- 172.21.15.70:0/3036891624 >> 172.21.15.70:0/3036891624 conn(0x7f99240fcc50 msgr2=0x7f99240ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.857+0000 7f992b244700 1 -- 172.21.15.70:0/3036891624 shutdown_connections 2024-03-23T08:14:20.375 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.857+0000 7f992b244700 1 -- 172.21.15.70:0/3036891624 wait complete. 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.857+0000 7f992b244700 1 Processor -- start 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.858+0000 7f992b244700 1 -- start start 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.858+0000 7f992b244700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.858+0000 7f992b244700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9924113ab0 con 0x7f9924076cd0 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.858+0000 7f9928fe0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.858+0000 7f9928fe0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924113570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59310/0 (socket says 172.21.15.70:59310) 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.858+0000 7f9928fe0700 1 -- 172.21.15.70:0/1625863602 learned_addr learned my addr 172.21.15.70:0/1625863602 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.858+0000 7f9928fe0700 1 -- 172.21.15.70:0/1625863602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180094f0 con 0x7f9924076cd0 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.858+0000 7f9928fe0700 1 --2- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924113570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9918006d40 tx=0x7f9918009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.858+0000 7f99217fa700 1 -- 172.21.15.70:0/1625863602 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99180109b0 con 0x7f9924076cd0 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.859+0000 7f99217fa700 1 -- 172.21.15.70:0/1625863602 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f9918024440 con 0x7f9924076cd0 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.859+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9924113cb0 con 0x7f9924076cd0 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.859+0000 7f99217fa700 1 -- 172.21.15.70:0/1625863602 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f991801a5f0 con 0x7f9924076cd0 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.859+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9924114820 con 0x7f9924076cd0 2024-03-23T08:14:20.376 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.859+0000 7f99217fa700 1 -- 172.21.15.70:0/1625863602 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f991801a820 con 0x7f9924076cd0 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.859+0000 7f99217fa700 1 --2- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f99140367e0 0x7f9914038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.859+0000 7f99217fa700 1 -- 172.21.15.70:0/1625863602 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f991804b630 con 0x7f9924076cd0 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.860+0000 7f9923fff700 1 --2- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f99140367e0 0x7f9914038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.860+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992405f5f0 con 0x7f9924076cd0 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.860+0000 7f9923fff700 1 --2- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f99140367e0 0x7f9914038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9910009990 tx=0x7f9910006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:17.865+0000 7f99217fa700 1 -- 172.21.15.70:0/1625863602 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f991801aad0 con 0x7f9924076cd0 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:18.052+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi070", "addr": "172.21.15.70", "target": ["mon-mgr", ""]}) v1 -- 0x7f99240fde70 con 0x7f99140367e0 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.732+0000 7f99217fa700 1 -- 172.21.15.70:0/1625863602 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f99240fde70 con 0x7f99140367e0 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.734+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f99140367e0 msgr2=0x7f9914038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.734+0000 7f992b244700 1 --2- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f99140367e0 0x7f9914038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9910009990 tx=0x7f9910006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.734+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 msgr2=0x7f9924113570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.734+0000 7f992b244700 1 --2- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924113570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9918006d40 tx=0x7f9918009250 comp rx=0 tx=0).stop 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.735+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 shutdown_connections 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.735+0000 7f992b244700 1 --2- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f99140367e0 0x7f9914038c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.735+0000 7f992b244700 1 --2- 172.21.15.70:0/1625863602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9924076cd0 0x7f9924113570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.735+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 >> 172.21.15.70:0/1625863602 conn(0x7f99240fcc50 msgr2=0x7f99240fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:20.378 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.735+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 shutdown_connections 2024-03-23T08:14:20.378 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:19.735+0000 7f992b244700 1 -- 172.21.15.70:0/1625863602 wait complete. 2024-03-23T08:14:20.378 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying mon service with default placement... 2024-03-23T08:14:20.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:20 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:20.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:20 smithi070 ceph-mon[39699]: Added host smithi070 2024-03-23T08:14:20.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:20 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:14:22.193 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-23T08:14:22.193 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.897+0000 7f8e5b625700 1 Processor -- start 2024-03-23T08:14:22.193 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.897+0000 7f8e5b625700 1 -- start start 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e5b625700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54102990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e5b625700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54102ed0 con 0x7f8e54102580 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e593c1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e593c1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40970/0 (socket says 172.21.15.70:40970) 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e593c1700 1 -- 172.21.15.70:0/4173758221 learned_addr learned my addr 172.21.15.70:0/4173758221 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e593c1700 1 -- 172.21.15.70:0/4173758221 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e54103010 con 0x7f8e54102580 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e593c1700 1 --2- 172.21.15.70:0/4173758221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54102990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e440098e0 tx=0x7f8e44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da60af6699b9e68 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e4bfff700 1 -- 172.21.15.70:0/4173758221 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e4400b860 con 0x7f8e54102580 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e4bfff700 1 -- 172.21.15.70:0/4173758221 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f8e4400be30 con 0x7f8e54102580 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.898+0000 7f8e4bfff700 1 -- 172.21.15.70:0/4173758221 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e44010380 con 0x7f8e54102580 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.899+0000 7f8e5b625700 1 -- 172.21.15.70:0/4173758221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 msgr2=0x7f8e54102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.899+0000 7f8e5b625700 1 --2- 172.21.15.70:0/4173758221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54102990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e440098e0 tx=0x7f8e44009250 comp rx=0 tx=0).stop 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.899+0000 7f8e5b625700 1 -- 172.21.15.70:0/4173758221 shutdown_connections 2024-03-23T08:14:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.899+0000 7f8e5b625700 1 --2- 172.21.15.70:0/4173758221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54102990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.899+0000 7f8e5b625700 1 -- 172.21.15.70:0/4173758221 >> 172.21.15.70:0/4173758221 conn(0x7f8e540fdd00 msgr2=0x7f8e54100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.899+0000 7f8e5b625700 1 -- 172.21.15.70:0/4173758221 shutdown_connections 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.899+0000 7f8e5b625700 1 -- 172.21.15.70:0/4173758221 wait complete. 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.899+0000 7f8e5b625700 1 Processor -- start 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e5b625700 1 -- start start 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e5b625700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54078c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e5b625700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54079190 con 0x7f8e54102580 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e593c1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54078c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e593c1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54078c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40974/0 (socket says 172.21.15.70:40974) 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e593c1700 1 -- 172.21.15.70:0/591867611 learned_addr learned my addr 172.21.15.70:0/591867611 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e593c1700 1 -- 172.21.15.70:0/591867611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440094f0 con 0x7f8e54102580 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e593c1700 1 --2- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54078c50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006b00 tx=0x7f8e44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e49ffb700 1 -- 172.21.15.70:0/591867611 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e440109b0 con 0x7f8e54102580 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e49ffb700 1 -- 172.21.15.70:0/591867611 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f8e44024440 con 0x7f8e54102580 2024-03-23T08:14:22.195 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e54077200 con 0x7f8e54102580 2024-03-23T08:14:22.196 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.900+0000 7f8e49ffb700 1 -- 172.21.15.70:0/591867611 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e4401a5f0 con 0x7f8e54102580 2024-03-23T08:14:22.196 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.901+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e540776f0 con 0x7f8e54102580 2024-03-23T08:14:22.196 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.901+0000 7f8e49ffb700 1 -- 172.21.15.70:0/591867611 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f8e4401a8b0 con 0x7f8e54102580 2024-03-23T08:14:22.196 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.902+0000 7f8e49ffb700 1 --2- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f8e40036790 0x7f8e40038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:22.196 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.901+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5404bed0 con 0x7f8e54102580 2024-03-23T08:14:22.196 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.902+0000 7f8e49ffb700 1 -- 172.21.15.70:0/591867611 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8e4404aa70 con 0x7f8e54102580 2024-03-23T08:14:22.196 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.902+0000 7f8e58bc0700 1 --2- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f8e40036790 0x7f8e40038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:22.196 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.902+0000 7f8e58bc0700 1 --2- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f8e40036790 0x7f8e40038c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009990 tx=0x7f8e50006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:22.196 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:20.906+0000 7f8e49ffb700 1 -- 172.21.15.70:0/591867611 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8e44022340 con 0x7f8e54102580 2024-03-23T08:14:22.196 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:22 smithi070 ceph-mon[39699]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:22.196 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:22 smithi070 ceph-mon[39699]: Saving service mon spec with placement count:5 2024-03-23T08:14:22.196 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:22 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.103+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e54077e00 con 0x7f8e40036790 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.107+0000 7f8e49ffb700 1 -- 172.21.15.70:0/591867611 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8e54077e00 con 0x7f8e40036790 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f8e40036790 msgr2=0x7f8e40038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 --2- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f8e40036790 0x7f8e40038c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009990 tx=0x7f8e50006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 msgr2=0x7f8e54078c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 --2- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54078c50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006b00 tx=0x7f8e44009250 comp rx=0 tx=0).stop 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 shutdown_connections 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 --2- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f8e40036790 0x7f8e40038c40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 --2- 172.21.15.70:0/591867611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e54102580 0x7f8e54078c50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 >> 172.21.15.70:0/591867611 conn(0x7f8e540fdd00 msgr2=0x7f8e540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 shutdown_connections 2024-03-23T08:14:22.198 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:21.110+0000 7f8e5b625700 1 -- 172.21.15.70:0/591867611 wait complete. 2024-03-23T08:14:22.199 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying mgr service with default placement... 2024-03-23T08:14:23.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-23T08:14:23.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.865+0000 7f4c433b3700 1 Processor -- start 2024-03-23T08:14:23.970 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.865+0000 7f4c433b3700 1 -- start start 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.865+0000 7f4c433b3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c1051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.865+0000 7f4c433b3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c1056f0 con 0x7f4c3c076c80 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.866+0000 7f4c4114f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c1051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.866+0000 7f4c4114f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c1051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40976/0 (socket says 172.21.15.70:40976) 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.866+0000 7f4c4114f700 1 -- 172.21.15.70:0/1007656483 learned_addr learned my addr 172.21.15.70:0/1007656483 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.866+0000 7f4c4114f700 1 -- 172.21.15.70:0/1007656483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c105830 con 0x7f4c3c076c80 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.866+0000 7f4c4114f700 1 --2- 172.21.15.70:0/1007656483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c1051b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0098e0 tx=0x7f4c2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419c89501184eb06 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.866+0000 7f4c33fff700 1 -- 172.21.15.70:0/1007656483 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c2c00b860 con 0x7f4c3c076c80 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.866+0000 7f4c33fff700 1 -- 172.21.15.70:0/1007656483 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f4c2c00be30 con 0x7f4c3c076c80 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.866+0000 7f4c33fff700 1 -- 172.21.15.70:0/1007656483 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c2c010380 con 0x7f4c3c076c80 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 -- 172.21.15.70:0/1007656483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 msgr2=0x7f4c3c1051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 --2- 172.21.15.70:0/1007656483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c1051b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0098e0 tx=0x7f4c2c009250 comp rx=0 tx=0).stop 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 -- 172.21.15.70:0/1007656483 shutdown_connections 2024-03-23T08:14:23.971 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 --2- 172.21.15.70:0/1007656483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c1051b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 -- 172.21.15.70:0/1007656483 >> 172.21.15.70:0/1007656483 conn(0x7f4c3c0fcc60 msgr2=0x7f4c3c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 -- 172.21.15.70:0/1007656483 shutdown_connections 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 -- 172.21.15.70:0/1007656483 wait complete. 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 Processor -- start 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 -- start start 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.867+0000 7f4c433b3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c103040 con 0x7f4c3c076c80 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c4114f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c4114f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c102b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40990/0 (socket says 172.21.15.70:40990) 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c4114f700 1 -- 172.21.15.70:0/4172723512 learned_addr learned my addr 172.21.15.70:0/4172723512 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c4114f700 1 -- 172.21.15.70:0/4172723512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0094f0 con 0x7f4c3c076c80 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c4114f700 1 --2- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c102b00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c006b00 tx=0x7f4c2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c31ffb700 1 -- 172.21.15.70:0/4172723512 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c2c010a80 con 0x7f4c3c076c80 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c31ffb700 1 -- 172.21.15.70:0/4172723512 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f4c2c01a430 con 0x7f4c3c076c80 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c1010b0 con 0x7f4c3c076c80 2024-03-23T08:14:23.972 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c31ffb700 1 -- 172.21.15.70:0/4172723512 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c2c0245b0 con 0x7f4c3c076c80 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.868+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c1015a0 con 0x7f4c3c076c80 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.869+0000 7f4c31ffb700 1 -- 172.21.15.70:0/4172723512 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f4c2c0247e0 con 0x7f4c3c076c80 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.869+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c04bed0 con 0x7f4c3c076c80 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.869+0000 7f4c31ffb700 1 --2- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4c280367e0 0x7f4c28038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.869+0000 7f4c31ffb700 1 -- 172.21.15.70:0/4172723512 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4c2c04acc0 con 0x7f4c3c076c80 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.869+0000 7f4c4094e700 1 --2- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4c280367e0 0x7f4c28038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.870+0000 7f4c4094e700 1 --2- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4c280367e0 0x7f4c28038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009990 tx=0x7f4c38006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:22.874+0000 7f4c31ffb700 1 -- 172.21.15.70:0/4172723512 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4c2c02e440 con 0x7f4c3c076c80 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.075+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c3c04bed0 con 0x7f4c280367e0 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.079+0000 7f4c31ffb700 1 -- 172.21.15.70:0/4172723512 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4c3c04bed0 con 0x7f4c280367e0 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.081+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4c280367e0 msgr2=0x7f4c28038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.081+0000 7f4c433b3700 1 --2- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4c280367e0 0x7f4c28038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009990 tx=0x7f4c38006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.081+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 msgr2=0x7f4c3c102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:23.973 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.081+0000 7f4c433b3700 1 --2- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c102b00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c006b00 tx=0x7f4c2c009250 comp rx=0 tx=0).stop 2024-03-23T08:14:23.974 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.081+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 shutdown_connections 2024-03-23T08:14:23.974 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.081+0000 7f4c433b3700 1 --2- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4c280367e0 0x7f4c28038c90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:23.974 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.081+0000 7f4c433b3700 1 --2- 172.21.15.70:0/4172723512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c3c076c80 0x7f4c3c102b00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:23.974 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.081+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 >> 172.21.15.70:0/4172723512 conn(0x7f4c3c0fcc60 msgr2=0x7f4c3c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:23.974 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.082+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 shutdown_connections 2024-03-23T08:14:23.974 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:23.082+0000 7f4c433b3700 1 -- 172.21.15.70:0/4172723512 wait complete. 2024-03-23T08:14:23.974 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying crash service with default placement... 2024-03-23T08:14:24.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:24 smithi070 ceph-mon[39699]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:24 smithi070 ceph-mon[39699]: Saving service mgr spec with placement count:2 2024-03-23T08:14:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:24 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:24 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:25.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-23T08:14:25.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.525+0000 7fbfa1beb700 1 Processor -- start 2024-03-23T08:14:25.385 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.525+0000 7fbfa1beb700 1 -- start start 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.525+0000 7fbfa1beb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c1075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.525+0000 7fbfa1beb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c102fe0 con 0x7fbf9c076c40 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.525+0000 7fbf9b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c1075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.525+0000 7fbf9b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c1075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40996/0 (socket says 172.21.15.70:40996) 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.525+0000 7fbf9b7fe700 1 -- 172.21.15.70:0/2491681752 learned_addr learned my addr 172.21.15.70:0/2491681752 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.525+0000 7fbf9b7fe700 1 -- 172.21.15.70:0/2491681752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c107b30 con 0x7fbf9c076c40 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.526+0000 7fbf9b7fe700 1 --2- 172.21.15.70:0/2491681752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c1075f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0098e0 tx=0x7fbf8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0779f18b90d9e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.526+0000 7fbf9a7fc700 1 -- 172.21.15.70:0/2491681752 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf8c00b860 con 0x7fbf9c076c40 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.526+0000 7fbf9a7fc700 1 -- 172.21.15.70:0/2491681752 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf8c00be70 con 0x7fbf9c076c40 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.526+0000 7fbf9a7fc700 1 -- 172.21.15.70:0/2491681752 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf8c010380 con 0x7fbf9c076c40 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.527+0000 7fbfa1beb700 1 -- 172.21.15.70:0/2491681752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 msgr2=0x7fbf9c1075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.527+0000 7fbfa1beb700 1 --2- 172.21.15.70:0/2491681752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c1075f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0098e0 tx=0x7fbf8c009250 comp rx=0 tx=0).stop 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.527+0000 7fbfa1beb700 1 -- 172.21.15.70:0/2491681752 shutdown_connections 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.527+0000 7fbfa1beb700 1 --2- 172.21.15.70:0/2491681752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c1075f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.527+0000 7fbfa1beb700 1 -- 172.21.15.70:0/2491681752 >> 172.21.15.70:0/2491681752 conn(0x7fbf9c0fcc60 msgr2=0x7fbf9c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.527+0000 7fbfa1beb700 1 -- 172.21.15.70:0/2491681752 shutdown_connections 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.527+0000 7fbfa1beb700 1 -- 172.21.15.70:0/2491681752 wait complete. 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.527+0000 7fbfa1beb700 1 Processor -- start 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbfa1beb700 1 -- start start 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbfa1beb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c113560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbfa1beb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c113aa0 con 0x7fbf9c076c40 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbf9b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c113560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbf9b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c113560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41012/0 (socket says 172.21.15.70:41012) 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbf9b7fe700 1 -- 172.21.15.70:0/1108604575 learned_addr learned my addr 172.21.15.70:0/1108604575 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbf9b7fe700 1 -- 172.21.15.70:0/1108604575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c0094f0 con 0x7fbf9c076c40 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbf9b7fe700 1 --2- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c113560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c006d40 tx=0x7fbf8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbf83fff700 1 -- 172.21.15.70:0/1108604575 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf8c010960 con 0x7fbf9c076c40 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbf83fff700 1 -- 172.21.15.70:0/1108604575 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf8c01a430 con 0x7fbf9c076c40 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.528+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c113d00 con 0x7fbf9c076c40 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.529+0000 7fbf83fff700 1 -- 172.21.15.70:0/1108604575 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf8c0245b0 con 0x7fbf9c076c40 2024-03-23T08:14:25.387 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.529+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c114810 con 0x7fbf9c076c40 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.530+0000 7fbf83fff700 1 -- 172.21.15.70:0/1108604575 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbf8c02d3f0 con 0x7fbf9c076c40 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.530+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c05f5f0 con 0x7fbf9c076c40 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.530+0000 7fbf83fff700 1 --2- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fbf840367e0 0x7fbf84038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.530+0000 7fbf83fff700 1 -- 172.21.15.70:0/1108604575 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbf8c04a240 con 0x7fbf9c076c40 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.530+0000 7fbf9affd700 1 --2- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fbf840367e0 0x7fbf84038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.530+0000 7fbf9affd700 1 --2- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fbf840367e0 0x7fbf84038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009990 tx=0x7fbf90006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.535+0000 7fbf83fff700 1 -- 172.21.15.70:0/1108604575 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbf8c02b8e0 con 0x7fbf9c076c40 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.759+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf9c0fde50 con 0x7fbf840367e0 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.763+0000 7fbf83fff700 1 -- 172.21.15.70:0/1108604575 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fbf9c0fde50 con 0x7fbf840367e0 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.765+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fbf840367e0 msgr2=0x7fbf84038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.765+0000 7fbfa1beb700 1 --2- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fbf840367e0 0x7fbf84038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009990 tx=0x7fbf90006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.765+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 msgr2=0x7fbf9c113560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.765+0000 7fbfa1beb700 1 --2- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c113560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c006d40 tx=0x7fbf8c009250 comp rx=0 tx=0).stop 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.766+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 shutdown_connections 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.766+0000 7fbfa1beb700 1 --2- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fbf840367e0 0x7fbf84038c90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.766+0000 7fbfa1beb700 1 --2- 172.21.15.70:0/1108604575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c076c40 0x7fbf9c113560 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:25.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.766+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 >> 172.21.15.70:0/1108604575 conn(0x7fbf9c0fcc60 msgr2=0x7fbf9c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:25.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.766+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 shutdown_connections 2024-03-23T08:14:25.389 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:24.766+0000 7fbfa1beb700 1 -- 172.21.15.70:0/1108604575 wait complete. 2024-03-23T08:14:25.389 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying prometheus service with default placement... 2024-03-23T08:14:25.389 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:25 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:25.389 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:25 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:26.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:26 smithi070 ceph-mon[39699]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:26.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:26 smithi070 ceph-mon[39699]: Saving service crash spec with placement * 2024-03-23T08:14:26.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:26 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.990+0000 7fd38cf15700 1 Processor -- start 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.991+0000 7fd38cf15700 1 -- start start 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.991+0000 7fd38cf15700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.991+0000 7fd38cf15700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd388104530 con 0x7fd388103be0 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.991+0000 7fd38659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.991+0000 7fd38659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388103ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41024/0 (socket says 172.21.15.70:41024) 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.991+0000 7fd38659c700 1 -- 172.21.15.70:0/2521437923 learned_addr learned my addr 172.21.15.70:0/2521437923 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.991+0000 7fd38659c700 1 -- 172.21.15.70:0/2521437923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd388104670 con 0x7fd388103be0 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.991+0000 7fd38659c700 1 --2- 172.21.15.70:0/2521437923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388103ff0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0098e0 tx=0x7fd37c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92b4eda0fe21a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38559a700 1 -- 172.21.15.70:0/2521437923 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd37c00b860 con 0x7fd388103be0 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38559a700 1 -- 172.21.15.70:0/2521437923 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd37c00be70 con 0x7fd388103be0 2024-03-23T08:14:26.794 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38559a700 1 -- 172.21.15.70:0/2521437923 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd37c010380 con 0x7fd388103be0 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38cf15700 1 -- 172.21.15.70:0/2521437923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 msgr2=0x7fd388103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38cf15700 1 --2- 172.21.15.70:0/2521437923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388103ff0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0098e0 tx=0x7fd37c009250 comp rx=0 tx=0).stop 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38cf15700 1 -- 172.21.15.70:0/2521437923 shutdown_connections 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38cf15700 1 --2- 172.21.15.70:0/2521437923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388103ff0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38cf15700 1 -- 172.21.15.70:0/2521437923 >> 172.21.15.70:0/2521437923 conn(0x7fd3880fc400 msgr2=0x7fd3880fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38cf15700 1 -- 172.21.15.70:0/2521437923 shutdown_connections 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.992+0000 7fd38cf15700 1 -- 172.21.15.70:0/2521437923 wait complete. 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd38cf15700 1 Processor -- start 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd38cf15700 1 -- start start 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd38cf15700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388111360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd38cf15700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881118a0 con 0x7fd388103be0 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd38659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388111360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd38659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388111360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41038/0 (socket says 172.21.15.70:41038) 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd38659c700 1 -- 172.21.15.70:0/3489050777 learned_addr learned my addr 172.21.15.70:0/3489050777 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:26.795 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd38659c700 1 -- 172.21.15.70:0/3489050777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0094f0 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd38659c700 1 --2- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388111360 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd37c011ca0 tx=0x7fd37c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.993+0000 7fd376ffd700 1 -- 172.21.15.70:0/3489050777 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd37c010850 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.994+0000 7fd376ffd700 1 -- 172.21.15.70:0/3489050777 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd37c010e60 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.994+0000 7fd376ffd700 1 -- 172.21.15.70:0/3489050777 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd37c01ae20 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.994+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd388111aa0 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.994+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd388112610 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.995+0000 7fd376ffd700 1 -- 172.21.15.70:0/3489050777 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd37c0109b0 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.995+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38805f5f0 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.995+0000 7fd376ffd700 1 --2- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd3700367e0 0x7fd370038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.995+0000 7fd376ffd700 1 -- 172.21.15.70:0/3489050777 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd37c04a3a0 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.995+0000 7fd385d9b700 1 --2- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd3700367e0 0x7fd370038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.995+0000 7fd385d9b700 1 --2- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd3700367e0 0x7fd370038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd378009990 tx=0x7fd378006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:25.999+0000 7fd376ffd700 1 -- 172.21.15.70:0/3489050777 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd37c01a6a0 con 0x7fd388103be0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.196+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3880fd580 con 0x7fd3700367e0 2024-03-23T08:14:26.796 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.200+0000 7fd376ffd700 1 -- 172.21.15.70:0/3489050777 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fd3880fd580 con 0x7fd3700367e0 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd3700367e0 msgr2=0x7fd370038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 --2- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd3700367e0 0x7fd370038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd378009990 tx=0x7fd378006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 msgr2=0x7fd388111360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 --2- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388111360 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd37c011ca0 tx=0x7fd37c009250 comp rx=0 tx=0).stop 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 shutdown_connections 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 --2- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd3700367e0 0x7fd370038c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 --2- 172.21.15.70:0/3489050777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd388103be0 0x7fd388111360 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 >> 172.21.15.70:0/3489050777 conn(0x7fd3880fc400 msgr2=0x7fd3880fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 shutdown_connections 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:26.202+0000 7fd38cf15700 1 -- 172.21.15.70:0/3489050777 wait complete. 2024-03-23T08:14:26.797 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying grafana service with default placement... 2024-03-23T08:14:27.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:27 smithi070 ceph-mon[39699]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:27.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:27 smithi070 ceph-mon[39699]: Saving service prometheus spec with placement count:1 2024-03-23T08:14:27.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:27 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:28.775 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-23T08:14:28.775 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.932+0000 7f16e7b37700 1 Processor -- start 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.933+0000 7f16e7b37700 1 -- start start 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.933+0000 7f16e7b37700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0107600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.933+0000 7f16e7b37700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e0102ff0 con 0x7f16e0076c00 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.933+0000 7f16e58d3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0107600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.933+0000 7f16e58d3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0107600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41042/0 (socket says 172.21.15.70:41042) 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.933+0000 7f16e58d3700 1 -- 172.21.15.70:0/1429152812 learned_addr learned my addr 172.21.15.70:0/1429152812 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.933+0000 7f16e58d3700 1 -- 172.21.15.70:0/1429152812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e0107b40 con 0x7f16e0076c00 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.933+0000 7f16e58d3700 1 --2- 172.21.15.70:0/1429152812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0107600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16d00098e0 tx=0x7f16d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e88cd9234bf7ff3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e48d1700 1 -- 172.21.15.70:0/1429152812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16d000b860 con 0x7f16e0076c00 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e48d1700 1 -- 172.21.15.70:0/1429152812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16d000be70 con 0x7f16e0076c00 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e48d1700 1 -- 172.21.15.70:0/1429152812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16d0010380 con 0x7f16e0076c00 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e7b37700 1 -- 172.21.15.70:0/1429152812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 msgr2=0x7f16e0107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e7b37700 1 --2- 172.21.15.70:0/1429152812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0107600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16d00098e0 tx=0x7f16d0009250 comp rx=0 tx=0).stop 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e7b37700 1 -- 172.21.15.70:0/1429152812 shutdown_connections 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e7b37700 1 --2- 172.21.15.70:0/1429152812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0107600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:28.776 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e7b37700 1 -- 172.21.15.70:0/1429152812 >> 172.21.15.70:0/1429152812 conn(0x7f16e00fcc50 msgr2=0x7f16e00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e7b37700 1 -- 172.21.15.70:0/1429152812 shutdown_connections 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.934+0000 7f16e7b37700 1 -- 172.21.15.70:0/1429152812 wait complete. 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.935+0000 7f16e7b37700 1 Processor -- start 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.935+0000 7f16e7b37700 1 -- start start 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.935+0000 7f16e7b37700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0113590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.935+0000 7f16e7b37700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e0113ad0 con 0x7f16e0076c00 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.935+0000 7f16e58d3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0113590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.935+0000 7f16e58d3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0113590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41044/0 (socket says 172.21.15.70:41044) 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.935+0000 7f16e58d3700 1 -- 172.21.15.70:0/2478901717 learned_addr learned my addr 172.21.15.70:0/2478901717 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.935+0000 7f16e58d3700 1 -- 172.21.15.70:0/2478901717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d00094f0 con 0x7f16e0076c00 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.936+0000 7f16e58d3700 1 --2- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0113590 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16d0006d40 tx=0x7f16d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.936+0000 7f16d67fc700 1 -- 172.21.15.70:0/2478901717 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16d0010960 con 0x7f16e0076c00 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.936+0000 7f16d67fc700 1 -- 172.21.15.70:0/2478901717 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16d001a430 con 0x7f16e0076c00 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.936+0000 7f16d67fc700 1 -- 172.21.15.70:0/2478901717 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16d00245b0 con 0x7f16e0076c00 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.936+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e0113cd0 con 0x7f16e0076c00 2024-03-23T08:14:28.777 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.936+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e0114840 con 0x7f16e0076c00 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.937+0000 7f16d67fc700 1 -- 172.21.15.70:0/2478901717 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f16d001a9f0 con 0x7f16e0076c00 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.937+0000 7f16d67fc700 1 --2- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f16cc0367e0 0x7f16cc038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.937+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e005f5f0 con 0x7f16e0076c00 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.937+0000 7f16d67fc700 1 -- 172.21.15.70:0/2478901717 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f16d004a250 con 0x7f16e0076c00 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.937+0000 7f16e50d2700 1 --2- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f16cc0367e0 0x7f16cc038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.937+0000 7f16e50d2700 1 --2- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f16cc0367e0 0x7f16cc038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009990 tx=0x7f16dc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:27.942+0000 7f16d67fc700 1 -- 172.21.15.70:0/2478901717 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f16d002b4d0 con 0x7f16e0076c00 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.163+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f16e00fdea0 con 0x7f16cc0367e0 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.167+0000 7f16d67fc700 1 -- 172.21.15.70:0/2478901717 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f16e00fdea0 con 0x7f16cc0367e0 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.169+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f16cc0367e0 msgr2=0x7f16cc038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.169+0000 7f16e7b37700 1 --2- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f16cc0367e0 0x7f16cc038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009990 tx=0x7f16dc006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.169+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 msgr2=0x7f16e0113590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:28.778 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.169+0000 7f16e7b37700 1 --2- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0113590 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16d0006d40 tx=0x7f16d0009250 comp rx=0 tx=0).stop 2024-03-23T08:14:28.779 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.169+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 shutdown_connections 2024-03-23T08:14:28.779 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.169+0000 7f16e7b37700 1 --2- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f16cc0367e0 0x7f16cc038c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:28.779 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.169+0000 7f16e7b37700 1 --2- 172.21.15.70:0/2478901717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16e0076c00 0x7f16e0113590 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:28.779 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.169+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 >> 172.21.15.70:0/2478901717 conn(0x7f16e00fcc50 msgr2=0x7f16e00fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:28.779 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.170+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 shutdown_connections 2024-03-23T08:14:28.779 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:28.170+0000 7f16e7b37700 1 -- 172.21.15.70:0/2478901717 wait complete. 2024-03-23T08:14:28.779 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying node-exporter service with default placement... 2024-03-23T08:14:29.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:28 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:29.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:28 smithi070 ceph-mon[39699]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:29.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:28 smithi070 ceph-mon[39699]: Saving service grafana spec with placement count:1 2024-03-23T08:14:29.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:28 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:30.811 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-23T08:14:30.811 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.417+0000 7fd8dfbab700 1 Processor -- start 2024-03-23T08:14:30.811 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.417+0000 7fd8dfbab700 1 -- start start 2024-03-23T08:14:30.811 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.417+0000 7fd8dfbab700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d81018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:30.811 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.417+0000 7fd8dfbab700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d8101e30 con 0x7fd8d81014e0 2024-03-23T08:14:30.811 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.418+0000 7fd8dd947700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d81018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.418+0000 7fd8dd947700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d81018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42472/0 (socket says 172.21.15.70:42472) 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.418+0000 7fd8dd947700 1 -- 172.21.15.70:0/344376257 learned_addr learned my addr 172.21.15.70:0/344376257 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.418+0000 7fd8dd947700 1 -- 172.21.15.70:0/344376257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d8101f70 con 0x7fd8d81014e0 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.418+0000 7fd8dd947700 1 --2- 172.21.15.70:0/344376257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d81018f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80098e0 tx=0x7fd8c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4c8d40706b3093 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.418+0000 7fd8dc945700 1 -- 172.21.15.70:0/344376257 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8c800b860 con 0x7fd8d81014e0 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.418+0000 7fd8dc945700 1 -- 172.21.15.70:0/344376257 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8c800be70 con 0x7fd8d81014e0 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.418+0000 7fd8dc945700 1 -- 172.21.15.70:0/344376257 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8c8010380 con 0x7fd8d81014e0 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 -- 172.21.15.70:0/344376257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 msgr2=0x7fd8d81018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 --2- 172.21.15.70:0/344376257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d81018f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80098e0 tx=0x7fd8c8009250 comp rx=0 tx=0).stop 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 -- 172.21.15.70:0/344376257 shutdown_connections 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 --2- 172.21.15.70:0/344376257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d81018f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 -- 172.21.15.70:0/344376257 >> 172.21.15.70:0/344376257 conn(0x7fd8d80fcc60 msgr2=0x7fd8d80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 -- 172.21.15.70:0/344376257 shutdown_connections 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 -- 172.21.15.70:0/344376257 wait complete. 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 Processor -- start 2024-03-23T08:14:30.812 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 -- start start 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d8076b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dfbab700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d8114080 con 0x7fd8d81014e0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dd947700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d8076b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.419+0000 7fd8dd947700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d8076b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42478/0 (socket says 172.21.15.70:42478) 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.420+0000 7fd8dd947700 1 -- 172.21.15.70:0/1774587435 learned_addr learned my addr 172.21.15.70:0/1774587435 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.420+0000 7fd8dd947700 1 -- 172.21.15.70:0/1774587435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c80094f0 con 0x7fd8d81014e0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.420+0000 7fd8dd947700 1 --2- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d8076b90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8011ca0 tx=0x7fd8c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.420+0000 7fd8ce7fc700 1 -- 172.21.15.70:0/1774587435 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8c8010850 con 0x7fd8d81014e0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.420+0000 7fd8ce7fc700 1 -- 172.21.15.70:0/1774587435 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8c8010e60 con 0x7fd8d81014e0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.420+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d81141c0 con 0x7fd8d81014e0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.420+0000 7fd8ce7fc700 1 -- 172.21.15.70:0/1774587435 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8c801ae20 con 0x7fd8d81014e0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.420+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d81146b0 con 0x7fd8d81014e0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.421+0000 7fd8ce7fc700 1 -- 172.21.15.70:0/1774587435 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd8c802d3f0 con 0x7fd8d81014e0 2024-03-23T08:14:30.813 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.421+0000 7fd8ce7fc700 1 --2- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd8c40367e0 0x7fd8c4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:30.814 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:30 smithi070 ceph-mon[39699]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:30.814 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:30 smithi070 ceph-mon[39699]: Saving service node-exporter spec with placement * 2024-03-23T08:14:30.814 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:30 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:30.814 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:30 smithi070 ceph-mon[39699]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:14:30.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.421+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d804bed0 con 0x7fd8d81014e0 2024-03-23T08:14:30.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.421+0000 7fd8ce7fc700 1 -- 172.21.15.70:0/1774587435 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd8c804a570 con 0x7fd8d81014e0 2024-03-23T08:14:30.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.421+0000 7fd8dd146700 1 --2- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd8c40367e0 0x7fd8c4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:30.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.421+0000 7fd8dd146700 1 --2- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd8c40367e0 0x7fd8c4038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009990 tx=0x7fd8d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:30.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.426+0000 7fd8ce7fc700 1 -- 172.21.15.70:0/1774587435 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd8c802b8e0 con 0x7fd8d81014e0 2024-03-23T08:14:30.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.627+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8d81148f0 con 0x7fd8c40367e0 2024-03-23T08:14:30.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.631+0000 7fd8ce7fc700 1 -- 172.21.15.70:0/1774587435 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fd8d81148f0 con 0x7fd8c40367e0 2024-03-23T08:14:30.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd8c40367e0 msgr2=0x7fd8c4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:30.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 --2- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd8c40367e0 0x7fd8c4038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009990 tx=0x7fd8d4006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:30.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 msgr2=0x7fd8d8076b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:30.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 --2- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d8076b90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8011ca0 tx=0x7fd8c8009250 comp rx=0 tx=0).stop 2024-03-23T08:14:30.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 shutdown_connections 2024-03-23T08:14:30.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 --2- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fd8c40367e0 0x7fd8c4038c90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:30.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 --2- 172.21.15.70:0/1774587435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8d81014e0 0x7fd8d8076b90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:30.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 >> 172.21.15.70:0/1774587435 conn(0x7fd8d80fcc60 msgr2=0x7fd8d80fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:30.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 shutdown_connections 2024-03-23T08:14:30.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:29.633+0000 7fd8dfbab700 1 -- 172.21.15.70:0/1774587435 wait complete. 2024-03-23T08:14:30.816 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying alertmanager service with default placement... 2024-03-23T08:14:32.210 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-23T08:14:32.210 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.357+0000 7f931fac6700 1 Processor -- start 2024-03-23T08:14:32.210 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.357+0000 7f931fac6700 1 -- start start 2024-03-23T08:14:32.210 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.357+0000 7f931fac6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f93181018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:32.210 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.357+0000 7f931fac6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9318101e30 con 0x7f93181014e0 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.357+0000 7f931d862700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f93181018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.357+0000 7f931d862700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f93181018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42486/0 (socket says 172.21.15.70:42486) 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.357+0000 7f931d862700 1 -- 172.21.15.70:0/882713366 learned_addr learned my addr 172.21.15.70:0/882713366 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.357+0000 7f931d862700 1 -- 172.21.15.70:0/882713366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9318101f70 con 0x7f93181014e0 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931d862700 1 --2- 172.21.15.70:0/882713366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f93181018f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93140098e0 tx=0x7f9314009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2910d6737efac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931c860700 1 -- 172.21.15.70:0/882713366 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f931400b860 con 0x7f93181014e0 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931c860700 1 -- 172.21.15.70:0/882713366 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f931400be70 con 0x7f93181014e0 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931c860700 1 -- 172.21.15.70:0/882713366 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9314010380 con 0x7f93181014e0 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931fac6700 1 -- 172.21.15.70:0/882713366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 msgr2=0x7f93181018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931fac6700 1 --2- 172.21.15.70:0/882713366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f93181018f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93140098e0 tx=0x7f9314009250 comp rx=0 tx=0).stop 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931fac6700 1 -- 172.21.15.70:0/882713366 shutdown_connections 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931fac6700 1 --2- 172.21.15.70:0/882713366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f93181018f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931fac6700 1 -- 172.21.15.70:0/882713366 >> 172.21.15.70:0/882713366 conn(0x7f93180fcc60 msgr2=0x7f93180ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931fac6700 1 -- 172.21.15.70:0/882713366 shutdown_connections 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.358+0000 7f931fac6700 1 -- 172.21.15.70:0/882713366 wait complete. 2024-03-23T08:14:32.211 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.359+0000 7f931fac6700 1 Processor -- start 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.359+0000 7f931fac6700 1 -- start start 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.359+0000 7f931fac6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f9318107740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.359+0000 7f931fac6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9318107c80 con 0x7f93181014e0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.359+0000 7f931d862700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f9318107740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.359+0000 7f931d862700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f9318107740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42494/0 (socket says 172.21.15.70:42494) 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.359+0000 7f931d862700 1 -- 172.21.15.70:0/3364434647 learned_addr learned my addr 172.21.15.70:0/3364434647 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.359+0000 7f931d862700 1 -- 172.21.15.70:0/3364434647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93140094f0 con 0x7f93181014e0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.360+0000 7f931d862700 1 --2- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f9318107740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f931400c010 tx=0x7f9314009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.360+0000 7f930e7fc700 1 -- 172.21.15.70:0/3364434647 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9314010960 con 0x7f93181014e0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.360+0000 7f930e7fc700 1 -- 172.21.15.70:0/3364434647 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f931401a430 con 0x7f93181014e0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.360+0000 7f930e7fc700 1 -- 172.21.15.70:0/3364434647 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9314019580 con 0x7f93181014e0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.360+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9318105cf0 con 0x7f93181014e0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.360+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93181061e0 con 0x7f93181014e0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.361+0000 7f930e7fc700 1 -- 172.21.15.70:0/3364434647 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f93140197b0 con 0x7f93181014e0 2024-03-23T08:14:32.212 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.361+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931804bed0 con 0x7f93181014e0 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.361+0000 7f930e7fc700 1 --2- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f93040367e0 0x7f9304038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.361+0000 7f930e7fc700 1 -- 172.21.15.70:0/3364434647 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f931404a660 con 0x7f93181014e0 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.361+0000 7f931d061700 1 --2- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f93040367e0 0x7f9304038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.361+0000 7f931d061700 1 --2- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f93040367e0 0x7f9304038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9308009990 tx=0x7f9308006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.365+0000 7f930e7fc700 1 -- 172.21.15.70:0/3364434647 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f931402bb80 con 0x7f93181014e0 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.559+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f931810b3c0 con 0x7f93040367e0 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.563+0000 7f930e7fc700 1 -- 172.21.15.70:0/3364434647 <== mgr.14118 v2:172.21.15.70:6800/2149030412 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f931810b3c0 con 0x7f93040367e0 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f93040367e0 msgr2=0x7f9304038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 --2- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f93040367e0 0x7f9304038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9308009990 tx=0x7f9308006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 msgr2=0x7f9318107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 --2- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f9318107740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f931400c010 tx=0x7f9314009250 comp rx=0 tx=0).stop 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 shutdown_connections 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 --2- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f93040367e0 0x7f9304038c90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 --2- 172.21.15.70:0/3364434647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93181014e0 0x7f9318107740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:32.213 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 >> 172.21.15.70:0/3364434647 conn(0x7f93180fcc60 msgr2=0x7f93180fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:32.214 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 shutdown_connections 2024-03-23T08:14:32.214 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:31.565+0000 7f931fac6700 1 -- 172.21.15.70:0/3364434647 wait complete. 2024-03-23T08:14:32.942 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:32 smithi070 ceph-mon[39699]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:32.942 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:32 smithi070 ceph-mon[39699]: Saving service alertmanager spec with placement count:1 2024-03-23T08:14:32.943 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:32 smithi070 ceph-mon[39699]: from='mgr.14118 172.21.15.70:0/3947576735' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:32.943 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:32 smithi070 ceph-mon[39699]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:14:33.390 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.718+0000 7f4f09ca7700 1 Processor -- start 2024-03-23T08:14:33.390 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.718+0000 7f4f09ca7700 1 -- start start 2024-03-23T08:14:33.390 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.718+0000 7f4f09ca7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04102990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:33.390 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.718+0000 7f4f09ca7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f04102ed0 con 0x7f4f04102580 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.718+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.718+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42506/0 (socket says 172.21.15.70:42506) 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.718+0000 7f4f037fe700 1 -- 172.21.15.70:0/2930249572 learned_addr learned my addr 172.21.15.70:0/2930249572 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.718+0000 7f4f037fe700 1 -- 172.21.15.70:0/2930249572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04103010 con 0x7f4f04102580 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.718+0000 7f4f037fe700 1 --2- 172.21.15.70:0/2930249572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04102990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40098e0 tx=0x7f4ef4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df14fd6ab9c2773f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f027fc700 1 -- 172.21.15.70:0/2930249572 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ef400b860 con 0x7f4f04102580 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f027fc700 1 -- 172.21.15.70:0/2930249572 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ef400be70 con 0x7f4f04102580 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f027fc700 1 -- 172.21.15.70:0/2930249572 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ef4010380 con 0x7f4f04102580 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f09ca7700 1 -- 172.21.15.70:0/2930249572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 msgr2=0x7f4f04102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f09ca7700 1 --2- 172.21.15.70:0/2930249572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04102990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40098e0 tx=0x7f4ef4009250 comp rx=0 tx=0).stop 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f09ca7700 1 -- 172.21.15.70:0/2930249572 shutdown_connections 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f09ca7700 1 --2- 172.21.15.70:0/2930249572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04102990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f09ca7700 1 -- 172.21.15.70:0/2930249572 >> 172.21.15.70:0/2930249572 conn(0x7f4f040fdd00 msgr2=0x7f4f04100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f09ca7700 1 -- 172.21.15.70:0/2930249572 shutdown_connections 2024-03-23T08:14:33.391 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.719+0000 7f4f09ca7700 1 -- 172.21.15.70:0/2930249572 wait complete. 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.720+0000 7f4f09ca7700 1 Processor -- start 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.720+0000 7f4f09ca7700 1 -- start start 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.720+0000 7f4f09ca7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04111370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.720+0000 7f4f09ca7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f041118b0 con 0x7f4f04102580 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.720+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04111370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.720+0000 7f4f037fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04111370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42514/0 (socket says 172.21.15.70:42514) 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.720+0000 7f4f037fe700 1 -- 172.21.15.70:0/1727284528 learned_addr learned my addr 172.21.15.70:0/1727284528 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.720+0000 7f4f037fe700 1 -- 172.21.15.70:0/1727284528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef40094f0 con 0x7f4f04102580 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.721+0000 7f4f037fe700 1 --2- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04111370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c010 tx=0x7f4ef4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.721+0000 7f4eebfff700 1 -- 172.21.15.70:0/1727284528 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ef4010960 con 0x7f4f04102580 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.721+0000 7f4eebfff700 1 -- 172.21.15.70:0/1727284528 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ef401a430 con 0x7f4f04102580 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.721+0000 7f4eebfff700 1 -- 172.21.15.70:0/1727284528 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ef4019580 con 0x7f4f04102580 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.721+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f04111ab0 con 0x7f4f04102580 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.721+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f04112620 con 0x7f4f04102580 2024-03-23T08:14:33.392 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.721+0000 7f4eebfff700 1 -- 172.21.15.70:0/1727284528 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f4ef4019800 con 0x7f4f04102580 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.722+0000 7f4eebfff700 1 --2- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4eec036790 0x7f4eec038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.722+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0405f5f0 con 0x7f4f04102580 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.722+0000 7f4eebfff700 1 -- 172.21.15.70:0/1727284528 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4ef404a5c0 con 0x7f4f04102580 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.722+0000 7f4f02ffd700 1 --2- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4eec036790 0x7f4eec038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.722+0000 7f4f02ffd700 1 --2- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4eec036790 0x7f4eec038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009990 tx=0x7f4ef8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.726+0000 7f4eebfff700 1 -- 172.21.15.70:0/1727284528 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4ef402bb30 con 0x7f4f04102580 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.914+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f4f0404bed0 con 0x7f4f04102580 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.917+0000 7f4eebfff700 1 -- 172.21.15.70:0/1727284528 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4ef4023e20 con 0x7f4f04102580 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4eec036790 msgr2=0x7f4eec038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 --2- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4eec036790 0x7f4eec038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009990 tx=0x7f4ef8006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 msgr2=0x7f4f04111370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 --2- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04111370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c010 tx=0x7f4ef4009250 comp rx=0 tx=0).stop 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 shutdown_connections 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 --2- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f4eec036790 0x7f4eec038c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:33.393 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 --2- 172.21.15.70:0/1727284528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f04102580 0x7f4f04111370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:33.394 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 >> 172.21.15.70:0/1727284528 conn(0x7f4f040fdd00 msgr2=0x7f4f040fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:33.394 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 shutdown_connections 2024-03-23T08:14:33.394 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:32.919+0000 7f4f09ca7700 1 -- 172.21.15.70:0/1727284528 wait complete. 2024-03-23T08:14:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:33 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1727284528' entity='client.admin' 2024-03-23T08:14:34.739 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.898+0000 7f006e7a9700 1 Processor -- start 2024-03-23T08:14:34.739 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.898+0000 7f006e7a9700 1 -- start start 2024-03-23T08:14:34.739 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.898+0000 7f006e7a9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068102960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.898+0000 7f006e7a9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068102ea0 con 0x7f0068102550 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.898+0000 7f0067fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.899+0000 7f0067fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068102960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42520/0 (socket says 172.21.15.70:42520) 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.899+0000 7f0067fff700 1 -- 172.21.15.70:0/1039764886 learned_addr learned my addr 172.21.15.70:0/1039764886 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.899+0000 7f0067fff700 1 -- 172.21.15.70:0/1039764886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068102fe0 con 0x7f0068102550 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.899+0000 7f0067fff700 1 --2- 172.21.15.70:0/1039764886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068102960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f00580098e0 tx=0x7f0058009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13987c1d31970d22 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.899+0000 7f0066ffd700 1 -- 172.21.15.70:0/1039764886 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f005800b860 con 0x7f0068102550 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.899+0000 7f0066ffd700 1 -- 172.21.15.70:0/1039764886 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f005800be70 con 0x7f0068102550 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.899+0000 7f0066ffd700 1 -- 172.21.15.70:0/1039764886 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0058010380 con 0x7f0068102550 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 -- 172.21.15.70:0/1039764886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 msgr2=0x7f0068102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 --2- 172.21.15.70:0/1039764886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068102960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f00580098e0 tx=0x7f0058009250 comp rx=0 tx=0).stop 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 -- 172.21.15.70:0/1039764886 shutdown_connections 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 --2- 172.21.15.70:0/1039764886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068102960 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 -- 172.21.15.70:0/1039764886 >> 172.21.15.70:0/1039764886 conn(0x7f00680fdd10 msgr2=0x7f0068100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:34.740 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 -- 172.21.15.70:0/1039764886 shutdown_connections 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 -- 172.21.15.70:0/1039764886 wait complete. 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 Processor -- start 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 -- start start 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068111440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.900+0000 7f006e7a9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068111980 con 0x7f0068102550 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f0067fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068111440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f0067fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068111440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42528/0 (socket says 172.21.15.70:42528) 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f0067fff700 1 -- 172.21.15.70:0/2393665773 learned_addr learned my addr 172.21.15.70:0/2393665773 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f0067fff700 1 -- 172.21.15.70:0/2393665773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00580094f0 con 0x7f0068102550 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f0067fff700 1 --2- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068111440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0058000c00 tx=0x7f0058009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f0064ff9700 1 -- 172.21.15.70:0/2393665773 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00580105f0 con 0x7f0068102550 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f0064ff9700 1 -- 172.21.15.70:0/2393665773 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0058010c00 con 0x7f0068102550 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f0064ff9700 1 -- 172.21.15.70:0/2393665773 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f005801ac70 con 0x7f0068102550 2024-03-23T08:14:34.741 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0068111b80 con 0x7f0068102550 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.901+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00681126f0 con 0x7f0068102550 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.902+0000 7f0064ff9700 1 -- 172.21.15.70:0/2393665773 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f0058025400 con 0x7f0068102550 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.902+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006805f5f0 con 0x7f0068102550 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.902+0000 7f0064ff9700 1 --2- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0054036790 0x7f0054038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.902+0000 7f0064ff9700 1 -- 172.21.15.70:0/2393665773 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f005804a2f0 con 0x7f0068102550 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.902+0000 7f00677fe700 1 --2- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0054036790 0x7f0054038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.902+0000 7f00677fe700 1 --2- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0054036790 0x7f0054038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0050009990 tx=0x7f0050006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:33.907+0000 7f0064ff9700 1 -- 172.21.15.70:0/2393665773 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f005802c8f0 con 0x7f0068102550 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.093+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f006804bed0 con 0x7f0068102550 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.096+0000 7f0064ff9700 1 -- 172.21.15.70:0/2393665773 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f0058020030 con 0x7f0068102550 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0054036790 msgr2=0x7f0054038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 --2- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0054036790 0x7f0054038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0050009990 tx=0x7f0050006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 msgr2=0x7f0068111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 --2- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068111440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0058000c00 tx=0x7f0058009250 comp rx=0 tx=0).stop 2024-03-23T08:14:34.742 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 shutdown_connections 2024-03-23T08:14:34.743 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 --2- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0054036790 0x7f0054038c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:34.743 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 --2- 172.21.15.70:0/2393665773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0068102550 0x7f0068111440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:34.743 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 >> 172.21.15.70:0/2393665773 conn(0x7f00680fdd10 msgr2=0x7f00680fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:34.743 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 shutdown_connections 2024-03-23T08:14:34.743 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:34.098+0000 7f006e7a9700 1 -- 172.21.15.70:0/2393665773 wait complete. 2024-03-23T08:14:34.743 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling the dashboard module... 2024-03-23T08:14:35.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:35 smithi070 ceph-mon[39699]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:14:35.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:35 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2393665773' entity='client.admin' 2024-03-23T08:14:36.430 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:36 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/227248373' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-23T08:14:36.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.202+0000 7f048b15d700 1 Processor -- start 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.202+0000 7f048b15d700 1 -- start start 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.202+0000 7f048b15d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0484076d00 0x7f04841050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.202+0000 7f048b15d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0484105630 con 0x7f0484076d00 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.203+0000 7f0488ef9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0484076d00 0x7f04841050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.203+0000 7f0488ef9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0484076d00 0x7f04841050f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42536/0 (socket says 172.21.15.70:42536) 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.203+0000 7f0488ef9700 1 -- 172.21.15.70:0/497900005 learned_addr learned my addr 172.21.15.70:0/497900005 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.203+0000 7f0488ef9700 1 -- 172.21.15.70:0/497900005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0484105770 con 0x7f0484076d00 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.203+0000 7f0488ef9700 1 --2- 172.21.15.70:0/497900005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0484076d00 0x7f04841050f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04780098e0 tx=0x7f0478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3019094eb727cd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.203+0000 7f04837fe700 1 -- 172.21.15.70:0/497900005 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f047800b860 con 0x7f0484076d00 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.203+0000 7f04837fe700 1 -- 172.21.15.70:0/497900005 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f047800be70 con 0x7f0484076d00 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.203+0000 7f04837fe700 1 -- 172.21.15.70:0/497900005 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0478010380 con 0x7f0484076d00 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.204+0000 7f048b15d700 1 -- 172.21.15.70:0/497900005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0484076d00 msgr2=0x7f04841050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.204+0000 7f048b15d700 1 --2- 172.21.15.70:0/497900005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0484076d00 0x7f04841050f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04780098e0 tx=0x7f0478009250 comp rx=0 tx=0).stop 2024-03-23T08:14:36.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.204+0000 7f048b15d700 1 -- 172.21.15.70:0/497900005 shutdown_connections 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.204+0000 7f048b15d700 1 --2- 172.21.15.70:0/497900005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0484076d00 0x7f04841050f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.204+0000 7f048b15d700 1 -- 172.21.15.70:0/497900005 >> 172.21.15.70:0/497900005 conn(0x7f04840fcb20 msgr2=0x7f04840fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.204+0000 7f048b15d700 1 -- 172.21.15.70:0/497900005 shutdown_connections 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.204+0000 7f048b15d700 1 -- 172.21.15.70:0/497900005 wait complete. 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.204+0000 7f048b15d700 1 Processor -- start 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.204+0000 7f048b15d700 1 -- start start 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.205+0000 7f048b15d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04841113a0 0x7f04841117b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.205+0000 7f048b15d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0484111cf0 con 0x7f04841113a0 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.205+0000 7f0488ef9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04841113a0 0x7f04841117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.205+0000 7f0488ef9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04841113a0 0x7f04841117b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42546/0 (socket says 172.21.15.70:42546) 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.205+0000 7f0488ef9700 1 -- 172.21.15.70:0/227248373 learned_addr learned my addr 172.21.15.70:0/227248373 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.205+0000 7f0488ef9700 1 -- 172.21.15.70:0/227248373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04780094f0 con 0x7f04841113a0 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.205+0000 7f0488ef9700 1 --2- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04841113a0 0x7f04841117b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0478006d40 tx=0x7f0478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.205+0000 7f04817fa700 1 -- 172.21.15.70:0/227248373 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0478010960 con 0x7f04841113a0 2024-03-23T08:14:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.206+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0484111ef0 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.206+0000 7f04817fa700 1 -- 172.21.15.70:0/227248373 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f047801a430 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.206+0000 7f04817fa700 1 -- 172.21.15.70:0/227248373 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f04780245b0 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.206+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04841128d0 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.207+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048404bed0 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.207+0000 7f04817fa700 1 -- 172.21.15.70:0/227248373 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f047801a9f0 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.207+0000 7f04817fa700 1 --2- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0474036790 0x7f0474038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.207+0000 7f04817fa700 1 -- 172.21.15.70:0/227248373 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f047804a5b0 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.207+0000 7f0483fff700 1 --2- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0474036790 0x7f0474038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.208+0000 7f0483fff700 1 --2- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0474036790 0x7f0474038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0470009990 tx=0x7f0470006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.212+0000 7f04817fa700 1 -- 172.21.15.70:0/227248373 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f047802bd50 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:35.447+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f0484112b10 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.098+0000 7f04817fa700 1 -- 172.21.15.70:0/227248373 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f0478049750 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.099+0000 7f04817fa700 1 -- 172.21.15.70:0/227248373 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f0478018350 con 0x7f04841113a0 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0474036790 msgr2=0x7f0474038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 --2- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0474036790 0x7f0474038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0470009990 tx=0x7f0470006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:36.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04841113a0 msgr2=0x7f04841117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:36.724 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 --2- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04841113a0 0x7f04841117b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0478006d40 tx=0x7f0478009250 comp rx=0 tx=0).stop 2024-03-23T08:14:36.724 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 shutdown_connections 2024-03-23T08:14:36.724 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 --2- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f0474036790 0x7f0474038c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:36.724 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 --2- 172.21.15.70:0/227248373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04841113a0 0x7f04841117b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:36.724 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 >> 172.21.15.70:0/227248373 conn(0x7f04840fcb20 msgr2=0x7f04840fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:36.724 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 shutdown_connections 2024-03-23T08:14:36.724 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:36.102+0000 7f048b15d700 1 -- 172.21.15.70:0/227248373 wait complete. 2024-03-23T08:14:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:37 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/227248373' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-23T08:14:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:37 smithi070 ceph-mon[39699]: mgrmap e9: smithi070.jhsbch(active, since 26s) 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "smithi070.jhsbch", 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.217+0000 7f27f1efc700 1 Processor -- start 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.217+0000 7f27f1efc700 1 -- start start 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.217+0000 7f27f1efc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec107650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:38.138 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.217+0000 7f27f1efc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec103040 con 0x7f27ec076be0 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.218+0000 7f27eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec107650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.218+0000 7f27eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec107650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42566/0 (socket says 172.21.15.70:42566) 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.218+0000 7f27eb7fe700 1 -- 172.21.15.70:0/2427682259 learned_addr learned my addr 172.21.15.70:0/2427682259 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.218+0000 7f27eb7fe700 1 -- 172.21.15.70:0/2427682259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec107b90 con 0x7f27ec076be0 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.218+0000 7f27eb7fe700 1 --2- 172.21.15.70:0/2427682259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec107650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0098e0 tx=0x7f27dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff7cb342a9d0d88 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.218+0000 7f27ea7fc700 1 -- 172.21.15.70:0/2427682259 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27dc00b860 con 0x7f27ec076be0 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.218+0000 7f27ea7fc700 1 -- 172.21.15.70:0/2427682259 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27dc00be70 con 0x7f27ec076be0 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.218+0000 7f27ea7fc700 1 -- 172.21.15.70:0/2427682259 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27dc010380 con 0x7f27ec076be0 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.219+0000 7f27f1efc700 1 -- 172.21.15.70:0/2427682259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 msgr2=0x7f27ec107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.219+0000 7f27f1efc700 1 --2- 172.21.15.70:0/2427682259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec107650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0098e0 tx=0x7f27dc009250 comp rx=0 tx=0).stop 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.219+0000 7f27f1efc700 1 -- 172.21.15.70:0/2427682259 shutdown_connections 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.219+0000 7f27f1efc700 1 --2- 172.21.15.70:0/2427682259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec107650 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.219+0000 7f27f1efc700 1 -- 172.21.15.70:0/2427682259 >> 172.21.15.70:0/2427682259 conn(0x7f27ec0fcc60 msgr2=0x7f27ec0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.219+0000 7f27f1efc700 1 -- 172.21.15.70:0/2427682259 shutdown_connections 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.219+0000 7f27f1efc700 1 -- 172.21.15.70:0/2427682259 wait complete. 2024-03-23T08:14:38.139 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.220+0000 7f27f1efc700 1 Processor -- start 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.220+0000 7f27f1efc700 1 -- start start 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.220+0000 7f27f1efc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec1135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.220+0000 7f27f1efc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec113ae0 con 0x7f27ec076be0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.220+0000 7f27eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec1135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.220+0000 7f27eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec1135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42572/0 (socket says 172.21.15.70:42572) 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.220+0000 7f27eb7fe700 1 -- 172.21.15.70:0/1248814739 learned_addr learned my addr 172.21.15.70:0/1248814739 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.220+0000 7f27eb7fe700 1 -- 172.21.15.70:0/1248814739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0094f0 con 0x7f27ec076be0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.220+0000 7f27eb7fe700 1 --2- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec1135a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006d40 tx=0x7f27dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.221+0000 7f27d3fff700 1 -- 172.21.15.70:0/1248814739 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27dc010960 con 0x7f27ec076be0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.221+0000 7f27d3fff700 1 -- 172.21.15.70:0/1248814739 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27dc01a430 con 0x7f27ec076be0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.221+0000 7f27d3fff700 1 -- 172.21.15.70:0/1248814739 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27dc0245b0 con 0x7f27ec076be0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.221+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec113ce0 con 0x7f27ec076be0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.221+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec114850 con 0x7f27ec076be0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.221+0000 7f27d3fff700 1 -- 172.21.15.70:0/1248814739 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f27dc02d3f0 con 0x7f27ec076be0 2024-03-23T08:14:38.140 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.221+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec05f5f0 con 0x7f27ec076be0 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.222+0000 7f27d3fff700 1 --2- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f27d40367e0 0x7f27d4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.222+0000 7f27d3fff700 1 -- 172.21.15.70:0/1248814739 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f27dc04a340 con 0x7f27ec076be0 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.222+0000 7f27eaffd700 1 -- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f27d40367e0 msgr2=0x7f27d4038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2149030412 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.222+0000 7f27eaffd700 1 --2- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f27d40367e0 0x7f27d4038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.230+0000 7f27d3fff700 1 -- 172.21.15.70:0/1248814739 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f27dc02b8e0 con 0x7f27ec076be0 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.423+0000 7f27eaffd700 1 -- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f27d40367e0 msgr2=0x7f27d4038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2149030412 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.423+0000 7f27eaffd700 1 --2- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f27d40367e0 0x7f27d4038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.476+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f27ec114b90 con 0x7f27ec076be0 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.476+0000 7f27d3fff700 1 -- 172.21.15.70:0/1248814739 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f27dc02b2f0 con 0x7f27ec076be0 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f27d40367e0 msgr2=0x7f27d4038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 --2- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f27d40367e0 0x7f27d4038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 msgr2=0x7f27ec1135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 --2- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec1135a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006d40 tx=0x7f27dc009250 comp rx=0 tx=0).stop 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 shutdown_connections 2024-03-23T08:14:38.141 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 --2- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7f27d40367e0 0x7f27d4038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:38.142 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 --2- 172.21.15.70:0/1248814739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec076be0 0x7f27ec1135a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:38.142 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 >> 172.21.15.70:0/1248814739 conn(0x7f27ec0fcc60 msgr2=0x7f27ec0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:38.142 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 shutdown_connections 2024-03-23T08:14:38.142 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:37.478+0000 7f27f1efc700 1 -- 172.21.15.70:0/1248814739 wait complete. 2024-03-23T08:14:38.142 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2024-03-23T08:14:38.142 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 9... 2024-03-23T08:14:38.367 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:38 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1248814739' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-23T08:14:42.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: Active manager daemon smithi070.jhsbch restarted 2024-03-23T08:14:42.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: Activating manager daemon smithi070.jhsbch 2024-03-23T08:14:42.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: osdmap e3: 0 total, 0 up, 0 in 2024-03-23T08:14:42.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: mgrmap e10: smithi070.jhsbch(active, starting, since 0.0033419s) 2024-03-23T08:14:42.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:14:42.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:14:42.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:14:42.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:14:42.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:14:42.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: Manager daemon smithi070.jhsbch is now available 2024-03-23T08:14:42.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:14:42.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:14:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:43 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:14:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:43 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/trash_purge_schedule"}]: dispatch 2024-03-23T08:14:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:43 smithi070 ceph-mon[39699]: [23/Mar/2024:08:14:42] ENGINE Bus STARTING 2024-03-23T08:14:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:43 smithi070 ceph-mon[39699]: [23/Mar/2024:08:14:43] ENGINE Serving on https://172.21.15.70:7150 2024-03-23T08:14:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:43 smithi070 ceph-mon[39699]: [23/Mar/2024:08:14:43] ENGINE Bus STARTED 2024-03-23T08:14:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:43 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:43 smithi070 ceph-mon[39699]: mgrmap e11: smithi070.jhsbch(active, since 1.00554s) 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.644+0000 7fac39b79700 1 Processor -- start 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.644+0000 7fac39b79700 1 -- start start 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.644+0000 7fac39b79700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac34077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.644+0000 7fac39b79700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac34079160 con 0x7fac34078d50 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.644+0000 7fac337fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac34077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.644+0000 7fac337fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac34077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50722/0 (socket says 172.21.15.70:50722) 2024-03-23T08:14:44.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.644+0000 7fac337fe700 1 -- 172.21.15.70:0/1049502670 learned_addr learned my addr 172.21.15.70:0/1049502670 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.645+0000 7fac337fe700 1 -- 172.21.15.70:0/1049502670 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340777a0 con 0x7fac34078d50 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.645+0000 7fac337fe700 1 --2- 172.21.15.70:0/1049502670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac34077260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac240098e0 tx=0x7fac24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39d8e004b38b733 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.645+0000 7fac327fc700 1 -- 172.21.15.70:0/1049502670 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac2400b860 con 0x7fac34078d50 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.645+0000 7fac327fc700 1 -- 172.21.15.70:0/1049502670 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac2400be70 con 0x7fac34078d50 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.645+0000 7fac327fc700 1 -- 172.21.15.70:0/1049502670 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac24010380 con 0x7fac34078d50 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.645+0000 7fac39b79700 1 -- 172.21.15.70:0/1049502670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 msgr2=0x7fac34077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.645+0000 7fac39b79700 1 --2- 172.21.15.70:0/1049502670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac34077260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac240098e0 tx=0x7fac24009250 comp rx=0 tx=0).stop 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.646+0000 7fac39b79700 1 -- 172.21.15.70:0/1049502670 shutdown_connections 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.646+0000 7fac39b79700 1 --2- 172.21.15.70:0/1049502670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac34077260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.646+0000 7fac39b79700 1 -- 172.21.15.70:0/1049502670 >> 172.21.15.70:0/1049502670 conn(0x7fac340fdd00 msgr2=0x7fac34100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.646+0000 7fac39b79700 1 -- 172.21.15.70:0/1049502670 shutdown_connections 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.646+0000 7fac39b79700 1 -- 172.21.15.70:0/1049502670 wait complete. 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.646+0000 7fac39b79700 1 Processor -- start 2024-03-23T08:14:44.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.647+0000 7fac39b79700 1 -- start start 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.647+0000 7fac39b79700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac341135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.647+0000 7fac39b79700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac34113ae0 con 0x7fac34078d50 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.647+0000 7fac337fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac341135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.647+0000 7fac337fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac341135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50736/0 (socket says 172.21.15.70:50736) 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.647+0000 7fac337fe700 1 -- 172.21.15.70:0/616388904 learned_addr learned my addr 172.21.15.70:0/616388904 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.647+0000 7fac337fe700 1 -- 172.21.15.70:0/616388904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac240094f0 con 0x7fac34078d50 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.647+0000 7fac337fe700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac341135a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac240094c0 tx=0x7fac24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.647+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac24010850 con 0x7fac34078d50 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.648+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac34113ce0 con 0x7fac34078d50 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.648+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac24010e60 con 0x7fac34078d50 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.648+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac2401a430 con 0x7fac34078d50 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.648+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac34114850 con 0x7fac34078d50 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.648+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fac2401a6b0 con 0x7fac34078d50 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.649+0000 7fac1bfff700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 0x7fac1c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.649+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 --> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac3405f5f0 con 0x7fac1c0367e0 2024-03-23T08:14:44.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.649+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fac2404a640 con 0x7fac34078d50 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.649+0000 7fac32ffd700 1 -- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 msgr2=0x7fac1c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2149030412 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.649+0000 7fac32ffd700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 0x7fac1c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.850+0000 7fac32ffd700 1 -- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 msgr2=0x7fac1c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2149030412 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:38.850+0000 7fac32ffd700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 0x7fac1c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:39.251+0000 7fac32ffd700 1 -- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 msgr2=0x7fac1c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2149030412 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:39.251+0000 7fac32ffd700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 0x7fac1c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:40.052+0000 7fac32ffd700 1 -- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 msgr2=0x7fac1c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2149030412 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:40.052+0000 7fac32ffd700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 0x7fac1c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:41.653+0000 7fac32ffd700 1 -- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 msgr2=0x7fac1c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2149030412 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:41.653+0000 7fac32ffd700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 0x7fac1c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:42.565+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7fac240247b0 con 0x7fac34078d50 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:42.565+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 msgr2=0x7fac1c038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:42.565+0000 7fac1bfff700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 0x7fac1c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.566+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fac2404b850 con 0x7fac34078d50 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.566+0000 7fac1bfff700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fac1c039ff0 0x7fac1c03c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.566+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 --> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac3405f5f0 con 0x7fac1c039ff0 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.567+0000 7fac32ffd700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fac1c039ff0 0x7fac1c03c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.567+0000 7fac32ffd700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fac1c039ff0 0x7fac1c03c3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fac280039d0 tx=0x7fac280092f0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.568+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 <== mgr.14160 v2:172.21.15.70:6800/2732684403 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fac3405f5f0 con 0x7fac1c039ff0 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 --> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fac34114170 con 0x7fac1c039ff0 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac1bfff700 1 -- 172.21.15.70:0/616388904 <== mgr.14160 v2:172.21.15.70:6800/2732684403 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fac34114170 con 0x7fac1c039ff0 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fac1c039ff0 msgr2=0x7fac1c03c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fac1c039ff0 0x7fac1c03c3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fac280039d0 tx=0x7fac280092f0 comp rx=0 tx=0).stop 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 msgr2=0x7fac341135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac341135a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac240094c0 tx=0x7fac24009250 comp rx=0 tx=0).stop 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 shutdown_connections 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac34078d50 0x7fac341135a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fac1c039ff0 0x7fac1c03c3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 --2- 172.21.15.70:0/616388904 >> [v2:172.21.15.70:6800/2149030412,v1:172.21.15.70:6801/2149030412] conn(0x7fac1c0367e0 0x7fac1c038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 >> 172.21.15.70:0/616388904 conn(0x7fac340fdd00 msgr2=0x7fac340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 shutdown_connections 2024-03-23T08:14:44.185 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:43.574+0000 7fac39b79700 1 -- 172.21.15.70:0/616388904 wait complete. 2024-03-23T08:14:44.186 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 9 is available 2024-03-23T08:14:44.186 INFO:teuthology.orchestra.run.smithi070.stdout:Generating a dashboard self-signed certificate... 2024-03-23T08:14:44.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:44 smithi070 ceph-mon[39699]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-23T08:14:44.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:44 smithi070 ceph-mon[39699]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-23T08:14:44.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:44 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:45.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-23T08:14:45.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.809+0000 7fcc4c4bf700 1 Processor -- start 2024-03-23T08:14:45.710 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.810+0000 7fcc4c4bf700 1 -- start start 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.810+0000 7fcc4c4bf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.810+0000 7fcc4c4bf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc44104510 con 0x7fcc44103bc0 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.810+0000 7fcc4a25b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.810+0000 7fcc4a25b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44103fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50798/0 (socket says 172.21.15.70:50798) 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.810+0000 7fcc4a25b700 1 -- 172.21.15.70:0/859989403 learned_addr learned my addr 172.21.15.70:0/859989403 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.810+0000 7fcc4a25b700 1 -- 172.21.15.70:0/859989403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44104650 con 0x7fcc44103bc0 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc4a25b700 1 --2- 172.21.15.70:0/859989403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44103fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc400098e0 tx=0x7fcc40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba29ce3934d25a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc49259700 1 -- 172.21.15.70:0/859989403 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc4000b860 con 0x7fcc44103bc0 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc49259700 1 -- 172.21.15.70:0/859989403 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc4000be70 con 0x7fcc44103bc0 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc49259700 1 -- 172.21.15.70:0/859989403 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc40010380 con 0x7fcc44103bc0 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/859989403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 msgr2=0x7fcc44103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc4c4bf700 1 --2- 172.21.15.70:0/859989403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44103fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc400098e0 tx=0x7fcc40009250 comp rx=0 tx=0).stop 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/859989403 shutdown_connections 2024-03-23T08:14:45.711 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc4c4bf700 1 --2- 172.21.15.70:0/859989403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44103fd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/859989403 >> 172.21.15.70:0/859989403 conn(0x7fcc440fc380 msgr2=0x7fcc440fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/859989403 shutdown_connections 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.811+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/859989403 wait complete. 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.812+0000 7fcc4c4bf700 1 Processor -- start 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.812+0000 7fcc4c4bf700 1 -- start start 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.812+0000 7fcc4c4bf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44111400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.812+0000 7fcc4c4bf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc44111940 con 0x7fcc44103bc0 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.812+0000 7fcc4a25b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44111400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.812+0000 7fcc4a25b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44111400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50804/0 (socket says 172.21.15.70:50804) 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.812+0000 7fcc4a25b700 1 -- 172.21.15.70:0/3251454720 learned_addr learned my addr 172.21.15.70:0/3251454720 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.813+0000 7fcc4a25b700 1 -- 172.21.15.70:0/3251454720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400094f0 con 0x7fcc44103bc0 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.813+0000 7fcc4a25b700 1 --2- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44111400 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcc40011ca0 tx=0x7fcc40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.813+0000 7fcc3affd700 1 -- 172.21.15.70:0/3251454720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc40010850 con 0x7fcc44103bc0 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.813+0000 7fcc3affd700 1 -- 172.21.15.70:0/3251454720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc40010e60 con 0x7fcc44103bc0 2024-03-23T08:14:45.712 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.813+0000 7fcc3affd700 1 -- 172.21.15.70:0/3251454720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc4001ae70 con 0x7fcc44103bc0 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.813+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc44111b40 con 0x7fcc44103bc0 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.813+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc441126b0 con 0x7fcc44103bc0 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.814+0000 7fcc3affd700 1 -- 172.21.15.70:0/3251454720 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fcc400109b0 con 0x7fcc44103bc0 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.814+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4405f5f0 con 0x7fcc44103bc0 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.814+0000 7fcc3affd700 1 --2- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fcc30036750 0x7fcc30038c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.814+0000 7fcc3affd700 1 -- 172.21.15.70:0/3251454720 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcc4004a4b0 con 0x7fcc44103bc0 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.814+0000 7fcc49a5a700 1 --2- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fcc30036750 0x7fcc30038c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.815+0000 7fcc49a5a700 1 --2- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fcc30036750 0x7fcc30038c00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009990 tx=0x7fcc34006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:44.819+0000 7fcc3affd700 1 -- 172.21.15.70:0/3251454720 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcc40018ba0 con 0x7fcc44103bc0 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.044+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 --> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc440fd600 con 0x7fcc30036750 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.112+0000 7fcc3affd700 1 -- 172.21.15.70:0/3251454720 <== mgr.14160 v2:172.21.15.70:6800/2732684403 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fcc440fd600 con 0x7fcc30036750 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.115+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fcc30036750 msgr2=0x7fcc30038c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.115+0000 7fcc4c4bf700 1 --2- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fcc30036750 0x7fcc30038c00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009990 tx=0x7fcc34006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.115+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 msgr2=0x7fcc44111400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:45.713 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.115+0000 7fcc4c4bf700 1 --2- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44111400 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcc40011ca0 tx=0x7fcc40009250 comp rx=0 tx=0).stop 2024-03-23T08:14:45.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.115+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 shutdown_connections 2024-03-23T08:14:45.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.115+0000 7fcc4c4bf700 1 --2- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fcc30036750 0x7fcc30038c00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:45.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.115+0000 7fcc4c4bf700 1 --2- 172.21.15.70:0/3251454720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc44103bc0 0x7fcc44111400 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:45.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.115+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 >> 172.21.15.70:0/3251454720 conn(0x7fcc440fc380 msgr2=0x7fcc440fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:45.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.115+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 shutdown_connections 2024-03-23T08:14:45.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:45.116+0000 7fcc4c4bf700 1 -- 172.21.15.70:0/3251454720 wait complete. 2024-03-23T08:14:45.714 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial admin user... 2024-03-23T08:14:46.416 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:46 smithi070 ceph-mon[39699]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:46.416 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:46 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:46.416 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:46 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:46.417 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:46 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:46.417 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:46 smithi070 ceph-mon[39699]: mgrmap e12: smithi070.jhsbch(active, since 2s) 2024-03-23T08:14:47.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$TcAi91AQf6HJkhRTdpjEEOZCOOnuCQ6g2Gyw9FfnCvovcRmslaTl6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711181686, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-23T08:14:47.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.238+0000 7f8944d35700 1 Processor -- start 2024-03-23T08:14:47.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.238+0000 7f8944d35700 1 -- start start 2024-03-23T08:14:47.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.238+0000 7f8944d35700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f89401018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:47.896 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.238+0000 7f8944d35700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940101e30 con 0x7f89401014e0 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.239+0000 7f893e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f89401018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.239+0000 7f893e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f89401018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50808/0 (socket says 172.21.15.70:50808) 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.239+0000 7f893e59c700 1 -- 172.21.15.70:0/2869263873 learned_addr learned my addr 172.21.15.70:0/2869263873 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.239+0000 7f893e59c700 1 -- 172.21.15.70:0/2869263873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8940101f70 con 0x7f89401014e0 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.239+0000 7f893e59c700 1 --2- 172.21.15.70:0/2869263873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f89401018f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f893000bb80 tx=0x7f893000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b798132de3c8decf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.239+0000 7f893dd9b700 1 -- 172.21.15.70:0/2869263873 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f893000dd00 con 0x7f89401014e0 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.239+0000 7f893dd9b700 1 -- 172.21.15.70:0/2869263873 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f893000de60 con 0x7f89401014e0 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.239+0000 7f893dd9b700 1 -- 172.21.15.70:0/2869263873 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89300188d0 con 0x7f89401014e0 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 -- 172.21.15.70:0/2869263873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 msgr2=0x7f89401018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 --2- 172.21.15.70:0/2869263873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f89401018f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f893000bb80 tx=0x7f893000b4f0 comp rx=0 tx=0).stop 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 -- 172.21.15.70:0/2869263873 shutdown_connections 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 --2- 172.21.15.70:0/2869263873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f89401018f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 -- 172.21.15.70:0/2869263873 >> 172.21.15.70:0/2869263873 conn(0x7f89400fcc60 msgr2=0x7f89400ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 -- 172.21.15.70:0/2869263873 shutdown_connections 2024-03-23T08:14:47.897 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 -- 172.21.15.70:0/2869263873 wait complete. 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 Processor -- start 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 -- start start 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f8940113530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.240+0000 7f8944d35700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940113a70 con 0x7f89401014e0 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f893e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f8940113530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f893e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f8940113530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50824/0 (socket says 172.21.15.70:50824) 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f893e59c700 1 -- 172.21.15.70:0/1952224464 learned_addr learned my addr 172.21.15.70:0/1952224464 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f893e59c700 1 -- 172.21.15.70:0/1952224464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893000b790 con 0x7f89401014e0 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f893e59c700 1 --2- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f8940113530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8930006b00 tx=0x7f893000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f8936ffd700 1 -- 172.21.15.70:0/1952224464 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8930018e40 con 0x7f89401014e0 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f8936ffd700 1 -- 172.21.15.70:0/1952224464 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8930010c00 con 0x7f89401014e0 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8940113c70 con 0x7f89401014e0 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f8936ffd700 1 -- 172.21.15.70:0/1952224464 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8930021550 con 0x7f89401014e0 2024-03-23T08:14:47.898 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.241+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89401147e0 con 0x7f89401014e0 2024-03-23T08:14:47.899 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.242+0000 7f8936ffd700 1 -- 172.21.15.70:0/1952224464 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f89300217c0 con 0x7f89401014e0 2024-03-23T08:14:47.899 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.242+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894005f5f0 con 0x7f89401014e0 2024-03-23T08:14:47.899 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.242+0000 7f8936ffd700 1 --2- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f892c036830 0x7f892c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:47.899 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.242+0000 7f8936ffd700 1 -- 172.21.15.70:0/1952224464 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8930014070 con 0x7f89401014e0 2024-03-23T08:14:47.899 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:47 smithi070 ceph-mon[39699]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:14:47.899 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:47 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.242+0000 7f8937fff700 1 --2- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f892c036830 0x7f892c038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.243+0000 7f8937fff700 1 --2- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f892c036830 0x7f892c038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8928009990 tx=0x7f8928006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.247+0000 7f8936ffd700 1 -- 172.21.15.70:0/1952224464 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f893000edc0 con 0x7f89401014e0 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.457+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 --> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f894005f410 con 0x7f892c036830 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.680+0000 7f8936ffd700 1 -- 172.21.15.70:0/1952224464 <== mgr.14160 v2:172.21.15.70:6800/2732684403 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f894005f410 con 0x7f892c036830 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.682+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f892c036830 msgr2=0x7f892c038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.683+0000 7f8944d35700 1 --2- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f892c036830 0x7f892c038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8928009990 tx=0x7f8928006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.683+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 msgr2=0x7f8940113530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.683+0000 7f8944d35700 1 --2- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f8940113530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8930006b00 tx=0x7f893000b4f0 comp rx=0 tx=0).stop 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.683+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 shutdown_connections 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.683+0000 7f8944d35700 1 --2- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f892c036830 0x7f892c038ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:47.901 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.683+0000 7f8944d35700 1 --2- 172.21.15.70:0/1952224464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89401014e0 0x7f8940113530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:47.902 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.683+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 >> 172.21.15.70:0/1952224464 conn(0x7f89400fcc60 msgr2=0x7f8940105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:47.902 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.683+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 shutdown_connections 2024-03-23T08:14:47.902 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:46.683+0000 7f8944d35700 1 -- 172.21.15.70:0/1952224464 wait complete. 2024-03-23T08:14:47.902 INFO:teuthology.orchestra.run.smithi070.stdout:Fetching dashboard port number... 2024-03-23T08:14:48.946 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:48 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2124056340' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 8443 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.419+0000 7f5e2fcf2700 1 Processor -- start 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2fcf2700 1 -- start start 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2fcf2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e28076d00 0x7f5e281050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2fcf2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e28105630 con 0x7f5e28076d00 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2da8e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e28076d00 0x7f5e281050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2da8e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e28076d00 0x7f5e281050f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40304/0 (socket says 172.21.15.70:40304) 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2da8e700 1 -- 172.21.15.70:0/1565485795 learned_addr learned my addr 172.21.15.70:0/1565485795 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2da8e700 1 -- 172.21.15.70:0/1565485795 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e28105770 con 0x7f5e28076d00 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2da8e700 1 --2- 172.21.15.70:0/1565485795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e28076d00 0x7f5e281050f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009e50 tx=0x7f5e18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2f4618968bdc73 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:49.179 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2ca8c700 1 -- 172.21.15.70:0/1565485795 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e180092e0 con 0x7f5e28076d00 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2ca8c700 1 -- 172.21.15.70:0/1565485795 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e18012430 con 0x7f5e28076d00 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.420+0000 7f5e2ca8c700 1 -- 172.21.15.70:0/1565485795 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e180075c0 con 0x7f5e28076d00 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.421+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/1565485795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e28076d00 msgr2=0x7f5e281050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.421+0000 7f5e2fcf2700 1 --2- 172.21.15.70:0/1565485795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e28076d00 0x7f5e281050f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009e50 tx=0x7f5e18009840 comp rx=0 tx=0).stop 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.421+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/1565485795 shutdown_connections 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.421+0000 7f5e2fcf2700 1 --2- 172.21.15.70:0/1565485795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e28076d00 0x7f5e281050f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.421+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/1565485795 >> 172.21.15.70:0/1565485795 conn(0x7f5e280fcb20 msgr2=0x7f5e280fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.421+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/1565485795 shutdown_connections 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.421+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/1565485795 wait complete. 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.421+0000 7f5e2fcf2700 1 Processor -- start 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.421+0000 7f5e2fcf2700 1 -- start start 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e2fcf2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e281113c0 0x7f5e281117d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e2fcf2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e28111d10 con 0x7f5e281113c0 2024-03-23T08:14:49.180 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e2da8e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e281113c0 0x7f5e281117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e2da8e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e281113c0 0x7f5e281117d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40318/0 (socket says 172.21.15.70:40318) 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e2da8e700 1 -- 172.21.15.70:0/2124056340 learned_addr learned my addr 172.21.15.70:0/2124056340 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e2da8e700 1 -- 172.21.15.70:0/2124056340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18009ae0 con 0x7f5e281113c0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e2da8e700 1 --2- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e281113c0 0x7f5e281117d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009fd0 tx=0x7f5e18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e267fc700 1 -- 172.21.15.70:0/2124056340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e180092e0 con 0x7f5e281113c0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e28111f10 con 0x7f5e281113c0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e267fc700 1 -- 172.21.15.70:0/2124056340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e18012920 con 0x7f5e281113c0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e267fc700 1 -- 172.21.15.70:0/2124056340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e180258e0 con 0x7f5e281113c0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.422+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e281128f0 con 0x7f5e281113c0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.423+0000 7f5e267fc700 1 -- 172.21.15.70:0/2124056340 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f5e18019050 con 0x7f5e281113c0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.423+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2804bed0 con 0x7f5e281113c0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.423+0000 7f5e267fc700 1 --2- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f5e100367e0 0x7f5e10038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.423+0000 7f5e267fc700 1 -- 172.21.15.70:0/2124056340 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5e1804a200 con 0x7f5e281113c0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.423+0000 7f5e2d28d700 1 --2- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f5e100367e0 0x7f5e10038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:49.181 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.424+0000 7f5e2d28d700 1 --2- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f5e100367e0 0x7f5e10038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009990 tx=0x7f5e1c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.428+0000 7f5e267fc700 1 -- 172.21.15.70:0/2124056340 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5e180236e0 con 0x7f5e281113c0 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.611+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f5e28106550 con 0x7f5e281113c0 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.611+0000 7f5e267fc700 1 -- 172.21.15.70:0/2124056340 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f5e18020030 con 0x7f5e281113c0 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.613+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f5e100367e0 msgr2=0x7f5e10038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.613+0000 7f5e2fcf2700 1 --2- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f5e100367e0 0x7f5e10038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009990 tx=0x7f5e1c006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.613+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e281113c0 msgr2=0x7f5e281117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.613+0000 7f5e2fcf2700 1 --2- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e281113c0 0x7f5e281117d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009fd0 tx=0x7f5e18009840 comp rx=0 tx=0).stop 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.613+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 shutdown_connections 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.613+0000 7f5e2fcf2700 1 --2- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f5e100367e0 0x7f5e10038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.613+0000 7f5e2fcf2700 1 --2- 172.21.15.70:0/2124056340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e281113c0 0x7f5e281117d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.613+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 >> 172.21.15.70:0/2124056340 conn(0x7f5e280fcb20 msgr2=0x7f5e280fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.613+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 shutdown_connections 2024-03-23T08:14:49.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:48.614+0000 7f5e2fcf2700 1 -- 172.21.15.70:0/2124056340 wait complete. 2024-03-23T08:14:49.185 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-23T08:14:49.185 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-03-23T08:14:49.193 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-23T08:14:49.193 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-03-23T08:14:49.193 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-03-23T08:14:49.193 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-23T08:14:49.195 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph Dashboard is now available at: 2024-03-23T08:14:49.195 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:49.195 INFO:teuthology.orchestra.run.smithi070.stdout: URL: https://smithi070.front.sepia.ceph.com:8443/ 2024-03-23T08:14:49.195 INFO:teuthology.orchestra.run.smithi070.stdout: User: admin 2024-03-23T08:14:49.195 INFO:teuthology.orchestra.run.smithi070.stdout: Password: toi6d084gw 2024-03-23T08:14:49.195 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:49.196 INFO:teuthology.orchestra.run.smithi070.stdout:Saving cluster configuration to /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config directory 2024-03-23T08:14:49.196 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling autotune for osd_memory_target 2024-03-23T08:14:49.939 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:49 smithi070 ceph-mon[39699]: mgrmap e13: smithi070.jhsbch(active, since 6s) 2024-03-23T08:14:50.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.720+0000 7f46f3101700 1 Processor -- start 2024-03-23T08:14:50.560 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.720+0000 7f46f3101700 1 -- start start 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.720+0000 7f46f3101700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.720+0000 7f46f3101700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec076d20 con 0x7f46ec1030c0 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.720+0000 7f46f0e9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.720+0000 7f46f0e9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1054a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40330/0 (socket says 172.21.15.70:40330) 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.720+0000 7f46f0e9d700 1 -- 172.21.15.70:0/2359901630 learned_addr learned my addr 172.21.15.70:0/2359901630 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.721+0000 7f46f0e9d700 1 -- 172.21.15.70:0/2359901630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec076e60 con 0x7f46ec1030c0 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.721+0000 7f46f0e9d700 1 --2- 172.21.15.70:0/2359901630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1054a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46e00098e0 tx=0x7f46e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e041c226f95f8acb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.721+0000 7f46eb7fe700 1 -- 172.21.15.70:0/2359901630 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46e000b860 con 0x7f46ec1030c0 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.721+0000 7f46eb7fe700 1 -- 172.21.15.70:0/2359901630 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46e000be70 con 0x7f46ec1030c0 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.721+0000 7f46eb7fe700 1 -- 172.21.15.70:0/2359901630 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46e0010380 con 0x7f46ec1030c0 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 -- 172.21.15.70:0/2359901630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 msgr2=0x7f46ec1054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 --2- 172.21.15.70:0/2359901630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1054a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46e00098e0 tx=0x7f46e0009250 comp rx=0 tx=0).stop 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 -- 172.21.15.70:0/2359901630 shutdown_connections 2024-03-23T08:14:50.561 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 --2- 172.21.15.70:0/2359901630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1054a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 -- 172.21.15.70:0/2359901630 >> 172.21.15.70:0/2359901630 conn(0x7f46ec0fcb20 msgr2=0x7f46ec0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 -- 172.21.15.70:0/2359901630 shutdown_connections 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 -- 172.21.15.70:0/2359901630 wait complete. 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 Processor -- start 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 -- start start 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1113d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.722+0000 7f46f3101700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec111910 con 0x7f46ec1030c0 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46f0e9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46f0e9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1113d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40346/0 (socket says 172.21.15.70:40346) 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46f0e9d700 1 -- 172.21.15.70:0/467975618 learned_addr learned my addr 172.21.15.70:0/467975618 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46f0e9d700 1 -- 172.21.15.70:0/467975618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00094f0 con 0x7f46ec1030c0 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46f0e9d700 1 --2- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1113d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46e00094c0 tx=0x7f46e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46e97fa700 1 -- 172.21.15.70:0/467975618 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46e0010850 con 0x7f46ec1030c0 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46e97fa700 1 -- 172.21.15.70:0/467975618 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46e0010e60 con 0x7f46ec1030c0 2024-03-23T08:14:50.562 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ec111b10 con 0x7f46ec1030c0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46e97fa700 1 -- 172.21.15.70:0/467975618 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46e001a430 con 0x7f46ec1030c0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.723+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec112680 con 0x7f46ec1030c0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.724+0000 7f46e97fa700 1 -- 172.21.15.70:0/467975618 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f46e001a6b0 con 0x7f46ec1030c0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.724+0000 7f46e97fa700 1 --2- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f46dc036880 0x7f46dc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.724+0000 7f46e97fa700 1 -- 172.21.15.70:0/467975618 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f46e004a8d0 con 0x7f46ec1030c0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.724+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ec05f5f0 con 0x7f46ec1030c0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.724+0000 7f46ebfff700 1 --2- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f46dc036880 0x7f46dc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.724+0000 7f46ebfff700 1 --2- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f46dc036880 0x7f46dc038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009990 tx=0x7f46d8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.729+0000 7f46e97fa700 1 -- 172.21.15.70:0/467975618 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f46e00189a0 con 0x7f46ec1030c0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.913+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f46ec04bed0 con 0x7f46ec1030c0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.914+0000 7f46e97fa700 1 -- 172.21.15.70:0/467975618 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f46e0018350 con 0x7f46ec1030c0 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f46dc036880 msgr2=0x7f46dc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 --2- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f46dc036880 0x7f46dc038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009990 tx=0x7f46d8006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 msgr2=0x7f46ec1113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 --2- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1113d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46e00094c0 tx=0x7f46e0009250 comp rx=0 tx=0).stop 2024-03-23T08:14:50.563 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 shutdown_connections 2024-03-23T08:14:50.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 --2- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f46dc036880 0x7f46dc038d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:50.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 --2- 172.21.15.70:0/467975618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46ec1030c0 0x7f46ec1113d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:50.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 >> 172.21.15.70:0/467975618 conn(0x7f46ec0fcb20 msgr2=0x7f46ec0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:50.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 shutdown_connections 2024-03-23T08:14:50.564 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:49.916+0000 7f46f3101700 1 -- 172.21.15.70:0/467975618 wait complete. 2024-03-23T08:14:52.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.066+0000 7f50af4eb700 1 Processor -- start 2024-03-23T08:14:52.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.066+0000 7f50af4eb700 1 -- start start 2024-03-23T08:14:52.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.066+0000 7f50af4eb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.066+0000 7f50af4eb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81056e0 con 0x7f50a8076c30 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.066+0000 7f50ad287700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.067+0000 7f50ad287700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40362/0 (socket says 172.21.15.70:40362) 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.067+0000 7f50ad287700 1 -- 172.21.15.70:0/368572980 learned_addr learned my addr 172.21.15.70:0/368572980 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.067+0000 7f50ad287700 1 -- 172.21.15.70:0/368572980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a8105820 con 0x7f50a8076c30 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.067+0000 7f50ad287700 1 --2- 172.21.15.70:0/368572980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81051a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50980098e0 tx=0x7f5098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f8dcbddee6c820 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.067+0000 7f509ffff700 1 -- 172.21.15.70:0/368572980 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f509800b860 con 0x7f50a8076c30 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.067+0000 7f509ffff700 1 -- 172.21.15.70:0/368572980 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f509800be70 con 0x7f50a8076c30 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.067+0000 7f509ffff700 1 -- 172.21.15.70:0/368572980 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5098010380 con 0x7f50a8076c30 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.068+0000 7f50af4eb700 1 -- 172.21.15.70:0/368572980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 msgr2=0x7f50a81051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.068+0000 7f50af4eb700 1 --2- 172.21.15.70:0/368572980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81051a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50980098e0 tx=0x7f5098009250 comp rx=0 tx=0).stop 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.068+0000 7f50af4eb700 1 -- 172.21.15.70:0/368572980 shutdown_connections 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.068+0000 7f50af4eb700 1 --2- 172.21.15.70:0/368572980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81051a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.068+0000 7f50af4eb700 1 -- 172.21.15.70:0/368572980 >> 172.21.15.70:0/368572980 conn(0x7f50a80fcc30 msgr2=0x7f50a80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.068+0000 7f50af4eb700 1 -- 172.21.15.70:0/368572980 shutdown_connections 2024-03-23T08:14:52.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.068+0000 7f50af4eb700 1 -- 172.21.15.70:0/368572980 wait complete. 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.068+0000 7f50af4eb700 1 Processor -- start 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.068+0000 7f50af4eb700 1 -- start start 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f50af4eb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81113b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f50af4eb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81118f0 con 0x7f50a8076c30 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f50ad287700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f50ad287700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81113b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40370/0 (socket says 172.21.15.70:40370) 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f50ad287700 1 -- 172.21.15.70:0/3708294154 learned_addr learned my addr 172.21.15.70:0/3708294154 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f50ad287700 1 -- 172.21.15.70:0/3708294154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50980094f0 con 0x7f50a8076c30 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f50ad287700 1 --2- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81113b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5098006d40 tx=0x7f5098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f509dffb700 1 -- 172.21.15.70:0/3708294154 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5098010960 con 0x7f50a8076c30 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f509dffb700 1 -- 172.21.15.70:0/3708294154 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f509801a430 con 0x7f50a8076c30 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a8111af0 con 0x7f50a8076c30 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f509dffb700 1 -- 172.21.15.70:0/3708294154 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50980245b0 con 0x7f50a8076c30 2024-03-23T08:14:52.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.069+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a8112660 con 0x7f50a8076c30 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.070+0000 7f509dffb700 1 -- 172.21.15.70:0/3708294154 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f509802d3f0 con 0x7f50a8076c30 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.070+0000 7f509dffb700 1 --2- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f50940368d0 0x7f5094038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.070+0000 7f509dffb700 1 -- 172.21.15.70:0/3708294154 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f509804a530 con 0x7f50a8076c30 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.070+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a805f5f0 con 0x7f50a8076c30 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.070+0000 7f50aca86700 1 --2- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f50940368d0 0x7f5094038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.071+0000 7f50aca86700 1 --2- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f50940368d0 0x7f5094038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009990 tx=0x7f50a4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.075+0000 7f509dffb700 1 -- 172.21.15.70:0/3708294154 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f509802b8c0 con 0x7f50a8076c30 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.358+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f50a8106540 con 0x7f50a8076c30 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.361+0000 7f509dffb700 1 -- 172.21.15.70:0/3708294154 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f509802b2f0 con 0x7f50a8076c30 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.362+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f50940368d0 msgr2=0x7f5094038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.362+0000 7f50af4eb700 1 --2- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f50940368d0 0x7f5094038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009990 tx=0x7f50a4006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.362+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 msgr2=0x7f50a81113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.362+0000 7f50af4eb700 1 --2- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81113b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5098006d40 tx=0x7f5098009250 comp rx=0 tx=0).stop 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.363+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 shutdown_connections 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.363+0000 7f50af4eb700 1 --2- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f50940368d0 0x7f5094038d80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.363+0000 7f50af4eb700 1 --2- 172.21.15.70:0/3708294154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50a8076c30 0x7f50a81113b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.363+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 >> 172.21.15.70:0/3708294154 conn(0x7f50a80fcc30 msgr2=0x7f50a80fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.363+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 shutdown_connections 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-03-23T08:14:51.363+0000 7f50af4eb700 1 -- 172.21.15.70:0/3708294154 wait complete. 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout:Or, if you are only running a single cluster on this host: 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: ceph telemetry on 2024-03-23T08:14:52.020 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:52.021 INFO:teuthology.orchestra.run.smithi070.stdout:For more information see: 2024-03-23T08:14:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-23T08:14:52.021 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:14:52.021 INFO:teuthology.orchestra.run.smithi070.stdout:Bootstrap complete. 2024-03-23T08:14:52.048 INFO:tasks.cephadm:Fetching config... 2024-03-23T08:14:52.048 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:14:52.048 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-23T08:14:52.065 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-23T08:14:52.065 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:14:52.065 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-23T08:14:52.123 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-23T08:14:52.123 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:14:52.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/keyring of=/dev/stdout 2024-03-23T08:14:52.191 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-23T08:14:52.191 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:14:52.191 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-23T08:14:52.249 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-23T08:14:52.249 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ5BcJEIdUPCuMj1sWw+F0JrjnuDJpkaU3bRcgrOFpSpziJCJCh1ZKYfFil6zXvCwZNVGBE0VlK5zQIgYP7gomKbgD7OpokajMdkh5J6o7CWSZf4Ehk/d6wDFv8Q4fdmN83NJdJ07xDEHaT+yKmB/Pvs8hUBSDedfBBUpvwpnK/Kd5KsP0sBn3FRTZPal3z/5IjQB93iaZb2vWjSh3RHuN6J5RvelP2VCgnlRUZ8o55tXc2VP9O4zNNmxt9PFPzYp2AZXNfjwd8v65sn42a3gWyygZrLdESi/RCBT3RcSYQDKQmReEcrP+2kA7xU5vF6rC01qApgz7Te/WV/y5uPkflHo0SzUmfiO2aiMqOdwl/5fCZZjsKfDw0zRMkhOwTADjtrqtfj0UDjChKSh135OzYEKl6QHcrWD+RopmqlsE6rOPScgEUCD+BPT5zGPWdSOXRvn+lPGcacQMgiaOJkIvZO7YwaDOy5LmBKuCBuvEWnWxsYl14fRI3QqdWecbmCk= ceph-2762dd54-e8ed-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T08:14:52.325 INFO:teuthology.orchestra.run.smithi070.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ5BcJEIdUPCuMj1sWw+F0JrjnuDJpkaU3bRcgrOFpSpziJCJCh1ZKYfFil6zXvCwZNVGBE0VlK5zQIgYP7gomKbgD7OpokajMdkh5J6o7CWSZf4Ehk/d6wDFv8Q4fdmN83NJdJ07xDEHaT+yKmB/Pvs8hUBSDedfBBUpvwpnK/Kd5KsP0sBn3FRTZPal3z/5IjQB93iaZb2vWjSh3RHuN6J5RvelP2VCgnlRUZ8o55tXc2VP9O4zNNmxt9PFPzYp2AZXNfjwd8v65sn42a3gWyygZrLdESi/RCBT3RcSYQDKQmReEcrP+2kA7xU5vF6rC01qApgz7Te/WV/y5uPkflHo0SzUmfiO2aiMqOdwl/5fCZZjsKfDw0zRMkhOwTADjtrqtfj0UDjChKSh135OzYEKl6QHcrWD+RopmqlsE6rOPScgEUCD+BPT5zGPWdSOXRvn+lPGcacQMgiaOJkIvZO7YwaDOy5LmBKuCBuvEWnWxsYl14fRI3QqdWecbmCk= ceph-2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:14:52.336 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ5BcJEIdUPCuMj1sWw+F0JrjnuDJpkaU3bRcgrOFpSpziJCJCh1ZKYfFil6zXvCwZNVGBE0VlK5zQIgYP7gomKbgD7OpokajMdkh5J6o7CWSZf4Ehk/d6wDFv8Q4fdmN83NJdJ07xDEHaT+yKmB/Pvs8hUBSDedfBBUpvwpnK/Kd5KsP0sBn3FRTZPal3z/5IjQB93iaZb2vWjSh3RHuN6J5RvelP2VCgnlRUZ8o55tXc2VP9O4zNNmxt9PFPzYp2AZXNfjwd8v65sn42a3gWyygZrLdESi/RCBT3RcSYQDKQmReEcrP+2kA7xU5vF6rC01qApgz7Te/WV/y5uPkflHo0SzUmfiO2aiMqOdwl/5fCZZjsKfDw0zRMkhOwTADjtrqtfj0UDjChKSh135OzYEKl6QHcrWD+RopmqlsE6rOPScgEUCD+BPT5zGPWdSOXRvn+lPGcacQMgiaOJkIvZO7YwaDOy5LmBKuCBuvEWnWxsYl14fRI3QqdWecbmCk= ceph-2762dd54-e8ed-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-23T08:14:52.379 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ5BcJEIdUPCuMj1sWw+F0JrjnuDJpkaU3bRcgrOFpSpziJCJCh1ZKYfFil6zXvCwZNVGBE0VlK5zQIgYP7gomKbgD7OpokajMdkh5J6o7CWSZf4Ehk/d6wDFv8Q4fdmN83NJdJ07xDEHaT+yKmB/Pvs8hUBSDedfBBUpvwpnK/Kd5KsP0sBn3FRTZPal3z/5IjQB93iaZb2vWjSh3RHuN6J5RvelP2VCgnlRUZ8o55tXc2VP9O4zNNmxt9PFPzYp2AZXNfjwd8v65sn42a3gWyygZrLdESi/RCBT3RcSYQDKQmReEcrP+2kA7xU5vF6rC01qApgz7Te/WV/y5uPkflHo0SzUmfiO2aiMqOdwl/5fCZZjsKfDw0zRMkhOwTADjtrqtfj0UDjChKSh135OzYEKl6QHcrWD+RopmqlsE6rOPScgEUCD+BPT5zGPWdSOXRvn+lPGcacQMgiaOJkIvZO7YwaDOy5LmBKuCBuvEWnWxsYl14fRI3QqdWecbmCk= ceph-2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:14:52.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-23T08:14:52.655 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:14:52.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:52 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3708294154' entity='client.admin' 2024-03-23T08:14:55.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.145+0000 7f27010db700 1 -- 172.21.15.70:0/2144718689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 msgr2=0x7f26fc0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:55.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.145+0000 7f27010db700 1 --2- 172.21.15.70:0/2144718689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 0x7f26fc0fce70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0048c0 tx=0x7f26ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:14:55.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.145+0000 7f27010db700 1 -- 172.21.15.70:0/2144718689 shutdown_connections 2024-03-23T08:14:55.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.145+0000 7f27010db700 1 --2- 172.21.15.70:0/2144718689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 0x7f26fc0fce70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:55.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.145+0000 7f27010db700 1 -- 172.21.15.70:0/2144718689 >> 172.21.15.70:0/2144718689 conn(0x7f26fc0f81c0 msgr2=0x7f26fc0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:55.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.145+0000 7f27010db700 1 -- 172.21.15.70:0/2144718689 shutdown_connections 2024-03-23T08:14:55.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.145+0000 7f27010db700 1 -- 172.21.15.70:0/2144718689 wait complete. 2024-03-23T08:14:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.146+0000 7f27010db700 1 Processor -- start 2024-03-23T08:14:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.146+0000 7f27010db700 1 -- start start 2024-03-23T08:14:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.146+0000 7f27010db700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 0x7f26fc10cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.146+0000 7f27010db700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc10d530 con 0x7f26fc0fca60 2024-03-23T08:14:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.146+0000 7f26fad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 0x7f26fc10cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.146+0000 7f26fad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 0x7f26fc10cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40392/0 (socket says 172.21.15.70:40392) 2024-03-23T08:14:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.146+0000 7f26fad9d700 1 -- 172.21.15.70:0/947211566 learned_addr learned my addr 172.21.15.70:0/947211566 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.146+0000 7f26fad9d700 1 -- 172.21.15.70:0/947211566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec005040 con 0x7f26fc0fca60 2024-03-23T08:14:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.146+0000 7f26fad9d700 1 --2- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 0x7f26fc10cff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009300 tx=0x7f26ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.147+0000 7f26f37fe700 1 -- 172.21.15.70:0/947211566 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26ec009600 con 0x7f26fc0fca60 2024-03-23T08:14:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.147+0000 7f26f37fe700 1 -- 172.21.15.70:0/947211566 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26ec009760 con 0x7f26fc0fca60 2024-03-23T08:14:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.147+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc10d730 con 0x7f26fc0fca60 2024-03-23T08:14:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.147+0000 7f26f37fe700 1 -- 172.21.15.70:0/947211566 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26ec01c5b0 con 0x7f26fc0fca60 2024-03-23T08:14:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.147+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc10e2a0 con 0x7f26fc0fca60 2024-03-23T08:14:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.147+0000 7f26f37fe700 1 -- 172.21.15.70:0/947211566 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f26ec01c870 con 0x7f26fc0fca60 2024-03-23T08:14:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.148+0000 7f26f37fe700 1 --2- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f26e8036880 0x7f26e8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.148+0000 7f26f37fe700 1 -- 172.21.15.70:0/947211566 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f26ec049a80 con 0x7f26fc0fca60 2024-03-23T08:14:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.148+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc102e30 con 0x7f26fc0fca60 2024-03-23T08:14:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.148+0000 7f26fa59c700 1 --2- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f26e8036880 0x7f26e8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.148+0000 7f26fa59c700 1 --2- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f26e8036880 0x7f26e8038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009990 tx=0x7f26e4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:55.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.152+0000 7f26f37fe700 1 -- 172.21.15.70:0/947211566 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f26ec011db0 con 0x7f26fc0fca60 2024-03-23T08:14:55.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.342+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f26fc05f5f0 con 0x7f26fc0fca60 2024-03-23T08:14:55.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.345+0000 7f26f37fe700 1 -- 172.21.15.70:0/947211566 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f26ec023990 con 0x7f26fc0fca60 2024-03-23T08:14:55.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f26e8036880 msgr2=0x7f26e8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:55.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 --2- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f26e8036880 0x7f26e8038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009990 tx=0x7f26e4006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:55.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 msgr2=0x7f26fc10cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:55.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 --2- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 0x7f26fc10cff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009300 tx=0x7f26ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:14:55.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 shutdown_connections 2024-03-23T08:14:55.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 --2- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f26e8036880 0x7f26e8038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:55.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 --2- 172.21.15.70:0/947211566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26fc0fca60 0x7f26fc10cff0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:55.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 >> 172.21.15.70:0/947211566 conn(0x7f26fc0f81c0 msgr2=0x7f26fc0f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:55.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 shutdown_connections 2024-03-23T08:14:55.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:55.347+0000 7f27010db700 1 -- 172.21.15.70:0/947211566 wait complete. 2024-03-23T08:14:55.468 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:55 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:55.468 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:55 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:55.469 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:55 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:14:55.469 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:55 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:14:55.469 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:55 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:14:55.469 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:55 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-23T08:14:55.469 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:55 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:14:55.469 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:55 smithi070 ceph-mon[39699]: Deploying daemon crash.smithi070 on smithi070 2024-03-23T08:14:56.023 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-23T08:14:56.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-23T08:14:56.318 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:14:56.609 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:14:56 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/947211566' entity='client.admin' 2024-03-23T08:14:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.164+0000 7f8fcda0e700 1 -- 172.21.15.70:0/3460387283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 msgr2=0x7f8fc80fbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.164+0000 7f8fcda0e700 1 --2- 172.21.15.70:0/3460387283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 0x7f8fc80fbd60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0048c0 tx=0x7f8fbc009e30 comp rx=0 tx=0).stop 2024-03-23T08:14:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.164+0000 7f8fcda0e700 1 -- 172.21.15.70:0/3460387283 shutdown_connections 2024-03-23T08:14:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.164+0000 7f8fcda0e700 1 --2- 172.21.15.70:0/3460387283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 0x7f8fc80fbd60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.164+0000 7f8fcda0e700 1 -- 172.21.15.70:0/3460387283 >> 172.21.15.70:0/3460387283 conn(0x7f8fc80f7090 msgr2=0x7f8fc80f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.164+0000 7f8fcda0e700 1 -- 172.21.15.70:0/3460387283 shutdown_connections 2024-03-23T08:14:59.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.164+0000 7f8fcda0e700 1 -- 172.21.15.70:0/3460387283 wait complete. 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fcda0e700 1 Processor -- start 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fcda0e700 1 -- start start 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fcda0e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 0x7f8fc810bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fcda0e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc810c400 con 0x7f8fc80fb950 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fc6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 0x7f8fc810bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fc6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 0x7f8fc810bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53862/0 (socket says 172.21.15.70:53862) 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fc6ffd700 1 -- 172.21.15.70:0/1195570286 learned_addr learned my addr 172.21.15.70:0/1195570286 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fc6ffd700 1 -- 172.21.15.70:0/1195570286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc005040 con 0x7f8fc80fb950 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fc6ffd700 1 --2- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 0x7f8fc810bec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009300 tx=0x7f8fbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8faf7fe700 1 -- 172.21.15.70:0/1195570286 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8fbc009600 con 0x7f8fc80fb950 2024-03-23T08:14:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc810c600 con 0x7f8fc80fb950 2024-03-23T08:14:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8faf7fe700 1 -- 172.21.15.70:0/1195570286 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8fbc009760 con 0x7f8fc80fb950 2024-03-23T08:14:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.165+0000 7f8faf7fe700 1 -- 172.21.15.70:0/1195570286 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8fbc01c5b0 con 0x7f8fc80fb950 2024-03-23T08:14:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.166+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc810d170 con 0x7f8fc80fb950 2024-03-23T08:14:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.166+0000 7f8faf7fe700 1 -- 172.21.15.70:0/1195570286 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f8fbc01c870 con 0x7f8fc80fb950 2024-03-23T08:14:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.166+0000 7f8faf7fe700 1 --2- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8fb0036880 0x7f8fb0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:14:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.166+0000 7f8faf7fe700 1 -- 172.21.15.70:0/1195570286 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8fbc049aa0 con 0x7f8fc80fb950 2024-03-23T08:14:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.167+0000 7f8fc67fc700 1 --2- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8fb0036880 0x7f8fb0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:14:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.167+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc8101d20 con 0x7f8fc80fb950 2024-03-23T08:14:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.167+0000 7f8fc67fc700 1 --2- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8fb0036880 0x7f8fb0038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009990 tx=0x7f8fb8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:14:59.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.171+0000 7f8faf7fe700 1 -- 172.21.15.70:0/1195570286 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8fbc011db0 con 0x7f8fc80fb950 2024-03-23T08:14:59.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.367+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 --> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fc8101700 con 0x7f8fb0036880 2024-03-23T08:14:59.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.371+0000 7f8faf7fe700 1 -- 172.21.15.70:0/1195570286 <== mgr.14160 v2:172.21.15.70:6800/2732684403 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8fb8009590 con 0x7f8fb0036880 2024-03-23T08:14:59.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.373+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8fb0036880 msgr2=0x7f8fb0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.373+0000 7f8fcda0e700 1 --2- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8fb0036880 0x7f8fb0038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009990 tx=0x7f8fb8006e40 comp rx=0 tx=0).stop 2024-03-23T08:14:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.373+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 msgr2=0x7f8fc810bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:14:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.373+0000 7f8fcda0e700 1 --2- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 0x7f8fc810bec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009300 tx=0x7f8fbc009e30 comp rx=0 tx=0).stop 2024-03-23T08:14:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.373+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 shutdown_connections 2024-03-23T08:14:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.373+0000 7f8fcda0e700 1 --2- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8fb0036880 0x7f8fb0038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.373+0000 7f8fcda0e700 1 --2- 172.21.15.70:0/1195570286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc80fb950 0x7f8fc810bec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:14:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.373+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 >> 172.21.15.70:0/1195570286 conn(0x7f8fc80f7090 msgr2=0x7f8fc80f7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:14:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.374+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 shutdown_connections 2024-03-23T08:14:59.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:14:59.374+0000 7f8fcda0e700 1 -- 172.21.15.70:0/1195570286 wait complete. 2024-03-23T08:15:00.220 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-03-23T08:15:00.221 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:15:00.221 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-03-23T08:15:00.242 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:15:00.242 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:15:00.303 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-03-23T08:15:00.303 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch host add smithi193 2024-03-23T08:15:00.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:00 smithi070 ceph-mon[39699]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:15:00.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:00 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:00.841 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:15:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:01 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:01.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:01 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:01.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:01 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:01.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:01 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:01.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:01 smithi070 ceph-mon[39699]: Deploying daemon node-exporter.smithi070 on smithi070 2024-03-23T08:15:02.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.499+0000 7fef93fdf700 1 -- 172.21.15.70:0/2316458601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c0fbf70 msgr2=0x7fef8c0fc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:02.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.499+0000 7fef93fdf700 1 --2- 172.21.15.70:0/2316458601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c0fbf70 0x7fef8c0fc380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0048c0 tx=0x7fef7c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.499+0000 7fef93fdf700 1 -- 172.21.15.70:0/2316458601 shutdown_connections 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.499+0000 7fef93fdf700 1 --2- 172.21.15.70:0/2316458601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c0fbf70 0x7fef8c0fc380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.499+0000 7fef93fdf700 1 -- 172.21.15.70:0/2316458601 >> 172.21.15.70:0/2316458601 conn(0x7fef8c0f6820 msgr2=0x7fef8c0f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.499+0000 7fef93fdf700 1 -- 172.21.15.70:0/2316458601 shutdown_connections 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.499+0000 7fef93fdf700 1 -- 172.21.15.70:0/2316458601 wait complete. 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef93fdf700 1 Processor -- start 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef93fdf700 1 -- start start 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef93fdf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c10c010 0x7fef8c10c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef93fdf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c10c960 con 0x7fef8c10c010 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef91d7b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c10c010 0x7fef8c10c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef91d7b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c10c010 0x7fef8c10c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53900/0 (socket says 172.21.15.70:53900) 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef91d7b700 1 -- 172.21.15.70:0/2156090125 learned_addr learned my addr 172.21.15.70:0/2156090125 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef91d7b700 1 -- 172.21.15.70:0/2156090125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c005040 con 0x7fef8c10c010 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef91d7b700 1 --2- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c10c010 0x7fef8c10c420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009300 tx=0x7fef7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:02.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef827fc700 1 -- 172.21.15.70:0/2156090125 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef7c004e60 con 0x7fef8c10c010 2024-03-23T08:15:02.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef827fc700 1 -- 172.21.15.70:0/2156090125 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef7c013420 con 0x7fef8c10c010 2024-03-23T08:15:02.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef827fc700 1 -- 172.21.15.70:0/2156090125 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef7c01c5b0 con 0x7fef8c10c010 2024-03-23T08:15:02.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c10cb60 con 0x7fef8c10c010 2024-03-23T08:15:02.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.500+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c10d540 con 0x7fef8c10c010 2024-03-23T08:15:02.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.501+0000 7fef827fc700 1 -- 172.21.15.70:0/2156090125 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fef7c01c710 con 0x7fef8c10c010 2024-03-23T08:15:02.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.501+0000 7fef827fc700 1 --2- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef780368d0 0x7fef78038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:02.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.501+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c05f5f0 con 0x7fef8c10c010 2024-03-23T08:15:02.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.501+0000 7fef827fc700 1 -- 172.21.15.70:0/2156090125 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fef7c049800 con 0x7fef8c10c010 2024-03-23T08:15:02.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.501+0000 7fef9157a700 1 --2- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef780368d0 0x7fef78038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:02.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.502+0000 7fef9157a700 1 --2- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef780368d0 0x7fef78038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef88009990 tx=0x7fef88006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:02.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.506+0000 7fef827fc700 1 -- 172.21.15.70:0/2156090125 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fef7c024b50 con 0x7fef8c10c010 2024-03-23T08:15:02.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:02.696+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 --> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) v1 -- 0x7fef8c10e060 con 0x7fef780368d0 2024-03-23T08:15:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:03 smithi070 ceph-mon[39699]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:03 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:03 smithi070 ceph-mon[39699]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:15:04.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.430+0000 7fef827fc700 1 -- 172.21.15.70:0/2156090125 <== mgr.14160 v2:172.21.15.70:6800/2732684403 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fef8c10e060 con 0x7fef780368d0 2024-03-23T08:15:04.430 INFO:teuthology.orchestra.run.smithi070.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-03-23T08:15:04.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.432+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef780368d0 msgr2=0x7fef78038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:04.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.432+0000 7fef93fdf700 1 --2- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef780368d0 0x7fef78038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef88009990 tx=0x7fef88006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:04.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.432+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c10c010 msgr2=0x7fef8c10c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:04.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.432+0000 7fef93fdf700 1 --2- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c10c010 0x7fef8c10c420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009300 tx=0x7fef7c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:04.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.432+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 shutdown_connections 2024-03-23T08:15:04.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.432+0000 7fef93fdf700 1 --2- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef780368d0 0x7fef78038d80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:04.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.432+0000 7fef93fdf700 1 --2- 172.21.15.70:0/2156090125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef8c10c010 0x7fef8c10c420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:04.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.432+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 >> 172.21.15.70:0/2156090125 conn(0x7fef8c0f6820 msgr2=0x7fef8c0f70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:04.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.433+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 shutdown_connections 2024-03-23T08:15:04.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:04.433+0000 7fef93fdf700 1 -- 172.21.15.70:0/2156090125 wait complete. 2024-03-23T08:15:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:04 smithi070 ceph-mon[39699]: Deploying cephadm binary to smithi193 2024-03-23T08:15:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:04 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:05.268 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-23T08:15:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:15:05.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:05 smithi070 ceph-mon[39699]: Added host smithi193 2024-03-23T08:15:05.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:05 smithi070 ceph-mon[39699]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:07.154 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:06 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:07.155 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:06 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:07.155 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:06 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:07.155 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:06 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:07.155 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:06 smithi070 ceph-mon[39699]: Deploying daemon alertmanager.smithi070 on smithi070 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.833+0000 7fd041bec700 1 -- 172.21.15.70:0/1361514521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 msgr2=0x7fd03c0fbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.833+0000 7fd041bec700 1 --2- 172.21.15.70:0/1361514521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 0x7fd03c0fbd60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0048c0 tx=0x7fd02c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.833+0000 7fd041bec700 1 -- 172.21.15.70:0/1361514521 shutdown_connections 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.833+0000 7fd041bec700 1 --2- 172.21.15.70:0/1361514521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 0x7fd03c0fbd60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.833+0000 7fd041bec700 1 -- 172.21.15.70:0/1361514521 >> 172.21.15.70:0/1361514521 conn(0x7fd03c0f7090 msgr2=0x7fd03c0f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.833+0000 7fd041bec700 1 -- 172.21.15.70:0/1361514521 shutdown_connections 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.833+0000 7fd041bec700 1 -- 172.21.15.70:0/1361514521 wait complete. 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.833+0000 7fd041bec700 1 Processor -- start 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd041bec700 1 -- start start 2024-03-23T08:15:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd041bec700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 0x7fd03c066840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd041bec700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c10fd20 con 0x7fd03c0fb950 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd03b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 0x7fd03c066840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd03b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 0x7fd03c066840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53914/0 (socket says 172.21.15.70:53914) 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd03b7fe700 1 -- 172.21.15.70:0/406482685 learned_addr learned my addr 172.21.15.70:0/406482685 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd03b7fe700 1 -- 172.21.15.70:0/406482685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c005040 con 0x7fd03c0fb950 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd03b7fe700 1 --2- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 0x7fd03c066840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd02c000c00 tx=0x7fd02c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd023fff700 1 -- 172.21.15.70:0/406482685 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd02c009620 con 0x7fd03c0fb950 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c10fe60 con 0x7fd03c0fb950 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd023fff700 1 -- 172.21.15.70:0/406482685 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd02c009780 con 0x7fd03c0fb950 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd023fff700 1 -- 172.21.15.70:0/406482685 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd02c01c5b0 con 0x7fd03c0fb950 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.834+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c110350 con 0x7fd03c0fb950 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.835+0000 7fd023fff700 1 -- 172.21.15.70:0/406482685 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fd02c01c7f0 con 0x7fd03c0fb950 2024-03-23T08:15:07.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.835+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c105760 con 0x7fd03c0fb950 2024-03-23T08:15:07.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.835+0000 7fd023fff700 1 --2- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fd024036880 0x7fd024038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:07.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.836+0000 7fd023fff700 1 -- 172.21.15.70:0/406482685 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd02c0499f0 con 0x7fd03c0fb950 2024-03-23T08:15:07.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.836+0000 7fd03affd700 1 --2- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fd024036880 0x7fd024038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:07.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.836+0000 7fd03affd700 1 --2- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fd024036880 0x7fd024038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd030009990 tx=0x7fd030006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:07.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:07.841+0000 7fd023fff700 1 -- 172.21.15.70:0/406482685 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd02c025920 con 0x7fd03c0fb950 2024-03-23T08:15:08.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.034+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 --> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd03c05e400 con 0x7fd024036880 2024-03-23T08:15:08.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.035+0000 7fd023fff700 1 -- 172.21.15.70:0/406482685 <== mgr.14160 v2:172.21.15.70:6800/2732684403 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fd03c05e400 con 0x7fd024036880 2024-03-23T08:15:08.035 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:15:08.035 INFO:teuthology.orchestra.run.smithi070.stdout:[{"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-03-23T08:15:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fd024036880 msgr2=0x7fd024038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 --2- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fd024036880 0x7fd024038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd030009990 tx=0x7fd030006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 msgr2=0x7fd03c066840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 --2- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 0x7fd03c066840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd02c000c00 tx=0x7fd02c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 shutdown_connections 2024-03-23T08:15:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 --2- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fd024036880 0x7fd024038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 --2- 172.21.15.70:0/406482685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd03c0fb950 0x7fd03c066840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 >> 172.21.15.70:0/406482685 conn(0x7fd03c0f7090 msgr2=0x7fd03c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 shutdown_connections 2024-03-23T08:15:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:08.037+0000 7fd041bec700 1 -- 172.21.15.70:0/406482685 wait complete. 2024-03-23T08:15:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:07 smithi070 ceph-mon[39699]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:07 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:08 smithi070 ceph-mon[39699]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T08:15:09.625 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-23T08:15:09.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-23T08:15:10.182 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:15:10.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:09 smithi070 ceph-mon[39699]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:12.203 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:11 smithi070 ceph-mon[39699]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:12.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.396+0000 7f3b33df9700 1 -- 172.21.15.70:0/3562568781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 msgr2=0x7f3b2c0fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:12.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.396+0000 7f3b33df9700 1 --2- 172.21.15.70:0/3562568781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 0x7f3b2c0fce60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0048c0 tx=0x7f3b1c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:12.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.396+0000 7f3b33df9700 1 -- 172.21.15.70:0/3562568781 shutdown_connections 2024-03-23T08:15:12.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.396+0000 7f3b33df9700 1 --2- 172.21.15.70:0/3562568781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 0x7f3b2c0fce60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:12.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.396+0000 7f3b33df9700 1 -- 172.21.15.70:0/3562568781 >> 172.21.15.70:0/3562568781 conn(0x7f3b2c0f8190 msgr2=0x7f3b2c0fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:12.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.396+0000 7f3b33df9700 1 -- 172.21.15.70:0/3562568781 shutdown_connections 2024-03-23T08:15:12.405 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.396+0000 7f3b33df9700 1 -- 172.21.15.70:0/3562568781 wait complete. 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b33df9700 1 Processor -- start 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b33df9700 1 -- start start 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b33df9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 0x7f3b2c10cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b33df9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c10d4e0 con 0x7f3b2c0fca50 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b31b95700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 0x7f3b2c10cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b31b95700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 0x7f3b2c10cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52476/0 (socket says 172.21.15.70:52476) 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b31b95700 1 -- 172.21.15.70:0/3360015604 learned_addr learned my addr 172.21.15.70:0/3360015604 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b31b95700 1 -- 172.21.15.70:0/3360015604 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c005040 con 0x7f3b2c0fca50 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b31b95700 1 --2- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 0x7f3b2c10cfa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009300 tx=0x7f3b1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.397+0000 7f3b227fc700 1 -- 172.21.15.70:0/3360015604 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b1c009600 con 0x7f3b2c0fca50 2024-03-23T08:15:12.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.398+0000 7f3b227fc700 1 -- 172.21.15.70:0/3360015604 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b1c009760 con 0x7f3b2c0fca50 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.398+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c10d6e0 con 0x7f3b2c0fca50 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.398+0000 7f3b227fc700 1 -- 172.21.15.70:0/3360015604 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b1c01c5b0 con 0x7f3b2c0fca50 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.398+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c10e250 con 0x7f3b2c0fca50 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.398+0000 7f3b227fc700 1 -- 172.21.15.70:0/3360015604 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f3b1c013420 con 0x7f3b2c0fca50 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.398+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2c102e20 con 0x7f3b2c0fca50 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.398+0000 7f3b227fc700 1 --2- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f3b18036880 0x7f3b18038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.399+0000 7f3b227fc700 1 -- 172.21.15.70:0/3360015604 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3b1c049aa0 con 0x7f3b2c0fca50 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.399+0000 7f3b31394700 1 --2- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f3b18036880 0x7f3b18038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.399+0000 7f3b31394700 1 --2- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f3b18036880 0x7f3b18038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009990 tx=0x7f3b28006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:12.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.403+0000 7f3b227fc700 1 -- 172.21.15.70:0/3360015604 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3b1c013960 con 0x7f3b2c0fca50 2024-03-23T08:15:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.594+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3b2c10ec60 con 0x7f3b2c0fca50 2024-03-23T08:15:12.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.949+0000 7f3b227fc700 1 -- 172.21.15.70:0/3360015604 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3b1c011370 con 0x7f3b2c0fca50 2024-03-23T08:15:12.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.951+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f3b18036880 msgr2=0x7f3b18038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:12.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.951+0000 7f3b33df9700 1 --2- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f3b18036880 0x7f3b18038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009990 tx=0x7f3b28006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:12.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.951+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 msgr2=0x7f3b2c10cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:12.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.951+0000 7f3b33df9700 1 --2- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 0x7f3b2c10cfa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009300 tx=0x7f3b1c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:12.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.951+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 shutdown_connections 2024-03-23T08:15:12.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.951+0000 7f3b33df9700 1 --2- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f3b18036880 0x7f3b18038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:12.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.951+0000 7f3b33df9700 1 --2- 172.21.15.70:0/3360015604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b2c0fca50 0x7f3b2c10cfa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:12.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.951+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 >> 172.21.15.70:0/3360015604 conn(0x7f3b2c0f8190 msgr2=0x7f3b2c0f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:12.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.952+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 shutdown_connections 2024-03-23T08:15:12.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:15:12.952+0000 7f3b33df9700 1 -- 172.21.15.70:0/3360015604 wait complete. 2024-03-23T08:15:12.953 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2024-03-23T08:15:13.180 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:12 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3360015604' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T08:15:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:13 smithi070 ceph-mon[39699]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:13 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3360015604' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T08:15:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:13 smithi070 ceph-mon[39699]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T08:15:14.311 INFO:tasks.cephadm:Adding mon.smithi070 on smithi070 2024-03-23T08:15:14.311 INFO:tasks.cephadm:Adding mon.smithi193 on smithi193 2024-03-23T08:15:14.311 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch apply mon '2;smithi070:172.21.15.70=smithi070;smithi193:172.21.15.193=smithi193' 2024-03-23T08:15:14.604 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:14.666 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:16.218 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.216+0000 7f75940ad700 1 -- 172.21.15.193:0/2407355567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 msgr2=0x7f758c0fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:16.218 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.216+0000 7f75940ad700 1 --2- 172.21.15.193:0/2407355567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 0x7f758c0fce50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f757c0048c0 tx=0x7f757c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:16.218 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.216+0000 7f75940ad700 1 -- 172.21.15.193:0/2407355567 shutdown_connections 2024-03-23T08:15:16.218 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.216+0000 7f75940ad700 1 --2- 172.21.15.193:0/2407355567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 0x7f758c0fce50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:16.218 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.216+0000 7f75940ad700 1 -- 172.21.15.193:0/2407355567 >> 172.21.15.193:0/2407355567 conn(0x7f758c0f81c0 msgr2=0x7f758c0fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:16.218 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.216+0000 7f75940ad700 1 -- 172.21.15.193:0/2407355567 shutdown_connections 2024-03-23T08:15:16.218 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.216+0000 7f75940ad700 1 -- 172.21.15.193:0/2407355567 wait complete. 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.217+0000 7f75940ad700 1 Processor -- start 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.217+0000 7f75940ad700 1 -- start start 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.217+0000 7f75940ad700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 0x7f758c10cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.217+0000 7f75940ad700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c10d4f0 con 0x7f758c0fca40 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.217+0000 7f7591e49700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 0x7f758c10cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.217+0000 7f7591e49700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 0x7f758c10cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:60726/0 (socket says 172.21.15.193:60726) 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.217+0000 7f7591e49700 1 -- 172.21.15.193:0/1233140802 learned_addr learned my addr 172.21.15.193:0/1233140802 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.217+0000 7f7591e49700 1 -- 172.21.15.193:0/1233140802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c005040 con 0x7f758c0fca40 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.218+0000 7f7591e49700 1 --2- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 0x7f758c10cfb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f757c000c00 tx=0x7f757c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.218+0000 7f75827fc700 1 -- 172.21.15.193:0/1233140802 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f757c009620 con 0x7f758c0fca40 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.218+0000 7f75827fc700 1 -- 172.21.15.193:0/1233140802 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f757c009780 con 0x7f758c0fca40 2024-03-23T08:15:16.219 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.218+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c10d6f0 con 0x7f758c0fca40 2024-03-23T08:15:16.220 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.218+0000 7f75827fc700 1 -- 172.21.15.193:0/1233140802 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f757c01c5b0 con 0x7f758c0fca40 2024-03-23T08:15:16.220 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.218+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c10e260 con 0x7f758c0fca40 2024-03-23T08:15:16.221 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.219+0000 7f75827fc700 1 -- 172.21.15.193:0/1233140802 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f757c01c7a0 con 0x7f758c0fca40 2024-03-23T08:15:16.221 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.219+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c102e10 con 0x7f758c0fca40 2024-03-23T08:15:16.221 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.219+0000 7f75827fc700 1 --2- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f75780368d0 0x7f7578038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:16.221 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.219+0000 7f75827fc700 1 -- 172.21.15.193:0/1233140802 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f757c049d00 con 0x7f758c0fca40 2024-03-23T08:15:16.221 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.219+0000 7f7591648700 1 --2- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f75780368d0 0x7f7578038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:16.221 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.219+0000 7f7591648700 1 --2- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f75780368d0 0x7f7578038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7588009990 tx=0x7f7588006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:16.224 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.224+0000 7f75827fc700 1 -- 172.21.15.193:0/1233140802 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f757c011db0 con 0x7f758c0fca40 2024-03-23T08:15:16.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:15 smithi070 ceph-mon[39699]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:16.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.418+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 --> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi070:172.21.15.70=smithi070;smithi193:172.21.15.193=smithi193", "target": ["mon-mgr", ""]}) v1 -- 0x7f758c10e550 con 0x7f75780368d0 2024-03-23T08:15:16.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.423+0000 7f75827fc700 1 -- 172.21.15.193:0/1233140802 <== mgr.14160 v2:172.21.15.70:6800/2732684403 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f758c10e550 con 0x7f75780368d0 2024-03-23T08:15:16.424 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.425+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f75780368d0 msgr2=0x7f7578038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.425+0000 7f75940ad700 1 --2- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f75780368d0 0x7f7578038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7588009990 tx=0x7f7588006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.425+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 msgr2=0x7f758c10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.425+0000 7f75940ad700 1 --2- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 0x7f758c10cfb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f757c000c00 tx=0x7f757c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.425+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 shutdown_connections 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.425+0000 7f75940ad700 1 --2- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f75780368d0 0x7f7578038d80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.425+0000 7f75940ad700 1 --2- 172.21.15.193:0/1233140802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f758c0fca40 0x7f758c10cfb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.425+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 >> 172.21.15.193:0/1233140802 conn(0x7f758c0f81c0 msgr2=0x7f758c0f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.425+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 shutdown_connections 2024-03-23T08:15:16.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:16.426+0000 7f75940ad700 1 -- 172.21.15.193:0/1233140802 wait complete. 2024-03-23T08:15:16.965 DEBUG:teuthology.orchestra.run.smithi193:mon.smithi193> sudo journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi193.service 2024-03-23T08:15:16.967 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:16.967 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:16.996 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:-- Logs begin at Sat 2024-03-23 08:01:04 UTC. -- 2024-03-23T08:15:17.262 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:17.323 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:17.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:17.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:17.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:17.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:17.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:17.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:17.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-23T08:15:17.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-23T08:15:17.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:17.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: Deploying daemon grafana.smithi070 on smithi070 2024-03-23T08:15:17.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi070:172.21.15.70=smithi070;smithi193:172.21.15.193=smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:15:17.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: Saving service mon spec with placement smithi070:172.21.15.70=smithi070;smithi193:172.21.15.193=smithi193;count:2 2024-03-23T08:15:17.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:17.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:17 smithi070 ceph-mon[39699]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:18.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:18 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:19.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.037+0000 7f24fe1dd700 1 -- 172.21.15.193:0/585818382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 msgr2=0x7f24f80fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:19.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.037+0000 7f24fe1dd700 1 --2- 172.21.15.193:0/585818382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 0x7f24f80fce80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24e00048c0 tx=0x7f24e0009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:19.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.037+0000 7f24fe1dd700 1 -- 172.21.15.193:0/585818382 shutdown_connections 2024-03-23T08:15:19.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.037+0000 7f24fe1dd700 1 --2- 172.21.15.193:0/585818382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 0x7f24f80fce80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:19.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.037+0000 7f24fe1dd700 1 -- 172.21.15.193:0/585818382 >> 172.21.15.193:0/585818382 conn(0x7f24f80f81d0 msgr2=0x7f24f80fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:19.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.037+0000 7f24fe1dd700 1 -- 172.21.15.193:0/585818382 shutdown_connections 2024-03-23T08:15:19.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.037+0000 7f24fe1dd700 1 -- 172.21.15.193:0/585818382 wait complete. 2024-03-23T08:15:19.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.038+0000 7f24fe1dd700 1 Processor -- start 2024-03-23T08:15:19.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.038+0000 7f24fe1dd700 1 -- start start 2024-03-23T08:15:19.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.038+0000 7f24fe1dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 0x7f24f810cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:19.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.038+0000 7f24fe1dd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f810d510 con 0x7f24f80fca70 2024-03-23T08:15:19.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.038+0000 7f24f77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 0x7f24f810cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:19.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.038+0000 7f24f77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 0x7f24f810cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:60736/0 (socket says 172.21.15.193:60736) 2024-03-23T08:15:19.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.038+0000 7f24f77fe700 1 -- 172.21.15.193:0/2077098823 learned_addr learned my addr 172.21.15.193:0/2077098823 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:19.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.038+0000 7f24f77fe700 1 -- 172.21.15.193:0/2077098823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e0005040 con 0x7f24f80fca70 2024-03-23T08:15:19.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.039+0000 7f24f77fe700 1 --2- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 0x7f24f810cfd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000c00 tx=0x7f24e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.039+0000 7f24effff700 1 -- 172.21.15.193:0/2077098823 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24e0009620 con 0x7f24f80fca70 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.039+0000 7f24effff700 1 -- 172.21.15.193:0/2077098823 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24e0009780 con 0x7f24f80fca70 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.039+0000 7f24effff700 1 -- 172.21.15.193:0/2077098823 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24e001c5b0 con 0x7f24f80fca70 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.039+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f810d710 con 0x7f24f80fca70 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.039+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f810e280 con 0x7f24f80fca70 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.040+0000 7f24effff700 1 -- 172.21.15.193:0/2077098823 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f24e001c7f0 con 0x7f24f80fca70 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.040+0000 7f24effff700 1 --2- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f24e4036880 0x7f24e4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.040+0000 7f24effff700 1 -- 172.21.15.193:0/2077098823 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f24e0049b70 con 0x7f24f80fca70 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.040+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f8102e40 con 0x7f24f80fca70 2024-03-23T08:15:19.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.040+0000 7f24f6ffd700 1 --2- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f24e4036880 0x7f24e4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:19.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.040+0000 7f24f6ffd700 1 --2- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f24e4036880 0x7f24e4038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009990 tx=0x7f24e8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:19.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.045+0000 7f24effff700 1 -- 172.21.15.193:0/2077098823 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f24e0011db0 con 0x7f24f80fca70 2024-03-23T08:15:19.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.300+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f24f805f5f0 con 0x7f24f80fca70 2024-03-23T08:15:19.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.300+0000 7f24effff700 1 -- 172.21.15.193:0/2077098823 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f24f805f5f0 con 0x7f24f80fca70 2024-03-23T08:15:19.301 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:19.301 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:19.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f24e4036880 msgr2=0x7f24e4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:19.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 --2- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f24e4036880 0x7f24e4038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009990 tx=0x7f24e8006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:19.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 msgr2=0x7f24f810cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:19.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 --2- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 0x7f24f810cfd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000c00 tx=0x7f24e0009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:19.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 shutdown_connections 2024-03-23T08:15:19.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 --2- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f24e4036880 0x7f24e4038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:19.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 --2- 172.21.15.193:0/2077098823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24f80fca70 0x7f24f810cfd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:19.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 >> 172.21.15.193:0/2077098823 conn(0x7f24f80f81d0 msgr2=0x7f24f80f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:19.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 shutdown_connections 2024-03-23T08:15:19.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:19.302+0000 7f24fe1dd700 1 -- 172.21.15.193:0/2077098823 wait complete. 2024-03-23T08:15:19.303 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:19.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:19 smithi070 ceph-mon[39699]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:19.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:19 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/2077098823' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:20.978 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:20.978 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:21.268 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:21.329 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:21.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:21 smithi070 ceph-mon[39699]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:22.933 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.933+0000 7f9de85e8700 1 -- 172.21.15.193:0/4241273661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 msgr2=0x7f9de00ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:22.934 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.933+0000 7f9de85e8700 1 --2- 172.21.15.193:0/4241273661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 0x7f9de00ffaa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00048c0 tx=0x7f9dd0009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:22.934 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.933+0000 7f9de85e8700 1 -- 172.21.15.193:0/4241273661 shutdown_connections 2024-03-23T08:15:22.934 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.933+0000 7f9de85e8700 1 --2- 172.21.15.193:0/4241273661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 0x7f9de00ffaa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:22.934 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.933+0000 7f9de85e8700 1 -- 172.21.15.193:0/4241273661 >> 172.21.15.193:0/4241273661 conn(0x7f9de00f70c0 msgr2=0x7f9de00f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:22.934 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.933+0000 7f9de85e8700 1 -- 172.21.15.193:0/4241273661 shutdown_connections 2024-03-23T08:15:22.934 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.933+0000 7f9de85e8700 1 -- 172.21.15.193:0/4241273661 wait complete. 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.933+0000 7f9de85e8700 1 Processor -- start 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.934+0000 7f9de85e8700 1 -- start start 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.934+0000 7f9de85e8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 0x7f9de00fd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.934+0000 7f9de85e8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00fd560 con 0x7f9de00fd6c0 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.934+0000 7f9de6384700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 0x7f9de00fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.934+0000 7f9de6384700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 0x7f9de00fd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:60760/0 (socket says 172.21.15.193:60760) 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.934+0000 7f9de6384700 1 -- 172.21.15.193:0/4016647340 learned_addr learned my addr 172.21.15.193:0/4016647340 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.934+0000 7f9de6384700 1 -- 172.21.15.193:0/4016647340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd0005040 con 0x7f9de00fd6c0 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.935+0000 7f9de6384700 1 --2- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 0x7f9de00fd020 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000b3b0 tx=0x7f9dd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.935+0000 7f9dd6ffd700 1 -- 172.21.15.193:0/4016647340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dd00095d0 con 0x7f9de00fd6c0 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.935+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de00fb630 con 0x7f9de00fd6c0 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.935+0000 7f9dd6ffd700 1 -- 172.21.15.193:0/4016647340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9dd0009730 con 0x7f9de00fd6c0 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.935+0000 7f9dd6ffd700 1 -- 172.21.15.193:0/4016647340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dd001c5b0 con 0x7f9de00fd6c0 2024-03-23T08:15:22.936 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.935+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de00fbb20 con 0x7f9de00fd6c0 2024-03-23T08:15:22.937 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.936+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de0102e10 con 0x7f9de00fd6c0 2024-03-23T08:15:22.937 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.936+0000 7f9dd6ffd700 1 -- 172.21.15.193:0/4016647340 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f9dd0013960 con 0x7f9de00fd6c0 2024-03-23T08:15:22.937 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.936+0000 7f9dd6ffd700 1 --2- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f9dcc036880 0x7f9dcc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:22.937 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.936+0000 7f9dd6ffd700 1 -- 172.21.15.193:0/4016647340 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9dd0049a40 con 0x7f9de00fd6c0 2024-03-23T08:15:22.937 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.936+0000 7f9de5b83700 1 --2- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f9dcc036880 0x7f9dcc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:22.937 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.936+0000 7f9de5b83700 1 --2- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f9dcc036880 0x7f9dcc038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009990 tx=0x7f9ddc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:22.941 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:22.941+0000 7f9dd6ffd700 1 -- 172.21.15.193:0/4016647340 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9dd0011550 con 0x7f9de00fd6c0 2024-03-23T08:15:23.198 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.197+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9de005f5f0 con 0x7f9de00fd6c0 2024-03-23T08:15:23.198 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.198+0000 7f9dd6ffd700 1 -- 172.21.15.193:0/4016647340 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9dd0011370 con 0x7f9de00fd6c0 2024-03-23T08:15:23.199 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:23.199 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:23.201 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f9dcc036880 msgr2=0x7f9dcc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:23.201 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 --2- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f9dcc036880 0x7f9dcc038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009990 tx=0x7f9ddc006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:23.201 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 msgr2=0x7f9de00fd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:23.201 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 --2- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 0x7f9de00fd020 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000b3b0 tx=0x7f9dd0009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:23.202 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 shutdown_connections 2024-03-23T08:15:23.202 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 --2- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f9dcc036880 0x7f9dcc038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:23.202 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 --2- 172.21.15.193:0/4016647340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9de00fd6c0 0x7f9de00fd020 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:23.202 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 >> 172.21.15.193:0/4016647340 conn(0x7f9de00f70c0 msgr2=0x7f9de00f79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:23.202 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 shutdown_connections 2024-03-23T08:15:23.202 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:23.200+0000 7f9de85e8700 1 -- 172.21.15.193:0/4016647340 wait complete. 2024-03-23T08:15:23.202 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:23.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:23 smithi070 ceph-mon[39699]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:23 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/4016647340' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:24.835 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:24.835 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:25.126 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:25.187 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:25.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:25 smithi070 ceph-mon[39699]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:26.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.735+0000 7f839de0e700 1 -- 172.21.15.193:0/636390430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 msgr2=0x7f83980fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:26.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.735+0000 7f839de0e700 1 --2- 172.21.15.193:0/636390430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 0x7f83980fce50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f83880048c0 tx=0x7f8388009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.735+0000 7f839de0e700 1 -- 172.21.15.193:0/636390430 shutdown_connections 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.735+0000 7f839de0e700 1 --2- 172.21.15.193:0/636390430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 0x7f83980fce50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.735+0000 7f839de0e700 1 -- 172.21.15.193:0/636390430 >> 172.21.15.193:0/636390430 conn(0x7f83980f81c0 msgr2=0x7f83980fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.735+0000 7f839de0e700 1 -- 172.21.15.193:0/636390430 shutdown_connections 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.736+0000 7f839de0e700 1 -- 172.21.15.193:0/636390430 wait complete. 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.736+0000 7f839de0e700 1 Processor -- start 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.736+0000 7f839de0e700 1 -- start start 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.736+0000 7f839de0e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 0x7f8398071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.736+0000 7f839de0e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83980722a0 con 0x7f83980fca40 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.736+0000 7f83977fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 0x7f8398071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:26.738 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.736+0000 7f83977fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 0x7f8398071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:41566/0 (socket says 172.21.15.193:41566) 2024-03-23T08:15:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.736+0000 7f83977fe700 1 -- 172.21.15.193:0/2317385160 learned_addr learned my addr 172.21.15.193:0/2317385160 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.737+0000 7f83977fe700 1 -- 172.21.15.193:0/2317385160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388005040 con 0x7f83980fca40 2024-03-23T08:15:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.737+0000 7f83977fe700 1 --2- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 0x7f8398071d60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8388000c00 tx=0x7f8388009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.737+0000 7f837ffff700 1 -- 172.21.15.193:0/2317385160 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8388009620 con 0x7f83980fca40 2024-03-23T08:15:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.737+0000 7f837ffff700 1 -- 172.21.15.193:0/2317385160 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8388009780 con 0x7f83980fca40 2024-03-23T08:15:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.737+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8398070370 con 0x7f83980fca40 2024-03-23T08:15:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.737+0000 7f837ffff700 1 -- 172.21.15.193:0/2317385160 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f838801c5b0 con 0x7f83980fca40 2024-03-23T08:15:26.739 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.737+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8398070860 con 0x7f83980fca40 2024-03-23T08:15:26.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.738+0000 7f837ffff700 1 -- 172.21.15.193:0/2317385160 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f838801c7f0 con 0x7f83980fca40 2024-03-23T08:15:26.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.738+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839805f5f0 con 0x7f83980fca40 2024-03-23T08:15:26.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.738+0000 7f837ffff700 1 --2- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8380036880 0x7f8380038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:26.740 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.738+0000 7f837ffff700 1 -- 172.21.15.193:0/2317385160 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8388049b90 con 0x7f83980fca40 2024-03-23T08:15:26.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.738+0000 7f8396ffd700 1 --2- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8380036880 0x7f8380038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:26.741 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.739+0000 7f8396ffd700 1 --2- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8380036880 0x7f8380038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f838c009990 tx=0x7f838c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:26.743 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:26.744+0000 7f837ffff700 1 -- 172.21.15.193:0/2317385160 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8388011550 con 0x7f83980fca40 2024-03-23T08:15:27.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.002+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8398106b60 con 0x7f83980fca40 2024-03-23T08:15:27.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.003+0000 7f837ffff700 1 -- 172.21.15.193:0/2317385160 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f8388011370 con 0x7f83980fca40 2024-03-23T08:15:27.003 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:27.004 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:27.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8380036880 msgr2=0x7f8380038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:27.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 --2- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8380036880 0x7f8380038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f838c009990 tx=0x7f838c006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:27.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 msgr2=0x7f8398071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:27.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 --2- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 0x7f8398071d60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8388000c00 tx=0x7f8388009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:27.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 shutdown_connections 2024-03-23T08:15:27.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 --2- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8380036880 0x7f8380038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:27.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 --2- 172.21.15.193:0/2317385160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83980fca40 0x7f8398071d60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:27.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 >> 172.21.15.193:0/2317385160 conn(0x7f83980f81c0 msgr2=0x7f83980f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:27.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 shutdown_connections 2024-03-23T08:15:27.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:27.006+0000 7f839de0e700 1 -- 172.21.15.193:0/2317385160 wait complete. 2024-03-23T08:15:27.006 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:27.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:27 smithi070 ceph-mon[39699]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:27.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:27 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/2317385160' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:28.603 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:28.603 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:28.890 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:28.954 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:29 smithi070 ceph-mon[39699]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:29 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:29 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:29 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:29 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:29 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:29 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.664+0000 7f82a6d7b700 1 -- 172.21.15.193:0/1166136970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 msgr2=0x7f82a00fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.664+0000 7f82a6d7b700 1 --2- 172.21.15.193:0/1166136970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 0x7f82a00fbd70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f82940048c0 tx=0x7f8294009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.664+0000 7f82a6d7b700 1 -- 172.21.15.193:0/1166136970 shutdown_connections 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.664+0000 7f82a6d7b700 1 --2- 172.21.15.193:0/1166136970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 0x7f82a00fbd70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.664+0000 7f82a6d7b700 1 -- 172.21.15.193:0/1166136970 >> 172.21.15.193:0/1166136970 conn(0x7f82a00f70c0 msgr2=0x7f82a00f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.664+0000 7f82a6d7b700 1 -- 172.21.15.193:0/1166136970 shutdown_connections 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.664+0000 7f82a6d7b700 1 -- 172.21.15.193:0/1166136970 wait complete. 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.664+0000 7f82a6d7b700 1 Processor -- start 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.665+0000 7f82a6d7b700 1 -- start start 2024-03-23T08:15:30.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.665+0000 7f82a6d7b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 0x7f82a010d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.665+0000 7f82a6d7b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a010d560 con 0x7f82a00fb960 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.665+0000 7f82a4b17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 0x7f82a010d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.665+0000 7f82a4b17700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 0x7f82a010d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:41578/0 (socket says 172.21.15.193:41578) 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.665+0000 7f82a4b17700 1 -- 172.21.15.193:0/3020989521 learned_addr learned my addr 172.21.15.193:0/3020989521 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.665+0000 7f82a4b17700 1 -- 172.21.15.193:0/3020989521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294005040 con 0x7f82a00fb960 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.665+0000 7f82a4b17700 1 --2- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 0x7f82a010d020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f8294009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.666+0000 7f829d7fa700 1 -- 172.21.15.193:0/3020989521 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8294009620 con 0x7f82a00fb960 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.666+0000 7f829d7fa700 1 -- 172.21.15.193:0/3020989521 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8294009780 con 0x7f82a00fb960 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.666+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a010d760 con 0x7f82a00fb960 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.666+0000 7f829d7fa700 1 -- 172.21.15.193:0/3020989521 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f829401c5b0 con 0x7f82a00fb960 2024-03-23T08:15:30.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.666+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a010e280 con 0x7f82a00fb960 2024-03-23T08:15:30.667 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.666+0000 7f829d7fa700 1 -- 172.21.15.193:0/3020989521 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f829401c7f0 con 0x7f82a00fb960 2024-03-23T08:15:30.667 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.667+0000 7f829d7fa700 1 --2- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8290036880 0x7f8290038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:30.667 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.667+0000 7f829d7fa700 1 -- 172.21.15.193:0/3020989521 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8294049b30 con 0x7f82a00fb960 2024-03-23T08:15:30.667 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.667+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a0101d30 con 0x7f82a00fb960 2024-03-23T08:15:30.667 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.667+0000 7f829ffff700 1 --2- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8290036880 0x7f8290038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:30.668 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.667+0000 7f829ffff700 1 --2- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8290036880 0x7f8290038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f828c009990 tx=0x7f828c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:30.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.672+0000 7f829d7fa700 1 -- 172.21.15.193:0/3020989521 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8294011db0 con 0x7f82a00fb960 2024-03-23T08:15:30.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:30 smithi070 ceph-mon[39699]: Deploying daemon prometheus.smithi070 on smithi070 2024-03-23T08:15:30.927 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.928+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f82a005f5f0 con 0x7f82a00fb960 2024-03-23T08:15:30.928 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.929+0000 7f829d7fa700 1 -- 172.21.15.193:0/3020989521 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f82a005f5f0 con 0x7f82a00fb960 2024-03-23T08:15:30.928 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:30.928 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8290036880 msgr2=0x7f8290038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 --2- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8290036880 0x7f8290038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f828c009990 tx=0x7f828c006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 msgr2=0x7f82a010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 --2- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 0x7f82a010d020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8294000c00 tx=0x7f8294009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 shutdown_connections 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 --2- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f8290036880 0x7f8290038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 --2- 172.21.15.193:0/3020989521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82a00fb960 0x7f82a010d020 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 >> 172.21.15.193:0/3020989521 conn(0x7f82a00f70c0 msgr2=0x7f82a0065900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 shutdown_connections 2024-03-23T08:15:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:30.931+0000 7f82a6d7b700 1 -- 172.21.15.193:0/3020989521 wait complete. 2024-03-23T08:15:30.931 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:31 smithi070 ceph-mon[39699]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:31 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/3020989521' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:32.545 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:32.545 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:32.829 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:32.898 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:33 smithi070 ceph-mon[39699]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:33 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:34.588 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.588+0000 7f4ca3c59700 1 -- 172.21.15.193:0/1241535086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 msgr2=0x7f4c9c0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.588+0000 7f4ca3c59700 1 --2- 172.21.15.193:0/1241535086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 0x7f4c9c0fce70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4c980048c0 tx=0x7f4c98009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.588+0000 7f4ca3c59700 1 -- 172.21.15.193:0/1241535086 shutdown_connections 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.588+0000 7f4ca3c59700 1 --2- 172.21.15.193:0/1241535086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 0x7f4c9c0fce70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.588+0000 7f4ca3c59700 1 -- 172.21.15.193:0/1241535086 >> 172.21.15.193:0/1241535086 conn(0x7f4c9c0f81c0 msgr2=0x7f4c9c0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.588+0000 7f4ca3c59700 1 -- 172.21.15.193:0/1241535086 shutdown_connections 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.588+0000 7f4ca3c59700 1 -- 172.21.15.193:0/1241535086 wait complete. 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.589+0000 7f4ca3c59700 1 Processor -- start 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.589+0000 7f4ca3c59700 1 -- start start 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.589+0000 7f4ca3c59700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 0x7f4c9c10cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:34.589 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.589+0000 7f4ca3c59700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c10d4f0 con 0x7f4c9c0fca60 2024-03-23T08:15:34.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.589+0000 7f4ca19f5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 0x7f4c9c10cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:34.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.589+0000 7f4ca19f5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 0x7f4c9c10cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:51738/0 (socket says 172.21.15.193:51738) 2024-03-23T08:15:34.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.589+0000 7f4ca19f5700 1 -- 172.21.15.193:0/2351481237 learned_addr learned my addr 172.21.15.193:0/2351481237 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:34.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.590+0000 7f4ca19f5700 1 -- 172.21.15.193:0/2351481237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98005040 con 0x7f4c9c0fca60 2024-03-23T08:15:34.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.590+0000 7f4ca19f5700 1 --2- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 0x7f4c9c10cfb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c98000c00 tx=0x7f4c98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:34.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.590+0000 7f4c8e7fc700 1 -- 172.21.15.193:0/2351481237 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c98009620 con 0x7f4c9c0fca60 2024-03-23T08:15:34.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.590+0000 7f4c8e7fc700 1 -- 172.21.15.193:0/2351481237 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c98009780 con 0x7f4c9c0fca60 2024-03-23T08:15:34.591 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.590+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c10d6f0 con 0x7f4c9c0fca60 2024-03-23T08:15:34.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.590+0000 7f4c8e7fc700 1 -- 172.21.15.193:0/2351481237 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c9801c5b0 con 0x7f4c9c0fca60 2024-03-23T08:15:34.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.590+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c10e260 con 0x7f4c9c0fca60 2024-03-23T08:15:34.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.591+0000 7f4c8e7fc700 1 -- 172.21.15.193:0/2351481237 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f4c9801c7f0 con 0x7f4c9c0fca60 2024-03-23T08:15:34.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.591+0000 7f4c8e7fc700 1 --2- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f4c88036880 0x7f4c88038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:34.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.591+0000 7f4c8e7fc700 1 -- 172.21.15.193:0/2351481237 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4c98049b90 con 0x7f4c9c0fca60 2024-03-23T08:15:34.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.591+0000 7f4ca11f4700 1 --2- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f4c88036880 0x7f4c88038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:34.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.591+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c102e30 con 0x7f4c9c0fca60 2024-03-23T08:15:34.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.592+0000 7f4ca11f4700 1 --2- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f4c88036880 0x7f4c88038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009990 tx=0x7f4c90006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:34.596 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.597+0000 7f4c8e7fc700 1 -- 172.21.15.193:0/2351481237 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4c98011550 con 0x7f4c9c0fca60 2024-03-23T08:15:34.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.858+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4c9c05f5f0 con 0x7f4c9c0fca60 2024-03-23T08:15:34.858 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.859+0000 7f4c8e7fc700 1 -- 172.21.15.193:0/2351481237 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4c9c05f5f0 con 0x7f4c9c0fca60 2024-03-23T08:15:34.858 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:34.858 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:34.860 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f4c88036880 msgr2=0x7f4c88038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:34.860 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 --2- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f4c88036880 0x7f4c88038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009990 tx=0x7f4c90006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:34.860 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 msgr2=0x7f4c9c10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:34.860 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 --2- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 0x7f4c9c10cfb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c98000c00 tx=0x7f4c98009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:34.860 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 shutdown_connections 2024-03-23T08:15:34.860 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 --2- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f4c88036880 0x7f4c88038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:34.861 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 --2- 172.21.15.193:0/2351481237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0fca60 0x7f4c9c10cfb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:34.861 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 >> 172.21.15.193:0/2351481237 conn(0x7f4c9c0f81c0 msgr2=0x7f4c9c0f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:34.861 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 shutdown_connections 2024-03-23T08:15:34.861 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:34.861+0000 7f4ca3c59700 1 -- 172.21.15.193:0/2351481237 wait complete. 2024-03-23T08:15:34.861 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:35.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:35 smithi070 ceph-mon[39699]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:35.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:35 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/2351481237' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:36.423 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:36.423 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:36.725 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:36.792 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:37.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:37 smithi070 ceph-mon[39699]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.415+0000 7fdcdc62a700 1 -- 172.21.15.193:0/1331091898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 msgr2=0x7fdcd40fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.415+0000 7fdcdc62a700 1 --2- 172.21.15.193:0/1331091898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 0x7fdcd40fe4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40048c0 tx=0x7fdcc4009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.415+0000 7fdcdc62a700 1 -- 172.21.15.193:0/1331091898 shutdown_connections 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.415+0000 7fdcdc62a700 1 --2- 172.21.15.193:0/1331091898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 0x7fdcd40fe4c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.415+0000 7fdcdc62a700 1 -- 172.21.15.193:0/1331091898 >> 172.21.15.193:0/1331091898 conn(0x7fdcd40f6850 msgr2=0x7fdcd40f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.415+0000 7fdcdc62a700 1 -- 172.21.15.193:0/1331091898 shutdown_connections 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.415+0000 7fdcdc62a700 1 -- 172.21.15.193:0/1331091898 wait complete. 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.416+0000 7fdcdc62a700 1 Processor -- start 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.416+0000 7fdcdc62a700 1 -- start start 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.416+0000 7fdcdc62a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 0x7fdcd40fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:38.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.416+0000 7fdcdc62a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40fdd00 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.417 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.416+0000 7fdcda3c6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 0x7fdcd40fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:38.417 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.416+0000 7fdcda3c6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 0x7fdcd40fd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:51754/0 (socket says 172.21.15.193:51754) 2024-03-23T08:15:38.417 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.416+0000 7fdcda3c6700 1 -- 172.21.15.193:0/4104933386 learned_addr learned my addr 172.21.15.193:0/4104933386 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.416+0000 7fdcda3c6700 1 -- 172.21.15.193:0/4104933386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4005040 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.416+0000 7fdcda3c6700 1 --2- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 0x7fdcd40fd7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4000c00 tx=0x7fdcc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.417+0000 7fdccaffd700 1 -- 172.21.15.193:0/4104933386 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcc4009620 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.417+0000 7fdccaffd700 1 -- 172.21.15.193:0/4104933386 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdcc4009780 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.417+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd40fbdd0 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.417+0000 7fdccaffd700 1 -- 172.21.15.193:0/4104933386 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcc401c5b0 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.417+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd40fc2c0 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.417+0000 7fdccaffd700 1 -- 172.21.15.193:0/4104933386 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fdcc401c7a0 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.418+0000 7fdccaffd700 1 --2- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fdcc00368d0 0x7fdcc0038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.418+0000 7fdccaffd700 1 -- 172.21.15.193:0/4104933386 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdcc4049ab0 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.418+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd405f5f0 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.418+0000 7fdcd9bc5700 1 --2- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fdcc00368d0 0x7fdcc0038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:38.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.418+0000 7fdcd9bc5700 1 --2- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fdcc00368d0 0x7fdcc0038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009990 tx=0x7fdcd0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:38.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.423+0000 7fdccaffd700 1 -- 172.21.15.193:0/4104933386 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdcc4011db0 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.681 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.682+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdcd4106b90 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.682 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.683+0000 7fdccaffd700 1 -- 172.21.15.193:0/4104933386 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fdcc4034090 con 0x7fdcd40fe0b0 2024-03-23T08:15:38.682 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:38.682 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:38.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fdcc00368d0 msgr2=0x7fdcc0038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:38.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 --2- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fdcc00368d0 0x7fdcc0038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009990 tx=0x7fdcd0006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:38.684 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 msgr2=0x7fdcd40fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:38.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 --2- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 0x7fdcd40fd7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4000c00 tx=0x7fdcc4009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:38.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 shutdown_connections 2024-03-23T08:15:38.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 --2- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fdcc00368d0 0x7fdcc0038d80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:38.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 --2- 172.21.15.193:0/4104933386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcd40fe0b0 0x7fdcd40fd7c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:38.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 >> 172.21.15.193:0/4104933386 conn(0x7fdcd40f6850 msgr2=0x7fdcd40f70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:38.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 shutdown_connections 2024-03-23T08:15:38.685 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:38.685+0000 7fdcdc62a700 1 -- 172.21.15.193:0/4104933386 wait complete. 2024-03-23T08:15:38.685 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:39.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:39 smithi070 ceph-mon[39699]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:39.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:39 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/4104933386' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:40.367 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:40.415 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:40.699 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:40.760 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:42.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.267+0000 7fef7eafc700 1 -- 172.21.15.193:0/1674436663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 msgr2=0x7fef780fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:42.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.267+0000 7fef7eafc700 1 --2- 172.21.15.193:0/1674436663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 0x7fef780fce60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0048c0 tx=0x7fef6c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:42.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.268+0000 7fef7eafc700 1 -- 172.21.15.193:0/1674436663 shutdown_connections 2024-03-23T08:15:42.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.268+0000 7fef7eafc700 1 --2- 172.21.15.193:0/1674436663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 0x7fef780fce60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:42.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.268+0000 7fef7eafc700 1 -- 172.21.15.193:0/1674436663 >> 172.21.15.193:0/1674436663 conn(0x7fef780f81d0 msgr2=0x7fef780fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:42.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.268+0000 7fef7eafc700 1 -- 172.21.15.193:0/1674436663 shutdown_connections 2024-03-23T08:15:42.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.268+0000 7fef7eafc700 1 -- 172.21.15.193:0/1674436663 wait complete. 2024-03-23T08:15:42.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.268+0000 7fef7eafc700 1 Processor -- start 2024-03-23T08:15:42.269 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.268+0000 7fef7eafc700 1 -- start start 2024-03-23T08:15:42.269 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.268+0000 7fef7eafc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 0x7fef7810cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.268+0000 7fef7eafc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7810d4c0 con 0x7fef780fca50 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef7c898700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 0x7fef7810cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef7c898700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 0x7fef7810cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:51764/0 (socket says 172.21.15.193:51764) 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef7c898700 1 -- 172.21.15.193:0/1948221740 learned_addr learned my addr 172.21.15.193:0/1948221740 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef7c898700 1 -- 172.21.15.193:0/1948221740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c005040 con 0x7fef780fca50 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef7c898700 1 --2- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 0x7fef7810cf80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef6c000c00 tx=0x7fef6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef757fa700 1 -- 172.21.15.193:0/1948221740 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef6c009620 con 0x7fef780fca50 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7810d6c0 con 0x7fef780fca50 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef757fa700 1 -- 172.21.15.193:0/1948221740 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef6c009780 con 0x7fef780fca50 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef757fa700 1 -- 172.21.15.193:0/1948221740 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef6c01c5b0 con 0x7fef780fca50 2024-03-23T08:15:42.270 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.269+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7810e230 con 0x7fef780fca50 2024-03-23T08:15:42.271 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.270+0000 7fef757fa700 1 -- 172.21.15.193:0/1948221740 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7fef6c01c7f0 con 0x7fef780fca50 2024-03-23T08:15:42.271 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.270+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef78102e20 con 0x7fef780fca50 2024-03-23T08:15:42.271 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.270+0000 7fef757fa700 1 --2- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef680368d0 0x7fef68038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:42.271 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.270+0000 7fef757fa700 1 -- 172.21.15.193:0/1948221740 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fef6c049d20 con 0x7fef780fca50 2024-03-23T08:15:42.271 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.270+0000 7fef77fff700 1 -- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef680368d0 msgr2=0x7fef68038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2732684403 2024-03-23T08:15:42.271 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.270+0000 7fef77fff700 1 --2- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef680368d0 0x7fef68038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:15:42.275 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.275+0000 7fef757fa700 1 -- 172.21.15.193:0/1948221740 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fef6c011dd0 con 0x7fef780fca50 2024-03-23T08:15:42.358 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:42 smithi070 ceph-mon[39699]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:15:42.358 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:42.358 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:42.359 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:42.359 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:42 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-23T08:15:42.471 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.471+0000 7fef77fff700 1 -- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef680368d0 msgr2=0x7fef68038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2732684403 2024-03-23T08:15:42.471 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.471+0000 7fef77fff700 1 --2- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef680368d0 0x7fef68038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T08:15:42.530 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.530+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fef7810ec40 con 0x7fef780fca50 2024-03-23T08:15:42.531 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.531+0000 7fef757fa700 1 -- 172.21.15.193:0/1948221740 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fef6c011370 con 0x7fef780fca50 2024-03-23T08:15:42.531 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:42.531 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:42.534 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef680368d0 msgr2=0x7fef68038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:15:42.534 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 --2- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef680368d0 0x7fef68038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:42.534 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 msgr2=0x7fef7810cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:42.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 --2- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 0x7fef7810cf80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef6c000c00 tx=0x7fef6c009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:42.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 shutdown_connections 2024-03-23T08:15:42.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 --2- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7fef680368d0 0x7fef68038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:42.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 --2- 172.21.15.193:0/1948221740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780fca50 0x7fef7810cf80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:42.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 >> 172.21.15.193:0/1948221740 conn(0x7fef780f81d0 msgr2=0x7fef780f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:42.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 shutdown_connections 2024-03-23T08:15:42.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:42.533+0000 7fef7eafc700 1 -- 172.21.15.193:0/1948221740 wait complete. 2024-03-23T08:15:42.535 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:43.371 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:43 smithi070 ceph-mon[39699]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-23T08:15:43.371 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:43 smithi070 ceph-mon[39699]: mgrmap e14: smithi070.jhsbch(active, since 59s) 2024-03-23T08:15:43.371 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:43 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/1948221740' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:44.189 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:44.190 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:44.480 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:44.542 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:46.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.087+0000 7f15ae285700 1 -- 172.21.15.193:0/2905731408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 msgr2=0x7f15a80fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:46.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.087+0000 7f15ae285700 1 --2- 172.21.15.193:0/2905731408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 0x7f15a80fd460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f15980048c0 tx=0x7f1598009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:46.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.087+0000 7f15ae285700 1 -- 172.21.15.193:0/2905731408 shutdown_connections 2024-03-23T08:15:46.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.087+0000 7f15ae285700 1 --2- 172.21.15.193:0/2905731408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 0x7f15a80fd460 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:46.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.087+0000 7f15ae285700 1 -- 172.21.15.193:0/2905731408 >> 172.21.15.193:0/2905731408 conn(0x7f15a80f70d0 msgr2=0x7f15a80f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:46.089 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.087+0000 7f15ae285700 1 -- 172.21.15.193:0/2905731408 shutdown_connections 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.087+0000 7f15ae285700 1 -- 172.21.15.193:0/2905731408 wait complete. 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.088+0000 7f15ae285700 1 Processor -- start 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.088+0000 7f15ae285700 1 -- start start 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.088+0000 7f15ae285700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 0x7f15a8065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.088+0000 7f15ae285700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a810fcd0 con 0x7f15a80fd050 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.088+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 0x7f15a8065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.088+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 0x7f15a8065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:53320/0 (socket says 172.21.15.193:53320) 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.088+0000 7f15a7fff700 1 -- 172.21.15.193:0/3049400417 learned_addr learned my addr 172.21.15.193:0/3049400417 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.089+0000 7f15a7fff700 1 -- 172.21.15.193:0/3049400417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598005040 con 0x7f15a80fd050 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.089+0000 7f15a7fff700 1 --2- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 0x7f15a8065bd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1598009300 tx=0x7f1598009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.089+0000 7f15a4ff9700 1 -- 172.21.15.193:0/3049400417 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1598009600 con 0x7f15a80fd050 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.089+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a810fe10 con 0x7f15a80fd050 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.089+0000 7f15a4ff9700 1 -- 172.21.15.193:0/3049400417 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1598009760 con 0x7f15a80fd050 2024-03-23T08:15:46.090 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.089+0000 7f15a4ff9700 1 -- 172.21.15.193:0/3049400417 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f159801c5b0 con 0x7f15a80fd050 2024-03-23T08:15:46.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.089+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a8110300 con 0x7f15a80fd050 2024-03-23T08:15:46.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.090+0000 7f15a4ff9700 1 -- 172.21.15.193:0/3049400417 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f159801c870 con 0x7f15a80fd050 2024-03-23T08:15:46.091 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.090+0000 7f15a4ff9700 1 --2- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f15900368d0 0x7f1590038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:46.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.090+0000 7f15a4ff9700 1 -- 172.21.15.193:0/3049400417 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1598049bc0 con 0x7f15a80fd050 2024-03-23T08:15:46.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.090+0000 7f15a77fe700 1 -- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f15900368d0 msgr2=0x7f1590038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2732684403 2024-03-23T08:15:46.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.090+0000 7f15a77fe700 1 --2- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f15900368d0 0x7f1590038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:15:46.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.090+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a8106060 con 0x7f15a80fd050 2024-03-23T08:15:46.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.095+0000 7f15a4ff9700 1 -- 172.21.15.193:0/3049400417 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1598011bd0 con 0x7f15a80fd050 2024-03-23T08:15:46.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.291+0000 7f15a77fe700 1 -- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f15900368d0 msgr2=0x7f1590038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.70:6800/2732684403 2024-03-23T08:15:46.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.291+0000 7f15a77fe700 1 --2- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f15900368d0 0x7f1590038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-23T08:15:46.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.352+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f15a8101d30 con 0x7f15a80fd050 2024-03-23T08:15:46.353 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.353+0000 7f15a4ff9700 1 -- 172.21.15.193:0/3049400417 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f15980117e0 con 0x7f15a80fd050 2024-03-23T08:15:46.353 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:46.353 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:46.354 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.354+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f15900368d0 msgr2=0x7f1590038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:15:46.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.354+0000 7f15ae285700 1 --2- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f15900368d0 0x7f1590038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:46.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.354+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 msgr2=0x7f15a8065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:46.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.354+0000 7f15ae285700 1 --2- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 0x7f15a8065bd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1598009300 tx=0x7f1598009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:46.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.354+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 shutdown_connections 2024-03-23T08:15:46.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.354+0000 7f15ae285700 1 --2- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:6800/2732684403,v1:172.21.15.70:6801/2732684403] conn(0x7f15900368d0 0x7f1590038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:46.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.354+0000 7f15ae285700 1 --2- 172.21.15.193:0/3049400417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80fd050 0x7f15a8065bd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:46.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.355+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 >> 172.21.15.193:0/3049400417 conn(0x7f15a80f70d0 msgr2=0x7f15a80f7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:46.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.355+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 shutdown_connections 2024-03-23T08:15:46.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:46.355+0000 7f15ae285700 1 -- 172.21.15.193:0/3049400417 wait complete. 2024-03-23T08:15:46.356 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:46.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:46 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/3049400417' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:48.000 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:48.000 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:48.290 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:48.357 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:48.538 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: Active manager daemon smithi070.jhsbch restarted 2024-03-23T08:15:48.538 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: Activating manager daemon smithi070.jhsbch 2024-03-23T08:15:48.538 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: osdmap e5: 0 total, 0 up, 0 in 2024-03-23T08:15:48.538 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: mgrmap e15: smithi070.jhsbch(active, starting, since 0.00299483s) 2024-03-23T08:15:48.538 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:15:48.538 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:15:48.539 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:15:48.539 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:15:48.539 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:15:48.539 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: Manager daemon smithi070.jhsbch is now available 2024-03-23T08:15:48.539 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:15:48.539 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:15:49.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:49.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:15:49.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:15:49.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/trash_purge_schedule"}]: dispatch 2024-03-23T08:15:49.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:49.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:49.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:49 smithi070 ceph-mon[39699]: mgrmap e16: smithi070.jhsbch(active, since 1.00502s) 2024-03-23T08:15:50.116 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.116+0000 7f80c3ba0700 1 -- 172.21.15.193:0/2930121390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 msgr2=0x7f80bc0fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:50.117 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.116+0000 7f80c3ba0700 1 --2- 172.21.15.193:0/2930121390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 0x7f80bc0fce60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f80b80048c0 tx=0x7f80b8009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:50.117 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.116+0000 7f80c3ba0700 1 -- 172.21.15.193:0/2930121390 shutdown_connections 2024-03-23T08:15:50.117 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.116+0000 7f80c3ba0700 1 --2- 172.21.15.193:0/2930121390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 0x7f80bc0fce60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:50.117 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.116+0000 7f80c3ba0700 1 -- 172.21.15.193:0/2930121390 >> 172.21.15.193:0/2930121390 conn(0x7f80bc0f81d0 msgr2=0x7f80bc0fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:50.117 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.116+0000 7f80c3ba0700 1 -- 172.21.15.193:0/2930121390 shutdown_connections 2024-03-23T08:15:50.117 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.116+0000 7f80c3ba0700 1 -- 172.21.15.193:0/2930121390 wait complete. 2024-03-23T08:15:50.118 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.117+0000 7f80c3ba0700 1 Processor -- start 2024-03-23T08:15:50.118 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.117+0000 7f80c3ba0700 1 -- start start 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.117+0000 7f80c3ba0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 0x7f80bc10cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.117+0000 7f80c3ba0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc10d4c0 con 0x7f80bc0fca50 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.117+0000 7f80c193c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 0x7f80bc10cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.117+0000 7f80c193c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 0x7f80bc10cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:53338/0 (socket says 172.21.15.193:53338) 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.117+0000 7f80c193c700 1 -- 172.21.15.193:0/1347393043 learned_addr learned my addr 172.21.15.193:0/1347393043 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.118+0000 7f80c193c700 1 -- 172.21.15.193:0/1347393043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b8005040 con 0x7f80bc0fca50 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.118+0000 7f80c193c700 1 --2- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 0x7f80bc10cf80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009300 tx=0x7f80b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.118+0000 7f80b27fc700 1 -- 172.21.15.193:0/1347393043 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80b8009600 con 0x7f80bc0fca50 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.118+0000 7f80b27fc700 1 -- 172.21.15.193:0/1347393043 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f80b8009760 con 0x7f80bc0fca50 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.118+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80bc10d6c0 con 0x7f80bc0fca50 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.118+0000 7f80b27fc700 1 -- 172.21.15.193:0/1347393043 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80b801c5b0 con 0x7f80bc0fca50 2024-03-23T08:15:50.119 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.118+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80bc10e230 con 0x7f80bc0fca50 2024-03-23T08:15:50.120 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.119+0000 7f80b27fc700 1 -- 172.21.15.193:0/1347393043 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 43289+0+0 (secure 0 0 0) 0x7f80b801c870 con 0x7f80bc0fca50 2024-03-23T08:15:50.120 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.119+0000 7f80b27fc700 1 --2- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f80a8036710 0x7f80a8038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:50.120 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.119+0000 7f80b27fc700 1 -- 172.21.15.193:0/1347393043 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f80b8049980 con 0x7f80bc0fca50 2024-03-23T08:15:50.120 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.119+0000 7f80c113b700 1 --2- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f80a8036710 0x7f80a8038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:50.121 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.119+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80bc102e20 con 0x7f80bc0fca50 2024-03-23T08:15:50.121 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.120+0000 7f80c113b700 1 --2- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f80a8036710 0x7f80a8038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009990 tx=0x7f80ac006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:50.125 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.125+0000 7f80b27fc700 1 -- 172.21.15.193:0/1347393043 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f80b8023300 con 0x7f80bc0fca50 2024-03-23T08:15:50.403 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.403+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f80bc05f5f0 con 0x7f80bc0fca50 2024-03-23T08:15:50.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.404+0000 7f80b27fc700 1 -- 172.21.15.193:0/1347393043 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f80b8011b60 con 0x7f80bc0fca50 2024-03-23T08:15:50.404 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:50.404 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:50.406 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f80a8036710 msgr2=0x7f80a8038bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:50.406 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 --2- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f80a8036710 0x7f80a8038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009990 tx=0x7f80ac006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:50.406 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 msgr2=0x7f80bc10cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:50.406 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 --2- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 0x7f80bc10cf80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009300 tx=0x7f80b8009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:50.406 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 shutdown_connections 2024-03-23T08:15:50.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 --2- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f80a8036710 0x7f80a8038bc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:50.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 --2- 172.21.15.193:0/1347393043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80bc0fca50 0x7f80bc10cf80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:50.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 >> 172.21.15.193:0/1347393043 conn(0x7f80bc0f81d0 msgr2=0x7f80bc0f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:50.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 shutdown_connections 2024-03-23T08:15:50.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:50.406+0000 7f80c3ba0700 1 -- 172.21.15.193:0/1347393043 wait complete. 2024-03-23T08:15:50.407 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:50 smithi070 ceph-mon[39699]: [23/Mar/2024:08:15:49] ENGINE Bus STARTING 2024-03-23T08:15:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:50 smithi070 ceph-mon[39699]: [23/Mar/2024:08:15:49] ENGINE Serving on https://172.21.15.70:7150 2024-03-23T08:15:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:50 smithi070 ceph-mon[39699]: [23/Mar/2024:08:15:49] ENGINE Bus STARTED 2024-03-23T08:15:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:50 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/1347393043' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:51.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:51 smithi070 ceph-mon[39699]: mgrmap e17: smithi070.jhsbch(active, since 2s) 2024-03-23T08:15:52.049 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:52.049 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:52.696 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:52.745 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-23T08:15:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:15:55.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.149+0000 7f02b9913700 1 -- 172.21.15.193:0/4265279013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 msgr2=0x7f02b40fdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:55.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.149+0000 7f02b9913700 1 --2- 172.21.15.193:0/4265279013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 0x7f02b40fdeb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f02a40048c0 tx=0x7f02a4009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:55.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.149+0000 7f02b9913700 1 -- 172.21.15.193:0/4265279013 shutdown_connections 2024-03-23T08:15:55.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.149+0000 7f02b9913700 1 --2- 172.21.15.193:0/4265279013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 0x7f02b40fdeb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:55.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.149+0000 7f02b9913700 1 -- 172.21.15.193:0/4265279013 >> 172.21.15.193:0/4265279013 conn(0x7f02b4071800 msgr2=0x7f02b4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:55.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.149+0000 7f02b9913700 1 -- 172.21.15.193:0/4265279013 shutdown_connections 2024-03-23T08:15:55.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.149+0000 7f02b9913700 1 -- 172.21.15.193:0/4265279013 wait complete. 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.150+0000 7f02b9913700 1 Processor -- start 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.150+0000 7f02b9913700 1 -- start start 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.150+0000 7f02b9913700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 0x7f02b41130b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.150+0000 7f02b9913700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b41135f0 con 0x7f02b40fdae0 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.150+0000 7f02b2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 0x7f02b41130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.150+0000 7f02b2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 0x7f02b41130b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:47954/0 (socket says 172.21.15.193:47954) 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.150+0000 7f02b2ffd700 1 -- 172.21.15.193:0/1819255870 learned_addr learned my addr 172.21.15.193:0/1819255870 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.151+0000 7f02b2ffd700 1 -- 172.21.15.193:0/1819255870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a4005040 con 0x7f02b40fdae0 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.151+0000 7f02b2ffd700 1 --2- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 0x7f02b41130b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f02a4007e80 tx=0x7f02a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.151+0000 7f029b7fe700 1 -- 172.21.15.193:0/1819255870 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02a40093c0 con 0x7f02b40fdae0 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.151+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b41137f0 con 0x7f02b40fdae0 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.151+0000 7f029b7fe700 1 -- 172.21.15.193:0/1819255870 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02a4013420 con 0x7f02b40fdae0 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.151+0000 7f029b7fe700 1 -- 172.21.15.193:0/1819255870 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02a401c5b0 con 0x7f02b40fdae0 2024-03-23T08:15:55.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.151+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b4113c60 con 0x7f02b40fdae0 2024-03-23T08:15:55.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.152+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b40658f0 con 0x7f02b40fdae0 2024-03-23T08:15:55.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.152+0000 7f029b7fe700 1 -- 172.21.15.193:0/1819255870 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f02a401c830 con 0x7f02b40fdae0 2024-03-23T08:15:55.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.152+0000 7f029b7fe700 1 --2- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f029c036970 0x7f029c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:55.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.152+0000 7f029b7fe700 1 -- 172.21.15.193:0/1819255870 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f02a4049c10 con 0x7f02b40fdae0 2024-03-23T08:15:55.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.152+0000 7f02b27fc700 1 --2- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f029c036970 0x7f029c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:55.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.153+0000 7f02b27fc700 1 --2- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f029c036970 0x7f029c038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009990 tx=0x7f02a8006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:55.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.157+0000 7f029b7fe700 1 -- 172.21.15.193:0/1819255870 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f02a4023300 con 0x7f02b40fdae0 2024-03-23T08:15:55.452 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.452+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f02b404bed0 con 0x7f02b40fdae0 2024-03-23T08:15:55.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.452+0000 7f029b7fe700 1 -- 172.21.15.193:0/1819255870 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f02a4012e40 con 0x7f02b40fdae0 2024-03-23T08:15:55.453 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:55.453 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:55.455 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f029c036970 msgr2=0x7f029c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:55.455 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 --2- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f029c036970 0x7f029c038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009990 tx=0x7f02a8006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:55.455 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 msgr2=0x7f02b41130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:55.455 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 --2- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 0x7f02b41130b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f02a4007e80 tx=0x7f02a4009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:55.455 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 shutdown_connections 2024-03-23T08:15:55.456 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 --2- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f029c036970 0x7f029c038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:55.456 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 --2- 172.21.15.193:0/1819255870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b40fdae0 0x7f02b41130b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:55.456 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 >> 172.21.15.193:0/1819255870 conn(0x7f02b4071800 msgr2=0x7f02b4103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:55.456 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 shutdown_connections 2024-03-23T08:15:55.456 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:55.455+0000 7f02b9913700 1 -- 172.21.15.193:0/1819255870 wait complete. 2024-03-23T08:15:55.456 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:55.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:55.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:55.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:15:55.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:15:55.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-23T08:15:55.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:15:56.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:56 smithi070 ceph-mon[39699]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:15:56.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:56 smithi070 ceph-mon[39699]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:15:56.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:56 smithi070 ceph-mon[39699]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:15:56.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:56 smithi070 ceph-mon[39699]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:15:56.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:56 smithi070 ceph-mon[39699]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:15:56.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:56 smithi070 ceph-mon[39699]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:15:56.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:56 smithi070 ceph-mon[39699]: Deploying daemon crash.smithi193 on smithi193 2024-03-23T08:15:56.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:15:56 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/1819255870' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:15:57.193 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:15:57.194 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:15:57.565 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:15:59.286 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.250+0000 7f8cd4e21700 1 -- 172.21.15.193:0/709097909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 msgr2=0x7f8cd0101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:59.286 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.250+0000 7f8cd4e21700 1 --2- 172.21.15.193:0/709097909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 0x7f8cd0101da0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00048c0 tx=0x7f8cc0009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:59.286 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.250+0000 7f8cd4e21700 1 -- 172.21.15.193:0/709097909 shutdown_connections 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.250+0000 7f8cd4e21700 1 --2- 172.21.15.193:0/709097909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 0x7f8cd0101da0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.250+0000 7f8cd4e21700 1 -- 172.21.15.193:0/709097909 >> 172.21.15.193:0/709097909 conn(0x7f8cd0071800 msgr2=0x7f8cd0071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.250+0000 7f8cd4e21700 1 -- 172.21.15.193:0/709097909 shutdown_connections 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.250+0000 7f8cd4e21700 1 -- 172.21.15.193:0/709097909 wait complete. 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.251+0000 7f8cd4e21700 1 Processor -- start 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.251+0000 7f8cd4e21700 1 -- start start 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.251+0000 7f8cd4e21700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 0x7f8cd0111f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.251+0000 7f8cd4e21700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01124b0 con 0x7f8cd01019d0 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cce59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 0x7f8cd0111f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cce59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 0x7f8cd0111f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:47958/0 (socket says 172.21.15.193:47958) 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cce59c700 1 -- 172.21.15.193:0/3047912467 learned_addr learned my addr 172.21.15.193:0/3047912467 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cce59c700 1 -- 172.21.15.193:0/3047912467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0005040 con 0x7f8cd01019d0 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cce59c700 1 --2- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 0x7f8cd0111f70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0007e80 tx=0x7f8cc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:59.287 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cbeffd700 1 -- 172.21.15.193:0/3047912467 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc00093c0 con 0x7f8cd01019d0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd01126b0 con 0x7f8cd01019d0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cbeffd700 1 -- 172.21.15.193:0/3047912467 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8cc0013420 con 0x7f8cd01019d0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cbeffd700 1 -- 172.21.15.193:0/3047912467 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc001c5b0 con 0x7f8cd01019d0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.252+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd0112ba0 con 0x7f8cd01019d0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.253+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd00fd0d0 con 0x7f8cd01019d0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.253+0000 7f8cbeffd700 1 -- 172.21.15.193:0/3047912467 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f8cc001c830 con 0x7f8cd01019d0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.253+0000 7f8cbeffd700 1 --2- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8cb8036970 0x7f8cb8038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.253+0000 7f8cbeffd700 1 -- 172.21.15.193:0/3047912467 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f8cc0049b50 con 0x7f8cd01019d0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.254+0000 7f8ccdd9b700 1 --2- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8cb8036970 0x7f8cb8038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.254+0000 7f8ccdd9b700 1 --2- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8cb8036970 0x7f8cb8038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009990 tx=0x7f8cc4006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:15:59.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.258+0000 7f8cbeffd700 1 -- 172.21.15.193:0/3047912467 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8cc0023300 con 0x7f8cd01019d0 2024-03-23T08:15:59.533 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.532+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8cd005f5f0 con 0x7f8cd01019d0 2024-03-23T08:15:59.533 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.533+0000 7f8cbeffd700 1 -- 172.21.15.193:0/3047912467 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f8cc0011b60 con 0x7f8cd01019d0 2024-03-23T08:15:59.533 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:15:59.534 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:15:59.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8cb8036970 msgr2=0x7f8cb8038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:59.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 --2- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8cb8036970 0x7f8cb8038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009990 tx=0x7f8cc4006e40 comp rx=0 tx=0).stop 2024-03-23T08:15:59.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 msgr2=0x7f8cd0111f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:15:59.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 --2- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 0x7f8cd0111f70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0007e80 tx=0x7f8cc0009e30 comp rx=0 tx=0).stop 2024-03-23T08:15:59.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 shutdown_connections 2024-03-23T08:15:59.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 --2- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8cb8036970 0x7f8cb8038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:59.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 --2- 172.21.15.193:0/3047912467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd01019d0 0x7f8cd0111f70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:15:59.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 >> 172.21.15.193:0/3047912467 conn(0x7f8cd0071800 msgr2=0x7f8cd0103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:15:59.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 shutdown_connections 2024-03-23T08:15:59.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:15:59.535+0000 7f8cd4e21700 1 -- 172.21.15.193:0/3047912467 wait complete. 2024-03-23T08:15:59.536 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:16:00.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:00 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:00.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:00 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:00.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:00 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:00.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:00 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:00.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:00 smithi070 ceph-mon[39699]: Deploying daemon node-exporter.smithi193 on smithi193 2024-03-23T08:16:00.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:00 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/3047912467' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:01.198 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:16:01.198 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:16:01.678 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:16:04.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.066+0000 7f7539509700 1 -- 172.21.15.193:0/2636095173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 msgr2=0x7f75341014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:04.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.066+0000 7f7539509700 1 --2- 172.21.15.193:0/2636095173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 0x7f75341014b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75240048c0 tx=0x7f7524009e30 comp rx=0 tx=0).stop 2024-03-23T08:16:04.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.066+0000 7f7539509700 1 -- 172.21.15.193:0/2636095173 shutdown_connections 2024-03-23T08:16:04.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.066+0000 7f7539509700 1 --2- 172.21.15.193:0/2636095173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 0x7f75341014b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:04.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.066+0000 7f7539509700 1 -- 172.21.15.193:0/2636095173 >> 172.21.15.193:0/2636095173 conn(0x7f75340f6c30 msgr2=0x7f75340f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:16:04.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.066+0000 7f7539509700 1 -- 172.21.15.193:0/2636095173 shutdown_connections 2024-03-23T08:16:04.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.066+0000 7f7539509700 1 -- 172.21.15.193:0/2636095173 wait complete. 2024-03-23T08:16:04.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.067+0000 7f7539509700 1 Processor -- start 2024-03-23T08:16:04.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.067+0000 7f7539509700 1 -- start start 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.067+0000 7f7539509700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 0x7f753410e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.067+0000 7f7539509700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7534105ad0 con 0x7f75341010e0 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.067+0000 7f7532ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 0x7f753410e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.067+0000 7f7532ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 0x7f753410e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:33144/0 (socket says 172.21.15.193:33144) 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.067+0000 7f7532ffd700 1 -- 172.21.15.193:0/1737807805 learned_addr learned my addr 172.21.15.193:0/1737807805 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.067+0000 7f7532ffd700 1 -- 172.21.15.193:0/1737807805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524005040 con 0x7f75341010e0 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.068+0000 7f7532ffd700 1 --2- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 0x7f753410e220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7524007e80 tx=0x7f7524009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.068+0000 7f751b7fe700 1 -- 172.21.15.193:0/1737807805 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f75240093c0 con 0x7f75341010e0 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.068+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753410e7c0 con 0x7f75341010e0 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.068+0000 7f751b7fe700 1 -- 172.21.15.193:0/1737807805 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7524013420 con 0x7f75341010e0 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.068+0000 7f751b7fe700 1 -- 172.21.15.193:0/1737807805 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f752401c5b0 con 0x7f75341010e0 2024-03-23T08:16:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.068+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753410ec30 con 0x7f75341010e0 2024-03-23T08:16:04.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.069+0000 7f751b7fe700 1 -- 172.21.15.193:0/1737807805 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f752401c830 con 0x7f75341010e0 2024-03-23T08:16:04.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.069+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753410eed0 con 0x7f75341010e0 2024-03-23T08:16:04.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.069+0000 7f751b7fe700 1 --2- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f751c036970 0x7f751c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:04.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.069+0000 7f751b7fe700 1 -- 172.21.15.193:0/1737807805 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f7524049c00 con 0x7f75341010e0 2024-03-23T08:16:04.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.069+0000 7f75327fc700 1 --2- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f751c036970 0x7f751c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:04.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.070+0000 7f75327fc700 1 --2- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f751c036970 0x7f751c038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7528009990 tx=0x7f7528006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:04.074 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.074+0000 7f751b7fe700 1 -- 172.21.15.193:0/1737807805 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f753410eed0 con 0x7f75341010e0 2024-03-23T08:16:04.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.344+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f753410eed0 con 0x7f75341010e0 2024-03-23T08:16:04.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.345+0000 7f751b7fe700 1 -- 172.21.15.193:0/1737807805 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f753410eed0 con 0x7f75341010e0 2024-03-23T08:16:04.346 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:16:04.346 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:16:04.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f751c036970 msgr2=0x7f751c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:04.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 --2- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f751c036970 0x7f751c038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7528009990 tx=0x7f7528006e40 comp rx=0 tx=0).stop 2024-03-23T08:16:04.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 msgr2=0x7f753410e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:04.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 --2- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 0x7f753410e220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7524007e80 tx=0x7f7524009e30 comp rx=0 tx=0).stop 2024-03-23T08:16:04.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 shutdown_connections 2024-03-23T08:16:04.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 --2- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f751c036970 0x7f751c038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:04.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 --2- 172.21.15.193:0/1737807805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75341010e0 0x7f753410e220 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:04.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 >> 172.21.15.193:0/1737807805 conn(0x7f75340f6c30 msgr2=0x7f75340f8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:16:04.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 shutdown_connections 2024-03-23T08:16:04.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:04.347+0000 7f7539509700 1 -- 172.21.15.193:0/1737807805 wait complete. 2024-03-23T08:16:04.348 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:04.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:04 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/1737807805' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:05.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:05 smithi070 ceph-mon[39699]: Deploying daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:16:06.228 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:16:06.228 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:16:06.516 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:16:08.409 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.408+0000 7fd1f171e700 1 -- 172.21.15.193:0/3223946840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 msgr2=0x7fd1ec0fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:08.409 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.408+0000 7fd1f171e700 1 --2- 172.21.15.193:0/3223946840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 0x7fd1ec0fde40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0048c0 tx=0x7fd1dc009e30 comp rx=0 tx=0).stop 2024-03-23T08:16:08.409 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.408+0000 7fd1f171e700 1 -- 172.21.15.193:0/3223946840 shutdown_connections 2024-03-23T08:16:08.409 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.408+0000 7fd1f171e700 1 --2- 172.21.15.193:0/3223946840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 0x7fd1ec0fde40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:08.410 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.408+0000 7fd1f171e700 1 -- 172.21.15.193:0/3223946840 >> 172.21.15.193:0/3223946840 conn(0x7fd1ec0f74e0 msgr2=0x7fd1ec0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:16:08.410 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.408+0000 7fd1f171e700 1 -- 172.21.15.193:0/3223946840 shutdown_connections 2024-03-23T08:16:08.410 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.408+0000 7fd1f171e700 1 -- 172.21.15.193:0/3223946840 wait complete. 2024-03-23T08:16:08.410 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.409+0000 7fd1f171e700 1 Processor -- start 2024-03-23T08:16:08.410 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.409+0000 7fd1f171e700 1 -- start start 2024-03-23T08:16:08.410 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.409+0000 7fd1f171e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 0x7fd1ec1130b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:08.410 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.409+0000 7fd1f171e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1135f0 con 0x7fd1ec0fda70 2024-03-23T08:16:08.411 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.409+0000 7fd1eaffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 0x7fd1ec1130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:08.411 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.410+0000 7fd1eaffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 0x7fd1ec1130b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.193:33188/0 (socket says 172.21.15.193:33188) 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.410+0000 7fd1eaffd700 1 -- 172.21.15.193:0/3244940719 learned_addr learned my addr 172.21.15.193:0/3244940719 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.410+0000 7fd1eaffd700 1 -- 172.21.15.193:0/3244940719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc005040 con 0x7fd1ec0fda70 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.410+0000 7fd1eaffd700 1 --2- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 0x7fd1ec1130b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc007e80 tx=0x7fd1dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.410+0000 7fd1d37fe700 1 -- 172.21.15.193:0/3244940719 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc0093c0 con 0x7fd1ec0fda70 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.410+0000 7fd1d37fe700 1 -- 172.21.15.193:0/3244940719 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd1dc013420 con 0x7fd1ec0fda70 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.410+0000 7fd1d37fe700 1 -- 172.21.15.193:0/3244940719 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1dc01c5b0 con 0x7fd1ec0fda70 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.411+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec1137f0 con 0x7fd1ec0fda70 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.411+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec113c60 con 0x7fd1ec0fda70 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.411+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec0665c0 con 0x7fd1ec0fda70 2024-03-23T08:16:08.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.412+0000 7fd1d37fe700 1 -- 172.21.15.193:0/3244940719 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fd1dc013a00 con 0x7fd1ec0fda70 2024-03-23T08:16:08.413 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.412+0000 7fd1d37fe700 1 --2- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1d40369c0 0x7fd1d4038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:08.413 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.412+0000 7fd1d37fe700 1 -- 172.21.15.193:0/3244940719 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd1dc049ac0 con 0x7fd1ec0fda70 2024-03-23T08:16:08.413 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.412+0000 7fd1ea7fc700 1 --2- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1d40369c0 0x7fd1d4038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:08.413 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.413+0000 7fd1ea7fc700 1 --2- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1d40369c0 0x7fd1d4038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009990 tx=0x7fd1e0006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:08.417 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.417+0000 7fd1d37fe700 1 -- 172.21.15.193:0/3244940719 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd1dc011960 con 0x7fd1ec0fda70 2024-03-23T08:16:08.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.715+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd1ec106cb0 con 0x7fd1ec0fda70 2024-03-23T08:16:08.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.716+0000 7fd1d37fe700 1 -- 172.21.15.193:0/3244940719 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd1dc011370 con 0x7fd1ec0fda70 2024-03-23T08:16:08.716 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:16:08.716 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:13:34.110495Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1d40369c0 msgr2=0x7fd1d4038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 --2- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1d40369c0 0x7fd1d4038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009990 tx=0x7fd1e0006e40 comp rx=0 tx=0).stop 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 msgr2=0x7fd1ec1130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 --2- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 0x7fd1ec1130b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc007e80 tx=0x7fd1dc009e30 comp rx=0 tx=0).stop 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 shutdown_connections 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 --2- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1d40369c0 0x7fd1d4038e70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 --2- 172.21.15.193:0/3244940719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1ec0fda70 0x7fd1ec1130b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 >> 172.21.15.193:0/3244940719 conn(0x7fd1ec0f74e0 msgr2=0x7fd1ec103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 shutdown_connections 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:08.718+0000 7fd1f171e700 1 -- 172.21.15.193:0/3244940719 wait complete. 2024-03-23T08:16:08.719 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-23T08:16:09.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:09.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:09.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:09.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:09.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:16:09.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:09.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: Deploying daemon mon.smithi193 on smithi193 2024-03-23T08:16:09.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:09.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:09.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:09 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/3244940719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.287 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 systemd[1]: Starting Ceph mon.smithi193 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:16:11.364 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-23T08:16:11.364 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-23T08:16:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:11 smithi070 ceph-mon[39699]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.828 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi193/config 2024-03-23T08:16:11.927 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T08:16:11.927 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable), process ceph-mon, pid 7 2024-03-23T08:16:11.927 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pidfile_write: ignore empty --pid-file 2024-03-23T08:16:11.927 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: load: jerasure load: lrc 2024-03-23T08:16:11.927 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: RocksDB version: 6.15.5 2024-03-23T08:16:11.927 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-23T08:16:11.927 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Compile date Mar 21 2024 2024-03-23T08:16:11.927 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: DB SUMMARY 2024-03-23T08:16:11.927 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: DB Session ID: HO13UXCFH6JX2ZJZZT3S 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: CURRENT file: CURRENT 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi193/store.db dir, Total Num: 0, files: 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi193/store.db: 000004.log size: 511 ; 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.error_if_exists: 0 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.create_if_missing: 0 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.paranoid_checks: 1 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.env: 0x560e9c250a80 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.fs: Posix File System 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.info_log: 0x560e9e291c00 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.statistics: (nil) 2024-03-23T08:16:11.928 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.use_fsync: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_log_file_size: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.allow_fallocate: 1 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.use_direct_reads: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.db_log_dir: 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi193/store.db 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T08:16:11.929 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.write_buffer_manager: 0x560e9f006ab0 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T08:16:11.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.unordered_write: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.row_cache: None 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.wal_filter: None 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.preserve_deletes: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.two_write_queues: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.atomic_flush: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T08:16:11.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.log_readahead_size: 0 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_background_jobs: 2 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_background_compactions: -1 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_subcompactions: 1 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T08:16:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_open_files: -1 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_background_flushes: -1 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Compression algorithms supported: 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: kZSTD supported: 0 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: kXpressCompression supported: 0 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: kZlibCompression supported: 1 2024-03-23T08:16:11.933 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: kSnappyCompression supported: 1 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: kLZ4Compression supported: 1 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: kBZip2Compression supported: 0 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000003 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.merge_operator: 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_filter: None 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560e9e2e6f58) 2024-03-23T08:16:11.934 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_type: 0 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_index_type: 0 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_shortening: 1 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: hash_index_allow_collision: 1 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: checksum: 1 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: no_block_cache: 0 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache: 0x560e9e319090 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_options: 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: capacity : 536870912 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: num_shard_bits : 4 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: strict_capacity_limit : 0 2024-03-23T08:16:11.935 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_compressed: (nil) 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: persistent_cache: (nil) 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size: 4096 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size_deviation: 10 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_restart_interval: 16 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_block_restart_interval: 1 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: metadata_block_size: 4096 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: partition_filters: 0 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: use_delta_encoding: 1 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: whole_key_filtering: 1 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: verify_compression: 0 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: format_version: 4 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_index_compression: 1 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_align: 0 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T08:16:11.936 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compression: NoCompression 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.num_levels: 7 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T08:16:11.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T08:16:11.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.arena_block_size: 4194304 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-23T08:16:11.939 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.table_properties_collectors: 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.inplace_update_support: 0 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T08:16:11.940 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.bloom_locality: 0 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.max_successive_merges: 0 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.ttl: 2592000 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.enable_blob_files: false 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.min_blob_size: 0 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T08:16:11.941 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711181771757642, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711181771758005, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711181771, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "276f4683-f710-41a0-adb9-72ebf177de1a", "db_session_id": "HO13UXCFH6JX2ZJZZT3S"}} 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711181771758427, "job": 1, "event": "recovery_finished"} 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi193/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x560e9e2ad180 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: DB pointer 0x560e9e382000 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-23T08:16:11.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** DB Stats ** 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:16:11.943 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:16:11.944 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T08:16:11.945 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193 does not exist in monmap, will attempt to join an existing cluster 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: starting mon.smithi193 rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi193 fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(???) e0 preinit fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).mds e1 new map 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).mds e1 print_map 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: e1 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:16:11.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: legacy client fscid: -1 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: No filesystems configured 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Deploying cephadm binary to smithi193 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Added host smithi193 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Deploying daemon alertmanager.smithi070 on smithi070 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3360015604' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-23T08:16:11.948 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3360015604' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: osdmap e4: 0 total, 0 up, 0 in 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Deploying daemon grafana.smithi070 on smithi070 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi070:172.21.15.70=smithi070;smithi193:172.21.15.193=smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Saving service mon spec with placement smithi070:172.21.15.70=smithi070;smithi193:172.21.15.193=smithi193;count:2 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.949 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/2077098823' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/4016647340' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/2317385160' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Deploying daemon prometheus.smithi070 on smithi070 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/3020989521' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/2351481237' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/4104933386' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14160 172.21.15.70:0/1841040323' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mgrmap e14: smithi070.jhsbch(active, since 59s) 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/1948221740' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/3049400417' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Active manager daemon smithi070.jhsbch restarted 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Activating manager daemon smithi070.jhsbch 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: osdmap e5: 0 total, 0 up, 0 in 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mgrmap e15: smithi070.jhsbch(active, starting, since 0.00299483s) 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:16:11.952 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Manager daemon smithi070.jhsbch is now available 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/trash_purge_schedule"}]: dispatch 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mgrmap e16: smithi070.jhsbch(active, since 1.00502s) 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: [23/Mar/2024:08:15:49] ENGINE Bus STARTING 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: [23/Mar/2024:08:15:49] ENGINE Serving on https://172.21.15.70:7150 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: [23/Mar/2024:08:15:49] ENGINE Bus STARTED 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/1347393043' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mgrmap e17: smithi070.jhsbch(active, since 2s) 2024-03-23T08:16:11.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:16:11.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.955 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Deploying daemon crash.smithi193 on smithi193 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/1819255870' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.956 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Deploying daemon node-exporter.smithi193 on smithi193 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/3047912467' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/1737807805' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.957 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Deploying daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:16:11.958 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.958 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.958 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.958 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.958 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:16:11.958 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:11.958 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: Deploying daemon mon.smithi193 on smithi193 2024-03-23T08:16:11.958 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.958 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:11.959 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/3244940719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:11.959 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:11.959 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 ceph-mon[48370]: mon.smithi193@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-03-23T08:16:11.959 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 bash[48306]: e0bad724d3200fe4fc458a9061fe3cbaf5b9b70aacabf2568272674f15497026 2024-03-23T08:16:11.959 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:11 smithi193 systemd[1]: Started Ceph mon.smithi193 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:16:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.151+0000 7fe7460b6700 1 -- 172.21.15.193:0/2776447838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe740101c50 msgr2=0x7fe7280055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.151+0000 7fe7460b6700 1 --2- 172.21.15.193:0/2776447838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe740101c50 0x7fe7280055d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe730007ed0 tx=0x7fe73000b040 comp rx=0 tx=0).stop 2024-03-23T08:16:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.151+0000 7fe7460b6700 1 -- 172.21.15.193:0/2776447838 shutdown_connections 2024-03-23T08:16:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.151+0000 7fe7460b6700 1 --2- 172.21.15.193:0/2776447838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe740101c50 0x7fe7280055d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.151+0000 7fe7460b6700 1 -- 172.21.15.193:0/2776447838 >> 172.21.15.193:0/2776447838 conn(0x7fe7400719d0 msgr2=0x7fe740071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:16:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.151+0000 7fe7460b6700 1 -- 172.21.15.193:0/2776447838 shutdown_connections 2024-03-23T08:16:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.151+0000 7fe7460b6700 1 -- 172.21.15.193:0/2776447838 wait complete. 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe7460b6700 1 Processor -- start 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe7460b6700 1 -- start start 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe7460b6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe740101c50 0x7fe740112090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe7460b6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 0x7fe74010bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe7460b6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe740112be0 con 0x7fe740101c50 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe7460b6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74010c530 con 0x7fe7401125d0 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe73effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 0x7fe74010bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe73effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 0x7fe74010bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:45302/0 (socket says 172.21.15.193:45302) 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe73effd700 1 -- 172.21.15.193:0/1933714887 learned_addr learned my addr 172.21.15.193:0/1933714887 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe73effd700 1 -- 172.21.15.193:0/1933714887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 msgr2=0x7fe74010bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T08:16:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe73effd700 1 -- 172.21.15.193:0/1933714887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 msgr2=0x7fe74010bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe73f7fe700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe740101c50 0x7fe740112090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe73effd700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 0x7fe74010bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.152+0000 7fe73effd700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 0x7fe74010bff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.153+0000 7fe73f7fe700 1 -- 172.21.15.193:0/1933714887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 msgr2=0x7fe74010bff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.153+0000 7fe73f7fe700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 0x7fe74010bff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.153+0000 7fe73f7fe700 1 -- 172.21.15.193:0/1933714887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730005040 con 0x7fe740101c50 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.153+0000 7fe73f7fe700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe740101c50 0x7fe740112090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe730004f50 tx=0x7fe73000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.153+0000 7fe727fff700 1 -- 172.21.15.193:0/1933714887 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe730009540 con 0x7fe740101c50 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.153+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74010c7b0 con 0x7fe740101c50 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.153+0000 7fe727fff700 1 -- 172.21.15.193:0/1933714887 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe730026070 con 0x7fe740101c50 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.153+0000 7fe727fff700 1 -- 172.21.15.193:0/1933714887 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe730021460 con 0x7fe740101c50 2024-03-23T08:16:17.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.153+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74010cd70 con 0x7fe740101c50 2024-03-23T08:16:17.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.154+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74005f5f0 con 0x7fe740101c50 2024-03-23T08:16:17.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.155+0000 7fe727fff700 1 -- 172.21.15.193:0/1933714887 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fe7300096a0 con 0x7fe740101c50 2024-03-23T08:16:17.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.155+0000 7fe727fff700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe72c069320 0x7fe72c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:17.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.155+0000 7fe727fff700 1 -- 172.21.15.193:0/1933714887 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe7300881c0 con 0x7fe740101c50 2024-03-23T08:16:17.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.155+0000 7fe73effd700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe72c069320 0x7fe72c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:17.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.156+0000 7fe73effd700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe72c069320 0x7fe72c06b7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe734006fd0 tx=0x7fe73400b3d0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:17.160 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.160+0000 7fe727fff700 1 -- 172.21.15.193:0/1933714887 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe730058fe0 con 0x7fe740101c50 2024-03-23T08:16:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:16:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: mon.smithi070 calling monitor election 2024-03-23T08:16:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: mon.smithi193 calling monitor election 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/292012651' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: mon.smithi070 is new leader, mons smithi070,smithi193 in quorum (ranks 0,1) 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: monmap e2: 2 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],smithi193=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: fsmap 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: osdmap e5: 0 total, 0 up, 0 in 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: mgrmap e17: smithi070.jhsbch(active, since 28s) 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: overall HEALTH_OK 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/292012651' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/292012651' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:16:17.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/292012651' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:16:17.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:16:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: mon.smithi070 calling monitor election 2024-03-23T08:16:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: mon.smithi193 calling monitor election 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/292012651' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: mon.smithi070 is new leader, mons smithi070,smithi193 in quorum (ranks 0,1) 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: monmap e2: 2 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],smithi193=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: fsmap 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: osdmap e5: 0 total, 0 up, 0 in 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: mgrmap e17: smithi070.jhsbch(active, since 28s) 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: overall HEALTH_OK 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:16:17.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/292012651' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:16:17.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/292012651' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:16:17.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/292012651' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:16:17.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:17.460 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.459+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe74004bed0 con 0x7fe740101c50 2024-03-23T08:16:17.460 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.460+0000 7fe727fff700 1 -- 172.21.15.193:0/1933714887 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7fe730030020 con 0x7fe740101c50 2024-03-23T08:16:17.461 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-23T08:16:17.461 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","modified":"2024-03-23T08:16:11.790633Z","created":"2024-03-23T08:13:34.110495Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi193","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-23T08:16:17.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.462+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe72c069320 msgr2=0x7fe72c06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:17.462 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.462+0000 7fe7460b6700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe72c069320 0x7fe72c06b7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe734006fd0 tx=0x7fe73400b3d0 comp rx=0 tx=0).stop 2024-03-23T08:16:17.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.462+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe740101c50 msgr2=0x7fe740112090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:17.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.462+0000 7fe7460b6700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe740101c50 0x7fe740112090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe730004f50 tx=0x7fe73000b040 comp rx=0 tx=0).stop 2024-03-23T08:16:17.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.462+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 shutdown_connections 2024-03-23T08:16:17.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.462+0000 7fe7460b6700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe72c069320 0x7fe72c06b7d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:17.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.462+0000 7fe7460b6700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe740101c50 0x7fe740112090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:17.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.462+0000 7fe7460b6700 1 --2- 172.21.15.193:0/1933714887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7401125d0 0x7fe74010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:17.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.462+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 >> 172.21.15.193:0/1933714887 conn(0x7fe7400719d0 msgr2=0x7fe7400fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:16:17.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.463+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 shutdown_connections 2024-03-23T08:16:17.464 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:16:17.463+0000 7fe7460b6700 1 -- 172.21.15.193:0/1933714887 wait complete. 2024-03-23T08:16:17.464 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-03-23T08:16:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:17 smithi070 ceph-mon[39699]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:17 smithi070 ceph-mon[39699]: mgrmap e18: smithi070.jhsbch(active, since 28s), standbys: smithi193.zgpdbu 2024-03-23T08:16:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:17 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi193.zgpdbu", "id": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:16:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:17 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:16:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:17 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/1933714887' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:17 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:18.297 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:17 smithi193 ceph-mon[48370]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:18.297 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:17 smithi193 ceph-mon[48370]: mgrmap e18: smithi070.jhsbch(active, since 28s), standbys: smithi193.zgpdbu 2024-03-23T08:16:18.297 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:17 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi193.zgpdbu", "id": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:16:18.298 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:17 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:16:18.298 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:17 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/1933714887' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-23T08:16:18.298 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:17 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:16:18.317 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-23T08:16:18.317 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-23T08:16:18.594 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:16:19.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:18 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:16:19.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:18 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:19.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:18 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:16:19.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:18 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:20.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:19 smithi070 ceph-mon[39699]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:20.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:20.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:20.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:20.144 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:16:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:19 smithi193 ceph-mon[48370]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:20.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:16:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.409+0000 7f4df6899700 1 -- 172.21.15.70:0/1604482886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 msgr2=0x7f4df0101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.409+0000 7f4df6899700 1 --2- 172.21.15.70:0/1604482886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 0x7f4df0101da0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80048c0 tx=0x7f4dd8009e30 comp rx=0 tx=0).stop 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.409+0000 7f4df6899700 1 -- 172.21.15.70:0/1604482886 shutdown_connections 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.409+0000 7f4df6899700 1 --2- 172.21.15.70:0/1604482886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 0x7f4df0101da0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.409+0000 7f4df6899700 1 -- 172.21.15.70:0/1604482886 >> 172.21.15.70:0/1604482886 conn(0x7f4df0071800 msgr2=0x7f4df0071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.409+0000 7f4df6899700 1 -- 172.21.15.70:0/1604482886 shutdown_connections 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.409+0000 7f4df6899700 1 -- 172.21.15.70:0/1604482886 wait complete. 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.409+0000 7f4df6899700 1 Processor -- start 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4df6899700 1 -- start start 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4df6899700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 0x7f4df01120b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4df6899700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4df01125f0 0x7f4df010c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4df6899700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0112c00 con 0x7f4df01019d0 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4df6899700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df010c550 con 0x7f4df01125f0 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4deffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 0x7f4df01120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4deffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 0x7f4df01120b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40752/0 (socket says 172.21.15.70:40752) 2024-03-23T08:16:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4deffff700 1 -- 172.21.15.70:0/3170531002 learned_addr learned my addr 172.21.15.70:0/3170531002 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4deffff700 1 -- 172.21.15.70:0/3170531002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4df01125f0 msgr2=0x7f4df010c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4deffff700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4df01125f0 0x7f4df010c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4deffff700 1 -- 172.21.15.70:0/3170531002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd8005040 con 0x7f4df01019d0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4deffff700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 0x7f4df01120b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800b3b0 tx=0x7f4dd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.410+0000 7f4decff9700 1 -- 172.21.15.70:0/3170531002 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dd800e040 con 0x7f4df01019d0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.411+0000 7f4df6899700 1 -- 172.21.15.70:0/3170531002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df010c7d0 con 0x7f4df01019d0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.411+0000 7f4df6899700 1 -- 172.21.15.70:0/3170531002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df010cd90 con 0x7f4df01019d0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.411+0000 7f4decff9700 1 -- 172.21.15.70:0/3170531002 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4dd8013420 con 0x7f4df01019d0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.411+0000 7f4decff9700 1 -- 172.21.15.70:0/3170531002 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dd8025640 con 0x7f4df01019d0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.411+0000 7f4de67fc700 1 -- 172.21.15.70:0/3170531002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd40052b0 con 0x7f4df01019d0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.412+0000 7f4decff9700 1 -- 172.21.15.70:0/3170531002 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f4dd80139e0 con 0x7f4df01019d0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.412+0000 7f4decff9700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4ddc069320 0x7f4ddc06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.412+0000 7f4decff9700 1 -- 172.21.15.70:0/3170531002 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f4dd80872b0 con 0x7f4df01019d0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.412+0000 7f4def7fe700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4ddc069320 0x7f4ddc06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.413+0000 7f4def7fe700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4ddc069320 0x7f4ddc06b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009930 tx=0x7f4de0006de0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:20.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.417+0000 7f4decff9700 1 -- 172.21.15.70:0/3170531002 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4dd8058150 con 0x7f4df01019d0 2024-03-23T08:16:20.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.618+0000 7f4de67fc700 1 -- 172.21.15.70:0/3170531002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4dd4005560 con 0x7f4df01019d0 2024-03-23T08:16:20.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.618+0000 7f4decff9700 1 -- 172.21.15.70:0/3170531002 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f4dd8086a60 con 0x7f4df01019d0 2024-03-23T08:16:20.619 INFO:teuthology.orchestra.run.smithi070.stdout:# minimal ceph.conf for 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:16:20.619 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-03-23T08:16:20.619 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:16:20.619 INFO:teuthology.orchestra.run.smithi070.stdout: mon_host = [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.620+0000 7f4de67fc700 1 -- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4ddc069320 msgr2=0x7f4ddc06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.620+0000 7f4de67fc700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4ddc069320 0x7f4ddc06b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009930 tx=0x7f4de0006de0 comp rx=0 tx=0).stop 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.620+0000 7f4de67fc700 1 -- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 msgr2=0x7f4df01120b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.620+0000 7f4de67fc700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 0x7f4df01120b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800b3b0 tx=0x7f4dd8009e30 comp rx=0 tx=0).stop 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.620+0000 7f4de67fc700 1 -- 172.21.15.70:0/3170531002 shutdown_connections 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.620+0000 7f4de67fc700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4ddc069320 0x7f4ddc06b7d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.620+0000 7f4de67fc700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4df01019d0 0x7f4df01120b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.620+0000 7f4de67fc700 1 --2- 172.21.15.70:0/3170531002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4df01125f0 0x7f4df010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.620+0000 7f4de67fc700 1 -- 172.21.15.70:0/3170531002 >> 172.21.15.70:0/3170531002 conn(0x7f4df0071800 msgr2=0x7f4df01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.621+0000 7f4de67fc700 1 -- 172.21.15.70:0/3170531002 shutdown_connections 2024-03-23T08:16:20.621 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:20.621+0000 7f4de67fc700 1 -- 172.21.15.70:0/3170531002 wait complete. 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:16:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:16:21.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:21.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:21 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3170531002' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:21.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:16:21.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:16:21.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:21 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3170531002' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:21.660 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-23T08:16:21.660 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:16:21.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T08:16:21.689 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:16:21.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:16:21.756 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:16:21.756 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-03-23T08:16:21.793 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:16:21.793 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:16:21.864 INFO:tasks.cephadm:Deploying OSDs... 2024-03-23T08:16:21.864 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:16:21.864 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T08:16:21.882 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T08:16:21.882 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-03-23T08:16:21.939 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T08:16:21.939 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T08:16:21.939 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 165844 Links: 1 2024-03-23T08:16:21.939 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:16:21.939 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T08:16:21.939 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-03-23 08:15:39.673048158 +0000 2024-03-23T08:16:21.939 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-03-23 08:07:58.142567878 +0000 2024-03-23T08:16:21.940 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-03-23 08:07:58.142567878 +0000 2024-03-23T08:16:21.940 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2024-03-23T08:16:21.940 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T08:16:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-03-23T08:16:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-03-23T08:16:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000112797 s, 4.5 MB/s 2024-03-23T08:16:22.005 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T08:16:22.064 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-03-23T08:16:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:22 smithi193 ceph-mon[48370]: Reconfiguring mon.smithi070 (unknown last config time)... 2024-03-23T08:16:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:22 smithi193 ceph-mon[48370]: Reconfiguring daemon mon.smithi070 on smithi070 2024-03-23T08:16:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:22 smithi193 ceph-mon[48370]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:22.123 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T08:16:22.123 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T08:16:22.123 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 168126 Links: 1 2024-03-23T08:16:22.124 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:16:22.124 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T08:16:22.124 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-03-23 08:15:39.673048158 +0000 2024-03-23T08:16:22.124 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-03-23 08:07:58.486557628 +0000 2024-03-23T08:16:22.124 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-03-23 08:07:58.486557628 +0000 2024-03-23T08:16:22.124 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2024-03-23T08:16:22.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T08:16:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-03-23T08:16:22.190 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-03-23T08:16:22.191 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 8.7933e-05 s, 5.8 MB/s 2024-03-23T08:16:22.191 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T08:16:22.250 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-03-23T08:16:22.307 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T08:16:22.307 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T08:16:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 169133 Links: 1 2024-03-23T08:16:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:16:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T08:16:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-03-23 08:15:39.673048158 +0000 2024-03-23T08:16:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-03-23 08:07:58.816547795 +0000 2024-03-23T08:16:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-03-23 08:07:58.816547795 +0000 2024-03-23T08:16:22.308 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2024-03-23T08:16:22.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T08:16:22.371 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:22 smithi070 ceph-mon[39699]: Reconfiguring mon.smithi070 (unknown last config time)... 2024-03-23T08:16:22.371 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:22 smithi070 ceph-mon[39699]: Reconfiguring daemon mon.smithi070 on smithi070 2024-03-23T08:16:22.371 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:22 smithi070 ceph-mon[39699]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:22.373 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-03-23T08:16:22.373 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-03-23T08:16:22.373 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 9.4259e-05 s, 5.4 MB/s 2024-03-23T08:16:22.374 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T08:16:22.433 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-03-23T08:16:22.491 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T08:16:22.492 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T08:16:22.492 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 6h/6d Inode: 172078 Links: 1 2024-03-23T08:16:22.492 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:16:22.492 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T08:16:22.492 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-03-23 08:15:39.673048158 +0000 2024-03-23T08:16:22.492 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-03-23 08:07:59.146537962 +0000 2024-03-23T08:16:22.492 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-03-23 08:07:59.146537962 +0000 2024-03-23T08:16:22.492 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2024-03-23T08:16:22.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T08:16:22.558 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-03-23T08:16:22.558 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-03-23T08:16:22.558 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000101811 s, 5.0 MB/s 2024-03-23T08:16:22.559 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T08:16:22.617 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:16:22.617 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T08:16:22.639 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T08:16:22.639 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-03-23T08:16:22.701 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T08:16:22.701 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T08:16:22.701 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 6h/6d Inode: 171069 Links: 1 2024-03-23T08:16:22.701 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:16:22.701 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T08:16:22.701 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-03-23 08:16:11.239522062 +0000 2024-03-23T08:16:22.701 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-03-23 08:07:58.517832290 +0000 2024-03-23T08:16:22.701 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-03-23 08:07:58.517832290 +0000 2024-03-23T08:16:22.701 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2024-03-23T08:16:22.701 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T08:16:22.770 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-03-23T08:16:22.771 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-03-23T08:16:22.771 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000239817 s, 2.1 MB/s 2024-03-23T08:16:22.771 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T08:16:22.832 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-03-23T08:16:22.892 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T08:16:22.892 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T08:16:22.892 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 6h/6d Inode: 169128 Links: 1 2024-03-23T08:16:22.892 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:16:22.893 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T08:16:22.893 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-03-23 08:16:11.239522062 +0000 2024-03-23T08:16:22.893 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-03-23 08:07:58.845827190 +0000 2024-03-23T08:16:22.893 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-03-23 08:07:58.845827190 +0000 2024-03-23T08:16:22.893 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2024-03-23T08:16:22.893 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T08:16:22.961 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-03-23T08:16:22.961 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-03-23T08:16:22.961 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00019854 s, 2.6 MB/s 2024-03-23T08:16:22.962 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T08:16:23.021 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-03-23T08:16:23.080 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T08:16:23.080 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T08:16:23.080 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 6h/6d Inode: 167295 Links: 1 2024-03-23T08:16:23.080 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:16:23.080 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T08:16:23.080 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-03-23 08:16:11.239522062 +0000 2024-03-23T08:16:23.080 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-03-23 08:07:59.180821981 +0000 2024-03-23T08:16:23.080 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-03-23 08:07:59.180821981 +0000 2024-03-23T08:16:23.081 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2024-03-23T08:16:23.081 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T08:16:23.148 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-03-23T08:16:23.148 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-03-23T08:16:23.148 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000165166 s, 3.1 MB/s 2024-03-23T08:16:23.149 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T08:16:23.209 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-03-23T08:16:23.267 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T08:16:23.268 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T08:16:23.268 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 6h/6d Inode: 172194 Links: 1 2024-03-23T08:16:23.268 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:16:23.268 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-03-23T08:16:23.268 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-03-23 08:16:11.240522047 +0000 2024-03-23T08:16:23.268 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-03-23 08:07:59.501816990 +0000 2024-03-23T08:16:23.268 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-03-23 08:07:59.501816990 +0000 2024-03-23T08:16:23.268 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2024-03-23T08:16:23.268 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T08:16:23.334 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-03-23T08:16:23.334 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-03-23T08:16:23.334 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000232658 s, 2.2 MB/s 2024-03-23T08:16:23.335 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T08:16:23.393 INFO:tasks.cephadm:Deploying osd.0 on smithi070 with /dev/vg_nvme/lv_4... 2024-03-23T08:16:23.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T08:16:23.419 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:23 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:23.420 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:23 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:23.420 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:23 smithi070 ceph-mon[39699]: Reconfiguring mgr.smithi070.jhsbch (unknown last config time)... 2024-03-23T08:16:23.420 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:23 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.jhsbch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:16:23.420 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:23 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:16:23.420 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:23 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:23.420 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:23 smithi070 ceph-mon[39699]: Reconfiguring daemon mgr.smithi070.jhsbch on smithi070 2024-03-23T08:16:23.420 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:23 smithi070 ceph-mon[39699]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:23.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:23 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:23.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:23 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:23.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:23 smithi193 ceph-mon[48370]: Reconfiguring mgr.smithi070.jhsbch (unknown last config time)... 2024-03-23T08:16:23.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:23 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.jhsbch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:16:23.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:23 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:16:23.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:23 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:23.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:23 smithi193 ceph-mon[48370]: Reconfiguring daemon mgr.smithi070.jhsbch on smithi070 2024-03-23T08:16:23.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:23 smithi193 ceph-mon[48370]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:23.664 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:16:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:24 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:24 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:24 smithi070 ceph-mon[39699]: Reconfiguring crash.smithi070 (monmap changed)... 2024-03-23T08:16:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:24 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:16:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:24 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:24 smithi070 ceph-mon[39699]: Reconfiguring daemon crash.smithi070 on smithi070 2024-03-23T08:16:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:24 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:24 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:24 smithi193 ceph-mon[48370]: Reconfiguring crash.smithi070 (monmap changed)... 2024-03-23T08:16:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:24 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:16:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:24 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:24 smithi193 ceph-mon[48370]: Reconfiguring daemon crash.smithi070 on smithi070 2024-03-23T08:16:25.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:25 smithi070 ceph-mon[39699]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:25.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:25 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:25.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:25 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:25 smithi193 ceph-mon[48370]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:25 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:25 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:26 smithi070 ceph-mon[39699]: Reconfiguring alertmanager.smithi070 (dependencies changed)... 2024-03-23T08:16:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:26 smithi070 ceph-mon[39699]: Reconfiguring daemon alertmanager.smithi070 on smithi070 2024-03-23T08:16:27.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:26 smithi193 ceph-mon[48370]: Reconfiguring alertmanager.smithi070 (dependencies changed)... 2024-03-23T08:16:27.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:26 smithi193 ceph-mon[48370]: Reconfiguring daemon alertmanager.smithi070 on smithi070 2024-03-23T08:16:27.140 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T08:16:27.140 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T08:16:27.140 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-03-23T08:16:27.140 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-03-23T08:16:27.140 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00818706 s, 1.3 GB/s 2024-03-23T08:16:27.140 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-03-23T08:16:27.976 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:27 smithi070 ceph-mon[39699]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:27 smithi193 ceph-mon[48370]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:28.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_4 2024-03-23T08:16:29.603 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:16:29.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:29 smithi070 ceph-mon[39699]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:29 smithi193 ceph-mon[48370]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.819+0000 7f0a59ff8700 1 -- 172.21.15.70:0/3357740400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fcfa0 msgr2=0x7f0a540fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.819+0000 7f0a59ff8700 1 --2- 172.21.15.70:0/3357740400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fcfa0 0x7f0a540fd410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a44004940 tx=0x7f0a44009e30 comp rx=0 tx=0).stop 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.819+0000 7f0a59ff8700 1 -- 172.21.15.70:0/3357740400 shutdown_connections 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.819+0000 7f0a59ff8700 1 --2- 172.21.15.70:0/3357740400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fcfa0 0x7f0a540fd410 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.819+0000 7f0a59ff8700 1 --2- 172.21.15.70:0/3357740400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a540fbe30 0x7f0a540fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.819+0000 7f0a59ff8700 1 -- 172.21.15.70:0/3357740400 >> 172.21.15.70:0/3357740400 conn(0x7f0a540f7480 msgr2=0x7f0a540f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.819+0000 7f0a59ff8700 1 -- 172.21.15.70:0/3357740400 shutdown_connections 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.819+0000 7f0a59ff8700 1 -- 172.21.15.70:0/3357740400 wait complete. 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.820+0000 7f0a59ff8700 1 Processor -- start 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.820+0000 7f0a59ff8700 1 -- start start 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.820+0000 7f0a59ff8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fbe30 0x7f0a5410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.820+0000 7f0a59ff8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a540fcfa0 0x7f0a5410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.820+0000 7f0a59ff8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5410e4f0 con 0x7f0a540fbe30 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.820+0000 7f0a59ff8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5410ece0 con 0x7f0a540fcfa0 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.820+0000 7f0a537fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fbe30 0x7f0a5410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.820+0000 7f0a537fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fbe30 0x7f0a5410d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35310/0 (socket says 172.21.15.70:35310) 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.820+0000 7f0a537fe700 1 -- 172.21.15.70:0/977534375 learned_addr learned my addr 172.21.15.70:0/977534375 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a52ffd700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a540fcfa0 0x7f0a5410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a537fe700 1 -- 172.21.15.70:0/977534375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a540fcfa0 msgr2=0x7f0a5410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a537fe700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a540fcfa0 0x7f0a5410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a537fe700 1 -- 172.21.15.70:0/977534375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a44005040 con 0x7f0a540fbe30 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a537fe700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fbe30 0x7f0a5410d920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800b790 tx=0x7f0a48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a3bfff700 1 -- 172.21.15.70:0/977534375 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a4800f770 con 0x7f0a540fbe30 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a3bfff700 1 -- 172.21.15.70:0/977534375 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a4800fd80 con 0x7f0a540fbe30 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a3bfff700 1 -- 172.21.15.70:0/977534375 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a4800d310 con 0x7f0a540fbe30 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5410efc0 con 0x7f0a540fbe30 2024-03-23T08:16:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.821+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5410f580 con 0x7f0a540fbe30 2024-03-23T08:16:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.822+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a54101b30 con 0x7f0a540fbe30 2024-03-23T08:16:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.823+0000 7f0a3bfff700 1 -- 172.21.15.70:0/977534375 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f0a48010420 con 0x7f0a540fbe30 2024-03-23T08:16:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.823+0000 7f0a3bfff700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0a3c069210 0x7f0a3c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:16:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.823+0000 7f0a3bfff700 1 -- 172.21.15.70:0/977534375 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0a48085f20 con 0x7f0a540fbe30 2024-03-23T08:16:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.823+0000 7f0a52ffd700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0a3c069210 0x7f0a3c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:16:31.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.823+0000 7f0a52ffd700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0a3c069210 0x7f0a3c06b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400b3b0 tx=0x7f0a44009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:16:31.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:31.827+0000 7f0a3bfff700 1 -- 172.21.15.70:0/977534375 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0a48056e60 con 0x7f0a540fbe30 2024-03-23T08:16:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:31 smithi070 ceph-mon[39699]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:32.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:16:32.023+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a54104940 con 0x7f0a3c069210 2024-03-23T08:16:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:31 smithi193 ceph-mon[48370]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:32.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:32 smithi070 ceph-mon[39699]: from='client.14240 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:16:32.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:32 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:16:32.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:32 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:16:32.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:32 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:32 smithi193 ceph-mon[48370]: from='client.14240 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:16:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:32 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:16:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:32 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:16:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:32 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:34 smithi070 ceph-mon[39699]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:34 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:34 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:34 smithi070 ceph-mon[39699]: Reconfiguring grafana.smithi070 (dependencies changed)... 2024-03-23T08:16:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:34 smithi070 ceph-mon[39699]: Reconfiguring daemon grafana.smithi070 on smithi070 2024-03-23T08:16:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:34 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:16:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:34 smithi193 ceph-mon[48370]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:34 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:34 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:34 smithi193 ceph-mon[48370]: Reconfiguring grafana.smithi070 (dependencies changed)... 2024-03-23T08:16:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:34 smithi193 ceph-mon[48370]: Reconfiguring daemon grafana.smithi070 on smithi070 2024-03-23T08:16:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:34 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:16:36.191 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:36 smithi070 ceph-mon[39699]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:36.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:36 smithi193 ceph-mon[48370]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:38.294 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:38 smithi070 ceph-mon[39699]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:38 smithi193 ceph-mon[48370]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:39.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:39 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:39.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:39 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:39.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:39 smithi070 ceph-mon[39699]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-03-23T08:16:39.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:39 smithi070 ceph-mon[39699]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-03-23T08:16:39.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:39 smithi070 ceph-mon[39699]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:39.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:39 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:39.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:39 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:39.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:39 smithi193 ceph-mon[48370]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-03-23T08:16:39.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:39 smithi193 ceph-mon[48370]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-03-23T08:16:39.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:39 smithi193 ceph-mon[48370]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:40 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2472121578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a71163c1-e6c6-490d-ade2-f345e4b22ee6"}]: dispatch 2024-03-23T08:16:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:40 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2472121578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a71163c1-e6c6-490d-ade2-f345e4b22ee6"}]': finished 2024-03-23T08:16:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:40 smithi070 ceph-mon[39699]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T08:16:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:40 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:16:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:40 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2754459147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:16:40.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:40 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2472121578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a71163c1-e6c6-490d-ade2-f345e4b22ee6"}]: dispatch 2024-03-23T08:16:40.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:40 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2472121578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a71163c1-e6c6-490d-ade2-f345e4b22ee6"}]': finished 2024-03-23T08:16:40.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:40 smithi193 ceph-mon[48370]: osdmap e6: 1 total, 0 up, 1 in 2024-03-23T08:16:40.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:40 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:16:40.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:40 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2754459147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:16:41.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:41 smithi070 ceph-mon[39699]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:41.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:41 smithi193 ceph-mon[48370]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:43.630 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:43 smithi070 ceph-mon[39699]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:43.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:43 smithi193 ceph-mon[48370]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:45.715 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:45 smithi070 ceph-mon[39699]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:45.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:45 smithi193 ceph-mon[48370]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:47.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:47 smithi070 ceph-mon[39699]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:47.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:47 smithi193 ceph-mon[48370]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:48.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:16:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:16:49.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:49 smithi070 ceph-mon[39699]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:49.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:49.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:49.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:16:49.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:49.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:49 smithi193 ceph-mon[48370]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:49.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:49 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:49.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:49 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:49.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:49 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:16:49.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:49 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:50.580 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:50 smithi070 ceph-mon[39699]: Reconfiguring crash.smithi193 (monmap changed)... 2024-03-23T08:16:50.580 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:50 smithi070 ceph-mon[39699]: Reconfiguring daemon crash.smithi193 on smithi193 2024-03-23T08:16:50.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:50 smithi193 ceph-mon[48370]: Reconfiguring crash.smithi193 (monmap changed)... 2024-03-23T08:16:50.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:50 smithi193 ceph-mon[48370]: Reconfiguring daemon crash.smithi193 on smithi193 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: Deploying daemon osd.0 on smithi070 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: Reconfiguring mgr.smithi193.zgpdbu (monmap changed)... 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:51 smithi070 ceph-mon[39699]: Reconfiguring daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:16:51.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:51.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T08:16:51.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:51.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: Deploying daemon osd.0 on smithi070 2024-03-23T08:16:51.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:51.809 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:51.809 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: Reconfiguring mgr.smithi193.zgpdbu (monmap changed)... 2024-03-23T08:16:51.809 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:16:51.809 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:16:51.809 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:51.809 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:51 smithi193 ceph-mon[48370]: Reconfiguring daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:16:53.273 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:53.273 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:53.273 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:53 smithi193 ceph-mon[48370]: Reconfiguring mon.smithi193 (monmap changed)... 2024-03-23T08:16:53.273 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:16:53.273 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:16:53.273 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:53.273 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:53 smithi193 ceph-mon[48370]: Reconfiguring daemon mon.smithi193 on smithi193 2024-03-23T08:16:53.273 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:53 smithi193 ceph-mon[48370]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:53.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:53.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:53.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:53 smithi070 ceph-mon[39699]: Reconfiguring mon.smithi193 (monmap changed)... 2024-03-23T08:16:53.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:16:53.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:16:53.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:16:53.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:53 smithi070 ceph-mon[39699]: Reconfiguring daemon mon.smithi193 on smithi193 2024-03-23T08:16:53.343 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:53 smithi070 ceph-mon[39699]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:16:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:16:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-03-23T08:16:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi070.front.sepia.ceph.com:3000"}]: dispatch 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi070.front.sepia.ceph.com:3000"}]: dispatch 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi070.front.sepia.ceph.com:9095"}]: dispatch 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi070.front.sepia.ceph.com:9095"}]: dispatch 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:16:54.817 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi070.front.sepia.ceph.com:3000"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi070.front.sepia.ceph.com:3000"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:16:54.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi070.front.sepia.ceph.com:9095"}]: dispatch 2024-03-23T08:16:54.819 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi070.front.sepia.ceph.com:9095"}]: dispatch 2024-03-23T08:16:54.819 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:54.819 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:16:55.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:55 smithi070 ceph-mon[39699]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:55.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:55.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:55 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:55 smithi193 ceph-mon[48370]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:55 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:55 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:57.726 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:57 smithi070 ceph-mon[39699]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:57.726 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:57.726 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:57.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:57 smithi193 ceph-mon[48370]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:57.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:57.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:16:59.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:16:59 smithi193 ceph-mon[48370]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:16:59.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:16:59 smithi070 ceph-mon[39699]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:17:00.813 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:00 smithi070 ceph-mon[39699]: from='osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T08:17:00.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:00 smithi193 ceph-mon[48370]: from='osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T08:17:00.852 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 0 on host 'smithi070' 2024-03-23T08:17:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.852+0000 7f0a3bfff700 1 -- 172.21.15.70:0/977534375 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0a54104940 con 0x7f0a3c069210 2024-03-23T08:17:00.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0a3c069210 msgr2=0x7f0a3c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:00.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0a3c069210 0x7f0a3c06b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400b3b0 tx=0x7f0a44009e30 comp rx=0 tx=0).stop 2024-03-23T08:17:00.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fbe30 msgr2=0x7f0a5410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:00.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fbe30 0x7f0a5410d920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800b790 tx=0x7f0a48009250 comp rx=0 tx=0).stop 2024-03-23T08:17:00.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 shutdown_connections 2024-03-23T08:17:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0a3c069210 0x7f0a3c06b6c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a540fbe30 0x7f0a5410d920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 --2- 172.21.15.70:0/977534375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a540fcfa0 0x7f0a5410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 >> 172.21.15.70:0/977534375 conn(0x7f0a540f7480 msgr2=0x7f0a54066440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:17:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 shutdown_connections 2024-03-23T08:17:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:00.854+0000 7f0a59ff8700 1 -- 172.21.15.70:0/977534375 wait complete. 2024-03-23T08:17:01.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:01 smithi070 ceph-mon[39699]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:17:01.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:01 smithi070 ceph-mon[39699]: from='osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T08:17:01.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:01 smithi070 ceph-mon[39699]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T08:17:01.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:01 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:17:01.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:01 smithi070 ceph-mon[39699]: from='osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:17:01.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:01 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:01.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:01 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:01.803 DEBUG:teuthology.orchestra.run.smithi070:osd.0> sudo journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.0.service 2024-03-23T08:17:01.805 INFO:tasks.cephadm:Deploying osd.1 on smithi070 with /dev/vg_nvme/lv_3... 2024-03-23T08:17:01.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T08:17:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:01 smithi193 ceph-mon[48370]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:17:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:01 smithi193 ceph-mon[48370]: from='osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T08:17:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:01 smithi193 ceph-mon[48370]: osdmap e7: 1 total, 0 up, 1 in 2024-03-23T08:17:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:01 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:17:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:01 smithi193 ceph-mon[48370]: from='osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:17:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:01 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:01 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:01.834 INFO:journalctl@ceph.osd.0.smithi070.stdout:-- Logs begin at Sat 2024-03-23 08:00:59 UTC. -- 2024-03-23T08:17:02.118 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:17:02.669 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:02 smithi070 ceph-mon[39699]: from='osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-03-23T08:17:02.669 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:02 smithi070 ceph-mon[39699]: osdmap e8: 1 total, 0 up, 1 in 2024-03-23T08:17:02.669 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:02 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:17:02.670 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:02 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:17:02.670 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:02 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:02.670 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:02 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:02 smithi193 ceph-mon[48370]: from='osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-03-23T08:17:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:02 smithi193 ceph-mon[48370]: osdmap e8: 1 total, 0 up, 1 in 2024-03-23T08:17:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:02 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:17:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:02 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:17:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:02 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:02 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:03.583 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:03 smithi070 ceph-mon[39699]: purged_snaps scrub starts 2024-03-23T08:17:03.583 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:03 smithi070 ceph-mon[39699]: purged_snaps scrub ok 2024-03-23T08:17:03.583 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:03 smithi070 ceph-mon[39699]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:17:03.583 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:03 smithi070 ceph-mon[39699]: osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] boot 2024-03-23T08:17:03.583 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:03 smithi070 ceph-mon[39699]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T08:17:03.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:17:03.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:17:03.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:03 smithi193 ceph-mon[48370]: purged_snaps scrub starts 2024-03-23T08:17:03.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:03 smithi193 ceph-mon[48370]: purged_snaps scrub ok 2024-03-23T08:17:03.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:03 smithi193 ceph-mon[48370]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-23T08:17:03.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:03 smithi193 ceph-mon[48370]: osd.0 [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] boot 2024-03-23T08:17:03.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:03 smithi193 ceph-mon[48370]: osdmap e9: 1 total, 1 up, 1 in 2024-03-23T08:17:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:17:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:17:04.366 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T08:17:04.366 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T08:17:04.366 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-03-23T08:17:04.366 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-03-23T08:17:04.366 INFO:teuthology.orchestra.run.smithi070.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00823207 s, 1.3 GB/s 2024-03-23T08:17:04.367 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-03-23T08:17:05.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:04 smithi070 ceph-mon[39699]: osdmap e10: 1 total, 1 up, 1 in 2024-03-23T08:17:05.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_3 2024-03-23T08:17:05.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:04 smithi193 ceph-mon[48370]: osdmap e10: 1 total, 1 up, 1 in 2024-03-23T08:17:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:17:05.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:05 smithi070 ceph-mon[39699]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:05 smithi193 ceph-mon[48370]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:08.202 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:07 smithi070 ceph-mon[39699]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:07 smithi193 ceph-mon[48370]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:08.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.481+0000 7f592def1700 1 -- 172.21.15.70:0/3073268569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fcee0 msgr2=0x7f59280fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:08.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.481+0000 7f592def1700 1 --2- 172.21.15.70:0/3073268569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fcee0 0x7f59280fd310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f591c004940 tx=0x7f591c009e30 comp rx=0 tx=0).stop 2024-03-23T08:17:08.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.481+0000 7f592def1700 1 -- 172.21.15.70:0/3073268569 shutdown_connections 2024-03-23T08:17:08.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.481+0000 7f592def1700 1 --2- 172.21.15.70:0/3073268569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59280fe0d0 0x7f59280fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:08.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.481+0000 7f592def1700 1 --2- 172.21.15.70:0/3073268569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fcee0 0x7f59280fd310 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:08.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.481+0000 7f592def1700 1 -- 172.21.15.70:0/3073268569 >> 172.21.15.70:0/3073268569 conn(0x7f59280f8560 msgr2=0x7f59280fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.481+0000 7f592def1700 1 -- 172.21.15.70:0/3073268569 shutdown_connections 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.481+0000 7f592def1700 1 -- 172.21.15.70:0/3073268569 wait complete. 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f592def1700 1 Processor -- start 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f592def1700 1 -- start start 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f592def1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59280fcee0 0x7f5928071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f592def1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fe0d0 0x7f5928070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f592def1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59280708c0 con 0x7f59280fe0d0 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f592def1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928070a30 con 0x7f59280fcee0 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f5926ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fe0d0 0x7f5928070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f5926ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fe0d0 0x7f5928070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47934/0 (socket says 172.21.15.70:47934) 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f5926ffd700 1 -- 172.21.15.70:0/2012264923 learned_addr learned my addr 172.21.15.70:0/2012264923 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f59277fe700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59280fcee0 0x7f5928071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f5926ffd700 1 -- 172.21.15.70:0/2012264923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59280fcee0 msgr2=0x7f5928071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:08.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f5926ffd700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59280fcee0 0x7f5928071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:08.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f5926ffd700 1 -- 172.21.15.70:0/2012264923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c005040 con 0x7f59280fe0d0 2024-03-23T08:17:08.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.482+0000 7f5926ffd700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fe0d0 0x7f5928070310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5918002a80 tx=0x7f59180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:17:08.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.483+0000 7f590ffff700 1 -- 172.21.15.70:0/2012264923 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5918007cc0 con 0x7f59280fe0d0 2024-03-23T08:17:08.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.483+0000 7f590ffff700 1 -- 172.21.15.70:0/2012264923 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f591800f040 con 0x7f59280fe0d0 2024-03-23T08:17:08.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.483+0000 7f590ffff700 1 -- 172.21.15.70:0/2012264923 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f591800da60 con 0x7f59280fe0d0 2024-03-23T08:17:08.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.483+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5928070d10 con 0x7f59280fe0d0 2024-03-23T08:17:08.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.483+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59281158f0 con 0x7f59280fe0d0 2024-03-23T08:17:08.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.484+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592805f5f0 con 0x7f59280fe0d0 2024-03-23T08:17:08.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.484+0000 7f590ffff700 1 -- 172.21.15.70:0/2012264923 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f591800dbc0 con 0x7f59280fe0d0 2024-03-23T08:17:08.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.485+0000 7f590ffff700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5910069210 0x7f591006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:17:08.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.485+0000 7f590ffff700 1 -- 172.21.15.70:0/2012264923 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f5918085f70 con 0x7f59280fe0d0 2024-03-23T08:17:08.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.485+0000 7f59277fe700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5910069210 0x7f591006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:17:08.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.485+0000 7f59277fe700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5910069210 0x7f591006b6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5928071bd0 tx=0x7f591c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:17:08.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.489+0000 7f590ffff700 1 -- 172.21.15.70:0/2012264923 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f59180530a0 con 0x7f59280fe0d0 2024-03-23T08:17:08.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:08.697+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f59281024e0 con 0x7f5910069210 2024-03-23T08:17:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: Detected new or changed devices on smithi070 2024-03-23T08:17:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:17:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: Adjusting osd_memory_target on smithi070 to 12898M 2024-03-23T08:17:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:09.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:09.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:17:09.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:09.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:17:09.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:17:09.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:17:09.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: Detected new or changed devices on smithi070 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: Adjusting osd_memory_target on smithi070 to 12898M 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:17:09.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:17:09.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:17:09.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:10 smithi193 ceph-mon[48370]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:10 smithi193 ceph-mon[48370]: from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:17:10.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:10 smithi070 ceph-mon[39699]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:10.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:10 smithi070 ceph-mon[39699]: from='client.14258 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:17:11.416 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:11 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:11.416 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:11 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:11.416 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:11 smithi070 ceph-mon[39699]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:11 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:11 smithi193 ceph-mon[48370]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:13.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:13 smithi070 ceph-mon[39699]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:13.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:13 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3189472856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0054cfce-6cdc-46f2-b434-ecd02d1ad193"}]: dispatch 2024-03-23T08:17:13.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:13 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3189472856' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0054cfce-6cdc-46f2-b434-ecd02d1ad193"}]': finished 2024-03-23T08:17:13.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:13 smithi070 ceph-mon[39699]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T08:17:13.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:13 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:13.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:13 smithi193 ceph-mon[48370]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:13.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:13 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3189472856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0054cfce-6cdc-46f2-b434-ecd02d1ad193"}]: dispatch 2024-03-23T08:17:13.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:13 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3189472856' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0054cfce-6cdc-46f2-b434-ecd02d1ad193"}]': finished 2024-03-23T08:17:13.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:13 smithi193 ceph-mon[48370]: osdmap e11: 2 total, 1 up, 2 in 2024-03-23T08:17:13.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:13 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:14.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:14 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/829152108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:17:14.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:14 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:14.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:14 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:14.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:14 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:14.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:14 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:17:14.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:14 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:14 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/829152108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:17:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:14 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:14 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:14 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:14 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:17:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:14 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:15 smithi193 ceph-mon[48370]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:15 smithi070 ceph-mon[39699]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:17 smithi070 ceph-mon[39699]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:17 smithi193 ceph-mon[48370]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:18.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:18 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:17:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:18 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:17:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:19 smithi193 ceph-mon[48370]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:19 smithi070 ceph-mon[39699]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:20.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:20 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T08:17:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:20 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:20 smithi070 ceph-mon[39699]: Deploying daemon osd.1 on smithi070 2024-03-23T08:17:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:20 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T08:17:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:20 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:20 smithi193 ceph-mon[48370]: Deploying daemon osd.1 on smithi070 2024-03-23T08:17:21.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:21 smithi070 ceph-mon[39699]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:21 smithi193 ceph-mon[48370]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:23 smithi193 ceph-mon[48370]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:24.098 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:23 smithi070 ceph-mon[39699]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:25 smithi070 ceph-mon[39699]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:25 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:25 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:25 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:17:26.297 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:25 smithi193 ceph-mon[48370]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:26.297 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:25 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:26.297 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:25 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:26.297 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:25 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:17:27.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:27 smithi070 ceph-mon[39699]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:27.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:27 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:27.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:27 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:27 smithi193 ceph-mon[48370]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:27 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:27 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:29.104 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 1 on host 'smithi070' 2024-03-23T08:17:29.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.103+0000 7f590ffff700 1 -- 172.21.15.70:0/2012264923 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f59281024e0 con 0x7f5910069210 2024-03-23T08:17:29.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.105+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5910069210 msgr2=0x7f591006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:29.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.105+0000 7f592def1700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5910069210 0x7f591006b6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5928071bd0 tx=0x7f591c009e30 comp rx=0 tx=0).stop 2024-03-23T08:17:29.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.106+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fe0d0 msgr2=0x7f5928070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:29.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.106+0000 7f592def1700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fe0d0 0x7f5928070310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5918002a80 tx=0x7f59180046b0 comp rx=0 tx=0).stop 2024-03-23T08:17:29.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.106+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 shutdown_connections 2024-03-23T08:17:29.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.106+0000 7f592def1700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59280fcee0 0x7f5928071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:29.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.106+0000 7f592def1700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5910069210 0x7f591006b6c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:29.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.106+0000 7f592def1700 1 --2- 172.21.15.70:0/2012264923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59280fe0d0 0x7f5928070310 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:29.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.106+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 >> 172.21.15.70:0/2012264923 conn(0x7f59280f8560 msgr2=0x7f59280f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:17:29.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.106+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 shutdown_connections 2024-03-23T08:17:29.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:29.106+0000 7f592def1700 1 -- 172.21.15.70:0/2012264923 wait complete. 2024-03-23T08:17:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:29 smithi070 ceph-mon[39699]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:29 smithi070 ceph-mon[39699]: from='osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T08:17:29.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:29 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:29.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:29 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:29 smithi193 ceph-mon[48370]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:29 smithi193 ceph-mon[48370]: from='osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T08:17:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:29 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:29 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:30.114 DEBUG:teuthology.orchestra.run.smithi070:osd.1> sudo journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.1.service 2024-03-23T08:17:30.117 INFO:tasks.cephadm:Deploying osd.2 on smithi070 with /dev/vg_nvme/lv_2... 2024-03-23T08:17:30.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T08:17:30.148 INFO:journalctl@ceph.osd.1.smithi070.stdout:-- Logs begin at Sat 2024-03-23 08:00:59 UTC. -- 2024-03-23T08:17:30.504 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:17:31.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:30 smithi193 ceph-mon[48370]: from='osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T08:17:31.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:30 smithi193 ceph-mon[48370]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T08:17:31.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:30 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:31.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:30 smithi193 ceph-mon[48370]: from='osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:17:31.233 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:17:30 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[72673]: 2024-03-23T08:17:30.817+0000 7fe9aa3a9700 -1 osd.1 0 waiting for initial osdmap 2024-03-23T08:17:31.233 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:17:30 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[72673]: 2024-03-23T08:17:30.822+0000 7fe9a31a0700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:17:31.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:30 smithi070 ceph-mon[39699]: from='osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T08:17:31.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:30 smithi070 ceph-mon[39699]: osdmap e12: 2 total, 1 up, 2 in 2024-03-23T08:17:31.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:30 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:31.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:30 smithi070 ceph-mon[39699]: from='osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:17:32.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:31 smithi193 ceph-mon[48370]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:32.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:31 smithi193 ceph-mon[48370]: from='osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-03-23T08:17:32.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:31 smithi193 ceph-mon[48370]: osdmap e13: 2 total, 1 up, 2 in 2024-03-23T08:17:32.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:31 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:32.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:31 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:32.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:31 smithi070 ceph-mon[39699]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:32.160 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:31 smithi070 ceph-mon[39699]: from='osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-03-23T08:17:32.160 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:31 smithi070 ceph-mon[39699]: osdmap e13: 2 total, 1 up, 2 in 2024-03-23T08:17:32.160 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:31 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:32.160 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:31 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:32 smithi070 ceph-mon[39699]: purged_snaps scrub starts 2024-03-23T08:17:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:32 smithi070 ceph-mon[39699]: purged_snaps scrub ok 2024-03-23T08:17:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:32 smithi070 ceph-mon[39699]: osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] boot 2024-03-23T08:17:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:32 smithi070 ceph-mon[39699]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T08:17:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:32 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:32 smithi193 ceph-mon[48370]: purged_snaps scrub starts 2024-03-23T08:17:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:32 smithi193 ceph-mon[48370]: purged_snaps scrub ok 2024-03-23T08:17:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:32 smithi193 ceph-mon[48370]: osd.1 [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] boot 2024-03-23T08:17:33.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:32 smithi193 ceph-mon[48370]: osdmap e14: 2 total, 2 up, 2 in 2024-03-23T08:17:33.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:32 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:17:33.333 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T08:17:33.333 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T08:17:33.333 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-03-23T08:17:33.333 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-03-23T08:17:33.333 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00844932 s, 1.2 GB/s 2024-03-23T08:17:33.333 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-03-23T08:17:33.971 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:33 smithi070 ceph-mon[39699]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:33.971 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:33 smithi070 ceph-mon[39699]: osdmap e15: 2 total, 2 up, 2 in 2024-03-23T08:17:33.971 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:33.971 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:33.972 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:17:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:33 smithi193 ceph-mon[48370]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-23T08:17:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:33 smithi193 ceph-mon[48370]: osdmap e15: 2 total, 2 up, 2 in 2024-03-23T08:17:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:17:34.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_2 2024-03-23T08:17:35.353 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:17:35.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:35 smithi070 ceph-mon[39699]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:36.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:35 smithi193 ceph-mon[48370]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:38.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.048+0000 7f20a6884700 1 -- 172.21.15.70:0/420759115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a01002e0 msgr2=0x7f20a01026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:38.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.048+0000 7f20a6884700 1 --2- 172.21.15.70:0/420759115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a01002e0 0x7f20a01026c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2094004940 tx=0x7f2094009e30 comp rx=0 tx=0).stop 2024-03-23T08:17:38.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.048+0000 7f20a6884700 1 -- 172.21.15.70:0/420759115 shutdown_connections 2024-03-23T08:17:38.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.048+0000 7f20a6884700 1 --2- 172.21.15.70:0/420759115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a01002e0 0x7f20a01026c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:38.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.048+0000 7f20a6884700 1 --2- 172.21.15.70:0/420759115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a00fd9c0 0x7f20a00ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:38.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.048+0000 7f20a6884700 1 -- 172.21.15.70:0/420759115 >> 172.21.15.70:0/420759115 conn(0x7f20a00f7480 msgr2=0x7f20a00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:17:38.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.048+0000 7f20a6884700 1 -- 172.21.15.70:0/420759115 shutdown_connections 2024-03-23T08:17:38.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.048+0000 7f20a6884700 1 -- 172.21.15.70:0/420759115 wait complete. 2024-03-23T08:17:38.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f20a6884700 1 Processor -- start 2024-03-23T08:17:38.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f20a6884700 1 -- start start 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f20a6884700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a00fd9c0 0x7f20a010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f20a6884700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a01002e0 0x7f20a010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f20a6884700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a010e480 con 0x7f20a00fd9c0 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f20a6884700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a010ecd0 con 0x7f20a01002e0 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f209ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a00fd9c0 0x7f20a010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f209ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a00fd9c0 0x7f20a010d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38358/0 (socket says 172.21.15.70:38358) 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f209ffff700 1 -- 172.21.15.70:0/311087487 learned_addr learned my addr 172.21.15.70:0/311087487 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.049+0000 7f209f7fe700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a01002e0 0x7f20a010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.050+0000 7f209ffff700 1 -- 172.21.15.70:0/311087487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a01002e0 msgr2=0x7f20a010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.050+0000 7f209ffff700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a01002e0 0x7f20a010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.050+0000 7f209ffff700 1 -- 172.21.15.70:0/311087487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2094005040 con 0x7f20a00fd9c0 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.050+0000 7f209ffff700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a00fd9c0 0x7f20a010d940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2090002a50 tx=0x7f2090004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.050+0000 7f209cff9700 1 -- 172.21.15.70:0/311087487 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2090007cc0 con 0x7f20a00fd9c0 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.050+0000 7f209cff9700 1 -- 172.21.15.70:0/311087487 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f209000f040 con 0x7f20a00fd9c0 2024-03-23T08:17:38.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.050+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a010efb0 con 0x7f20a00fd9c0 2024-03-23T08:17:38.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.050+0000 7f209cff9700 1 -- 172.21.15.70:0/311087487 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209000d920 con 0x7f20a00fd9c0 2024-03-23T08:17:38.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.050+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a010f4f0 con 0x7f20a00fd9c0 2024-03-23T08:17:38.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.051+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a005f5f0 con 0x7f20a00fd9c0 2024-03-23T08:17:38.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.051+0000 7f209cff9700 1 -- 172.21.15.70:0/311087487 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f2090009680 con 0x7f20a00fd9c0 2024-03-23T08:17:38.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.052+0000 7f209cff9700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2088069210 0x7f208806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:17:38.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.052+0000 7f209cff9700 1 -- 172.21.15.70:0/311087487 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f2090086140 con 0x7f20a00fd9c0 2024-03-23T08:17:38.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.052+0000 7f209f7fe700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2088069210 0x7f208806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:17:38.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.052+0000 7f209f7fe700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2088069210 0x7f208806b6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f209400b900 tx=0x7f2094009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:17:38.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.058+0000 7f209cff9700 1 -- 172.21.15.70:0/311087487 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2090056bd0 con 0x7f20a00fd9c0 2024-03-23T08:17:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:17:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:17:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:38.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:17:38.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:38.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:17:38.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:38.272+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f20a010f8c0 con 0x7f2088069210 2024-03-23T08:17:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:38.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:17:38.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:17:38.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:38.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:38.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:17:38.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:38.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:17:39.111 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:38 smithi070 ceph-mon[39699]: Detected new or changed devices on smithi070 2024-03-23T08:17:39.111 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:38 smithi070 ceph-mon[39699]: Adjusting osd_memory_target on smithi070 to 6449M 2024-03-23T08:17:39.111 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:17:39.111 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:17:39.112 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:38 smithi193 ceph-mon[48370]: Detected new or changed devices on smithi070 2024-03-23T08:17:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:38 smithi193 ceph-mon[48370]: Adjusting osd_memory_target on smithi070 to 6449M 2024-03-23T08:17:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:17:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:17:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:40.021 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:39 smithi070 ceph-mon[39699]: from='client.14276 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:17:40.021 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:39 smithi070 ceph-mon[39699]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:39 smithi193 ceph-mon[48370]: from='client.14276 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:17:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:39 smithi193 ceph-mon[48370]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:41 smithi070 ceph-mon[39699]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:41 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/548557185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24c96238-b75f-47c4-b8ef-3794e45e7d9c"}]: dispatch 2024-03-23T08:17:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:41 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/548557185' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24c96238-b75f-47c4-b8ef-3794e45e7d9c"}]': finished 2024-03-23T08:17:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:41 smithi070 ceph-mon[39699]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T08:17:42.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:41 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:17:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:41 smithi193 ceph-mon[48370]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:41 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/548557185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24c96238-b75f-47c4-b8ef-3794e45e7d9c"}]: dispatch 2024-03-23T08:17:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:41 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/548557185' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24c96238-b75f-47c4-b8ef-3794e45e7d9c"}]': finished 2024-03-23T08:17:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:41 smithi193 ceph-mon[48370]: osdmap e16: 3 total, 2 up, 3 in 2024-03-23T08:17:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:41 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:17:43.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:42 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1793554503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:17:43.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:42 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1793554503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:17:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:43 smithi070 ceph-mon[39699]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:43 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:43 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:43 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:43 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:17:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:43 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:43 smithi193 ceph-mon[48370]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:43 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:43 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:43 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:43 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:17:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:43 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:45.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:45 smithi070 ceph-mon[39699]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:46.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:45 smithi193 ceph-mon[48370]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:47 smithi070 ceph-mon[39699]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:47 smithi193 ceph-mon[48370]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:17:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T08:17:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:49.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:17:49.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T08:17:49.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:17:49.935 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:49 smithi070 ceph-mon[39699]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:49.936 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:49 smithi070 ceph-mon[39699]: Deploying daemon osd.2 on smithi070 2024-03-23T08:17:50.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:49 smithi193 ceph-mon[48370]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:50.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:49 smithi193 ceph-mon[48370]: Deploying daemon osd.2 on smithi070 2024-03-23T08:17:52.202 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:51 smithi070 ceph-mon[39699]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:51 smithi193 ceph-mon[48370]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:53.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:53 smithi070 ceph-mon[39699]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:53 smithi193 ceph-mon[48370]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:17:55.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:55.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:55.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:17:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:55 smithi070 ceph-mon[39699]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:55 smithi193 ceph-mon[48370]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:57.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.777+0000 7f209cff9700 1 -- 172.21.15.70:0/311087487 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f20a010f8c0 con 0x7f2088069210 2024-03-23T08:17:57.777 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 2 on host 'smithi070' 2024-03-23T08:17:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2088069210 msgr2=0x7f208806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2088069210 0x7f208806b6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f209400b900 tx=0x7f2094009e30 comp rx=0 tx=0).stop 2024-03-23T08:17:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a00fd9c0 msgr2=0x7f20a010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:17:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a00fd9c0 0x7f20a010d940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2090002a50 tx=0x7f2090004600 comp rx=0 tx=0).stop 2024-03-23T08:17:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 shutdown_connections 2024-03-23T08:17:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2088069210 0x7f208806b6c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:57.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20a00fd9c0 0x7f20a010d940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:57.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 --2- 172.21.15.70:0/311087487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a01002e0 0x7f20a010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:17:57.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 >> 172.21.15.70:0/311087487 conn(0x7f20a00f7480 msgr2=0x7f20a00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:17:57.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 shutdown_connections 2024-03-23T08:17:57.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:17:57.779+0000 7f20a6884700 1 -- 172.21.15.70:0/311087487 wait complete. 2024-03-23T08:17:58.203 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:57 smithi070 ceph-mon[39699]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:58.203 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:58.203 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:58.203 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:57 smithi070 ceph-mon[39699]: from='osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T08:17:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:57 smithi193 ceph-mon[48370]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:17:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:17:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:57 smithi193 ceph-mon[48370]: from='osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T08:17:58.487 DEBUG:teuthology.orchestra.run.smithi070:osd.2> sudo journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.2.service 2024-03-23T08:17:58.489 INFO:tasks.cephadm:Deploying osd.3 on smithi193 with /dev/vg_nvme/lv_4... 2024-03-23T08:17:58.489 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-23T08:17:58.519 INFO:journalctl@ceph.osd.2.smithi070.stdout:-- Logs begin at Sat 2024-03-23 08:00:59 UTC. -- 2024-03-23T08:17:58.759 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi193/config 2024-03-23T08:18:00.067 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:59 smithi193 ceph-mon[48370]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:18:00.067 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:59 smithi193 ceph-mon[48370]: from='osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T08:18:00.067 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:59 smithi193 ceph-mon[48370]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T08:18:00.067 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:59 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:00.067 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:17:59 smithi193 ceph-mon[48370]: from='osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:18:00.233 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:17:59 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[82465]: 2024-03-23T08:17:59.779+0000 7f3065897700 -1 osd.2 0 waiting for initial osdmap 2024-03-23T08:18:00.233 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:17:59 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[82465]: 2024-03-23T08:17:59.785+0000 7f3060692700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:18:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:59 smithi070 ceph-mon[39699]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:18:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:59 smithi070 ceph-mon[39699]: from='osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T08:18:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:59 smithi070 ceph-mon[39699]: osdmap e17: 3 total, 2 up, 3 in 2024-03-23T08:18:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:59 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:00.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:17:59 smithi070 ceph-mon[39699]: from='osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:18:00.887 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-23T08:18:00.887 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-23T08:18:00.888 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-03-23T08:18:00.888 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-03-23T08:18:00.888 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111126 s, 944 MB/s 2024-03-23T08:18:00.888 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-03-23T08:18:00.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:00 smithi070 ceph-mon[39699]: from='osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-03-23T08:18:00.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:00 smithi070 ceph-mon[39699]: osdmap e18: 3 total, 2 up, 3 in 2024-03-23T08:18:00.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:00 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:00.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:00 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:00 smithi193 ceph-mon[48370]: from='osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-03-23T08:18:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:00 smithi193 ceph-mon[48370]: osdmap e18: 3 total, 2 up, 3 in 2024-03-23T08:18:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:00 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:00 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:01.524 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-03-23T08:18:01.786 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi193/config 2024-03-23T08:18:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:01 smithi070 ceph-mon[39699]: purged_snaps scrub starts 2024-03-23T08:18:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:01 smithi070 ceph-mon[39699]: purged_snaps scrub ok 2024-03-23T08:18:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:01 smithi070 ceph-mon[39699]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:18:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:01 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:01 smithi070 ceph-mon[39699]: osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] boot 2024-03-23T08:18:01.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:01 smithi070 ceph-mon[39699]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T08:18:01.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:01 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:01.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:01 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:01.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:01 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:02.038 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:01 smithi193 ceph-mon[48370]: purged_snaps scrub starts 2024-03-23T08:18:02.038 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:01 smithi193 ceph-mon[48370]: purged_snaps scrub ok 2024-03-23T08:18:02.038 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:01 smithi193 ceph-mon[48370]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-23T08:18:02.039 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:01 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:02.039 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:01 smithi193 ceph-mon[48370]: osd.2 [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] boot 2024-03-23T08:18:02.039 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:01 smithi193 ceph-mon[48370]: osdmap e19: 3 total, 3 up, 3 in 2024-03-23T08:18:02.039 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:01 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:18:02.039 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:01 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:02.039 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:01 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:02.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:02 smithi193 ceph-mon[48370]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T08:18:02.871 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:02 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-23T08:18:02.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:02 smithi070 ceph-mon[39699]: osdmap e20: 3 total, 3 up, 3 in 2024-03-23T08:18:02.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:02 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-23T08:18:03.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.402+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1327835871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 msgr2=0x7f1a88102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:03.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.402+0000 7f1a8dcda700 1 --2- 172.21.15.193:0/1327835871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 0x7f1a88102320 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1a78004940 tx=0x7f1a78009e30 comp rx=0 tx=0).stop 2024-03-23T08:18:03.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.402+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1327835871 shutdown_connections 2024-03-23T08:18:03.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.402+0000 7f1a8dcda700 1 --2- 172.21.15.193:0/1327835871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 0x7f1a88102320 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:03.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.402+0000 7f1a8dcda700 1 --2- 172.21.15.193:0/1327835871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a880662e0 0x7f1a880ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:03.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.402+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1327835871 >> 172.21.15.193:0/1327835871 conn(0x7f1a880f7430 msgr2=0x7f1a880f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:18:03.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.402+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1327835871 shutdown_connections 2024-03-23T08:18:03.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.402+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1327835871 wait complete. 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a8dcda700 1 Processor -- start 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a8dcda700 1 -- start start 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a8dcda700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a880662e0 0x7f1a8810d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a8dcda700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 0x7f1a8810de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a8dcda700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8810e420 con 0x7f1a880662e0 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a8dcda700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8810ec70 con 0x7f1a880fff00 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a86ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 0x7f1a8810de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a86ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 0x7f1a8810de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:55792/0 (socket says 172.21.15.193:55792) 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a86ffd700 1 -- 172.21.15.193:0/1230560446 learned_addr learned my addr 172.21.15.193:0/1230560446 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a877fe700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a880662e0 0x7f1a8810d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a86ffd700 1 -- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a880662e0 msgr2=0x7f1a8810d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a86ffd700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a880662e0 0x7f1a8810d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.403+0000 7f1a86ffd700 1 -- 172.21.15.193:0/1230560446 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a78005040 con 0x7f1a880fff00 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.404+0000 7f1a86ffd700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 0x7f1a8810de20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1a780137b0 tx=0x7f1a78009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:18:03.405 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.404+0000 7f1a6ffff700 1 -- 172.21.15.193:0/1230560446 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7800e070 con 0x7f1a880fff00 2024-03-23T08:18:03.406 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.404+0000 7f1a6ffff700 1 -- 172.21.15.193:0/1230560446 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a7801c400 con 0x7f1a880fff00 2024-03-23T08:18:03.406 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.404+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8810eef0 con 0x7f1a880fff00 2024-03-23T08:18:03.406 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.404+0000 7f1a6ffff700 1 -- 172.21.15.193:0/1230560446 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7800b650 con 0x7f1a880fff00 2024-03-23T08:18:03.406 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.404+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8810f3d0 con 0x7f1a880fff00 2024-03-23T08:18:03.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.405+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8805f5f0 con 0x7f1a880fff00 2024-03-23T08:18:03.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.405+0000 7f1a6ffff700 1 -- 172.21.15.193:0/1230560446 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f1a7801c9e0 con 0x7f1a880fff00 2024-03-23T08:18:03.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.406+0000 7f1a6ffff700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f1a70069210 0x7f1a7006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:18:03.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.406+0000 7f1a6ffff700 1 -- 172.21.15.193:0/1230560446 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f1a78087dc0 con 0x7f1a880fff00 2024-03-23T08:18:03.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.406+0000 7f1a877fe700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f1a70069210 0x7f1a7006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:18:03.407 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.406+0000 7f1a877fe700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f1a70069210 0x7f1a7006b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009b80 tx=0x7f1a7c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:18:03.410 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.410+0000 7f1a6ffff700 1 -- 172.21.15.193:0/1230560446 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1a78058520 con 0x7f1a880fff00 2024-03-23T08:18:03.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:03.599+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a8810f700 con 0x7f1a70069210 2024-03-23T08:18:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:03 smithi193 ceph-mon[48370]: pgmap v75: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-23T08:18:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:03 smithi193 ceph-mon[48370]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T08:18:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T08:18:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:18:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:18:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:18:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:04.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:03 smithi070 ceph-mon[39699]: pgmap v75: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:04.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-23T08:18:04.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:03 smithi070 ceph-mon[39699]: osdmap e21: 3 total, 3 up, 3 in 2024-03-23T08:18:04.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-23T08:18:04.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:18:04.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:18:04.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:18:04.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: osdmap e22: 3 total, 3 up, 3 in 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:04.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:04.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:04.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:04.994 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: osdmap e22: 3 total, 3 up, 3 in 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:04.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:05.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:18:05 smithi070 sudo[88087]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T08:18:05.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:18:05 smithi070 sudo[88087]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:05.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:18:05 smithi070 sudo[88087]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:05.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:18:05 smithi070 sudo[88087]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:06.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:05 smithi193 ceph-mon[48370]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:06.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:05 smithi193 ceph-mon[48370]: Detected new or changed devices on smithi070 2024-03-23T08:18:06.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:05 smithi193 ceph-mon[48370]: Adjusting osd_memory_target on smithi070 to 4299M 2024-03-23T08:18:06.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:05 smithi193 ceph-mon[48370]: osdmap e23: 3 total, 3 up, 3 in 2024-03-23T08:18:06.113 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:18:05 smithi070 sudo[88158]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T08:18:06.113 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:18:05 smithi070 sudo[88158]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:06.113 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:18:05 smithi070 sudo[88158]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:06.113 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:18:05 smithi070 sudo[88158]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:06.113 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:05 smithi070 ceph-mon[39699]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:06.113 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:05 smithi070 ceph-mon[39699]: Detected new or changed devices on smithi070 2024-03-23T08:18:06.113 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:05 smithi070 ceph-mon[39699]: Adjusting osd_memory_target on smithi070 to 4299M 2024-03-23T08:18:06.113 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:05 smithi070 ceph-mon[39699]: osdmap e23: 3 total, 3 up, 3 in 2024-03-23T08:18:06.368 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88198]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T08:18:06.368 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88198]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:06.368 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88198]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:06.368 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88198]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:06.624 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88217]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T08:18:06.624 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88217]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:06.624 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88217]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:06.624 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88217]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:06.624 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88214]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T08:18:06.625 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:06.625 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88214]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:06.625 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88214]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:06.983 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88266]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T08:18:06.983 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88266]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:06.983 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88266]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:06.983 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:18:06 smithi070 sudo[88266]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:06 smithi070 ceph-mon[39699]: osdmap e24: 3 total, 3 up, 3 in 2024-03-23T08:18:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:06 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:06 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/1379580546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64e802c4-a037-4a42-8f0f-14756b62251b"}]: dispatch 2024-03-23T08:18:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:06 smithi070 ceph-mon[39699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64e802c4-a037-4a42-8f0f-14756b62251b"}]: dispatch 2024-03-23T08:18:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:06 smithi070 ceph-mon[39699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64e802c4-a037-4a42-8f0f-14756b62251b"}]': finished 2024-03-23T08:18:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:06 smithi070 ceph-mon[39699]: osdmap e25: 4 total, 3 up, 4 in 2024-03-23T08:18:06.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:06 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:06.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:06 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/667598039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:18:07.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:06 smithi193 ceph-mon[48370]: osdmap e24: 3 total, 3 up, 3 in 2024-03-23T08:18:07.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:06 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:07.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:06 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/1379580546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64e802c4-a037-4a42-8f0f-14756b62251b"}]: dispatch 2024-03-23T08:18:07.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:06 smithi193 ceph-mon[48370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64e802c4-a037-4a42-8f0f-14756b62251b"}]: dispatch 2024-03-23T08:18:07.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:06 smithi193 ceph-mon[48370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64e802c4-a037-4a42-8f0f-14756b62251b"}]': finished 2024-03-23T08:18:07.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:06 smithi193 ceph-mon[48370]: osdmap e25: 4 total, 3 up, 4 in 2024-03-23T08:18:07.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:06 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:07.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:06 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/667598039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:18:07.360 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 sudo[88297]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T08:18:07.360 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 sudo[88297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:07.360 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 sudo[88297]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:07.360 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 sudo[88297]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 sudo[88313]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T08:18:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 sudo[88313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 sudo[88313]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 sudo[88313]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:07.854 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 sudo[52741]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-23T08:18:07.855 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 sudo[52741]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:07.855 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 sudo[52741]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:07.855 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 sudo[52741]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:07.867 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:07.867+0000 7f1a6ffff700 1 -- 172.21.15.193:0/1230560446 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1a78054e60 con 0x7f1a880fff00 2024-03-23T08:18:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 ceph-mon[39699]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 ceph-mon[39699]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T08:18:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 ceph-mon[39699]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T08:18:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:18:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:18:08.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:18:08.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:07 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:18:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 sudo[52929]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-23T08:18:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 sudo[52929]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-23T08:18:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 sudo[52929]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-23T08:18:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 sudo[52929]: pam_unix(sudo:session): session closed for user root 2024-03-23T08:18:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 ceph-mon[48370]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:08.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 ceph-mon[48370]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T08:18:08.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 ceph-mon[48370]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T08:18:08.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:18:08.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:18:08.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:18:08.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:07 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:18:08.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:08 smithi070 ceph-mon[39699]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T08:18:08.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:08 smithi070 ceph-mon[39699]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T08:18:08.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:08 smithi070 ceph-mon[39699]: mgrmap e19: smithi070.jhsbch(active, since 2m), standbys: smithi193.zgpdbu 2024-03-23T08:18:08.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:08 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:08 smithi193 ceph-mon[48370]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-23T08:18:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:08 smithi193 ceph-mon[48370]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-23T08:18:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:08 smithi193 ceph-mon[48370]: mgrmap e19: smithi070.jhsbch(active, since 2m), standbys: smithi193.zgpdbu 2024-03-23T08:18:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:08 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:10.222 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:09 smithi193 ceph-mon[48370]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:10.222 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:10.222 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:10.222 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:10.223 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:10.223 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:09 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:09 smithi070 ceph-mon[39699]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:09 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:12.131 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:11 smithi193 ceph-mon[48370]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:11 smithi070 ceph-mon[39699]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:13 smithi193 ceph-mon[48370]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:13 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T08:18:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:13 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:13 smithi070 ceph-mon[39699]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:13 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T08:18:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:13 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:15.213 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:14 smithi193 ceph-mon[48370]: Deploying daemon osd.3 on smithi193 2024-03-23T08:18:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:14 smithi070 ceph-mon[39699]: Deploying daemon osd.3 on smithi193 2024-03-23T08:18:16.206 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:15 smithi193 ceph-mon[48370]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-23T08:18:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:15 smithi070 ceph-mon[39699]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-23T08:18:18.125 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:17 smithi193 ceph-mon[48370]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-23T08:18:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:17 smithi070 ceph-mon[39699]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-23T08:18:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:19 smithi070 ceph-mon[39699]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-23T08:18:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:18:19.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:19.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:19.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:19.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:19 smithi193 ceph-mon[48370]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-23T08:18:19.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:18:21.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:21 smithi070 ceph-mon[39699]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-23T08:18:21.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:21 smithi193 ceph-mon[48370]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-23T08:18:21.912 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.912+0000 7f1a6ffff700 1 -- 172.21.15.193:0/1230560446 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1a8810f700 con 0x7f1a70069210 2024-03-23T08:18:21.912 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 3 on host 'smithi193' 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f1a70069210 msgr2=0x7f1a7006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f1a70069210 0x7f1a7006b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009b80 tx=0x7f1a7c00b040 comp rx=0 tx=0).stop 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 msgr2=0x7f1a8810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 0x7f1a8810de20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1a780137b0 tx=0x7f1a78009e30 comp rx=0 tx=0).stop 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 shutdown_connections 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f1a70069210 0x7f1a7006b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a880662e0 0x7f1a8810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 --2- 172.21.15.193:0/1230560446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a880fff00 0x7f1a8810de20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 >> 172.21.15.193:0/1230560446 conn(0x7f1a880f7430 msgr2=0x7f1a880f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.914+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 shutdown_connections 2024-03-23T08:18:21.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:21.915+0000 7f1a8dcda700 1 -- 172.21.15.193:0/1230560446 wait complete. 2024-03-23T08:18:22.576 DEBUG:teuthology.orchestra.run.smithi193:osd.3> sudo journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.3.service 2024-03-23T08:18:22.582 INFO:tasks.cephadm:Deploying osd.4 on smithi193 with /dev/vg_nvme/lv_3... 2024-03-23T08:18:22.582 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-23T08:18:22.608 INFO:journalctl@ceph.osd.3.smithi193.stdout:-- Logs begin at Sat 2024-03-23 08:01:04 UTC. -- 2024-03-23T08:18:22.877 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi193/config 2024-03-23T08:18:23.160 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:22 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:23.160 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:22 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:23.160 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:22 smithi193 ceph-mon[48370]: from='osd.3 [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T08:18:23.160 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:22 smithi193 ceph-mon[48370]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T08:18:23.160 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:22 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:23.160 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:22 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:22 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:22 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:22 smithi070 ceph-mon[39699]: from='osd.3 [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T08:18:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:22 smithi070 ceph-mon[39699]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T08:18:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:22 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:22 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:23.990 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:18:23 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[54804]: 2024-03-23T08:18:23.897+0000 7f153e685700 -1 osd.3 0 waiting for initial osdmap 2024-03-23T08:18:23.991 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:18:23 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[54804]: 2024-03-23T08:18:23.903+0000 7f153747c700 -1 osd.3 27 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:18:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:23 smithi193 ceph-mon[48370]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-23T08:18:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:23 smithi193 ceph-mon[48370]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T08:18:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:23 smithi193 ceph-mon[48370]: osdmap e26: 4 total, 3 up, 4 in 2024-03-23T08:18:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:23 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:23 smithi193 ceph-mon[48370]: from='osd.3 [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:18:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:23 smithi193 ceph-mon[48370]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:18:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:23 smithi070 ceph-mon[39699]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-23T08:18:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:23 smithi070 ceph-mon[39699]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T08:18:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:23 smithi070 ceph-mon[39699]: osdmap e26: 4 total, 3 up, 4 in 2024-03-23T08:18:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:23 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:23 smithi070 ceph-mon[39699]: from='osd.3 [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:18:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:23 smithi070 ceph-mon[39699]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:18:25.070 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-23T08:18:25.070 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-23T08:18:25.070 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-03-23T08:18:25.070 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-03-23T08:18:25.070 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113713 s, 922 MB/s 2024-03-23T08:18:25.070 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-03-23T08:18:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:24 smithi193 ceph-mon[48370]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-23T08:18:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:24 smithi193 ceph-mon[48370]: osdmap e27: 4 total, 3 up, 4 in 2024-03-23T08:18:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:24 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:24 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:24 smithi193 ceph-mon[48370]: osd.3 [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] boot 2024-03-23T08:18:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:24 smithi193 ceph-mon[48370]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T08:18:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:24 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:24 smithi070 ceph-mon[39699]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-23T08:18:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:24 smithi070 ceph-mon[39699]: osdmap e27: 4 total, 3 up, 4 in 2024-03-23T08:18:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:24 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:24 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:24 smithi070 ceph-mon[39699]: osd.3 [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] boot 2024-03-23T08:18:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:24 smithi070 ceph-mon[39699]: osdmap e28: 4 total, 4 up, 4 in 2024-03-23T08:18:25.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:24 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:18:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:25 smithi193 ceph-mon[48370]: purged_snaps scrub starts 2024-03-23T08:18:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:25 smithi193 ceph-mon[48370]: purged_snaps scrub ok 2024-03-23T08:18:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:25 smithi193 ceph-mon[48370]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:25 smithi193 ceph-mon[48370]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T08:18:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:25 smithi070 ceph-mon[39699]: purged_snaps scrub starts 2024-03-23T08:18:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:25 smithi070 ceph-mon[39699]: purged_snaps scrub ok 2024-03-23T08:18:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:25 smithi070 ceph-mon[39699]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-23T08:18:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:25 smithi070 ceph-mon[39699]: osdmap e29: 4 total, 4 up, 4 in 2024-03-23T08:18:26.719 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-03-23T08:18:26.995 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi193/config 2024-03-23T08:18:27.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:27 smithi193 ceph-mon[48370]: osdmap e30: 4 total, 4 up, 4 in 2024-03-23T08:18:27.650 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:27 smithi193 ceph-mon[48370]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:27.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:27 smithi070 ceph-mon[39699]: osdmap e30: 4 total, 4 up, 4 in 2024-03-23T08:18:27.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:27 smithi070 ceph-mon[39699]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:28.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: osdmap e31: 4 total, 4 up, 4 in 2024-03-23T08:18:28.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: Detected new or changed devices on smithi193 2024-03-23T08:18:28.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:28.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:28.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:28.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: Adjusting osd_memory_target on smithi193 to 15970M 2024-03-23T08:18:28.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:28.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:28.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:28.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:28.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:28.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:28.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:28.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:28 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:28.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: osdmap e31: 4 total, 4 up, 4 in 2024-03-23T08:18:28.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: Detected new or changed devices on smithi193 2024-03-23T08:18:28.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:28.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:28.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:28.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: Adjusting osd_memory_target on smithi193 to 15970M 2024-03-23T08:18:28.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:28.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:28.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:28.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:28.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:28.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:28.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:28.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:28 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:29.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.067+0000 7f3952991700 1 -- 172.21.15.193:0/1799466600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c0fff90 msgr2=0x7f394c102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:29.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.067+0000 7f3952991700 1 --2- 172.21.15.193:0/1799466600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c0fff90 0x7f394c102370 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3940004940 tx=0x7f3940009e30 comp rx=0 tx=0).stop 2024-03-23T08:18:29.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.067+0000 7f3952991700 1 -- 172.21.15.193:0/1799466600 shutdown_connections 2024-03-23T08:18:29.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.067+0000 7f3952991700 1 --2- 172.21.15.193:0/1799466600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c0fff90 0x7f394c102370 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:29.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.067+0000 7f3952991700 1 --2- 172.21.15.193:0/1799466600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f394c066af0 0x7f394c0ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:29.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.067+0000 7f3952991700 1 -- 172.21.15.193:0/1799466600 >> 172.21.15.193:0/1799466600 conn(0x7f394c0f7440 msgr2=0x7f394c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:18:29.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.067+0000 7f3952991700 1 -- 172.21.15.193:0/1799466600 shutdown_connections 2024-03-23T08:18:29.070 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.067+0000 7f3952991700 1 -- 172.21.15.193:0/1799466600 wait complete. 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.068+0000 7f3952991700 1 Processor -- start 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.068+0000 7f3952991700 1 -- start start 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.068+0000 7f3952991700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c066af0 0x7f394c0fd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.068+0000 7f3952991700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f394c0fff90 0x7f394c0fb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.068+0000 7f3952991700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0fbf00 con 0x7f394c0fff90 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.068+0000 7f3952991700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0fc070 con 0x7f394c066af0 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.068+0000 7f394bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c066af0 0x7f394c0fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f394bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c066af0 0x7f394c0fd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:59454/0 (socket says 172.21.15.193:59454) 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f394bfff700 1 -- 172.21.15.193:0/3367465430 learned_addr learned my addr 172.21.15.193:0/3367465430 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f394b7fe700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f394c0fff90 0x7f394c0fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f394bfff700 1 -- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f394c0fff90 msgr2=0x7f394c0fb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f394bfff700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f394c0fff90 0x7f394c0fb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:29.071 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f394bfff700 1 -- 172.21.15.193:0/3367465430 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940005040 con 0x7f394c066af0 2024-03-23T08:18:29.072 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f394bfff700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c066af0 0x7f394c0fd480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c8b0 tx=0x7f393c004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:18:29.072 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f3948ff9700 1 -- 172.21.15.193:0/3367465430 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f393c0106f0 con 0x7f394c066af0 2024-03-23T08:18:29.072 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c0fc350 con 0x7f394c066af0 2024-03-23T08:18:29.072 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f3948ff9700 1 -- 172.21.15.193:0/3367465430 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f393c010d00 con 0x7f394c066af0 2024-03-23T08:18:29.072 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f3948ff9700 1 -- 172.21.15.193:0/3367465430 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f393c012dd0 con 0x7f394c066af0 2024-03-23T08:18:29.072 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.069+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c115940 con 0x7f394c066af0 2024-03-23T08:18:29.072 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.070+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c05f5f0 con 0x7f394c066af0 2024-03-23T08:18:29.073 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.071+0000 7f3948ff9700 1 -- 172.21.15.193:0/3367465430 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f393c012600 con 0x7f394c066af0 2024-03-23T08:18:29.073 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.071+0000 7f3948ff9700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f39340692a0 0x7f393406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:18:29.073 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.071+0000 7f3948ff9700 1 -- 172.21.15.193:0/3367465430 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f393c087590 con 0x7f394c066af0 2024-03-23T08:18:29.073 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.071+0000 7f394b7fe700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f39340692a0 0x7f393406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:18:29.073 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.072+0000 7f394b7fe700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f39340692a0 0x7f393406b750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f394000b900 tx=0x7f3940009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:18:29.075 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.075+0000 7f3948ff9700 1 -- 172.21.15.193:0/3367465430 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f393c053f60 con 0x7f394c066af0 2024-03-23T08:18:29.266 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:29.266+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f394c0f8f80 con 0x7f39340692a0 2024-03-23T08:18:29.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:29 smithi070 ceph-mon[39699]: osdmap e32: 4 total, 4 up, 4 in 2024-03-23T08:18:29.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:29 smithi070 ceph-mon[39699]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T08:18:29.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:29 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:18:29.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:29 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:18:29.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:29 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:29.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:29 smithi193 ceph-mon[48370]: osdmap e32: 4 total, 4 up, 4 in 2024-03-23T08:18:29.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:29 smithi193 ceph-mon[48370]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-23T08:18:29.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:29 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:18:29.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:29 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:18:29.808 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:29 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:30.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:30 smithi070 ceph-mon[39699]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:18:30.738 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:30 smithi193 ceph-mon[48370]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:18:31.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:31 smithi193 ceph-mon[48370]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-23T08:18:31.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:31 smithi070 ceph-mon[39699]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-03-23T08:18:32.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:32 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/3856900361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7"}]: dispatch 2024-03-23T08:18:32.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:32 smithi070 ceph-mon[39699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7"}]: dispatch 2024-03-23T08:18:32.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:32 smithi070 ceph-mon[39699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7"}]': finished 2024-03-23T08:18:32.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:32 smithi070 ceph-mon[39699]: osdmap e33: 5 total, 4 up, 5 in 2024-03-23T08:18:32.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:32 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:32 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/3856900361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7"}]: dispatch 2024-03-23T08:18:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:32 smithi193 ceph-mon[48370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7"}]: dispatch 2024-03-23T08:18:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:32 smithi193 ceph-mon[48370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7"}]': finished 2024-03-23T08:18:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:32 smithi193 ceph-mon[48370]: osdmap e33: 5 total, 4 up, 5 in 2024-03-23T08:18:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:32 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:33.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:33 smithi070 ceph-mon[39699]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T08:18:33.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:33 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/4142264291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:18:33.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:33 smithi193 ceph-mon[48370]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-23T08:18:33.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:33 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/4142264291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:18:34.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:34 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:18:34.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:34 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:18:35.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:35 smithi070 ceph-mon[39699]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-23T08:18:35.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:35 smithi193 ceph-mon[48370]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-23T08:18:37.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:37 smithi193 ceph-mon[48370]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:37.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:37 smithi070 ceph-mon[39699]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:39.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:39 smithi193 ceph-mon[48370]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:39.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:39 smithi070 ceph-mon[39699]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:40.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:40 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T08:18:40.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:40 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:40.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:40 smithi070 ceph-mon[39699]: Deploying daemon osd.4 on smithi193 2024-03-23T08:18:40.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:40 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T08:18:40.790 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:40 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:40.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:40 smithi193 ceph-mon[48370]: Deploying daemon osd.4 on smithi193 2024-03-23T08:18:41.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:41 smithi070 ceph-mon[39699]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:41.761 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:41 smithi193 ceph-mon[48370]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:43 smithi070 ceph-mon[39699]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:43.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:43 smithi193 ceph-mon[48370]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:45 smithi193 ceph-mon[48370]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:45 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:45 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:45 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:45 smithi070 ceph-mon[39699]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:45 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:45 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:45 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:48.002 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:47 smithi193 ceph-mon[48370]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:47 smithi070 ceph-mon[39699]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:48.686 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 4 on host 'smithi193' 2024-03-23T08:18:48.687 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.687+0000 7f3948ff9700 1 -- 172.21.15.193:0/3367465430 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f394c0f8f80 con 0x7f39340692a0 2024-03-23T08:18:48.689 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.690+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f39340692a0 msgr2=0x7f393406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.690+0000 7f3952991700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f39340692a0 0x7f393406b750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f394000b900 tx=0x7f3940009e30 comp rx=0 tx=0).stop 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.690+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c066af0 msgr2=0x7f394c0fd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.690+0000 7f3952991700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c066af0 0x7f394c0fd480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f393c00c8b0 tx=0x7f393c004470 comp rx=0 tx=0).stop 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.690+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 shutdown_connections 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.690+0000 7f3952991700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394c066af0 0x7f394c0fd480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.690+0000 7f3952991700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f39340692a0 0x7f393406b750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.690+0000 7f3952991700 1 --2- 172.21.15.193:0/3367465430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f394c0fff90 0x7f394c0fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.690+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 >> 172.21.15.193:0/3367465430 conn(0x7f394c0f7440 msgr2=0x7f394c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.691+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 shutdown_connections 2024-03-23T08:18:48.690 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:48.691+0000 7f3952991700 1 -- 172.21.15.193:0/3367465430 wait complete. 2024-03-23T08:18:49.046 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:18:49.046 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:48 smithi193 ceph-mon[48370]: from='osd.4 [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T08:18:49.046 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:48 smithi193 ceph-mon[48370]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T08:18:49.046 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:49.046 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:18:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:48 smithi070 ceph-mon[39699]: from='osd.4 [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T08:18:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:48 smithi070 ceph-mon[39699]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T08:18:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:49.330 DEBUG:teuthology.orchestra.run.smithi193:osd.4> sudo journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.4.service 2024-03-23T08:18:49.332 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_2... 2024-03-23T08:18:49.332 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-23T08:18:49.364 INFO:journalctl@ceph.osd.4.smithi193.stdout:-- Logs begin at Sat 2024-03-23 08:01:04 UTC. -- 2024-03-23T08:18:50.075 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[63545]: 2024-03-23T08:18:49.761+0000 7f9ceda7a700 -1 osd.4 0 waiting for initial osdmap 2024-03-23T08:18:50.075 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[63545]: 2024-03-23T08:18:49.769+0000 7f9ce9076700 -1 osd.4 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:18:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-mon[48370]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-mon[48370]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T08:18:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-mon[48370]: osdmap e34: 5 total, 4 up, 5 in 2024-03-23T08:18:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-mon[48370]: from='osd.4 [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:18:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-mon[48370]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:18:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:49 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:49 smithi070 ceph-mon[39699]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:49 smithi070 ceph-mon[39699]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T08:18:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:49 smithi070 ceph-mon[39699]: osdmap e34: 5 total, 4 up, 5 in 2024-03-23T08:18:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:49 smithi070 ceph-mon[39699]: from='osd.4 [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:18:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:49 smithi070 ceph-mon[39699]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:18:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:49 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:50.518 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi193/config 2024-03-23T08:18:50.772 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:50 smithi193 ceph-mon[48370]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-23T08:18:50.772 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:50 smithi193 ceph-mon[48370]: osdmap e35: 5 total, 4 up, 5 in 2024-03-23T08:18:50.773 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:50.773 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:50.773 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:50 smithi193 ceph-mon[48370]: osd.4 [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] boot 2024-03-23T08:18:50.773 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:50 smithi193 ceph-mon[48370]: osdmap e36: 5 total, 5 up, 5 in 2024-03-23T08:18:50.773 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:50 smithi070 ceph-mon[39699]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-23T08:18:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:50 smithi070 ceph-mon[39699]: osdmap e35: 5 total, 4 up, 5 in 2024-03-23T08:18:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:50 smithi070 ceph-mon[39699]: osd.4 [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] boot 2024-03-23T08:18:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:50 smithi070 ceph-mon[39699]: osdmap e36: 5 total, 5 up, 5 in 2024-03-23T08:18:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:18:51.818 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:51 smithi193 ceph-mon[48370]: purged_snaps scrub starts 2024-03-23T08:18:51.819 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:51 smithi193 ceph-mon[48370]: purged_snaps scrub ok 2024-03-23T08:18:51.819 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:51 smithi193 ceph-mon[48370]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:51 smithi070 ceph-mon[39699]: purged_snaps scrub starts 2024-03-23T08:18:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:51 smithi070 ceph-mon[39699]: purged_snaps scrub ok 2024-03-23T08:18:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:51 smithi070 ceph-mon[39699]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-23T08:18:52.956 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-23T08:18:52.956 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-23T08:18:52.956 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-03-23T08:18:52.956 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-03-23T08:18:52.956 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109611 s, 957 MB/s 2024-03-23T08:18:52.956 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-03-23T08:18:53.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:52 smithi193 ceph-mon[48370]: osdmap e37: 5 total, 5 up, 5 in 2024-03-23T08:18:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:52 smithi070 ceph-mon[39699]: osdmap e37: 5 total, 5 up, 5 in 2024-03-23T08:18:53.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:18:53.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:53.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:53.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:53.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:53.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:53.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:53.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:53.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:53.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:53.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:53.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:53.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:54.006 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-03-23T08:18:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:18:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:18:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:54.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:54.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:54.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:54.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:18:54.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:54.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:18:54.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:18:54.268 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi193/config 2024-03-23T08:18:55.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:54 smithi193 ceph-mon[48370]: Detected new or changed devices on smithi193 2024-03-23T08:18:55.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:54 smithi193 ceph-mon[48370]: Adjusting osd_memory_target on smithi193 to 7985M 2024-03-23T08:18:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:54 smithi070 ceph-mon[39699]: Detected new or changed devices on smithi193 2024-03-23T08:18:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:54 smithi070 ceph-mon[39699]: Adjusting osd_memory_target on smithi193 to 7985M 2024-03-23T08:18:55.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.961+0000 7efe09f21700 1 -- 172.21.15.193:0/3248340456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fe430 msgr2=0x7efe040fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.961+0000 7efe09f21700 1 --2- 172.21.15.193:0/3248340456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fe430 0x7efe040fe840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efdf4004940 tx=0x7efdf4009e30 comp rx=0 tx=0).stop 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.961+0000 7efe09f21700 1 -- 172.21.15.193:0/3248340456 shutdown_connections 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.961+0000 7efe09f21700 1 --2- 172.21.15.193:0/3248340456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040fed80 0x7efe040faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.961+0000 7efe09f21700 1 --2- 172.21.15.193:0/3248340456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fe430 0x7efe040fe840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.961+0000 7efe09f21700 1 -- 172.21.15.193:0/3248340456 >> 172.21.15.193:0/3248340456 conn(0x7efe040f6bd0 msgr2=0x7efe040f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.961+0000 7efe09f21700 1 -- 172.21.15.193:0/3248340456 shutdown_connections 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.961+0000 7efe09f21700 1 -- 172.21.15.193:0/3248340456 wait complete. 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.962+0000 7efe09f21700 1 Processor -- start 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.962+0000 7efe09f21700 1 -- start start 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.962+0000 7efe09f21700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040fe430 0x7efe0410d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.962+0000 7efe09f21700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fed80 0x7efe0410dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.962+0000 7efe09f21700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0410e320 con 0x7efe040fe430 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.962+0000 7efe09f21700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0410e490 con 0x7efe040fed80 2024-03-23T08:18:55.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.962+0000 7efe02ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fed80 0x7efe0410dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.962+0000 7efe02ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fed80 0x7efe0410dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:48614/0 (socket says 172.21.15.193:48614) 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.962+0000 7efe02ffd700 1 -- 172.21.15.193:0/4135618575 learned_addr learned my addr 172.21.15.193:0/4135618575 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efe037fe700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040fe430 0x7efe0410d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efe02ffd700 1 -- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040fe430 msgr2=0x7efe0410d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efe02ffd700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040fe430 0x7efe0410d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efe02ffd700 1 -- 172.21.15.193:0/4135618575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4005040 con 0x7efe040fed80 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efe037fe700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040fe430 0x7efe0410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efe02ffd700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fed80 0x7efe0410dd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efdf8002a50 tx=0x7efdf80046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efdebfff700 1 -- 172.21.15.193:0/4135618575 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf8007cc0 con 0x7efe040fed80 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efdebfff700 1 -- 172.21.15.193:0/4135618575 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efdf800f040 con 0x7efe040fed80 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe04112ef0 con 0x7efe040fed80 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efdebfff700 1 -- 172.21.15.193:0/4135618575 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf8009540 con 0x7efe040fed80 2024-03-23T08:18:55.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.963+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe04113480 con 0x7efe040fed80 2024-03-23T08:18:55.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.964+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0404c710 con 0x7efe040fed80 2024-03-23T08:18:55.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.964+0000 7efdebfff700 1 -- 172.21.15.193:0/4135618575 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7efdf80096a0 con 0x7efe040fed80 2024-03-23T08:18:55.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.965+0000 7efdebfff700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efdec0692a0 0x7efdec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:18:55.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.965+0000 7efdebfff700 1 -- 172.21.15.193:0/4135618575 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4202+0+0 (secure 0 0 0) 0x7efdf8086ad0 con 0x7efe040fed80 2024-03-23T08:18:55.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.965+0000 7efe037fe700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efdec0692a0 0x7efdec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:18:55.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.965+0000 7efe037fe700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efdec0692a0 0x7efdec06b750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b900 tx=0x7efdf4009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:18:55.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:55.969+0000 7efdebfff700 1 -- 172.21.15.193:0/4135618575 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efdf8056d00 con 0x7efe040fed80 2024-03-23T08:18:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:55 smithi193 ceph-mon[48370]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:18:56.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:18:56.158+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7efe04102550 con 0x7efdec0692a0 2024-03-23T08:18:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:55 smithi070 ceph-mon[39699]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:18:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:56 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:18:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:56 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:18:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:56 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:56 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-23T08:18:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:56 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-23T08:18:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:56 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:18:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:57 smithi193 ceph-mon[48370]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:18:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:57 smithi193 ceph-mon[48370]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:18:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:57 smithi070 ceph-mon[39699]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:18:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:57 smithi070 ceph-mon[39699]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:18:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:58 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/1276229408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57e8b767-4e62-41b1-8d96-caeaad28c929"}]: dispatch 2024-03-23T08:18:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:58 smithi193 ceph-mon[48370]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57e8b767-4e62-41b1-8d96-caeaad28c929"}]: dispatch 2024-03-23T08:18:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:58 smithi193 ceph-mon[48370]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57e8b767-4e62-41b1-8d96-caeaad28c929"}]': finished 2024-03-23T08:18:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:58 smithi193 ceph-mon[48370]: osdmap e38: 6 total, 5 up, 6 in 2024-03-23T08:18:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:58 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:18:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:58 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/1276229408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57e8b767-4e62-41b1-8d96-caeaad28c929"}]: dispatch 2024-03-23T08:18:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:58 smithi070 ceph-mon[39699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57e8b767-4e62-41b1-8d96-caeaad28c929"}]: dispatch 2024-03-23T08:18:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:58 smithi070 ceph-mon[39699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57e8b767-4e62-41b1-8d96-caeaad28c929"}]': finished 2024-03-23T08:18:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:58 smithi070 ceph-mon[39699]: osdmap e38: 6 total, 5 up, 6 in 2024-03-23T08:18:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:58 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:19:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:59 smithi193 ceph-mon[48370]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:18:59 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/4197164267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:19:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:59 smithi070 ceph-mon[39699]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:18:59 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/4197164267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-23T08:19:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:01 smithi193 ceph-mon[48370]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:01 smithi070 ceph-mon[39699]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:03 smithi193 ceph-mon[48370]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:19:04.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:03 smithi070 ceph-mon[39699]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:04.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:19:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:05 smithi193 ceph-mon[48370]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:06.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:05 smithi070 ceph-mon[39699]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:06.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:06 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T08:19:06.954 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:06 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:19:07.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:06 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T08:19:07.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:06 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:19:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:07 smithi193 ceph-mon[48370]: Deploying daemon osd.5 on smithi193 2024-03-23T08:19:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:07 smithi193 ceph-mon[48370]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:07 smithi070 ceph-mon[39699]: Deploying daemon osd.5 on smithi193 2024-03-23T08:19:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:07 smithi070 ceph-mon[39699]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:09 smithi193 ceph-mon[48370]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:09 smithi070 ceph-mon[39699]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:12.305 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:12 smithi193 ceph-mon[48370]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:12.306 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:12 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:12.306 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:12 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:12.306 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:12 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:19:12.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:12 smithi070 ceph-mon[39699]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:12.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:12 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:12.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:12 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:12.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:12 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:19:14.309 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:14 smithi193 ceph-mon[48370]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:14.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:14 smithi070 ceph-mon[39699]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:14.627 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-03-23T08:19:14.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.627+0000 7efdebfff700 1 -- 172.21.15.193:0/4135618575 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efe04102550 con 0x7efdec0692a0 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.629+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efdec0692a0 msgr2=0x7efdec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.629+0000 7efe09f21700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efdec0692a0 0x7efdec06b750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b900 tx=0x7efdf4009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.629+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fed80 msgr2=0x7efe0410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.629+0000 7efe09f21700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fed80 0x7efe0410dd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efdf8002a50 tx=0x7efdf80046b0 comp rx=0 tx=0).stop 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.629+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 shutdown_connections 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.629+0000 7efe09f21700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efdec0692a0 0x7efdec06b750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.629+0000 7efe09f21700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040fe430 0x7efe0410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.630+0000 7efe09f21700 1 --2- 172.21.15.193:0/4135618575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe040fed80 0x7efe0410dd20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.630+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 >> 172.21.15.193:0/4135618575 conn(0x7efe040f6bd0 msgr2=0x7efe040f82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.630+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 shutdown_connections 2024-03-23T08:19:14.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:14.630+0000 7efe09f21700 1 -- 172.21.15.193:0/4135618575 wait complete. 2024-03-23T08:19:15.260 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.5.service 2024-03-23T08:19:15.263 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-23T08:19:15.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-23T08:19:15.292 INFO:journalctl@ceph.osd.5.smithi193.stdout:-- Logs begin at Sat 2024-03-23 08:01:04 UTC. -- 2024-03-23T08:19:15.540 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:15.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:15 smithi070 ceph-mon[39699]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:15.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:15 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:15.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:15 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:15.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:15 smithi070 ceph-mon[39699]: from='osd.5 [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T08:19:15.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:15 smithi070 ceph-mon[39699]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T08:19:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:15 smithi193 ceph-mon[48370]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:15 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:15 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:15 smithi193 ceph-mon[48370]: from='osd.5 [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T08:19:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:15 smithi193 ceph-mon[48370]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T08:19:16.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:16 smithi070 ceph-mon[39699]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T08:19:16.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:16 smithi070 ceph-mon[39699]: osdmap e39: 6 total, 5 up, 6 in 2024-03-23T08:19:16.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:19:16.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:16 smithi070 ceph-mon[39699]: from='osd.5 [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:19:16.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:16 smithi070 ceph-mon[39699]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:19:16.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:16.951 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:16 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:16.961 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:19:16 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[71571]: 2024-03-23T08:19:16.606+0000 7f9728bcd700 -1 osd.5 0 waiting for initial osdmap 2024-03-23T08:19:16.961 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:19:16 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[71571]: 2024-03-23T08:19:16.614+0000 7f97241c9700 -1 osd.5 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:19:16.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:16 smithi193 ceph-mon[48370]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T08:19:16.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:16 smithi193 ceph-mon[48370]: osdmap e39: 6 total, 5 up, 6 in 2024-03-23T08:19:16.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:19:16.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:16 smithi193 ceph-mon[48370]: from='osd.5 [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:19:16.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:16 smithi193 ceph-mon[48370]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:19:16.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:16.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:16 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 -- 172.21.15.70:0/2635230091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fbe00 msgr2=0x7fd1500fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 --2- 172.21.15.70:0/2635230091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fbe00 0x7fd1500fc210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd144004940 tx=0x7fd144009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 -- 172.21.15.70:0/2635230091 shutdown_connections 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 --2- 172.21.15.70:0/2635230091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1500fcf70 0x7fd1500fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 --2- 172.21.15.70:0/2635230091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fbe00 0x7fd1500fc210 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 -- 172.21.15.70:0/2635230091 >> 172.21.15.70:0/2635230091 conn(0x7fd1500f74e0 msgr2=0x7fd1500f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 -- 172.21.15.70:0/2635230091 shutdown_connections 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 -- 172.21.15.70:0/2635230091 wait complete. 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 Processor -- start 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.475+0000 7fd155d4a700 1 -- start start 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd155d4a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1500fbe00 0x7fd15010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd155d4a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fcf70 0x7fd15010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd155d4a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15010e4f0 con 0x7fd1500fcf70 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd155d4a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15010ed40 con 0x7fd1500fbe00 2024-03-23T08:19:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd14effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fcf70 0x7fd15010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd14effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fcf70 0x7fd15010def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54962/0 (socket says 172.21.15.70:54962) 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd14effd700 1 -- 172.21.15.70:0/1409701018 learned_addr learned my addr 172.21.15.70:0/1409701018 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd14f7fe700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1500fbe00 0x7fd15010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd14effd700 1 -- 172.21.15.70:0/1409701018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1500fbe00 msgr2=0x7fd15010d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd14effd700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1500fbe00 0x7fd15010d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd14effd700 1 -- 172.21.15.70:0/1409701018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144005040 con 0x7fd1500fcf70 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd14effd700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fcf70 0x7fd15010def0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c8c0 tx=0x7fd1400044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd137fff700 1 -- 172.21.15.70:0/1409701018 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd14000a760 con 0x7fd1500fcf70 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15010f020 con 0x7fd1500fcf70 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd137fff700 1 -- 172.21.15.70:0/1409701018 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd14000ad70 con 0x7fd1500fcf70 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd137fff700 1 -- 172.21.15.70:0/1409701018 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1400113a0 con 0x7fd1500fcf70 2024-03-23T08:19:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.476+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15010f560 con 0x7fd1500fcf70 2024-03-23T08:19:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.477+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd150101b00 con 0x7fd1500fcf70 2024-03-23T08:19:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.478+0000 7fd137fff700 1 -- 172.21.15.70:0/1409701018 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd140012400 con 0x7fd1500fcf70 2024-03-23T08:19:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.478+0000 7fd137fff700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1380692b0 0x7fd13806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.478+0000 7fd137fff700 1 -- 172.21.15.70:0/1409701018 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4354+0+0 (secure 0 0 0) 0x7fd140086da0 con 0x7fd1500fcf70 2024-03-23T08:19:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.478+0000 7fd14f7fe700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1380692b0 0x7fd13806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:17.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.478+0000 7fd14f7fe700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1380692b0 0x7fd13806b760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd144009fd0 tx=0x7fd144009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:17.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.483+0000 7fd137fff700 1 -- 172.21.15.70:0/1409701018 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd140056fb0 con 0x7fd1500fcf70 2024-03-23T08:19:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.673+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd15005f5f0 con 0x7fd1500fcf70 2024-03-23T08:19:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.673+0000 7fd137fff700 1 -- 172.21.15.70:0/1409701018 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7fd14005c090 con 0x7fd1500fcf70 2024-03-23T08:19:17.673 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:19:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1380692b0 msgr2=0x7fd13806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1380692b0 0x7fd13806b760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd144009fd0 tx=0x7fd144009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fcf70 msgr2=0x7fd15010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fcf70 0x7fd15010def0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c8c0 tx=0x7fd1400044c0 comp rx=0 tx=0).stop 2024-03-23T08:19:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 shutdown_connections 2024-03-23T08:19:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1500fbe00 0x7fd15010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd1380692b0 0x7fd13806b760 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 --2- 172.21.15.70:0/1409701018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1500fcf70 0x7fd15010def0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 >> 172.21.15.70:0/1409701018 conn(0x7fd1500f74e0 msgr2=0x7fd150066560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 shutdown_connections 2024-03-23T08:19:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:17.675+0000 7fd155d4a700 1 -- 172.21.15.70:0/1409701018 wait complete. 2024-03-23T08:19:17.718 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:17 smithi193 ceph-mon[48370]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:17.718 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:17 smithi193 ceph-mon[48370]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-23T08:19:17.718 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:17 smithi193 ceph-mon[48370]: osdmap e40: 6 total, 5 up, 6 in 2024-03-23T08:19:17.718 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:17 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:19:17.719 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:17 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:19:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:17 smithi070 ceph-mon[39699]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:17 smithi070 ceph-mon[39699]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-23T08:19:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:17 smithi070 ceph-mon[39699]: osdmap e40: 6 total, 5 up, 6 in 2024-03-23T08:19:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:17 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:19:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:17 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:19:18.363 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1711181957,"num_in_osds":6,"osd_in_since":1711181938,"num_remapped_pgs":0} 2024-03-23T08:19:18.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T08:19:18.636 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:18.864 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:18 smithi070 ceph-mon[39699]: purged_snaps scrub starts 2024-03-23T08:19:18.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:18 smithi070 ceph-mon[39699]: purged_snaps scrub ok 2024-03-23T08:19:18.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:18 smithi070 ceph-mon[39699]: osd.5 [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] boot 2024-03-23T08:19:18.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:18 smithi070 ceph-mon[39699]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T08:19:18.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:18 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:19:18.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:18 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1409701018' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T08:19:18.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:18 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:19:18.899 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:18 smithi193 ceph-mon[48370]: purged_snaps scrub starts 2024-03-23T08:19:18.899 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:18 smithi193 ceph-mon[48370]: purged_snaps scrub ok 2024-03-23T08:19:18.900 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:18 smithi193 ceph-mon[48370]: osd.5 [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] boot 2024-03-23T08:19:18.900 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:18 smithi193 ceph-mon[48370]: osdmap e41: 6 total, 6 up, 6 in 2024-03-23T08:19:18.900 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:18 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:19:18.900 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:18 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1409701018' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-23T08:19:18.900 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:18 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:19:19.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:19.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T08:19:19.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:19.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:19.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:19:19.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:19:19.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:19:19.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:19.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:19:19.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:19:19.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:19.949 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: osdmap e42: 6 total, 6 up, 6 in 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:19:19.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:20.481 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.480+0000 7f4a32621700 1 -- 172.21.15.70:0/3207363031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 msgr2=0x7f4a2c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:20.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.480+0000 7f4a32621700 1 --2- 172.21.15.70:0/3207363031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 0x7f4a2c102140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c004940 tx=0x7f4a1c009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:20.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.480+0000 7f4a32621700 1 -- 172.21.15.70:0/3207363031 shutdown_connections 2024-03-23T08:19:20.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.480+0000 7f4a32621700 1 --2- 172.21.15.70:0/3207363031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a2c102710 0x7f4a2c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.480+0000 7f4a32621700 1 --2- 172.21.15.70:0/3207363031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 0x7f4a2c102140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.480+0000 7f4a32621700 1 -- 172.21.15.70:0/3207363031 >> 172.21.15.70:0/3207363031 conn(0x7f4a2c072bc0 msgr2=0x7f4a2c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.480+0000 7f4a32621700 1 -- 172.21.15.70:0/3207363031 shutdown_connections 2024-03-23T08:19:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.480+0000 7f4a32621700 1 -- 172.21.15.70:0/3207363031 wait complete. 2024-03-23T08:19:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a32621700 1 Processor -- start 2024-03-23T08:19:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a32621700 1 -- start start 2024-03-23T08:19:20.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a32621700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 0x7f4a2c1129b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:20.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a32621700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a2c102710 0x7f4a2c112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:20.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a32621700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c1135d0 con 0x7f4a2c101d70 2024-03-23T08:19:20.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a32621700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c10c930 con 0x7f4a2c102710 2024-03-23T08:19:20.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a2bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 0x7f4a2c1129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:20.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a2bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 0x7f4a2c1129b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58604/0 (socket says 172.21.15.70:58604) 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a2bfff700 1 -- 172.21.15.70:0/2412772617 learned_addr learned my addr 172.21.15.70:0/2412772617 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a2bfff700 1 -- 172.21.15.70:0/2412772617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a2c102710 msgr2=0x7f4a2c112ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a2bfff700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a2c102710 0x7f4a2c112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a2bfff700 1 -- 172.21.15.70:0/2412772617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c005040 con 0x7f4a2c101d70 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.481+0000 7f4a2bfff700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 0x7f4a2c1129b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00b3b0 tx=0x7f4a1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.482+0000 7f4a28ff9700 1 -- 172.21.15.70:0/2412772617 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a1c00e040 con 0x7f4a2c101d70 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.482+0000 7f4a28ff9700 1 -- 172.21.15.70:0/2412772617 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4a1c004290 con 0x7f4a2c101d70 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.482+0000 7f4a28ff9700 1 -- 172.21.15.70:0/2412772617 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a1c013660 con 0x7f4a2c101d70 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.482+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2c10cbb0 con 0x7f4a2c101d70 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.482+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2c10d110 con 0x7f4a2c101d70 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.483+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2c0fd4c0 con 0x7f4a2c101d70 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.484+0000 7f4a28ff9700 1 -- 172.21.15.70:0/2412772617 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4a1c00bd60 con 0x7f4a2c101d70 2024-03-23T08:19:20.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.484+0000 7f4a28ff9700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4a140692a0 0x7f4a1406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.484+0000 7f4a28ff9700 1 -- 172.21.15.70:0/2412772617 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4a1c08b870 con 0x7f4a2c101d70 2024-03-23T08:19:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.484+0000 7f4a2b7fe700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4a140692a0 0x7f4a1406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.484+0000 7f4a2b7fe700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4a140692a0 0x7f4a1406b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a20006fd0 tx=0x7f4a20006db0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.488+0000 7f4a28ff9700 1 -- 172.21.15.70:0/2412772617 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4a1c05b970 con 0x7f4a2c101d70 2024-03-23T08:19:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.672+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a2c05f630 con 0x7f4a2c101d70 2024-03-23T08:19:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.673+0000 7f4a28ff9700 1 -- 172.21.15.70:0/2412772617 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+11083 (secure 0 0 0) 0x7f4a1c05b500 con 0x7f4a2c101d70 2024-03-23T08:19:20.673 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:19:20.673 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":42,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","created":"2024-03-23T08:13:36.916195+0000","modified":"2024-03-23T08:19:18.609709+0000","last_up_change":"2024-03-23T08:19:17.604967+0000","last_in_change":"2024-03-23T08:18:58.641957+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T08:18:02.704319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a71163c1-e6c6-490d-ade2-f345e4b22ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6803","nonce":2404324767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6805","nonce":2404324767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6809","nonce":2404324767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6807","nonce":2404324767}]},"public_addr":"172.21.15.70:6803/2404324767","cluster_addr":"172.21.15.70:6805/2404324767","heartbeat_back_addr":"172.21.15.70:6809/2404324767","heartbeat_front_addr":"172.21.15.70:6807/2404324767","state":["exists","up"]},{"osd":1,"uuid":"0054cfce-6cdc-46f2-b434-ecd02d1ad193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6811","nonce":3805865717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6813","nonce":3805865717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6817","nonce":3805865717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6815","nonce":3805865717}]},"public_addr":"172.21.15.70:6811/3805865717","cluster_addr":"172.21.15.70:6813/3805865717","heartbeat_back_addr":"172.21.15.70:6817/3805865717","heartbeat_front_addr":"172.21.15.70:6815/3805865717","state":["exists","up"]},{"osd":2,"uuid":"24c96238-b75f-47c4-b8ef-3794e45e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6819","nonce":3317266675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6821","nonce":3317266675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6825","nonce":3317266675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6823","nonce":3317266675}]},"public_addr":"172.21.15.70:6819/3317266675","cluster_addr":"172.21.15.70:6821/3317266675","heartbeat_back_addr":"172.21.15.70:6825/3317266675","heartbeat_front_addr":"172.21.15.70:6823/3317266675","state":["exists","up"]},{"osd":3,"uuid":"64e802c4-a037-4a42-8f0f-14756b62251b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6801","nonce":1644042787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6803","nonce":1644042787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6807","nonce":1644042787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6805","nonce":1644042787}]},"public_addr":"172.21.15.193:6801/1644042787","cluster_addr":"172.21.15.193:6803/1644042787","heartbeat_back_addr":"172.21.15.193:6807/1644042787","heartbeat_front_addr":"172.21.15.193:6805/1644042787","state":["exists","up"]},{"osd":4,"uuid":"3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859463992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859463992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859463992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859463992}]},"public_addr":"172.21.15.193:6809/2859463992","cluster_addr":"172.21.15.193:6811/2859463992","heartbeat_back_addr":"172.21.15.193:6815/2859463992","heartbeat_front_addr":"172.21.15.193:6813/2859463992","state":["exists","up"]},{"osd":5,"uuid":"57e8b767-4e62-41b1-8d96-caeaad28c929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6817","nonce":863428472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6819","nonce":863428472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6823","nonce":863428472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6821","nonce":863428472}]},"public_addr":"172.21.15.193:6817/863428472","cluster_addr":"172.21.15.193:6819/863428472","heartbeat_back_addr":"172.21.15.193:6823/863428472","heartbeat_front_addr":"172.21.15.193:6821/863428472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:01.224711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:29.865012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:58.894993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:23.061992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:49.665476+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:19:16.008523+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/2209675146":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/1608386260":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/3721968086":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/562426678":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/3484360896":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:6801/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/3004659310":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/3991559747":"2024-03-24T08:15:48.390242+0000","172.21.15.70:6800/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6800/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/3894371824":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6801/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/370857700":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6800/2149030412":"2024-03-24T08:14:42.564176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4a140692a0 msgr2=0x7f4a1406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4a140692a0 0x7f4a1406b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a20006fd0 tx=0x7f4a20006db0 comp rx=0 tx=0).stop 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 msgr2=0x7f4a2c1129b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 0x7f4a2c1129b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00b3b0 tx=0x7f4a1c009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 shutdown_connections 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4a140692a0 0x7f4a1406b750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a2c101d70 0x7f4a2c1129b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 --2- 172.21.15.70:0/2412772617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a2c102710 0x7f4a2c112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 >> 172.21.15.70:0/2412772617 conn(0x7f4a2c072bc0 msgr2=0x7f4a2c0faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.675+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 shutdown_connections 2024-03-23T08:19:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:20.676+0000 7f4a32621700 1 -- 172.21.15.70:0/2412772617 wait complete. 2024-03-23T08:19:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:20 smithi070 ceph-mon[39699]: Detected new or changed devices on smithi193 2024-03-23T08:19:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:20 smithi070 ceph-mon[39699]: Adjusting osd_memory_target on smithi193 to 5323M 2024-03-23T08:19:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:20 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:19:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:20 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:19:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:20 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:19:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:20 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:20 smithi193 ceph-mon[48370]: Detected new or changed devices on smithi193 2024-03-23T08:19:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:20 smithi193 ceph-mon[48370]: Adjusting osd_memory_target on smithi193 to 5323M 2024-03-23T08:19:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:20 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:19:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:20 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:19:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:20 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:19:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:20 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:19:21.395 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T08:18:02.704319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-23T08:19:21.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-23T08:19:21.659 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:21.928 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:21 smithi070 ceph-mon[39699]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:21.928 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:21 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2412772617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:19:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:21 smithi193 ceph-mon[48370]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:21 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2412772617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:19:23.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.408+0000 7f6261999700 1 -- 172.21.15.70:0/3094067967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 msgr2=0x7f625c0fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:23.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.408+0000 7f6261999700 1 --2- 172.21.15.70:0/3094067967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 0x7f625c0fe2a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6254004940 tx=0x7f6254009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:23.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.408+0000 7f6261999700 1 -- 172.21.15.70:0/3094067967 shutdown_connections 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.408+0000 7f6261999700 1 --2- 172.21.15.70:0/3094067967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f625c0fe870 0x7f625c106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.408+0000 7f6261999700 1 --2- 172.21.15.70:0/3094067967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 0x7f625c0fe2a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.408+0000 7f6261999700 1 -- 172.21.15.70:0/3094067967 >> 172.21.15.70:0/3094067967 conn(0x7f625c072bc0 msgr2=0x7f625c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.408+0000 7f6261999700 1 -- 172.21.15.70:0/3094067967 shutdown_connections 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.408+0000 7f6261999700 1 -- 172.21.15.70:0/3094067967 wait complete. 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.409+0000 7f6261999700 1 Processor -- start 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.409+0000 7f6261999700 1 -- start start 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.409+0000 7f6261999700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 0x7f625c113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.409+0000 7f6261999700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f625c0fe870 0x7f625c113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.409+0000 7f6261999700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c114630 con 0x7f625c0fded0 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.409+0000 7f6261999700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c10d9e0 con 0x7f625c0fe870 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.409+0000 7f625affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 0x7f625c113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.409+0000 7f625affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 0x7f625c113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58612/0 (socket says 172.21.15.70:58612) 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.409+0000 7f625affd700 1 -- 172.21.15.70:0/2262425269 learned_addr learned my addr 172.21.15.70:0/2262425269 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f625a7fc700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f625c0fe870 0x7f625c113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:23.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f625affd700 1 -- 172.21.15.70:0/2262425269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f625c0fe870 msgr2=0x7f625c113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f625affd700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f625c0fe870 0x7f625c113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f625affd700 1 -- 172.21.15.70:0/2262425269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254005040 con 0x7f625c0fded0 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f625a7fc700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f625c0fe870 0x7f625c113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f625affd700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 0x7f625c113a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f625400b3b0 tx=0x7f6254009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f62437fe700 1 -- 172.21.15.70:0/2262425269 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f625400e040 con 0x7f625c0fded0 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f62437fe700 1 -- 172.21.15.70:0/2262425269 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6254013420 con 0x7f625c0fded0 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625c10dcc0 con 0x7f625c0fded0 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f62437fe700 1 -- 172.21.15.70:0/2262425269 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f625401c630 con 0x7f625c0fded0 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.410+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625c10e200 con 0x7f625c0fded0 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.411+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625c065df0 con 0x7f625c0fded0 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.411+0000 7f62437fe700 1 -- 172.21.15.70:0/2262425269 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6254013590 con 0x7f625c0fded0 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.412+0000 7f62437fe700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f62440692b0 0x7f624406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.412+0000 7f625a7fc700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f62440692b0 0x7f624406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:23.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.412+0000 7f62437fe700 1 -- 172.21.15.70:0/2262425269 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f62540880b0 con 0x7f625c0fded0 2024-03-23T08:19:23.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.412+0000 7f625a7fc700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f62440692b0 0x7f624406b760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6250006fd0 tx=0x7f6250008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:23.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.416+0000 7f62437fe700 1 -- 172.21.15.70:0/2262425269 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6254058130 con 0x7f625c0fded0 2024-03-23T08:19:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.601+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f625c05f5f0 con 0x7f625c0fded0 2024-03-23T08:19:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.601+0000 7f62437fe700 1 -- 172.21.15.70:0/2262425269 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6254057cc0 con 0x7f625c0fded0 2024-03-23T08:19:23.601 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-03-23T08:19:23.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f62440692b0 msgr2=0x7f624406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f62440692b0 0x7f624406b760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6250006fd0 tx=0x7f6250008040 comp rx=0 tx=0).stop 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 msgr2=0x7f625c113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 0x7f625c113a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f625400b3b0 tx=0x7f6254009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 shutdown_connections 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f62440692b0 0x7f624406b760 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f625c0fded0 0x7f625c113a10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 --2- 172.21.15.70:0/2262425269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f625c0fe870 0x7f625c113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 >> 172.21.15.70:0/2262425269 conn(0x7f625c072bc0 msgr2=0x7f625c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 shutdown_connections 2024-03-23T08:19:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:23.603+0000 7f6261999700 1 -- 172.21.15.70:0/2262425269 wait complete. 2024-03-23T08:19:23.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:23 smithi070 ceph-mon[39699]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:23.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:23 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2262425269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:19:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:23 smithi193 ceph-mon[48370]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:23 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2262425269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:19:24.333 INFO:tasks.cephadm:Setting up client nodes... 2024-03-23T08:19:24.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-23T08:19:24.600 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:25.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:25 smithi070 ceph-mon[39699]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:25 smithi193 ceph-mon[48370]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:26.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 -- 172.21.15.70:0/451265790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef41023d0 msgr2=0x7fbef41047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:26.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 --2- 172.21.15.70:0/451265790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef41023d0 0x7fbef41047b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbee0004940 tx=0x7fbee0009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:26.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 -- 172.21.15.70:0/451265790 shutdown_connections 2024-03-23T08:19:26.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 --2- 172.21.15.70:0/451265790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef41023d0 0x7fbef41047b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:26.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 --2- 172.21.15.70:0/451265790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbef4066200 0x7fbef4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:26.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 -- 172.21.15.70:0/451265790 >> 172.21.15.70:0/451265790 conn(0x7fbef40f7460 msgr2=0x7fbef40f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:26.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 -- 172.21.15.70:0/451265790 shutdown_connections 2024-03-23T08:19:26.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 -- 172.21.15.70:0/451265790 wait complete. 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 Processor -- start 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.344+0000 7fbefad92700 1 -- start start 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbefad92700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef4066200 0x7fbef410fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbefad92700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbef41023d0 0x7fbef4110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbefad92700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4110610 con 0x7fbef4066200 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbefad92700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4110780 con 0x7fbef41023d0 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef8b2e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef4066200 0x7fbef410fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef8b2e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef4066200 0x7fbef410fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58630/0 (socket says 172.21.15.70:58630) 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef8b2e700 1 -- 172.21.15.70:0/3041868693 learned_addr learned my addr 172.21.15.70:0/3041868693 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef3fff700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbef41023d0 0x7fbef4110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef8b2e700 1 -- 172.21.15.70:0/3041868693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbef41023d0 msgr2=0x7fbef4110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef8b2e700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbef41023d0 0x7fbef4110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:26.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef8b2e700 1 -- 172.21.15.70:0/3041868693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0005040 con 0x7fbef4066200 2024-03-23T08:19:26.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef8b2e700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef4066200 0x7fbef410fad0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbee8002a80 tx=0x7fbee8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:26.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef17fa700 1 -- 172.21.15.70:0/3041868693 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee8007cc0 con 0x7fbef4066200 2024-03-23T08:19:26.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef41151e0 con 0x7fbef4066200 2024-03-23T08:19:26.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef17fa700 1 -- 172.21.15.70:0/3041868693 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbee800f040 con 0x7fbef4066200 2024-03-23T08:19:26.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbef17fa700 1 -- 172.21.15.70:0/3041868693 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee800da20 con 0x7fbef4066200 2024-03-23T08:19:26.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.345+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef41157a0 con 0x7fbef4066200 2024-03-23T08:19:26.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.346+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef404c710 con 0x7fbef4066200 2024-03-23T08:19:26.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.347+0000 7fbef17fa700 1 -- 172.21.15.70:0/3041868693 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbee800db80 con 0x7fbef4066200 2024-03-23T08:19:26.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.347+0000 7fbef17fa700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbee40692a0 0x7fbee406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:26.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.347+0000 7fbef17fa700 1 -- 172.21.15.70:0/3041868693 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbee8086c40 con 0x7fbef4066200 2024-03-23T08:19:26.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.347+0000 7fbef3fff700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbee40692a0 0x7fbee406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:26.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.347+0000 7fbef3fff700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbee40692a0 0x7fbee406b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009fd0 tx=0x7fbee0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:26.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.351+0000 7fbef17fa700 1 -- 172.21.15.70:0/3041868693 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbee8056cc0 con 0x7fbef4066200 2024-03-23T08:19:26.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.612+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fbef41079e0 con 0x7fbef4066200 2024-03-23T08:19:26.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.616+0000 7fbef17fa700 1 -- 172.21.15.70:0/3041868693 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7fbee8056850 con 0x7fbef4066200 2024-03-23T08:19:26.616 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2024-03-23T08:19:26.617 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCOkP5leKihJBAAVQbCngb5P20FL9zpDVSDkA== 2024-03-23T08:19:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbee40692a0 msgr2=0x7fbee406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbee40692a0 0x7fbee406b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009fd0 tx=0x7fbee0009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef4066200 msgr2=0x7fbef410fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef4066200 0x7fbef410fad0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbee8002a80 tx=0x7fbee8004600 comp rx=0 tx=0).stop 2024-03-23T08:19:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 shutdown_connections 2024-03-23T08:19:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbee40692a0 0x7fbee406b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef4066200 0x7fbef410fad0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 --2- 172.21.15.70:0/3041868693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbef41023d0 0x7fbef4110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 >> 172.21.15.70:0/3041868693 conn(0x7fbef40f7460 msgr2=0x7fbef40f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 shutdown_connections 2024-03-23T08:19:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:26.618+0000 7fbefad92700 1 -- 172.21.15.70:0/3041868693 wait complete. 2024-03-23T08:19:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:26 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3041868693' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T08:19:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:26 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3041868693' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T08:19:27.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:26 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3041868693' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T08:19:27.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:26 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3041868693' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T08:19:27.423 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:19:27.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-23T08:19:27.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-23T08:19:27.465 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-23T08:19:27.741 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi193/config 2024-03-23T08:19:27.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:27 smithi193 ceph-mon[48370]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:27.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:27 smithi070 ceph-mon[39699]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.389+0000 7f00eb24f700 1 -- 172.21.15.193:0/2597885751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 msgr2=0x7f00e40fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.389+0000 7f00eb24f700 1 --2- 172.21.15.193:0/2597885751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 0x7f00e40fd290 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f00d8004940 tx=0x7f00d8009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.390+0000 7f00eb24f700 1 -- 172.21.15.193:0/2597885751 shutdown_connections 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.390+0000 7f00eb24f700 1 --2- 172.21.15.193:0/2597885751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e40fdff0 0x7f00e40fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.390+0000 7f00eb24f700 1 --2- 172.21.15.193:0/2597885751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 0x7f00e40fd290 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.390+0000 7f00eb24f700 1 -- 172.21.15.193:0/2597885751 >> 172.21.15.193:0/2597885751 conn(0x7f00e40f8580 msgr2=0x7f00e40fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.390+0000 7f00eb24f700 1 -- 172.21.15.193:0/2597885751 shutdown_connections 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.390+0000 7f00eb24f700 1 -- 172.21.15.193:0/2597885751 wait complete. 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.390+0000 7f00eb24f700 1 Processor -- start 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.390+0000 7f00eb24f700 1 -- start start 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00eb24f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 0x7f00e410d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00eb24f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e40fdff0 0x7f00e410dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:29.392 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00eb24f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e410e410 con 0x7f00e40fdff0 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00eb24f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e4112db0 con 0x7f00e40fce80 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e8feb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 0x7f00e410d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e8feb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 0x7f00e410d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:42362/0 (socket says 172.21.15.193:42362) 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e8feb700 1 -- 172.21.15.193:0/3004394526 learned_addr learned my addr 172.21.15.193:0/3004394526 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e3fff700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e40fdff0 0x7f00e410dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e8feb700 1 -- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e40fdff0 msgr2=0x7f00e410dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e8feb700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e40fdff0 0x7f00e410dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e8feb700 1 -- 172.21.15.193:0/3004394526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8005040 con 0x7f00e40fce80 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e8feb700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 0x7f00e410d840 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f00d800b3b0 tx=0x7f00d8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e17fa700 1 -- 172.21.15.193:0/3004394526 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d800e040 con 0x7f00e40fce80 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e4112fb0 con 0x7f00e40fce80 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e17fa700 1 -- 172.21.15.193:0/3004394526 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f00d800bc50 con 0x7f00e40fce80 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.391+0000 7f00e17fa700 1 -- 172.21.15.193:0/3004394526 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d801c630 con 0x7f00e40fce80 2024-03-23T08:19:29.393 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.392+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e4113490 con 0x7f00e40fce80 2024-03-23T08:19:29.394 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.392+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e404c710 con 0x7f00e40fce80 2024-03-23T08:19:29.394 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.393+0000 7f00e17fa700 1 -- 172.21.15.193:0/3004394526 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f00d8013420 con 0x7f00e40fce80 2024-03-23T08:19:29.394 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.393+0000 7f00e17fa700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00d40692a0 0x7f00d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:29.395 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.393+0000 7f00e17fa700 1 -- 172.21.15.193:0/3004394526 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f00d8087b40 con 0x7f00e40fce80 2024-03-23T08:19:29.395 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.393+0000 7f00e3fff700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00d40692a0 0x7f00d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:29.395 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.394+0000 7f00e3fff700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00d40692a0 0x7f00d406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009bf0 tx=0x7f00d00091a0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:29.397 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.398+0000 7f00e17fa700 1 -- 172.21.15.193:0/3004394526 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f00d8057c40 con 0x7f00e40fce80 2024-03-23T08:19:29.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.656+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f00e41136b0 con 0x7f00e40fce80 2024-03-23T08:19:29.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.661+0000 7f00e17fa700 1 -- 172.21.15.193:0/3004394526 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f00d80577d0 con 0x7f00e40fce80 2024-03-23T08:19:29.661 INFO:teuthology.orchestra.run.smithi193.stdout:[client.1] 2024-03-23T08:19:29.662 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCRkP5lwpFIJxAAI6g3zWooYQbRdDm9timupQ== 2024-03-23T08:19:29.663 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.663+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00d40692a0 msgr2=0x7f00d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.663+0000 7f00eb24f700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00d40692a0 0x7f00d406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009bf0 tx=0x7f00d00091a0 comp rx=0 tx=0).stop 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.663+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 msgr2=0x7f00e410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.663+0000 7f00eb24f700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 0x7f00e410d840 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f00d800b3b0 tx=0x7f00d8009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.664+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 shutdown_connections 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.664+0000 7f00eb24f700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e40fce80 0x7f00e410d840 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.664+0000 7f00eb24f700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00d40692a0 0x7f00d406b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.664+0000 7f00eb24f700 1 --2- 172.21.15.193:0/3004394526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00e40fdff0 0x7f00e410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.664+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 >> 172.21.15.193:0/3004394526 conn(0x7f00e40f8580 msgr2=0x7f00e40f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.664+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 shutdown_connections 2024-03-23T08:19:29.664 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-23T08:19:29.664+0000 7f00eb24f700 1 -- 172.21.15.193:0/3004394526 wait complete. 2024-03-23T08:19:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:29 smithi070 ceph-mon[39699]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:29 smithi070 ceph-mon[39699]: from='client.? 172.21.15.193:0/3004394526' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T08:19:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:29 smithi070 ceph-mon[39699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T08:19:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:29 smithi070 ceph-mon[39699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T08:19:30.006 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:29 smithi193 ceph-mon[48370]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:30.006 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:29 smithi193 ceph-mon[48370]: from='client.? 172.21.15.193:0/3004394526' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T08:19:30.006 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:29 smithi193 ceph-mon[48370]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-23T08:19:30.006 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:29 smithi193 ceph-mon[48370]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-23T08:19:30.332 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:19:30.332 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-23T08:19:30.333 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-23T08:19:30.373 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-23T08:19:30.373 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-23T08:19:30.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-23T08:19:30.650 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:31.885 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:31 smithi070 ceph-mon[39699]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:31 smithi193 ceph-mon[48370]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:32.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 -- 172.21.15.70:0/2183152931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 msgr2=0x7f51040fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:32.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 --2- 172.21.15.70:0/2183152931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 0x7f51040fc650 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5100004940 tx=0x7f5100009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:32.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 -- 172.21.15.70:0/2183152931 shutdown_connections 2024-03-23T08:19:32.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 --2- 172.21.15.70:0/2183152931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 0x7f51040fc650 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:32.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 --2- 172.21.15.70:0/2183152931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5104101e60 0x7f5104102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:32.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 -- 172.21.15.70:0/2183152931 >> 172.21.15.70:0/2183152931 conn(0x7f5104071530 msgr2=0x7f5104071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:32.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 -- 172.21.15.70:0/2183152931 shutdown_connections 2024-03-23T08:19:32.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 -- 172.21.15.70:0/2183152931 wait complete. 2024-03-23T08:19:32.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 Processor -- start 2024-03-23T08:19:32.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.429+0000 7f510bdcf700 1 -- start start 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510bdcf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5104101e60 0x7f510410ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510bdcf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 0x7f510410efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510bdcf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510410f4f0 con 0x7f5104102800 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510bdcf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510410f630 con 0x7f5104101e60 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510936a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 0x7f510410efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510936a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 0x7f510410efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37512/0 (socket says 172.21.15.70:37512) 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510936a700 1 -- 172.21.15.70:0/1063447460 learned_addr learned my addr 172.21.15.70:0/1063447460 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f5109b6b700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5104101e60 0x7f510410ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510936a700 1 -- 172.21.15.70:0/1063447460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5104101e60 msgr2=0x7f510410ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510936a700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5104101e60 0x7f510410ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510936a700 1 -- 172.21.15.70:0/1063447460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100005040 con 0x7f5104102800 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510936a700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 0x7f510410efb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5100009380 tx=0x7f5100009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f50fa7fc700 1 -- 172.21.15.70:0/1063447460 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f510000e040 con 0x7f5104102800 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510410f8b0 con 0x7f5104102800 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f50fa7fc700 1 -- 172.21.15.70:0/1063447460 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5100013420 con 0x7f5104102800 2024-03-23T08:19:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f50fa7fc700 1 -- 172.21.15.70:0/1063447460 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f510001c630 con 0x7f5104102800 2024-03-23T08:19:32.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.430+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5104117de0 con 0x7f5104102800 2024-03-23T08:19:32.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.431+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51040ff1c0 con 0x7f5104102800 2024-03-23T08:19:32.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.432+0000 7f50fa7fc700 1 -- 172.21.15.70:0/1063447460 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f510000ba80 con 0x7f5104102800 2024-03-23T08:19:32.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.432+0000 7f50fa7fc700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f50f00692a0 0x7f50f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:32.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.432+0000 7f50fa7fc700 1 -- 172.21.15.70:0/1063447460 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5100087f70 con 0x7f5104102800 2024-03-23T08:19:32.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.432+0000 7f5109b6b700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f50f00692a0 0x7f50f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:32.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.432+0000 7f5109b6b700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f50f00692a0 0x7f50f006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f50f4004f40 tx=0x7f50f400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:32.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.437+0000 7f50fa7fc700 1 -- 172.21.15.70:0/1063447460 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f51000ba190 con 0x7f5104102800 2024-03-23T08:19:32.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.671+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f51040fbb10 con 0x7f5104102800 2024-03-23T08:19:32.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.674+0000 7f50fa7fc700 1 -- 172.21.15.70:0/1063447460 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167166 (secure 0 0 0) 0x7f51000b7d10 con 0x7f5104102800 2024-03-23T08:19:32.676 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:19:32.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f50f00692a0 msgr2=0x7f50f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:32.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f50f00692a0 0x7f50f006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f50f4004f40 tx=0x7f50f400b040 comp rx=0 tx=0).stop 2024-03-23T08:19:32.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 msgr2=0x7f510410efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:32.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 0x7f510410efb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5100009380 tx=0x7f5100009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:32.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 shutdown_connections 2024-03-23T08:19:32.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5104101e60 0x7f510410ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:32.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f50f00692a0 0x7f50f006b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:32.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 --2- 172.21.15.70:0/1063447460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5104102800 0x7f510410efb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:32.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 >> 172.21.15.70:0/1063447460 conn(0x7f5104071530 msgr2=0x7f51040fa800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:32.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 shutdown_connections 2024-03-23T08:19:32.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:32.677+0000 7f510bdcf700 1 -- 172.21.15.70:0/1063447460 wait complete. 2024-03-23T08:19:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:32 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1063447460' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T08:19:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:32 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1063447460' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-23T08:19:33.424 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":19,"active_gid":14206,"active_name":"smithi070.jhsbch","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":3694824274},{"type":"v1","addr":"172.21.15.70:6801","nonce":3694824274}]},"active_addr":"172.21.15.70:6801/3694824274","active_change":"2024-03-23T08:15:48.390387+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14226,"name":"smithi193.zgpdbu","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.70:8443/","prometheus":"http://172.21.15.70:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3009100678}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":70452311}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1281398424}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":400165096}]}]} 2024-03-23T08:19:33.429 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-23T08:19:33.429 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-23T08:19:33.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T08:19:33.699 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:33.956 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:33 smithi070 ceph-mon[39699]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:33.956 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:19:34.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:33 smithi193 ceph-mon[48370]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:34.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:19:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.525+0000 7f3013f9c700 1 -- 172.21.15.70:0/2051717754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fde90 msgr2=0x7f300c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.525+0000 7f3013f9c700 1 --2- 172.21.15.70:0/2051717754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fde90 0x7f300c0fe260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc004940 tx=0x7f2ffc009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 -- 172.21.15.70:0/2051717754 shutdown_connections 2024-03-23T08:19:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 --2- 172.21.15.70:0/2051717754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c0fe830 0x7f300c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 --2- 172.21.15.70:0/2051717754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fde90 0x7f300c0fe260 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 -- 172.21.15.70:0/2051717754 >> 172.21.15.70:0/2051717754 conn(0x7f300c072bc0 msgr2=0x7f300c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 -- 172.21.15.70:0/2051717754 shutdown_connections 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 -- 172.21.15.70:0/2051717754 wait complete. 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 Processor -- start 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 -- start start 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c0fde90 0x7f300c113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fe830 0x7f300c114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c114680 con 0x7f300c0fe830 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3013f9c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c10da70 con 0x7f300c0fde90 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.526+0000 7f3011537700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fe830 0x7f300c114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f3011537700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fe830 0x7f300c114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37530/0 (socket says 172.21.15.70:37530) 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f3011537700 1 -- 172.21.15.70:0/902819804 learned_addr learned my addr 172.21.15.70:0/902819804 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f3011d38700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c0fde90 0x7f300c113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f3011537700 1 -- 172.21.15.70:0/902819804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c0fde90 msgr2=0x7f300c113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f3011537700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c0fde90 0x7f300c113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f3011537700 1 -- 172.21.15.70:0/902819804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc005040 con 0x7f300c0fe830 2024-03-23T08:19:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f3011537700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fe830 0x7f300c114030 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f300800c8c0 tx=0x7f30080044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f30027fc700 1 -- 172.21.15.70:0/902819804 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f300800a980 con 0x7f300c0fe830 2024-03-23T08:19:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c10dd50 con 0x7f300c0fe830 2024-03-23T08:19:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f30027fc700 1 -- 172.21.15.70:0/902819804 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f300800ce60 con 0x7f300c0fe830 2024-03-23T08:19:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f30027fc700 1 -- 172.21.15.70:0/902819804 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30080113a0 con 0x7f300c0fe830 2024-03-23T08:19:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.527+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c10e290 con 0x7f300c0fe830 2024-03-23T08:19:35.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.528+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c1019b0 con 0x7f300c0fe830 2024-03-23T08:19:35.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.528+0000 7f30027fc700 1 -- 172.21.15.70:0/902819804 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3008012400 con 0x7f300c0fe830 2024-03-23T08:19:35.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.529+0000 7f30027fc700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2ff80692b0 0x7f2ff806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:35.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.529+0000 7f3011d38700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2ff80692b0 0x7f2ff806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:35.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.529+0000 7f30027fc700 1 -- 172.21.15.70:0/902819804 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3008086f60 con 0x7f300c0fe830 2024-03-23T08:19:35.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.529+0000 7f3011d38700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2ff80692b0 0x7f2ff806b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00b3b0 tx=0x7f2ffc009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:35.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.533+0000 7f30027fc700 1 -- 172.21.15.70:0/902819804 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3008057060 con 0x7f300c0fe830 2024-03-23T08:19:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.723+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f300c10f140 con 0x7f300c0fe830 2024-03-23T08:19:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.723+0000 7f30027fc700 1 -- 172.21.15.70:0/902819804 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+11083 (secure 0 0 0) 0x7f3008056bf0 con 0x7f300c0fe830 2024-03-23T08:19:35.723 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:19:35.723 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":42,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","created":"2024-03-23T08:13:36.916195+0000","modified":"2024-03-23T08:19:18.609709+0000","last_up_change":"2024-03-23T08:19:17.604967+0000","last_in_change":"2024-03-23T08:18:58.641957+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T08:18:02.704319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a71163c1-e6c6-490d-ade2-f345e4b22ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6803","nonce":2404324767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6805","nonce":2404324767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6809","nonce":2404324767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6807","nonce":2404324767}]},"public_addr":"172.21.15.70:6803/2404324767","cluster_addr":"172.21.15.70:6805/2404324767","heartbeat_back_addr":"172.21.15.70:6809/2404324767","heartbeat_front_addr":"172.21.15.70:6807/2404324767","state":["exists","up"]},{"osd":1,"uuid":"0054cfce-6cdc-46f2-b434-ecd02d1ad193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6811","nonce":3805865717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6813","nonce":3805865717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6817","nonce":3805865717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6815","nonce":3805865717}]},"public_addr":"172.21.15.70:6811/3805865717","cluster_addr":"172.21.15.70:6813/3805865717","heartbeat_back_addr":"172.21.15.70:6817/3805865717","heartbeat_front_addr":"172.21.15.70:6815/3805865717","state":["exists","up"]},{"osd":2,"uuid":"24c96238-b75f-47c4-b8ef-3794e45e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6819","nonce":3317266675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6821","nonce":3317266675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6825","nonce":3317266675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6823","nonce":3317266675}]},"public_addr":"172.21.15.70:6819/3317266675","cluster_addr":"172.21.15.70:6821/3317266675","heartbeat_back_addr":"172.21.15.70:6825/3317266675","heartbeat_front_addr":"172.21.15.70:6823/3317266675","state":["exists","up"]},{"osd":3,"uuid":"64e802c4-a037-4a42-8f0f-14756b62251b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6801","nonce":1644042787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6803","nonce":1644042787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6807","nonce":1644042787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6805","nonce":1644042787}]},"public_addr":"172.21.15.193:6801/1644042787","cluster_addr":"172.21.15.193:6803/1644042787","heartbeat_back_addr":"172.21.15.193:6807/1644042787","heartbeat_front_addr":"172.21.15.193:6805/1644042787","state":["exists","up"]},{"osd":4,"uuid":"3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859463992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859463992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859463992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859463992}]},"public_addr":"172.21.15.193:6809/2859463992","cluster_addr":"172.21.15.193:6811/2859463992","heartbeat_back_addr":"172.21.15.193:6815/2859463992","heartbeat_front_addr":"172.21.15.193:6813/2859463992","state":["exists","up"]},{"osd":5,"uuid":"57e8b767-4e62-41b1-8d96-caeaad28c929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6817","nonce":863428472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6819","nonce":863428472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6823","nonce":863428472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6821","nonce":863428472}]},"public_addr":"172.21.15.193:6817/863428472","cluster_addr":"172.21.15.193:6819/863428472","heartbeat_back_addr":"172.21.15.193:6823/863428472","heartbeat_front_addr":"172.21.15.193:6821/863428472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:01.224711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:29.865012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:58.894993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:23.061992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:49.665476+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:19:16.008523+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/2209675146":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/1608386260":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/3721968086":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/562426678":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/3484360896":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:6801/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/3004659310":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/3991559747":"2024-03-24T08:15:48.390242+0000","172.21.15.70:6800/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6800/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/3894371824":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6801/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/370857700":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6800/2149030412":"2024-03-24T08:14:42.564176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T08:19:35.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.725+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2ff80692b0 msgr2=0x7f2ff806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.725+0000 7f3013f9c700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2ff80692b0 0x7f2ff806b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00b3b0 tx=0x7f2ffc009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.725+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fe830 msgr2=0x7f300c114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.725+0000 7f3013f9c700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fe830 0x7f300c114030 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f300800c8c0 tx=0x7f30080044c0 comp rx=0 tx=0).stop 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.725+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 shutdown_connections 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.725+0000 7f3013f9c700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c0fde90 0x7f300c113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.725+0000 7f3013f9c700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f2ff80692b0 0x7f2ff806b760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.725+0000 7f3013f9c700 1 --2- 172.21.15.70:0/902819804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300c0fe830 0x7f300c114030 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.725+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 >> 172.21.15.70:0/902819804 conn(0x7f300c072bc0 msgr2=0x7f300c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.726+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 shutdown_connections 2024-03-23T08:19:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:35.726+0000 7f3013f9c700 1 -- 172.21.15.70:0/902819804 wait complete. 2024-03-23T08:19:35.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:35 smithi070 ceph-mon[39699]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:36.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:35 smithi193 ceph-mon[48370]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:36.563 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-23T08:19:36.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-23T08:19:36.832 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:36.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:36 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/902819804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:19:37.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:36 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/902819804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:19:37.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:37 smithi070 ceph-mon[39699]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:37 smithi193 ceph-mon[48370]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:38.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.767+0000 7f7c213cf700 1 -- 172.21.15.70:0/3561296386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c101d30 msgr2=0x7f7c1c102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:38.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.767+0000 7f7c213cf700 1 --2- 172.21.15.70:0/3561296386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c101d30 0x7f7c1c102100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c004940 tx=0x7f7c0c009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:38.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.767+0000 7f7c213cf700 1 -- 172.21.15.70:0/3561296386 shutdown_connections 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.767+0000 7f7c213cf700 1 --2- 172.21.15.70:0/3561296386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c1c1026d0 0x7f7c1c0fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.767+0000 7f7c213cf700 1 --2- 172.21.15.70:0/3561296386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c101d30 0x7f7c1c102100 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.767+0000 7f7c213cf700 1 -- 172.21.15.70:0/3561296386 >> 172.21.15.70:0/3561296386 conn(0x7f7c1c0f7860 msgr2=0x7f7c1c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.767+0000 7f7c213cf700 1 -- 172.21.15.70:0/3561296386 shutdown_connections 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.767+0000 7f7c213cf700 1 -- 172.21.15.70:0/3561296386 wait complete. 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.768+0000 7f7c213cf700 1 Processor -- start 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.768+0000 7f7c213cf700 1 -- start start 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.768+0000 7f7c213cf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c1c101d30 0x7f7c1c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.768+0000 7f7c213cf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c1026d0 0x7f7c1c1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.768+0000 7f7c213cf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c104b70 con 0x7f7c1c1026d0 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.768+0000 7f7c213cf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c103610 con 0x7f7c1c101d30 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.768+0000 7f7c1a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c1026d0 0x7f7c1c1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.768+0000 7f7c1a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c1026d0 0x7f7c1c1030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50778/0 (socket says 172.21.15.70:50778) 2024-03-23T08:19:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.768+0000 7f7c1a7fc700 1 -- 172.21.15.70:0/4002649411 learned_addr learned my addr 172.21.15.70:0/4002649411 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c1affd700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c1c101d30 0x7f7c1c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c1a7fc700 1 -- 172.21.15.70:0/4002649411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c1c101d30 msgr2=0x7f7c1c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c1a7fc700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c1c101d30 0x7f7c1c102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c1a7fc700 1 -- 172.21.15.70:0/4002649411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c005040 con 0x7f7c1c1026d0 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c1a7fc700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c1026d0 0x7f7c1c1030d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000b790 tx=0x7f7c10009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c037fe700 1 -- 172.21.15.70:0/4002649411 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c1000f6f0 con 0x7f7c1c1026d0 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1c1038f0 con 0x7f7c1c1026d0 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c037fe700 1 -- 172.21.15.70:0/4002649411 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7c1000fd00 con 0x7f7c1c1026d0 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c037fe700 1 -- 172.21.15.70:0/4002649411 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c100103a0 con 0x7f7c1c1026d0 2024-03-23T08:19:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.769+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c11a120 con 0x7f7c1c1026d0 2024-03-23T08:19:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.770+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1c0fd480 con 0x7f7c1c1026d0 2024-03-23T08:19:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.770+0000 7f7c037fe700 1 -- 172.21.15.70:0/4002649411 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7c1000f850 con 0x7f7c1c1026d0 2024-03-23T08:19:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.771+0000 7f7c037fe700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f7c040692a0 0x7f7c0406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.771+0000 7f7c037fe700 1 -- 172.21.15.70:0/4002649411 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7c10086cc0 con 0x7f7c1c1026d0 2024-03-23T08:19:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.771+0000 7f7c1affd700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f7c040692a0 0x7f7c0406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.771+0000 7f7c1affd700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f7c040692a0 0x7f7c0406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c004540 tx=0x7f7c0c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:38.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.775+0000 7f7c037fe700 1 -- 172.21.15.70:0/4002649411 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7c100532b0 con 0x7f7c1c1026d0 2024-03-23T08:19:38.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.961+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c1c04c710 con 0x7f7c1c1026d0 2024-03-23T08:19:38.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.961+0000 7f7c037fe700 1 -- 172.21.15.70:0/4002649411 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+11083 (secure 0 0 0) 0x7f7c100568d0 con 0x7f7c1c1026d0 2024-03-23T08:19:38.962 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:19:38.962 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":42,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","created":"2024-03-23T08:13:36.916195+0000","modified":"2024-03-23T08:19:18.609709+0000","last_up_change":"2024-03-23T08:19:17.604967+0000","last_in_change":"2024-03-23T08:18:58.641957+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T08:18:02.704319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a71163c1-e6c6-490d-ade2-f345e4b22ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6803","nonce":2404324767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6805","nonce":2404324767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6809","nonce":2404324767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6807","nonce":2404324767}]},"public_addr":"172.21.15.70:6803/2404324767","cluster_addr":"172.21.15.70:6805/2404324767","heartbeat_back_addr":"172.21.15.70:6809/2404324767","heartbeat_front_addr":"172.21.15.70:6807/2404324767","state":["exists","up"]},{"osd":1,"uuid":"0054cfce-6cdc-46f2-b434-ecd02d1ad193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6811","nonce":3805865717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6813","nonce":3805865717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6817","nonce":3805865717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6815","nonce":3805865717}]},"public_addr":"172.21.15.70:6811/3805865717","cluster_addr":"172.21.15.70:6813/3805865717","heartbeat_back_addr":"172.21.15.70:6817/3805865717","heartbeat_front_addr":"172.21.15.70:6815/3805865717","state":["exists","up"]},{"osd":2,"uuid":"24c96238-b75f-47c4-b8ef-3794e45e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6819","nonce":3317266675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6821","nonce":3317266675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6825","nonce":3317266675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6823","nonce":3317266675}]},"public_addr":"172.21.15.70:6819/3317266675","cluster_addr":"172.21.15.70:6821/3317266675","heartbeat_back_addr":"172.21.15.70:6825/3317266675","heartbeat_front_addr":"172.21.15.70:6823/3317266675","state":["exists","up"]},{"osd":3,"uuid":"64e802c4-a037-4a42-8f0f-14756b62251b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6801","nonce":1644042787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6803","nonce":1644042787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6807","nonce":1644042787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6805","nonce":1644042787}]},"public_addr":"172.21.15.193:6801/1644042787","cluster_addr":"172.21.15.193:6803/1644042787","heartbeat_back_addr":"172.21.15.193:6807/1644042787","heartbeat_front_addr":"172.21.15.193:6805/1644042787","state":["exists","up"]},{"osd":4,"uuid":"3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859463992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859463992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859463992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859463992}]},"public_addr":"172.21.15.193:6809/2859463992","cluster_addr":"172.21.15.193:6811/2859463992","heartbeat_back_addr":"172.21.15.193:6815/2859463992","heartbeat_front_addr":"172.21.15.193:6813/2859463992","state":["exists","up"]},{"osd":5,"uuid":"57e8b767-4e62-41b1-8d96-caeaad28c929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6817","nonce":863428472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6819","nonce":863428472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6823","nonce":863428472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6821","nonce":863428472}]},"public_addr":"172.21.15.193:6817/863428472","cluster_addr":"172.21.15.193:6819/863428472","heartbeat_back_addr":"172.21.15.193:6823/863428472","heartbeat_front_addr":"172.21.15.193:6821/863428472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:01.224711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:29.865012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:58.894993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:23.061992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:49.665476+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:19:16.008523+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/2209675146":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/1608386260":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/3721968086":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/562426678":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/3484360896":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:6801/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/3004659310":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/3991559747":"2024-03-24T08:15:48.390242+0000","172.21.15.70:6800/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6800/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/3894371824":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6801/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/370857700":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6800/2149030412":"2024-03-24T08:14:42.564176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T08:19:38.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f7c040692a0 msgr2=0x7f7c0406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:38.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f7c040692a0 0x7f7c0406b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c004540 tx=0x7f7c0c009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:38.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c1026d0 msgr2=0x7f7c1c1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:38.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c1026d0 0x7f7c1c1030d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000b790 tx=0x7f7c10009220 comp rx=0 tx=0).stop 2024-03-23T08:19:38.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 shutdown_connections 2024-03-23T08:19:38.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c1c101d30 0x7f7c1c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:38.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f7c040692a0 0x7f7c0406b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:38.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 --2- 172.21.15.70:0/4002649411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c1c1026d0 0x7f7c1c1030d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:38.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 >> 172.21.15.70:0/4002649411 conn(0x7f7c1c0f7860 msgr2=0x7f7c1c0662d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:38.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 shutdown_connections 2024-03-23T08:19:38.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:38.963+0000 7f7c213cf700 1 -- 172.21.15.70:0/4002649411 wait complete. 2024-03-23T08:19:39.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-23T08:19:39.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-23T08:19:39.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-23T08:19:39.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-23T08:19:39.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-23T08:19:39.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-23T08:19:39.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:39 smithi070 ceph-mon[39699]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:39.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:39 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/4002649411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:19:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:40.011 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:39 smithi193 ceph-mon[48370]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:39 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/4002649411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:19:40.948 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:41 smithi070 ceph-mon[39699]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:41 smithi193 ceph-mon[48370]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.537+0000 7f6cc370d700 1 -- 172.21.15.70:0/1564326541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 msgr2=0x7f6cbc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.537+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1564326541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 0x7f6cbc0fce10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6cac004940 tx=0x7f6cac009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.537+0000 7f6cc370d700 1 -- 172.21.15.70:0/1564326541 shutdown_connections 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.537+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1564326541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 0x7f6cbc0fce10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.537+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1564326541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cbc102e00 0x7f6cbc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.537+0000 7f6cc370d700 1 -- 172.21.15.70:0/1564326541 >> 172.21.15.70:0/1564326541 conn(0x7f6cbc0f89b0 msgr2=0x7f6cbc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.537+0000 7f6cc370d700 1 -- 172.21.15.70:0/1564326541 shutdown_connections 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.537+0000 7f6cc370d700 1 -- 172.21.15.70:0/1564326541 wait complete. 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc370d700 1 Processor -- start 2024-03-23T08:19:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc370d700 1 -- start start 2024-03-23T08:19:43.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc370d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cbc102e00 0x7f6cbc113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:43.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc370d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 0x7f6cbc113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:43.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc370d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc114600 con 0x7f6cbc1037a0 2024-03-23T08:19:43.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc370d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc10d9f0 con 0x7f6cbc102e00 2024-03-23T08:19:43.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc0ca8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 0x7f6cbc113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc0ca8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 0x7f6cbc113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50802/0 (socket says 172.21.15.70:50802) 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc0ca8700 1 -- 172.21.15.70:0/1067116680 learned_addr learned my addr 172.21.15.70:0/1067116680 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc14a9700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cbc102e00 0x7f6cbc113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc0ca8700 1 -- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cbc102e00 msgr2=0x7f6cbc113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc0ca8700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cbc102e00 0x7f6cbc113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.538+0000 7f6cc0ca8700 1 -- 172.21.15.70:0/1067116680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac005040 con 0x7f6cbc1037a0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.539+0000 7f6cc0ca8700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 0x7f6cbc113fb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6cac000c00 tx=0x7f6cac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.539+0000 7f6cb1ffb700 1 -- 172.21.15.70:0/1067116680 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cac00e070 con 0x7f6cbc1037a0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.539+0000 7f6cb1ffb700 1 -- 172.21.15.70:0/1067116680 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6cac0093b0 con 0x7f6cbc1037a0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.539+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc10dc70 con 0x7f6cbc1037a0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.539+0000 7f6cb1ffb700 1 -- 172.21.15.70:0/1067116680 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cac01ccb0 con 0x7f6cbc1037a0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.539+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc10e1d0 con 0x7f6cbc1037a0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.540+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6cbc10e700 con 0x7f6cbc1037a0 2024-03-23T08:19:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.540+0000 7f6cb1ffb700 1 -- 172.21.15.70:0/1067116680 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6cac01c490 con 0x7f6cbc1037a0 2024-03-23T08:19:43.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.541+0000 7f6cb1ffb700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6ca80692a0 0x7f6ca806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.541+0000 7f6cc14a9700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6ca80692a0 0x7f6ca806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.541+0000 7f6cb1ffb700 1 -- 172.21.15.70:0/1067116680 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6cac088280 con 0x7f6cbc1037a0 2024-03-23T08:19:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.541+0000 7f6cb1ffb700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] conn(0x7f6ca806ef70 0x7f6ca8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.541+0000 7f6cb1ffb700 1 -- 172.21.15.70:0/1067116680 --> [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ca8071a30 con 0x7f6ca806ef70 2024-03-23T08:19:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.541+0000 7f6cb1ffb700 1 -- 172.21.15.70:0/1067116680 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cac08a830 con 0x7f6cbc1037a0 2024-03-23T08:19:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.541+0000 7f6cc14a9700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6ca80692a0 0x7f6ca806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009b80 tx=0x7f6cb800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.541+0000 7f6cc1caa700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] conn(0x7f6ca806ef70 0x7f6ca8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.541+0000 7f6cc1caa700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] conn(0x7f6ca806ef70 0x7f6ca8071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.542+0000 7f6cb1ffb700 1 -- 172.21.15.70:0/1067116680 <== osd.4 v2:172.21.15.193:6808/2859463992 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6ca8071a30 con 0x7f6ca806ef70 2024-03-23T08:19:43.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 --> [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6cbc0fa420 con 0x7f6ca806ef70 2024-03-23T08:19:43.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cb1ffb700 1 -- 172.21.15.70:0/1067116680 <== osd.4 v2:172.21.15.193:6808/2859463992 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6cbc0fa420 con 0x7f6ca806ef70 2024-03-23T08:19:43.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] conn(0x7f6ca806ef70 msgr2=0x7f6ca8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:43.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] conn(0x7f6ca806ef70 0x7f6ca8071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:43.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6ca80692a0 msgr2=0x7f6ca806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:43.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6ca80692a0 0x7f6ca806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009b80 tx=0x7f6cb800b040 comp rx=0 tx=0).stop 2024-03-23T08:19:43.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 msgr2=0x7f6cbc113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:43.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 0x7f6cbc113fb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6cac000c00 tx=0x7f6cac009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:43.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 shutdown_connections 2024-03-23T08:19:43.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:6808/2859463992,v1:172.21.15.193:6809/2859463992] conn(0x7f6ca806ef70 0x7f6ca8071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:43.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cbc102e00 0x7f6cbc113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:43.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6ca80692a0 0x7f6ca806b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:43.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 --2- 172.21.15.70:0/1067116680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cbc1037a0 0x7f6cbc113fb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:43.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.557+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 >> 172.21.15.70:0/1067116680 conn(0x7f6cbc0f89b0 msgr2=0x7f6cbc0f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:43.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.558+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 shutdown_connections 2024-03-23T08:19:43.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:43.558+0000 7f6cc370d700 1 -- 172.21.15.70:0/1067116680 wait complete. 2024-03-23T08:19:43.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:43 smithi070 ceph-mon[39699]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:44.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:43 smithi193 ceph-mon[48370]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.402+0000 7fb08cb50700 1 -- 172.21.15.70:0/1267361487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 msgr2=0x7fb0880fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.402+0000 7fb08cb50700 1 --2- 172.21.15.70:0/1267361487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 0x7fb0880fbd80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb078004940 tx=0x7fb078009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.402+0000 7fb08cb50700 1 -- 172.21.15.70:0/1267361487 shutdown_connections 2024-03-23T08:19:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.402+0000 7fb08cb50700 1 --2- 172.21.15.70:0/1267361487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 0x7fb0880fbd80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.402+0000 7fb08cb50700 1 --2- 172.21.15.70:0/1267361487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101d70 0x7fb088102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.402+0000 7fb08cb50700 1 -- 172.21.15.70:0/1267361487 >> 172.21.15.70:0/1267361487 conn(0x7fb088072bc0 msgr2=0x7fb088072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.402+0000 7fb08cb50700 1 -- 172.21.15.70:0/1267361487 shutdown_connections 2024-03-23T08:19:44.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.402+0000 7fb08cb50700 1 -- 172.21.15.70:0/1267361487 wait complete. 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb08cb50700 1 Processor -- start 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb08cb50700 1 -- start start 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb08cb50700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101d70 0x7fb08810ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb08cb50700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 0x7fb08810f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb08cb50700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08810f6b0 con 0x7fb088102710 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb08cb50700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08810f7f0 con 0x7fb088101d70 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb085d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 0x7fb08810f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb085d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 0x7fb08810f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50816/0 (socket says 172.21.15.70:50816) 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb085d9b700 1 -- 172.21.15.70:0/929804930 learned_addr learned my addr 172.21.15.70:0/929804930 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb085d9b700 1 -- 172.21.15.70:0/929804930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101d70 msgr2=0x7fb08810ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb085d9b700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101d70 0x7fb08810ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.403+0000 7fb085d9b700 1 -- 172.21.15.70:0/929804930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078005040 con 0x7fb088102710 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.404+0000 7fb085d9b700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 0x7fb08810f170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb0780093b0 tx=0x7fb078009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:44.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.404+0000 7fb076ffd700 1 -- 172.21.15.70:0/929804930 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07800e040 con 0x7fb088102710 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.404+0000 7fb076ffd700 1 -- 172.21.15.70:0/929804930 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb078013420 con 0x7fb088102710 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.404+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb088117b40 con 0x7fb088102710 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.404+0000 7fb076ffd700 1 -- 172.21.15.70:0/929804930 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07801c630 con 0x7fb088102710 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.404+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb088118100 con 0x7fb088102710 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.404+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb08805e700 con 0x7fb088102710 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.405+0000 7fb076ffd700 1 -- 172.21.15.70:0/929804930 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb0780139e0 con 0x7fb088102710 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.405+0000 7fb076ffd700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb0700692a0 0x7fb07006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.405+0000 7fb076ffd700 1 -- 172.21.15.70:0/929804930 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb078087f30 con 0x7fb088102710 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.405+0000 7fb08659c700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb0700692a0 0x7fb07006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.405+0000 7fb076ffd700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] conn(0x7fb07006ef70 0x7fb070071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.406+0000 7fb076ffd700 1 -- 172.21.15.70:0/929804930 --> [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb070071a30 con 0x7fb07006ef70 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.406+0000 7fb076ffd700 1 -- 172.21.15.70:0/929804930 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb07808a540 con 0x7fb088102710 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.406+0000 7fb08659c700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb0700692a0 0x7fb07006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb07c006fd0 tx=0x7fb07c006db0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:44.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.406+0000 7fb086d9d700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] conn(0x7fb07006ef70 0x7fb070071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:44.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.406+0000 7fb086d9d700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] conn(0x7fb07006ef70 0x7fb070071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:44.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.407+0000 7fb076ffd700 1 -- 172.21.15.70:0/929804930 <== osd.3 v2:172.21.15.193:6800/1644042787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb070071a30 con 0x7fb07006ef70 2024-03-23T08:19:44.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.422+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 --> [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb08805eb80 con 0x7fb07006ef70 2024-03-23T08:19:44.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.422+0000 7fb076ffd700 1 -- 172.21.15.70:0/929804930 <== osd.3 v2:172.21.15.193:6800/1644042787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb08805eb80 con 0x7fb07006ef70 2024-03-23T08:19:44.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 >> [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] conn(0x7fb07006ef70 msgr2=0x7fb070071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:44.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] conn(0x7fb07006ef70 0x7fb070071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:44.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb0700692a0 msgr2=0x7fb07006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:44.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb0700692a0 0x7fb07006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb07c006fd0 tx=0x7fb07c006db0 comp rx=0 tx=0).stop 2024-03-23T08:19:44.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 msgr2=0x7fb08810f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:44.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 0x7fb08810f170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb0780093b0 tx=0x7fb078009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 shutdown_connections 2024-03-23T08:19:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.193:6800/1644042787,v1:172.21.15.193:6801/1644042787] conn(0x7fb07006ef70 0x7fb070071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb088101d70 0x7fb08810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb0700692a0 0x7fb07006b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 --2- 172.21.15.70:0/929804930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb088102710 0x7fb08810f170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 >> 172.21.15.70:0/929804930 conn(0x7fb088072bc0 msgr2=0x7fb0880fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 shutdown_connections 2024-03-23T08:19:44.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:44.423+0000 7fb08cb50700 1 -- 172.21.15.70:0/929804930 wait complete. 2024-03-23T08:19:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.309+0000 7fbafaf07700 1 -- 172.21.15.70:0/119568928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf40fb4a0 msgr2=0x7fbaf40fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.309+0000 7fbafaf07700 1 --2- 172.21.15.70:0/119568928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf40fb4a0 0x7fbaf40fb910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbae0004940 tx=0x7fbae0009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 -- 172.21.15.70:0/119568928 shutdown_connections 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 --2- 172.21.15.70:0/119568928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf40fb4a0 0x7fbaf40fb910 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 --2- 172.21.15.70:0/119568928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf4102e70 0x7fbaf4103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 -- 172.21.15.70:0/119568928 >> 172.21.15.70:0/119568928 conn(0x7fbaf40714c0 msgr2=0x7fbaf40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 -- 172.21.15.70:0/119568928 shutdown_connections 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 -- 172.21.15.70:0/119568928 wait complete. 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 Processor -- start 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 -- start start 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf40fb4a0 0x7fbaf4113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf4102e70 0x7fbaf4113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.310+0000 7fbafaf07700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf4114600 con 0x7fbaf4102e70 2024-03-23T08:19:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbafaf07700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf410d9f0 con 0x7fbaf40fb4a0 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf4102e70 0x7fbaf4113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf4102e70 0x7fbaf4113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50828/0 (socket says 172.21.15.70:50828) 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf3fff700 1 -- 172.21.15.70:0/1787399601 learned_addr learned my addr 172.21.15.70:0/1787399601 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf8ca3700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf40fb4a0 0x7fbaf4113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf3fff700 1 -- 172.21.15.70:0/1787399601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf40fb4a0 msgr2=0x7fbaf4113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf3fff700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf40fb4a0 0x7fbaf4113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf3fff700 1 -- 172.21.15.70:0/1787399601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae0005040 con 0x7fbaf4102e70 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf3fff700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf4102e70 0x7fbaf4113fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009390 tx=0x7fbae0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf17fa700 1 -- 172.21.15.70:0/1787399601 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbae000e040 con 0x7fbaf4102e70 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf17fa700 1 -- 172.21.15.70:0/1787399601 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbae0013420 con 0x7fbaf4102e70 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbaf17fa700 1 -- 172.21.15.70:0/1787399601 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbae001c630 con 0x7fbaf4102e70 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf410dc70 con 0x7fbaf4102e70 2024-03-23T08:19:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.311+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf410e230 con 0x7fbaf4102e70 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.312+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbaf410e810 con 0x7fbaf4102e70 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.313+0000 7fbaf17fa700 1 -- 172.21.15.70:0/1787399601 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbae00139e0 con 0x7fbaf4102e70 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.313+0000 7fbaf17fa700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbae4069380 0x7fbae406b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.313+0000 7fbaf17fa700 1 -- 172.21.15.70:0/1787399601 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbae0088190 con 0x7fbaf4102e70 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.313+0000 7fbaf8ca3700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbae4069380 0x7fbae406b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.313+0000 7fbaf17fa700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] conn(0x7fbae406f050 0x7fbae4071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.313+0000 7fbaf17fa700 1 -- 172.21.15.70:0/1787399601 --> [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbae4071b10 con 0x7fbae406f050 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.313+0000 7fbaf17fa700 1 -- 172.21.15.70:0/1787399601 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbae0058270 con 0x7fbaf4102e70 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.313+0000 7fbaf94a4700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] conn(0x7fbae406f050 0x7fbae4071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.314+0000 7fbaf8ca3700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbae4069380 0x7fbae406b830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbae8007fd0 tx=0x7fbae8009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.314+0000 7fbaf94a4700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] conn(0x7fbae406f050 0x7fbae4071460 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.314+0000 7fbaf17fa700 1 -- 172.21.15.70:0/1787399601 <== osd.0 v2:172.21.15.70:6802/2404324767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbae4071b10 con 0x7fbae406f050 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.329+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 --> [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbaf405e6c0 con 0x7fbae406f050 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.329+0000 7fbaf17fa700 1 -- 172.21.15.70:0/1787399601 <== osd.0 v2:172.21.15.70:6802/2404324767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbaf405e6c0 con 0x7fbae406f050 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.329+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] conn(0x7fbae406f050 msgr2=0x7fbae4071460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.329+0000 7fbafaf07700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] conn(0x7fbae406f050 0x7fbae4071460 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.329+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbae4069380 msgr2=0x7fbae406b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.329+0000 7fbafaf07700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbae4069380 0x7fbae406b830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbae8007fd0 tx=0x7fbae8009040 comp rx=0 tx=0).stop 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.329+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf4102e70 msgr2=0x7fbaf4113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.329+0000 7fbafaf07700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf4102e70 0x7fbaf4113fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009390 tx=0x7fbae0009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.330+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 shutdown_connections 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.330+0000 7fbafaf07700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6802/2404324767,v1:172.21.15.70:6803/2404324767] conn(0x7fbae406f050 0x7fbae4071460 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.330+0000 7fbafaf07700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaf40fb4a0 0x7fbaf4113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.330+0000 7fbafaf07700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fbae4069380 0x7fbae406b830 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:45.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.330+0000 7fbafaf07700 1 --2- 172.21.15.70:0/1787399601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaf4102e70 0x7fbaf4113fb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:45.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.330+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 >> 172.21.15.70:0/1787399601 conn(0x7fbaf40714c0 msgr2=0x7fbaf40fa910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:45.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.330+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 shutdown_connections 2024-03-23T08:19:45.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:45.330+0000 7fbafaf07700 1 -- 172.21.15.70:0/1787399601 wait complete. 2024-03-23T08:19:45.880 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:45.961 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:45 smithi070 ceph-mon[39699]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:45 smithi193 ceph-mon[48370]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:46.378 INFO:teuthology.orchestra.run.smithi070.stdout:120259084306 2024-03-23T08:19:46.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-23T08:19:46.833 INFO:teuthology.orchestra.run.smithi070.stdout:154618822669 2024-03-23T08:19:46.834 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-23T08:19:47.809 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:47 smithi070 ceph-mon[39699]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:48.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:47 smithi193 ceph-mon[48370]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:19:49.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:19:49.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.307+0000 7f4496469700 1 -- 172.21.15.70:0/1514528243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 msgr2=0x7f44901018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:49.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.307+0000 7f4496469700 1 --2- 172.21.15.70:0/1514528243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 0x7f44901018c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4478004940 tx=0x7f4478009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:49.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.307+0000 7f4496469700 1 -- 172.21.15.70:0/1514528243 shutdown_connections 2024-03-23T08:19:49.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.307+0000 7f4496469700 1 --2- 172.21.15.70:0/1514528243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44900fb4a0 0x7f44900fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:49.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.307+0000 7f4496469700 1 --2- 172.21.15.70:0/1514528243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 0x7f44901018c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:49.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.307+0000 7f4496469700 1 -- 172.21.15.70:0/1514528243 >> 172.21.15.70:0/1514528243 conn(0x7f44900714c0 msgr2=0x7f44900718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:49.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.307+0000 7f4496469700 1 -- 172.21.15.70:0/1514528243 shutdown_connections 2024-03-23T08:19:49.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.307+0000 7f4496469700 1 -- 172.21.15.70:0/1514528243 wait complete. 2024-03-23T08:19:49.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f4496469700 1 Processor -- start 2024-03-23T08:19:49.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f4496469700 1 -- start start 2024-03-23T08:19:49.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f4496469700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44900fb4a0 0x7f4490111880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:49.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f4496469700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 0x7f4490111dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:49.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f4496469700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4490112410 con 0x7f44901014f0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f4496469700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449010b800 con 0x7f44900fb4a0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f448f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 0x7f4490111dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f448f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 0x7f4490111dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53562/0 (socket says 172.21.15.70:53562) 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f448f7fe700 1 -- 172.21.15.70:0/875131379 learned_addr learned my addr 172.21.15.70:0/875131379 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f448ffff700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44900fb4a0 0x7f4490111880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f448f7fe700 1 -- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44900fb4a0 msgr2=0x7f4490111880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f448f7fe700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44900fb4a0 0x7f4490111880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.308+0000 7f448f7fe700 1 -- 172.21.15.70:0/875131379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478005040 con 0x7f44901014f0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.309+0000 7f448f7fe700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 0x7f4490111dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44800041d0 tx=0x7f44800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.309+0000 7f448cff9700 1 -- 172.21.15.70:0/875131379 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f448000feb0 con 0x7f44901014f0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.309+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449010bae0 con 0x7f44901014f0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.309+0000 7f448cff9700 1 -- 172.21.15.70:0/875131379 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f448000a0d0 con 0x7f44901014f0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.309+0000 7f448cff9700 1 -- 172.21.15.70:0/875131379 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4480012690 con 0x7f44901014f0 2024-03-23T08:19:49.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.309+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449010c0a0 con 0x7f44901014f0 2024-03-23T08:19:49.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.310+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f449010c5c0 con 0x7f44901014f0 2024-03-23T08:19:49.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.310+0000 7f448cff9700 1 -- 172.21.15.70:0/875131379 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f448000a240 con 0x7f44901014f0 2024-03-23T08:19:49.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.310+0000 7f448cff9700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f447c0692a0 0x7f447c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.311+0000 7f448cff9700 1 -- 172.21.15.70:0/875131379 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4480089600 con 0x7f44901014f0 2024-03-23T08:19:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.311+0000 7f448ffff700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f447c0692a0 0x7f447c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.311+0000 7f448cff9700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] conn(0x7f447c06ef70 0x7f447c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.311+0000 7f448cff9700 1 -- 172.21.15.70:0/875131379 --> [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f447c071a30 con 0x7f447c06ef70 2024-03-23T08:19:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.311+0000 7f448cff9700 1 -- 172.21.15.70:0/875131379 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448008bb90 con 0x7f44901014f0 2024-03-23T08:19:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.311+0000 7f448ffff700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f447c0692a0 0x7f447c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4478004f90 tx=0x7f4478009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.311+0000 7f4494a06700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] conn(0x7f447c06ef70 0x7f447c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.311+0000 7f4494a06700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] conn(0x7f447c06ef70 0x7f447c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.312+0000 7f448cff9700 1 -- 172.21.15.70:0/875131379 <== osd.5 v2:172.21.15.193:6816/863428472 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f447c071a30 con 0x7f447c06ef70 2024-03-23T08:19:49.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 --> [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f44900f9e60 con 0x7f447c06ef70 2024-03-23T08:19:49.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f448cff9700 1 -- 172.21.15.70:0/875131379 <== osd.5 v2:172.21.15.193:6816/863428472 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f44900f9e60 con 0x7f447c06ef70 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] conn(0x7f447c06ef70 msgr2=0x7f447c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] conn(0x7f447c06ef70 0x7f447c071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f447c0692a0 msgr2=0x7f447c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f447c0692a0 0x7f447c06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4478004f90 tx=0x7f4478009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 msgr2=0x7f4490111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 0x7f4490111dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44800041d0 tx=0x7f44800046b0 comp rx=0 tx=0).stop 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 shutdown_connections 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:6816/863428472,v1:172.21.15.193:6817/863428472] conn(0x7f447c06ef70 0x7f447c071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44900fb4a0 0x7f4490111880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f447c0692a0 0x7f447c06b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:49.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 --2- 172.21.15.70:0/875131379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44901014f0 0x7f4490111dc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:49.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.327+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 >> 172.21.15.70:0/875131379 conn(0x7f44900714c0 msgr2=0x7f44900f9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:49.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.328+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 shutdown_connections 2024-03-23T08:19:49.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:49.328+0000 7f4496469700 1 -- 172.21.15.70:0/875131379 wait complete. 2024-03-23T08:19:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:49 smithi070 ceph-mon[39699]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:49 smithi193 ceph-mon[48370]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:50.113 INFO:teuthology.orchestra.run.smithi070.stdout:176093659145 2024-03-23T08:19:50.114 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-23T08:19:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.584+0000 7fb7d84c0700 1 -- 172.21.15.70:0/1320777910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 msgr2=0x7fb7d0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.584+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/1320777910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 0x7fb7d0102140 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc004940 tx=0x7fb7cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.584+0000 7fb7d84c0700 1 -- 172.21.15.70:0/1320777910 shutdown_connections 2024-03-23T08:19:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.584+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/1320777910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d0102710 0x7fb7d00fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.584+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/1320777910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 0x7fb7d0102140 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.584+0000 7fb7d84c0700 1 -- 172.21.15.70:0/1320777910 >> 172.21.15.70:0/1320777910 conn(0x7fb7d0072bc0 msgr2=0x7fb7d0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d84c0700 1 -- 172.21.15.70:0/1320777910 shutdown_connections 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d84c0700 1 -- 172.21.15.70:0/1320777910 wait complete. 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d84c0700 1 Processor -- start 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d84c0700 1 -- start start 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d84c0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 0x7fb7d010ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d84c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d0102710 0x7fb7d010ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d84c0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d010f650 con 0x7fb7d0101d70 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d84c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d010f7c0 con 0x7fb7d0102710 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d625c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 0x7fb7d010ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d625c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 0x7fb7d010ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53580/0 (socket says 172.21.15.70:53580) 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.585+0000 7fb7d625c700 1 -- 172.21.15.70:0/3650723636 learned_addr learned my addr 172.21.15.70:0/3650723636 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.586+0000 7fb7d625c700 1 -- 172.21.15.70:0/3650723636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d0102710 msgr2=0x7fb7d010ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.586+0000 7fb7d625c700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d0102710 0x7fb7d010ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.586+0000 7fb7d625c700 1 -- 172.21.15.70:0/3650723636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc005040 con 0x7fb7d0101d70 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.586+0000 7fb7d625c700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 0x7fb7d010ea30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b3b0 tx=0x7fb7cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.586+0000 7fb7c6ffd700 1 -- 172.21.15.70:0/3650723636 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7cc00e040 con 0x7fb7d0101d70 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.586+0000 7fb7c6ffd700 1 -- 172.21.15.70:0/3650723636 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7cc013420 con 0x7fb7d0101d70 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.586+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d0117aa0 con 0x7fb7d0101d70 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.586+0000 7fb7c6ffd700 1 -- 172.21.15.70:0/3650723636 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7cc01c630 con 0x7fb7d0101d70 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.586+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d0117fe0 con 0x7fb7d0101d70 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.587+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7d005e700 con 0x7fb7d0101d70 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.587+0000 7fb7c6ffd700 1 -- 172.21.15.70:0/3650723636 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb7cc013970 con 0x7fb7d0101d70 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.587+0000 7fb7c6ffd700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb7bc0692a0 0x7fb7bc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.588+0000 7fb7c6ffd700 1 -- 172.21.15.70:0/3650723636 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb7cc088090 con 0x7fb7d0101d70 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.588+0000 7fb7d5a5b700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb7bc0692a0 0x7fb7bc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.588+0000 7fb7c6ffd700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] conn(0x7fb7bc06ef70 0x7fb7bc071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.588+0000 7fb7c6ffd700 1 -- 172.21.15.70:0/3650723636 --> [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7bc071a30 con 0x7fb7bc06ef70 2024-03-23T08:19:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.588+0000 7fb7c6ffd700 1 -- 172.21.15.70:0/3650723636 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7cc08a680 con 0x7fb7d0101d70 2024-03-23T08:19:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.588+0000 7fb7d6a5d700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] conn(0x7fb7bc06ef70 0x7fb7bc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.588+0000 7fb7d5a5b700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb7bc0692a0 0x7fb7bc06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0006fd0 tx=0x7fb7c0006db0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.588+0000 7fb7d6a5d700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] conn(0x7fb7bc06ef70 0x7fb7bc071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.589+0000 7fb7c6ffd700 1 -- 172.21.15.70:0/3650723636 <== osd.1 v2:172.21.15.70:6810/3805865717 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb7bc071a30 con 0x7fb7bc06ef70 2024-03-23T08:19:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.604+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 --> [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7d005eb80 con 0x7fb7bc06ef70 2024-03-23T08:19:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.604+0000 7fb7c6ffd700 1 -- 172.21.15.70:0/3650723636 <== osd.1 v2:172.21.15.70:6810/3805865717 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb7d005eb80 con 0x7fb7bc06ef70 2024-03-23T08:19:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] conn(0x7fb7bc06ef70 msgr2=0x7fb7bc071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] conn(0x7fb7bc06ef70 0x7fb7bc071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb7bc0692a0 msgr2=0x7fb7bc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb7bc0692a0 0x7fb7bc06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0006fd0 tx=0x7fb7c0006db0 comp rx=0 tx=0).stop 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 msgr2=0x7fb7d010ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 0x7fb7d010ea30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b3b0 tx=0x7fb7cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 shutdown_connections 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6810/3805865717,v1:172.21.15.70:6811/3805865717] conn(0x7fb7bc06ef70 0x7fb7bc071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb7bc0692a0 0x7fb7bc06b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7d0101d70 0x7fb7d010ea30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 --2- 172.21.15.70:0/3650723636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d0102710 0x7fb7d010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 >> 172.21.15.70:0/3650723636 conn(0x7fb7d0072bc0 msgr2=0x7fb7d00faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 shutdown_connections 2024-03-23T08:19:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:50.605+0000 7fb7d84c0700 1 -- 172.21.15.70:0/3650723636 wait complete. 2024-03-23T08:19:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:38654705700 2024-03-23T08:19:51.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-23T08:19:51.612 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:51.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:51 smithi070 ceph-mon[39699]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:51.998 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:52.019 INFO:teuthology.orchestra.run.smithi070.stdout:60129542173 2024-03-23T08:19:52.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-23T08:19:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:51 smithi193 ceph-mon[48370]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:53.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:53 smithi070 ceph-mon[39699]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:53 smithi193 ceph-mon[48370]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.603+0000 7f3614321700 1 -- 172.21.15.70:0/3790528601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 msgr2=0x7f360c0fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.603+0000 7f3614321700 1 --2- 172.21.15.70:0/3790528601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 0x7f360c0fb940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3600004940 tx=0x7f3600009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.603+0000 7f3614321700 1 -- 172.21.15.70:0/3790528601 shutdown_connections 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.603+0000 7f3614321700 1 --2- 172.21.15.70:0/3790528601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 0x7f360c0fb940 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.603+0000 7f3614321700 1 --2- 172.21.15.70:0/3790528601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f360c102f50 0x7f360c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.603+0000 7f3614321700 1 -- 172.21.15.70:0/3790528601 >> 172.21.15.70:0/3790528601 conn(0x7f360c0714c0 msgr2=0x7f360c0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.603+0000 7f3614321700 1 -- 172.21.15.70:0/3790528601 shutdown_connections 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.603+0000 7f3614321700 1 -- 172.21.15.70:0/3790528601 wait complete. 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f3614321700 1 Processor -- start 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f3614321700 1 -- start start 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f3614321700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 0x7f360c1138c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f3614321700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f360c102f50 0x7f360c113e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f3614321700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c114450 con 0x7f360c0fb4d0 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f3614321700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c10da50 con 0x7f360c102f50 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f36120bd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 0x7f360c1138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f36120bd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 0x7f360c1138c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53590/0 (socket says 172.21.15.70:53590) 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f36120bd700 1 -- 172.21.15.70:0/3776153271 learned_addr learned my addr 172.21.15.70:0/3776153271 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f36118bc700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f360c102f50 0x7f360c113e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f36120bd700 1 -- 172.21.15.70:0/3776153271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f360c102f50 msgr2=0x7f360c113e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f36120bd700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f360c102f50 0x7f360c113e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f36120bd700 1 -- 172.21.15.70:0/3776153271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600005040 con 0x7f360c0fb4d0 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f36118bc700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f360c102f50 0x7f360c113e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f36120bd700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 0x7f360c1138c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f360400c8b0 tx=0x7f3604004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f360affd700 1 -- 172.21.15.70:0/3776153271 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f360400a760 con 0x7f360c0fb4d0 2024-03-23T08:19:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f360affd700 1 -- 172.21.15.70:0/3776153271 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f360400ad70 con 0x7f360c0fb4d0 2024-03-23T08:19:54.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c10dd30 con 0x7f360c0fb4d0 2024-03-23T08:19:54.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.604+0000 7f360affd700 1 -- 172.21.15.70:0/3776153271 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36040113a0 con 0x7f360c0fb4d0 2024-03-23T08:19:54.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.605+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c10e270 con 0x7f360c0fb4d0 2024-03-23T08:19:54.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.605+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f360c10e7f0 con 0x7f360c0fb4d0 2024-03-23T08:19:54.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.606+0000 7f360affd700 1 -- 172.21.15.70:0/3776153271 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3604012400 con 0x7f360c0fb4d0 2024-03-23T08:19:54.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.606+0000 7f360affd700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f35f40692a0 0x7f35f406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:54.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.606+0000 7f360affd700 1 -- 172.21.15.70:0/3776153271 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3604087ae0 con 0x7f360c0fb4d0 2024-03-23T08:19:54.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.607+0000 7f36118bc700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f35f40692a0 0x7f35f406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.607+0000 7f360affd700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] conn(0x7f35f406ef70 0x7f35f4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.607+0000 7f360affd700 1 -- 172.21.15.70:0/3776153271 --> [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35f4071a30 con 0x7f35f406ef70 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.607+0000 7f360affd700 1 -- 172.21.15.70:0/3776153271 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3604087ec0 con 0x7f360c0fb4d0 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.607+0000 7f36128be700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] conn(0x7f35f406ef70 0x7f35f4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.607+0000 7f36118bc700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f35f40692a0 0x7f35f406b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3600000c00 tx=0x7f3600009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.607+0000 7f36128be700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] conn(0x7f35f406ef70 0x7f35f4071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.608+0000 7f360affd700 1 -- 172.21.15.70:0/3776153271 <== osd.2 v2:172.21.15.70:6818/3317266675 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f35f4071a30 con 0x7f35f406ef70 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.622+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 --> [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f360c05e6c0 con 0x7f35f406ef70 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.622+0000 7f360affd700 1 -- 172.21.15.70:0/3776153271 <== osd.2 v2:172.21.15.70:6818/3317266675 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f360c05e6c0 con 0x7f35f406ef70 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.622+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] conn(0x7f35f406ef70 msgr2=0x7f35f4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.622+0000 7f3614321700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] conn(0x7f35f406ef70 0x7f35f4071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f35f40692a0 msgr2=0x7f35f406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f35f40692a0 0x7f35f406b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3600000c00 tx=0x7f3600009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:54.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 msgr2=0x7f360c1138c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:54.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 0x7f360c1138c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f360400c8b0 tx=0x7f3604004470 comp rx=0 tx=0).stop 2024-03-23T08:19:54.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 shutdown_connections 2024-03-23T08:19:54.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6818/3317266675,v1:172.21.15.70:6819/3317266675] conn(0x7f35f406ef70 0x7f35f4071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:54.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f35f40692a0 0x7f35f406b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:54.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f360c0fb4d0 0x7f360c1138c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:54.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 --2- 172.21.15.70:0/3776153271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f360c102f50 0x7f360c113e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:54.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 >> 172.21.15.70:0/3776153271 conn(0x7f360c0714c0 msgr2=0x7f360c0fa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:54.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 shutdown_connections 2024-03-23T08:19:54.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:54.623+0000 7f3614321700 1 -- 172.21.15.70:0/3776153271 wait complete. 2024-03-23T08:19:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.274+0000 7fb518bcc700 1 -- 172.21.15.70:0/1600768117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514101d70 msgr2=0x7fb514102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.274+0000 7fb518bcc700 1 --2- 172.21.15.70:0/1600768117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514101d70 0x7fb514102140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb504004940 tx=0x7fb504009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.274+0000 7fb518bcc700 1 -- 172.21.15.70:0/1600768117 shutdown_connections 2024-03-23T08:19:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.274+0000 7fb518bcc700 1 --2- 172.21.15.70:0/1600768117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb514102710 0x7fb5140fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.274+0000 7fb518bcc700 1 --2- 172.21.15.70:0/1600768117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514101d70 0x7fb514102140 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.274+0000 7fb518bcc700 1 -- 172.21.15.70:0/1600768117 >> 172.21.15.70:0/1600768117 conn(0x7fb514072bc0 msgr2=0x7fb514072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.274+0000 7fb518bcc700 1 -- 172.21.15.70:0/1600768117 shutdown_connections 2024-03-23T08:19:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.274+0000 7fb518bcc700 1 -- 172.21.15.70:0/1600768117 wait complete. 2024-03-23T08:19:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb518bcc700 1 Processor -- start 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb518bcc700 1 -- start start 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb518bcc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb514101d70 0x7fb51410ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb518bcc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514102710 0x7fb51410f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb518bcc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51410f660 con 0x7fb514102710 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb518bcc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51410f7a0 con 0x7fb514101d70 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb511d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514102710 0x7fb51410f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb511d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514102710 0x7fb51410f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53614/0 (socket says 172.21.15.70:53614) 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb511d9b700 1 -- 172.21.15.70:0/3159467740 learned_addr learned my addr 172.21.15.70:0/3159467740 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb511d9b700 1 -- 172.21.15.70:0/3159467740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb514101d70 msgr2=0x7fb51410ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb51259c700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb514101d70 0x7fb51410ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb511d9b700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb514101d70 0x7fb51410ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.275+0000 7fb511d9b700 1 -- 172.21.15.70:0/3159467740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504005040 con 0x7fb514102710 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.276+0000 7fb511d9b700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514102710 0x7fb51410f120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb508002a50 tx=0x7fb508004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.276+0000 7fb502ffd700 1 -- 172.21.15.70:0/3159467740 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb508007cc0 con 0x7fb514102710 2024-03-23T08:19:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.276+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb514117a90 con 0x7fb514102710 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.276+0000 7fb502ffd700 1 -- 172.21.15.70:0/3159467740 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb50800f040 con 0x7fb514102710 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.276+0000 7fb502ffd700 1 -- 172.21.15.70:0/3159467740 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb50800d9a0 con 0x7fb514102710 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.276+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb514118020 con 0x7fb514102710 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.277+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5140ff080 con 0x7fb514102710 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.277+0000 7fb502ffd700 1 -- 172.21.15.70:0/3159467740 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb50800dc60 con 0x7fb514102710 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.278+0000 7fb502ffd700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb4fc0692a0 0x7fb4fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.278+0000 7fb51259c700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb4fc0692a0 0x7fb4fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.278+0000 7fb502ffd700 1 -- 172.21.15.70:0/3159467740 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb508086ca0 con 0x7fb514102710 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.278+0000 7fb51259c700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb4fc0692a0 0x7fb4fc06b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb504004560 tx=0x7fb504009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.282+0000 7fb502ffd700 1 -- 172.21.15.70:0/3159467740 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb50808b070 con 0x7fb514102710 2024-03-23T08:19:55.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.482+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb514065e00 con 0x7fb514102710 2024-03-23T08:19:55.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.483+0000 7fb502ffd700 1 -- 172.21.15.70:0/3159467740 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb508056d20 con 0x7fb514102710 2024-03-23T08:19:55.483 INFO:teuthology.orchestra.run.smithi070.stdout:154618822671 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb4fc0692a0 msgr2=0x7fb4fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb4fc0692a0 0x7fb4fc06b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb504004560 tx=0x7fb504009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514102710 msgr2=0x7fb51410f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514102710 0x7fb51410f120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb508002a50 tx=0x7fb508004600 comp rx=0 tx=0).stop 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 shutdown_connections 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb514101d70 0x7fb51410ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fb4fc0692a0 0x7fb4fc06b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 --2- 172.21.15.70:0/3159467740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb514102710 0x7fb51410f120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 >> 172.21.15.70:0/3159467740 conn(0x7fb514072bc0 msgr2=0x7fb5140fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 shutdown_connections 2024-03-23T08:19:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:55.485+0000 7fb518bcc700 1 -- 172.21.15.70:0/3159467740 wait complete. 2024-03-23T08:19:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:55 smithi070 ceph-mon[39699]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:55 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3159467740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T08:19:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:55 smithi193 ceph-mon[48370]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:55 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3159467740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-23T08:19:57.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.093+0000 7f3a65eff700 1 -- 172.21.15.70:0/3519918732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 msgr2=0x7f3a600fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:57.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.093+0000 7f3a65eff700 1 --2- 172.21.15.70:0/3519918732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 0x7f3a600fbd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a54004940 tx=0x7f3a54009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:57.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.094+0000 7f3a65eff700 1 -- 172.21.15.70:0/3519918732 shutdown_connections 2024-03-23T08:19:57.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.094+0000 7f3a65eff700 1 --2- 172.21.15.70:0/3519918732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 0x7f3a600fbd30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.094+0000 7f3a65eff700 1 --2- 172.21.15.70:0/3519918732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a60101d20 0x7f3a601020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.094+0000 7f3a65eff700 1 -- 172.21.15.70:0/3519918732 >> 172.21.15.70:0/3519918732 conn(0x7f3a600f7850 msgr2=0x7f3a600f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:57.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.094+0000 7f3a65eff700 1 -- 172.21.15.70:0/3519918732 shutdown_connections 2024-03-23T08:19:57.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.094+0000 7f3a65eff700 1 -- 172.21.15.70:0/3519918732 wait complete. 2024-03-23T08:19:57.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.094+0000 7f3a65eff700 1 Processor -- start 2024-03-23T08:19:57.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.094+0000 7f3a65eff700 1 -- start start 2024-03-23T08:19:57.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a65eff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a60101d20 0x7f3a60113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:57.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a65eff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 0x7f3a60113d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:57.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a65eff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a601143b0 con 0x7f3a601026c0 2024-03-23T08:19:57.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a65eff700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6010d9b0 con 0x7f3a60101d20 2024-03-23T08:19:57.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a5effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 0x7f3a60113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:57.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a5effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 0x7f3a60113d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53618/0 (socket says 172.21.15.70:53618) 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a5effd700 1 -- 172.21.15.70:0/652739596 learned_addr learned my addr 172.21.15.70:0/652739596 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a5f7fe700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a60101d20 0x7f3a60113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a5effd700 1 -- 172.21.15.70:0/652739596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a60101d20 msgr2=0x7f3a60113820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a5effd700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a60101d20 0x7f3a60113820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a5effd700 1 -- 172.21.15.70:0/652739596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54005040 con 0x7f3a601026c0 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a5f7fe700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a60101d20 0x7f3a60113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a5effd700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 0x7f3a60113d60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3a54000c00 tx=0x7f3a54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a47fff700 1 -- 172.21.15.70:0/652739596 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a5400e070 con 0x7f3a601026c0 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6010dc30 con 0x7f3a601026c0 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a47fff700 1 -- 172.21.15.70:0/652739596 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a54013de0 con 0x7f3a601026c0 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.095+0000 7f3a47fff700 1 -- 172.21.15.70:0/652739596 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a5401cde0 con 0x7f3a601026c0 2024-03-23T08:19:57.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.096+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6010e190 con 0x7f3a601026c0 2024-03-23T08:19:57.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.097+0000 7f3a47fff700 1 -- 172.21.15.70:0/652739596 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3a54026460 con 0x7f3a601026c0 2024-03-23T08:19:57.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.097+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a600fd470 con 0x7f3a601026c0 2024-03-23T08:19:57.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.097+0000 7f3a47fff700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f3a480692a0 0x7f3a4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:57.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.097+0000 7f3a47fff700 1 -- 172.21.15.70:0/652739596 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3a54088310 con 0x7f3a601026c0 2024-03-23T08:19:57.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.097+0000 7f3a5f7fe700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f3a480692a0 0x7f3a4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:57.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.098+0000 7f3a5f7fe700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f3a480692a0 0x7f3a4806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a50004600 tx=0x7f3a5000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:57.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.102+0000 7f3a47fff700 1 -- 172.21.15.70:0/652739596 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3a54058480 con 0x7f3a601026c0 2024-03-23T08:19:57.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.292+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3a6004c710 con 0x7f3a601026c0 2024-03-23T08:19:57.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.293+0000 7f3a47fff700 1 -- 172.21.15.70:0/652739596 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a54058010 con 0x7f3a601026c0 2024-03-23T08:19:57.293 INFO:teuthology.orchestra.run.smithi070.stdout:120259084309 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f3a480692a0 msgr2=0x7f3a4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f3a480692a0 0x7f3a4806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a50004600 tx=0x7f3a5000b040 comp rx=0 tx=0).stop 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 msgr2=0x7f3a60113d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 0x7f3a60113d60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3a54000c00 tx=0x7f3a54009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 shutdown_connections 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a60101d20 0x7f3a60113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f3a480692a0 0x7f3a4806b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 --2- 172.21.15.70:0/652739596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a601026c0 0x7f3a60113d60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 >> 172.21.15.70:0/652739596 conn(0x7f3a600f7850 msgr2=0x7f3a600f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:57.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 shutdown_connections 2024-03-23T08:19:57.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.295+0000 7f3a65eff700 1 -- 172.21.15.70:0/652739596 wait complete. 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.646+0000 7fade0734700 1 -- 172.21.15.70:0/2151328885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd81037b0 msgr2=0x7fadd80fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.646+0000 7fade0734700 1 --2- 172.21.15.70:0/2151328885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd81037b0 0x7fadd80fce20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fadd4004940 tx=0x7fadd4009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.646+0000 7fade0734700 1 -- 172.21.15.70:0/2151328885 shutdown_connections 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.646+0000 7fade0734700 1 --2- 172.21.15.70:0/2151328885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd81037b0 0x7fadd80fce20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.646+0000 7fade0734700 1 --2- 172.21.15.70:0/2151328885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd8102e10 0x7fadd81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.646+0000 7fade0734700 1 -- 172.21.15.70:0/2151328885 >> 172.21.15.70:0/2151328885 conn(0x7fadd80f89a0 msgr2=0x7fadd80fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.646+0000 7fade0734700 1 -- 172.21.15.70:0/2151328885 shutdown_connections 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.646+0000 7fade0734700 1 -- 172.21.15.70:0/2151328885 wait complete. 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fade0734700 1 Processor -- start 2024-03-23T08:19:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fade0734700 1 -- start start 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fade0734700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8102e10 0x7fadd80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fade0734700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd81037b0 0x7fadd8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fade0734700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8070d90 con 0x7fadd8102e10 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fade0734700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8070ed0 con 0x7fadd81037b0 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fadde4d0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8102e10 0x7fadd80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fadde4d0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8102e10 0x7fadd80702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53646/0 (socket says 172.21.15.70:53646) 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fadde4d0700 1 -- 172.21.15.70:0/4217285852 learned_addr learned my addr 172.21.15.70:0/4217285852 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fadddccf700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd81037b0 0x7fadd8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fadde4d0700 1 -- 172.21.15.70:0/4217285852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd81037b0 msgr2=0x7fadd8070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fadde4d0700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd81037b0 0x7fadd8070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fadde4d0700 1 -- 172.21.15.70:0/4217285852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd4005040 con 0x7fadd8102e10 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.647+0000 7fadde4d0700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8102e10 0x7fadd80702c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fadc800eb70 tx=0x7fadc800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:57.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.648+0000 7fadceffd700 1 -- 172.21.15.70:0/4217285852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc8007cc0 con 0x7fadd8102e10 2024-03-23T08:19:57.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.648+0000 7fadceffd700 1 -- 172.21.15.70:0/4217285852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fadc800f040 con 0x7fadd8102e10 2024-03-23T08:19:57.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.648+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd8071120 con 0x7fadd8102e10 2024-03-23T08:19:57.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.648+0000 7fadceffd700 1 -- 172.21.15.70:0/4217285852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc80095f0 con 0x7fadd8102e10 2024-03-23T08:19:57.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.648+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd811a180 con 0x7fadd8102e10 2024-03-23T08:19:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.649+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd80fe560 con 0x7fadd8102e10 2024-03-23T08:19:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.649+0000 7fadceffd700 1 -- 172.21.15.70:0/4217285852 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fadc8009750 con 0x7fadd8102e10 2024-03-23T08:19:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.649+0000 7fadceffd700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fadc40692a0 0x7fadc406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:19:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.649+0000 7fadddccf700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fadc40692a0 0x7fadc406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:19:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.649+0000 7fadceffd700 1 -- 172.21.15.70:0/4217285852 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fadc8086a80 con 0x7fadd8102e10 2024-03-23T08:19:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.650+0000 7fadddccf700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fadc40692a0 0x7fadc406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fadd8071c00 tx=0x7fadd4009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:19:57.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.654+0000 7fadceffd700 1 -- 172.21.15.70:0/4217285852 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fadc8053070 con 0x7fadd8102e10 2024-03-23T08:19:57.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.843+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fadd805f5f0 con 0x7fadd8102e10 2024-03-23T08:19:57.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.843+0000 7fadceffd700 1 -- 172.21.15.70:0/4217285852 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fadc8056690 con 0x7fadd8102e10 2024-03-23T08:19:57.844 INFO:teuthology.orchestra.run.smithi070.stdout:176093659146 2024-03-23T08:19:57.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fadc40692a0 msgr2=0x7fadc406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:57.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fadc40692a0 0x7fadc406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fadd8071c00 tx=0x7fadd4009e30 comp rx=0 tx=0).stop 2024-03-23T08:19:57.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8102e10 msgr2=0x7fadd80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:19:57.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8102e10 0x7fadd80702c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fadc800eb70 tx=0x7fadc800b040 comp rx=0 tx=0).stop 2024-03-23T08:19:57.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 shutdown_connections 2024-03-23T08:19:57.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fadc40692a0 0x7fadc406b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8102e10 0x7fadd80702c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 --2- 172.21.15.70:0/4217285852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadd81037b0 0x7fadd8070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:19:57.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 >> 172.21.15.70:0/4217285852 conn(0x7fadd80f89a0 msgr2=0x7fadd80f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:19:57.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 shutdown_connections 2024-03-23T08:19:57.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:19:57.845+0000 7fade0734700 1 -- 172.21.15.70:0/4217285852 wait complete. 2024-03-23T08:19:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:57 smithi070 ceph-mon[39699]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:57 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/652739596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T08:19:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:57 smithi193 ceph-mon[48370]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:19:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:57 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/652739596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-23T08:19:58.091 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822669 got 154618822671 for osd.4 2024-03-23T08:19:58.091 DEBUG:teuthology.parallel:result is None 2024-03-23T08:19:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:58 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/4217285852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T08:19:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:58 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/4217285852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-23T08:19:59.334 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:59.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659145 got 176093659146 for osd.5 2024-03-23T08:19:59.362 DEBUG:teuthology.parallel:result is None 2024-03-23T08:19:59.764 INFO:teuthology.orchestra.run.smithi070.stdout:81604378650 2024-03-23T08:19:59.764 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-23T08:19:59.889 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:19:59.912 INFO:tasks.cephadm.ceph_manager.ceph:need seq 120259084306 got 120259084309 for osd.3 2024-03-23T08:19:59.912 DEBUG:teuthology.parallel:result is None 2024-03-23T08:19:59.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:19:59 smithi070 ceph-mon[39699]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:19:59 smithi193 ceph-mon[48370]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:00.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:00 smithi070 ceph-mon[39699]: overall HEALTH_OK 2024-03-23T08:20:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:00 smithi193 ceph-mon[48370]: overall HEALTH_OK 2024-03-23T08:20:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:01 smithi070 ceph-mon[39699]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:01 smithi193 ceph-mon[48370]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.305+0000 7f16c7a03700 1 -- 172.21.15.70:0/2848606979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 msgr2=0x7f16c0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.305+0000 7f16c7a03700 1 --2- 172.21.15.70:0/2848606979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 0x7f16c0102150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16bc004940 tx=0x7f16bc009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 -- 172.21.15.70:0/2848606979 shutdown_connections 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 --2- 172.21.15.70:0/2848606979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c0102720 0x7f16c00fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 --2- 172.21.15.70:0/2848606979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 0x7f16c0102150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 -- 172.21.15.70:0/2848606979 >> 172.21.15.70:0/2848606979 conn(0x7f16c00f78f0 msgr2=0x7f16c00f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 -- 172.21.15.70:0/2848606979 shutdown_connections 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 -- 172.21.15.70:0/2848606979 wait complete. 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 Processor -- start 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 -- start start 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 0x7f16c0113870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c0102720 0x7f16c0113db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:02.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0114490 con 0x7f16c0101d80 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c7a03700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c010da00 con 0x7f16c0102720 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c579f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 0x7f16c0113870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c579f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 0x7f16c0113870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49732/0 (socket says 172.21.15.70:49732) 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.306+0000 7f16c579f700 1 -- 172.21.15.70:0/1852541171 learned_addr learned my addr 172.21.15.70:0/1852541171 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16c4f9e700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c0102720 0x7f16c0113db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16c579f700 1 -- 172.21.15.70:0/1852541171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c0102720 msgr2=0x7f16c0113db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16c579f700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c0102720 0x7f16c0113db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16c579f700 1 -- 172.21.15.70:0/1852541171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc005040 con 0x7f16c0101d80 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16c579f700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 0x7f16c0113870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00b3b0 tx=0x7f16bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16b5ffb700 1 -- 172.21.15.70:0/1852541171 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16bc00e040 con 0x7f16c0101d80 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16b5ffb700 1 -- 172.21.15.70:0/1852541171 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16bc013420 con 0x7f16c0101d80 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16b5ffb700 1 -- 172.21.15.70:0/1852541171 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16bc01c630 con 0x7f16c0101d80 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c010dc80 con 0x7f16c0101d80 2024-03-23T08:20:02.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.307+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c010e240 con 0x7f16c0101d80 2024-03-23T08:20:02.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.308+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c00fd4d0 con 0x7f16c0101d80 2024-03-23T08:20:02.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.309+0000 7f16b5ffb700 1 -- 172.21.15.70:0/1852541171 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f16bc013970 con 0x7f16c0101d80 2024-03-23T08:20:02.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.309+0000 7f16b5ffb700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f16ac0692a0 0x7f16ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:02.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.309+0000 7f16b5ffb700 1 -- 172.21.15.70:0/1852541171 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f16bc088090 con 0x7f16c0101d80 2024-03-23T08:20:02.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.309+0000 7f16c4f9e700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f16ac0692a0 0x7f16ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:02.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.309+0000 7f16c4f9e700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f16ac0692a0 0x7f16ac06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006fd0 tx=0x7f16b0008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:02.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.313+0000 7f16b5ffb700 1 -- 172.21.15.70:0/1852541171 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f16bc058190 con 0x7f16c0101d80 2024-03-23T08:20:02.508 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.507+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f16c004c710 con 0x7f16c0101d80 2024-03-23T08:20:02.508 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.508+0000 7f16b5ffb700 1 -- 172.21.15.70:0/1852541171 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16bc057d20 con 0x7f16c0101d80 2024-03-23T08:20:02.508 INFO:teuthology.orchestra.run.smithi070.stdout:38654705703 2024-03-23T08:20:02.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f16ac0692a0 msgr2=0x7f16ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:02.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f16ac0692a0 0x7f16ac06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006fd0 tx=0x7f16b0008040 comp rx=0 tx=0).stop 2024-03-23T08:20:02.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 msgr2=0x7f16c0113870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:02.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 0x7f16c0113870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00b3b0 tx=0x7f16bc009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:02.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 shutdown_connections 2024-03-23T08:20:02.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f16ac0692a0 0x7f16ac06b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:02.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c0101d80 0x7f16c0113870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:02.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 --2- 172.21.15.70:0/1852541171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16c0102720 0x7f16c0113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:02.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 >> 172.21.15.70:0/1852541171 conn(0x7f16c00f78f0 msgr2=0x7f16c01062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:02.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 shutdown_connections 2024-03-23T08:20:02.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:02.510+0000 7f16c7a03700 1 -- 172.21.15.70:0/1852541171 wait complete. 2024-03-23T08:20:02.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:02 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1852541171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T08:20:03.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:02 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1852541171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-23T08:20:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.657+0000 7fe00ea80700 1 -- 172.21.15.70:0/1123798317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 msgr2=0x7fe0080fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:03.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.657+0000 7fe00ea80700 1 --2- 172.21.15.70:0/1123798317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 0x7fe0080fb990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdff4004940 tx=0x7fdff4009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:03.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.657+0000 7fe00ea80700 1 -- 172.21.15.70:0/1123798317 shutdown_connections 2024-03-23T08:20:03.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.657+0000 7fe00ea80700 1 --2- 172.21.15.70:0/1123798317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 0x7fe0080fb990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:03.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.657+0000 7fe00ea80700 1 --2- 172.21.15.70:0/1123798317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008102fd0 0x7fe0081033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:03.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.657+0000 7fe00ea80700 1 -- 172.21.15.70:0/1123798317 >> 172.21.15.70:0/1123798317 conn(0x7fe008071530 msgr2=0x7fe008071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:03.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.657+0000 7fe00ea80700 1 -- 172.21.15.70:0/1123798317 shutdown_connections 2024-03-23T08:20:03.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.657+0000 7fe00ea80700 1 -- 172.21.15.70:0/1123798317 wait complete. 2024-03-23T08:20:03.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.657+0000 7fe00ea80700 1 Processor -- start 2024-03-23T08:20:03.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00ea80700 1 -- start start 2024-03-23T08:20:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00ea80700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 0x7fe0081138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00ea80700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008102fd0 0x7fe008113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00ea80700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008114510 con 0x7fe0080fb540 2024-03-23T08:20:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00ea80700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00810da80 con 0x7fe008102fd0 2024-03-23T08:20:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00c81c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 0x7fe0081138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00c81c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 0x7fe0081138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49744/0 (socket says 172.21.15.70:49744) 2024-03-23T08:20:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00c81c700 1 -- 172.21.15.70:0/1381724833 learned_addr learned my addr 172.21.15.70:0/1381724833 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe007fff700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008102fd0 0x7fe008113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00c81c700 1 -- 172.21.15.70:0/1381724833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008102fd0 msgr2=0x7fe008113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00c81c700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008102fd0 0x7fe008113e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.658+0000 7fe00c81c700 1 -- 172.21.15.70:0/1381724833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff4005040 con 0x7fe0080fb540 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.659+0000 7fe00c81c700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 0x7fe0081138f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00b790 tx=0x7fdffc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.659+0000 7fe0057fa700 1 -- 172.21.15.70:0/1381724833 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdffc00f770 con 0x7fe0080fb540 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.659+0000 7fe0057fa700 1 -- 172.21.15.70:0/1381724833 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdffc00fd80 con 0x7fe0080fb540 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.659+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00810dd60 con 0x7fe0080fb540 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.659+0000 7fe0057fa700 1 -- 172.21.15.70:0/1381724833 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdffc00d310 con 0x7fe0080fb540 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.659+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00810e2a0 con 0x7fe0080fb540 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.660+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe008100270 con 0x7fe0080fb540 2024-03-23T08:20:03.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.660+0000 7fe0057fa700 1 -- 172.21.15.70:0/1381724833 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdffc010420 con 0x7fe0080fb540 2024-03-23T08:20:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.660+0000 7fe0057fa700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fdff8069370 0x7fdff806b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.661+0000 7fe007fff700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fdff8069370 0x7fdff806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.661+0000 7fe0057fa700 1 -- 172.21.15.70:0/1381724833 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdffc086db0 con 0x7fe0080fb540 2024-03-23T08:20:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.661+0000 7fe007fff700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fdff8069370 0x7fdff806b820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdff400b3b0 tx=0x7fdff4009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:03.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.665+0000 7fe0057fa700 1 -- 172.21.15.70:0/1381724833 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdffc056e90 con 0x7fe0080fb540 2024-03-23T08:20:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.856+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe00810ef70 con 0x7fe0080fb540 2024-03-23T08:20:03.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.856+0000 7fe0057fa700 1 -- 172.21.15.70:0/1381724833 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdffc056a20 con 0x7fe0080fb540 2024-03-23T08:20:03.856 INFO:teuthology.orchestra.run.smithi070.stdout:60129542175 2024-03-23T08:20:03.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fdff8069370 msgr2=0x7fdff806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:03.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fdff8069370 0x7fdff806b820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdff400b3b0 tx=0x7fdff4009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:03.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 msgr2=0x7fe0081138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:03.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 0x7fe0081138f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00b790 tx=0x7fdffc009250 comp rx=0 tx=0).stop 2024-03-23T08:20:03.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 shutdown_connections 2024-03-23T08:20:03.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fdff8069370 0x7fdff806b820 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:03.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0080fb540 0x7fe0081138f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:03.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 --2- 172.21.15.70:0/1381724833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008102fd0 0x7fe008113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:03.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 >> 172.21.15.70:0/1381724833 conn(0x7fe008071530 msgr2=0x7fe0080fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:03.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 shutdown_connections 2024-03-23T08:20:03.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:03.858+0000 7fe00ea80700 1 -- 172.21.15.70:0/1381724833 wait complete. 2024-03-23T08:20:03.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:03 smithi070 ceph-mon[39699]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:03.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:20:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:03 smithi193 ceph-mon[48370]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:20:04.139 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705703 for osd.0 2024-03-23T08:20:04.139 DEBUG:teuthology.parallel:result is None 2024-03-23T08:20:04.941 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:04 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1381724833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T08:20:04.959 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542175 for osd.1 2024-03-23T08:20:04.959 DEBUG:teuthology.parallel:result is None 2024-03-23T08:20:05.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:04 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1381724833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-23T08:20:05.928 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:05 smithi070 ceph-mon[39699]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:05 smithi193 ceph-mon[48370]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:06.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.178+0000 7f3353be8700 1 -- 172.21.15.70:0/1865867870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 msgr2=0x7f334c1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:06.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.178+0000 7f3353be8700 1 --2- 172.21.15.70:0/1865867870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 0x7f334c1031e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f333c004940 tx=0x7f333c009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:06.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.178+0000 7f3353be8700 1 -- 172.21.15.70:0/1865867870 shutdown_connections 2024-03-23T08:20:06.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.178+0000 7f3353be8700 1 --2- 172.21.15.70:0/1865867870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f334c1037b0 0x7f334c0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:06.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.178+0000 7f3353be8700 1 --2- 172.21.15.70:0/1865867870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 0x7f334c1031e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:06.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.178+0000 7f3353be8700 1 -- 172.21.15.70:0/1865867870 >> 172.21.15.70:0/1865867870 conn(0x7f334c0f8980 msgr2=0x7f334c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:06.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.178+0000 7f3353be8700 1 -- 172.21.15.70:0/1865867870 shutdown_connections 2024-03-23T08:20:06.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.178+0000 7f3353be8700 1 -- 172.21.15.70:0/1865867870 wait complete. 2024-03-23T08:20:06.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3353be8700 1 Processor -- start 2024-03-23T08:20:06.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3353be8700 1 -- start start 2024-03-23T08:20:06.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3353be8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 0x7f334c113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3353be8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f334c1037b0 0x7f334c114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3353be8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1146a0 con 0x7f334c102e10 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3353be8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c10da90 con 0x7f334c1037b0 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3351984700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 0x7f334c113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3351984700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 0x7f334c113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49760/0 (socket says 172.21.15.70:49760) 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3351984700 1 -- 172.21.15.70:0/2025638142 learned_addr learned my addr 172.21.15.70:0/2025638142 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3351183700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f334c1037b0 0x7f334c114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3351984700 1 -- 172.21.15.70:0/2025638142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f334c1037b0 msgr2=0x7f334c114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3351984700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f334c1037b0 0x7f334c114050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3351984700 1 -- 172.21.15.70:0/2025638142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c005040 con 0x7f334c102e10 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3351183700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f334c1037b0 0x7f334c114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.179+0000 7f3351984700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 0x7f334c113b10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f333c00b3b0 tx=0x7f333c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.180+0000 7f33427fc700 1 -- 172.21.15.70:0/2025638142 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333c00e040 con 0x7f334c102e10 2024-03-23T08:20:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.180+0000 7f33427fc700 1 -- 172.21.15.70:0/2025638142 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f333c013420 con 0x7f334c102e10 2024-03-23T08:20:06.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.180+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c10dd10 con 0x7f334c102e10 2024-03-23T08:20:06.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.180+0000 7f33427fc700 1 -- 172.21.15.70:0/2025638142 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333c01c630 con 0x7f334c102e10 2024-03-23T08:20:06.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.180+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c10e2d0 con 0x7f334c102e10 2024-03-23T08:20:06.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.181+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c0fe560 con 0x7f334c102e10 2024-03-23T08:20:06.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.181+0000 7f33427fc700 1 -- 172.21.15.70:0/2025638142 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f333c013970 con 0x7f334c102e10 2024-03-23T08:20:06.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.181+0000 7f33427fc700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f33380692a0 0x7f333806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:06.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.182+0000 7f33427fc700 1 -- 172.21.15.70:0/2025638142 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f333c087fc0 con 0x7f334c102e10 2024-03-23T08:20:06.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.182+0000 7f3351183700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f33380692a0 0x7f333806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:06.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.182+0000 7f3351183700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f33380692a0 0x7f333806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3348009fd0 tx=0x7f3348009ee0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:06.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.186+0000 7f33427fc700 1 -- 172.21.15.70:0/2025638142 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f333c058160 con 0x7f334c102e10 2024-03-23T08:20:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.371+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f334c04bed0 con 0x7f334c102e10 2024-03-23T08:20:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.371+0000 7f33427fc700 1 -- 172.21.15.70:0/2025638142 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f333c057cf0 con 0x7f334c102e10 2024-03-23T08:20:06.371 INFO:teuthology.orchestra.run.smithi070.stdout:81604378652 2024-03-23T08:20:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f33380692a0 msgr2=0x7f333806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f33380692a0 0x7f333806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3348009fd0 tx=0x7f3348009ee0 comp rx=0 tx=0).stop 2024-03-23T08:20:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 msgr2=0x7f334c113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 0x7f334c113b10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f333c00b3b0 tx=0x7f333c009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 shutdown_connections 2024-03-23T08:20:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f33380692a0 0x7f333806b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f334c102e10 0x7f334c113b10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 --2- 172.21.15.70:0/2025638142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f334c1037b0 0x7f334c114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 >> 172.21.15.70:0/2025638142 conn(0x7f334c0f8980 msgr2=0x7f334c0f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 shutdown_connections 2024-03-23T08:20:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:06.373+0000 7f3353be8700 1 -- 172.21.15.70:0/2025638142 wait complete. 2024-03-23T08:20:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:06 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2025638142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T08:20:07.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:06 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2025638142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-23T08:20:07.106 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378650 got 81604378652 for osd.2 2024-03-23T08:20:07.106 DEBUG:teuthology.parallel:result is None 2024-03-23T08:20:07.106 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-23T08:20:07.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T08:20:07.371 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:07.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:07 smithi070 ceph-mon[39699]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:07 smithi193 ceph-mon[48370]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.121+0000 7f4c8111d700 1 -- 172.21.15.70:0/175675167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c1037b0 msgr2=0x7f4c7c0fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.121+0000 7f4c8111d700 1 --2- 172.21.15.70:0/175675167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c1037b0 0x7f4c7c0fce20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c004940 tx=0x7f4c6c009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 -- 172.21.15.70:0/175675167 shutdown_connections 2024-03-23T08:20:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 --2- 172.21.15.70:0/175675167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c1037b0 0x7f4c7c0fce20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 --2- 172.21.15.70:0/175675167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c7c102e10 0x7f4c7c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 -- 172.21.15.70:0/175675167 >> 172.21.15.70:0/175675167 conn(0x7f4c7c0f89a0 msgr2=0x7f4c7c0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 -- 172.21.15.70:0/175675167 shutdown_connections 2024-03-23T08:20:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 -- 172.21.15.70:0/175675167 wait complete. 2024-03-23T08:20:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 Processor -- start 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 -- start start 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c102e10 0x7f4c7c113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.122+0000 7f4c8111d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c7c1037b0 0x7f4c7c114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c8111d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c114710 con 0x7f4c7c102e10 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c8111d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c10da70 con 0x7f4c7c1037b0 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c7ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c102e10 0x7f4c7c113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c7ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c102e10 0x7f4c7c113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50478/0 (socket says 172.21.15.70:50478) 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c7ad9d700 1 -- 172.21.15.70:0/3450250414 learned_addr learned my addr 172.21.15.70:0/3450250414 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c7ad9d700 1 -- 172.21.15.70:0/3450250414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c7c1037b0 msgr2=0x7f4c7c114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c7ad9d700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c7c1037b0 0x7f4c7c114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c7ad9d700 1 -- 172.21.15.70:0/3450250414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c005040 con 0x7f4c7c102e10 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c7ad9d700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c102e10 0x7f4c7c113af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000b790 tx=0x7f4c70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c637fe700 1 -- 172.21.15.70:0/3450250414 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7000f770 con 0x7f4c7c102e10 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c637fe700 1 -- 172.21.15.70:0/3450250414 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c7000fd80 con 0x7f4c7c102e10 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7c10dd50 con 0x7f4c7c102e10 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c637fe700 1 -- 172.21.15.70:0/3450250414 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7000d310 con 0x7f4c7c102e10 2024-03-23T08:20:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.123+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c10e310 con 0x7f4c7c102e10 2024-03-23T08:20:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.124+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c0fe560 con 0x7f4c7c102e10 2024-03-23T08:20:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.124+0000 7f4c637fe700 1 -- 172.21.15.70:0/3450250414 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4c70010420 con 0x7f4c7c102e10 2024-03-23T08:20:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.125+0000 7f4c637fe700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4c640692a0 0x7f4c6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.125+0000 7f4c637fe700 1 -- 172.21.15.70:0/3450250414 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4c70086df0 con 0x7f4c7c102e10 2024-03-23T08:20:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.125+0000 7f4c7a59c700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4c640692a0 0x7f4c6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.125+0000 7f4c7a59c700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4c640692a0 0x7f4c6406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c004f90 tx=0x7f4c6c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.129+0000 7f4c637fe700 1 -- 172.21.15.70:0/3450250414 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4c70056ef0 con 0x7f4c7c102e10 2024-03-23T08:20:09.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.310+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c7c109ec0 con 0x7f4c640692a0 2024-03-23T08:20:09.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.311+0000 7f4c637fe700 1 -- 172.21.15.70:0/3450250414 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27962 (secure 0 0 0) 0x7f4c6c007cc0 con 0x7f4c640692a0 2024-03-23T08:20:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:09.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4c640692a0 msgr2=0x7f4c6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:09.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4c640692a0 0x7f4c6406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c004f90 tx=0x7f4c6c009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c102e10 msgr2=0x7f4c7c113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c102e10 0x7f4c7c113af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000b790 tx=0x7f4c70009250 comp rx=0 tx=0).stop 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 shutdown_connections 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4c640692a0 0x7f4c6406b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c7c102e10 0x7f4c7c113af0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 --2- 172.21.15.70:0/3450250414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c7c1037b0 0x7f4c7c114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 >> 172.21.15.70:0/3450250414 conn(0x7f4c7c0f89a0 msgr2=0x7f4c7c0f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 shutdown_connections 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:09.313+0000 7f4c8111d700 1 -- 172.21.15.70:0/3450250414 wait complete. 2024-03-23T08:20:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-03-23T08:20:09.910 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:09 smithi070 ceph-mon[39699]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:09.911 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-03-23T08:20:08.438469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787696,"kb_used_data":3968,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683216,"statfs":{"total":575970213888,"available":574139613184,"internally_reserved":0,"allocated":4063232,"data_stored":2644380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001927"},"pg_stats":[{"pgid":"1.0","version":"25'117","reported_seq":191,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T08:19:17.608252+0000","last_change":"2024-03-23T08:18:50.760366+0000","last_active":"2024-03-23T08:19:17.608252+0000","last_peered":"2024-03-23T08:19:17.608252+0000","last_clean":"2024-03-23T08:19:17.608252+0000","last_became_active":"2024-03-23T08:18:50.759984+0000","last_became_peered":"2024-03-23T08:18:50.759984+0000","last_unstale":"2024-03-23T08:19:17.608252+0000","last_undegraded":"2024-03-23T08:19:17.608252+0000","last_fullsized":"2024-03-23T08:19:17.608252+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T08:18:02.790186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T08:18:02.790186+0000","last_clean_scrub_stamp":"2024-03-23T08:18:02.790186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:28:23.478029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":41,"seq":176093659148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447492,"statfs":{"total":95995035648,"available":95690231808,"internally_reserved":0,"allocated":380928,"data_stored":145647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]},{"osd":4,"up_from":36,"seq":154618822674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447492,"statfs":{"total":95995035648,"available":95690231808,"internally_reserved":0,"allocated":380928,"data_stored":145647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.40799999999999997}]},{"osd":1,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.36699999999999999}]},{"osd":2,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.433},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46000000000000002}]},{"osd":3,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.501},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":3,"up_from":28,"seq":120259084311,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":974848,"data_stored":736015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.497},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Mar 23 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.61699999999999999}]},{"osd":2,"last update":"Sat Mar 23 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.441},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447496,"statfs":{"total":95995035648,"available":95690235904,"internally_reserved":0,"allocated":376832,"data_stored":145041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 08:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.443,"5min":0.443,"15min":0.443},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.23799999999999999}]},{"osd":1,"last update":"Sat Mar 23 08:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":1.8540000000000001}]},{"osd":3,"last update":"Sat Mar 23 08:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60799999999999998}]},{"osd":4,"last update":"Sat Mar 23 08:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":974848,"data_stored":736015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 08:19:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.28100000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.40500000000000003}]},{"osd":2,"last update":"Sat Mar 23 08:19:06 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"last":0.442},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Mar 23 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat Mar 23 08:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":974848,"data_stored":736015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 08:19:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33600000000000002},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.47499999999999998,"5min":0.624,"15min":0.624},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.34799999999999998},"min":{"1min":0.26200000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48899999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.29399999999999998}]},{"osd":2,"last update":"Sat Mar 23 08:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36299999999999999,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.47299999999999998,"5min":0.495,"15min":0.495},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.375,"15min":0.38400000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48799999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.311}]},{"osd":3,"last update":"Sat Mar 23 08:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.46400000000000002}]},{"osd":4,"last update":"Sat Mar 23 08:19:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.497},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T08:20:09.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-23T08:20:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:09 smithi193 ceph-mon[48370]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:10.178 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:11.069 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:10 smithi070 ceph-mon[39699]: from='client.14410 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T08:20:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:10 smithi193 ceph-mon[48370]: from='client.14410 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T08:20:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.958+0000 7f66f4d3d700 1 -- 172.21.15.70:0/36772685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f00fbca0 msgr2=0x7f66f00fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.958+0000 7f66f4d3d700 1 --2- 172.21.15.70:0/36772685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f00fbca0 0x7f66f00fc110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f66e4004940 tx=0x7f66e4009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 -- 172.21.15.70:0/36772685 shutdown_connections 2024-03-23T08:20:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 --2- 172.21.15.70:0/36772685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f00fbca0 0x7f66f00fc110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 --2- 172.21.15.70:0/36772685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f0101cf0 0x7f66f01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 -- 172.21.15.70:0/36772685 >> 172.21.15.70:0/36772685 conn(0x7f66f0072bc0 msgr2=0x7f66f0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 -- 172.21.15.70:0/36772685 shutdown_connections 2024-03-23T08:20:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 -- 172.21.15.70:0/36772685 wait complete. 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 Processor -- start 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 -- start start 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f00fbca0 0x7f66f0102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f0101cf0 0x7f66f0102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01035e0 con 0x7f66f0101cf0 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66f4d3d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0103720 con 0x7f66f00fbca0 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66edd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f0101cf0 0x7f66f0102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66edd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f0101cf0 0x7f66f0102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50490/0 (socket says 172.21.15.70:50490) 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.959+0000 7f66edd9b700 1 -- 172.21.15.70:0/1496696390 learned_addr learned my addr 172.21.15.70:0/1496696390 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66edd9b700 1 -- 172.21.15.70:0/1496696390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f00fbca0 msgr2=0x7f66f0102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66ee59c700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f00fbca0 0x7f66f0102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66edd9b700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f00fbca0 0x7f66f0102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66edd9b700 1 -- 172.21.15.70:0/1496696390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e4005040 con 0x7f66f0101cf0 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66ee59c700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f00fbca0 0x7f66f0102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:20:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66edd9b700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f0101cf0 0x7f66f0102fc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f66e400b3b0 tx=0x7f66e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66deffd700 1 -- 172.21.15.70:0/1496696390 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e400e040 con 0x7f66f0101cf0 2024-03-23T08:20:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66deffd700 1 -- 172.21.15.70:0/1496696390 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66e4013420 con 0x7f66f0101cf0 2024-03-23T08:20:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f0119d00 con 0x7f66f0101cf0 2024-03-23T08:20:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66deffd700 1 -- 172.21.15.70:0/1496696390 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e401c630 con 0x7f66f0101cf0 2024-03-23T08:20:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.960+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f011a1e0 con 0x7f66f0101cf0 2024-03-23T08:20:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.961+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f00fd390 con 0x7f66f0101cf0 2024-03-23T08:20:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.961+0000 7f66deffd700 1 -- 172.21.15.70:0/1496696390 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f66e4013970 con 0x7f66f0101cf0 2024-03-23T08:20:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.962+0000 7f66deffd700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f66d80692a0 0x7f66d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.962+0000 7f66deffd700 1 -- 172.21.15.70:0/1496696390 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f66e4088190 con 0x7f66f0101cf0 2024-03-23T08:20:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.962+0000 7f66ee59c700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f66d80692a0 0x7f66d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.962+0000 7f66ee59c700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f66d80692a0 0x7f66d806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f66e0006fd0 tx=0x7f66e0008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:11.966+0000 7f66deffd700 1 -- 172.21.15.70:0/1496696390 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f66e4054760 con 0x7f66f0101cf0 2024-03-23T08:20:11.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:11 smithi070 ceph-mon[39699]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:11 smithi193 ceph-mon[48370]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:12.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.143+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66f004bed0 con 0x7f66d80692a0 2024-03-23T08:20:12.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.144+0000 7f66deffd700 1 -- 172.21.15.70:0/1496696390 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27975 (secure 0 0 0) 0x7f66f004bed0 con 0x7f66d80692a0 2024-03-23T08:20:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f66d80692a0 msgr2=0x7f66d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f66d80692a0 0x7f66d806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f66e0006fd0 tx=0x7f66e0008040 comp rx=0 tx=0).stop 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f0101cf0 msgr2=0x7f66f0102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f0101cf0 0x7f66f0102fc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f66e400b3b0 tx=0x7f66e4009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 shutdown_connections 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66f00fbca0 0x7f66f0102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f66d80692a0 0x7f66d806b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 --2- 172.21.15.70:0/1496696390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f0101cf0 0x7f66f0102fc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 >> 172.21.15.70:0/1496696390 conn(0x7f66f0072bc0 msgr2=0x7f66f0065600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 shutdown_connections 2024-03-23T08:20:12.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:12.146+0000 7f66f4d3d700 1 -- 172.21.15.70:0/1496696390 wait complete. 2024-03-23T08:20:12.148 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-03-23T08:20:12.885 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-03-23T08:20:10.438852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787696,"kb_used_data":3968,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683216,"statfs":{"total":575970213888,"available":574139613184,"internally_reserved":0,"allocated":4063232,"data_stored":2644380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002025"},"pg_stats":[{"pgid":"1.0","version":"25'117","reported_seq":191,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-23T08:19:17.608252+0000","last_change":"2024-03-23T08:18:50.760366+0000","last_active":"2024-03-23T08:19:17.608252+0000","last_peered":"2024-03-23T08:19:17.608252+0000","last_clean":"2024-03-23T08:19:17.608252+0000","last_became_active":"2024-03-23T08:18:50.759984+0000","last_became_peered":"2024-03-23T08:18:50.759984+0000","last_unstale":"2024-03-23T08:19:17.608252+0000","last_undegraded":"2024-03-23T08:19:17.608252+0000","last_fullsized":"2024-03-23T08:19:17.608252+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T08:18:02.790186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T08:18:02.790186+0000","last_clean_scrub_stamp":"2024-03-23T08:18:02.790186+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T08:28:23.478029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":41,"seq":176093659149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447492,"statfs":{"total":95995035648,"available":95690231808,"internally_reserved":0,"allocated":380928,"data_stored":145647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":4,"up_from":36,"seq":154618822674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447492,"statfs":{"total":95995035648,"available":95690231808,"internally_reserved":0,"allocated":380928,"data_stored":145647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.40799999999999997}]},{"osd":1,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.36699999999999999}]},{"osd":2,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.433},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46000000000000002}]},{"osd":3,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.501},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":3,"up_from":28,"seq":120259084312,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":974848,"data_stored":736015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.48299999999999998}]},{"osd":1,"last update":"Sat Mar 23 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Mar 23 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.495}]},{"osd":4,"last update":"Sat Mar 23 08:19:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.371},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447496,"statfs":{"total":95995035648,"available":95690235904,"internally_reserved":0,"allocated":376832,"data_stored":145041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 08:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.443,"5min":0.443,"15min":0.443},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.432,"5min":0.432,"15min":0.432},"last":0.23799999999999999}]},{"osd":1,"last update":"Sat Mar 23 08:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":1.8540000000000001}]},{"osd":3,"last update":"Sat Mar 23 08:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60799999999999998}]},{"osd":4,"last update":"Sat Mar 23 08:19:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":974848,"data_stored":736015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 23 08:19:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.312,"15min":0.30599999999999999},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.33600000000000002,"15min":0.33000000000000002},"min":{"1min":0.28100000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.46800000000000003}]},{"osd":2,"last update":"Sat Mar 23 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35399999999999998,"15min":0.35199999999999998},"min":{"1min":0.26000000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.47499999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.437}]},{"osd":3,"last update":"Sat Mar 23 08:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.77400000000000002}]},{"osd":4,"last update":"Sat Mar 23 08:19:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298240,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446912,"statfs":{"total":95995035648,"available":95689637888,"internally_reserved":0,"allocated":974848,"data_stored":736015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 23 08:19:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.33600000000000002},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.47499999999999998,"5min":0.624,"15min":0.624},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.34799999999999998},"min":{"1min":0.26200000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.48899999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.29399999999999998}]},{"osd":2,"last update":"Sat Mar 23 08:20:06 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.36299999999999999,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.47299999999999998,"5min":0.495,"15min":0.495},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.375,"15min":0.38400000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48799999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.311}]},{"osd":3,"last update":"Sat Mar 23 08:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.46400000000000002}]},{"osd":4,"last update":"Sat Mar 23 08:19:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.497},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T08:20:12.886 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-23T08:20:12.886 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-23T08:20:12.886 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-23T08:20:12.887 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-23T08:20:13.150 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:13.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:13 smithi070 ceph-mon[39699]: from='client.14414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T08:20:13.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:13 smithi070 ceph-mon[39699]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:13 smithi193 ceph-mon[48370]: from='client.14414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-23T08:20:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:13 smithi193 ceph-mon[48370]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 -- 172.21.15.70:0/1772634372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 msgr2=0x7f43201031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 --2- 172.21.15.70:0/1772634372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 0x7f43201031d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4310004940 tx=0x7f4310009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 -- 172.21.15.70:0/1772634372 shutdown_connections 2024-03-23T08:20:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 --2- 172.21.15.70:0/1772634372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43201037a0 0x7f43200fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 --2- 172.21.15.70:0/1772634372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 0x7f43201031d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 -- 172.21.15.70:0/1772634372 >> 172.21.15.70:0/1772634372 conn(0x7f43200f89b0 msgr2=0x7f43200fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 -- 172.21.15.70:0/1772634372 shutdown_connections 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 -- 172.21.15.70:0/1772634372 wait complete. 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 Processor -- start 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 -- start start 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 0x7f4320113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43201037a0 0x7f4320114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320114650 con 0x7f4320102e00 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f4325a5b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432010da40 con 0x7f43201037a0 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.265+0000 7f431effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 0x7f4320113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f431effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 0x7f4320113ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50508/0 (socket says 172.21.15.70:50508) 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f431effd700 1 -- 172.21.15.70:0/548506961 learned_addr learned my addr 172.21.15.70:0/548506961 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f4317fff700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43201037a0 0x7f4320114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f431effd700 1 -- 172.21.15.70:0/548506961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43201037a0 msgr2=0x7f4320114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f431effd700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43201037a0 0x7f4320114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f431effd700 1 -- 172.21.15.70:0/548506961 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310005040 con 0x7f4320102e00 2024-03-23T08:20:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f4317fff700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43201037a0 0x7f4320114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:20:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f431effd700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 0x7f4320113ac0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f431000b3b0 tx=0x7f4310009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f43177fe700 1 -- 172.21.15.70:0/548506961 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f431000e040 con 0x7f4320102e00 2024-03-23T08:20:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f43177fe700 1 -- 172.21.15.70:0/548506961 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4310013420 con 0x7f4320102e00 2024-03-23T08:20:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f43177fe700 1 -- 172.21.15.70:0/548506961 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f431001c630 con 0x7f4320102e00 2024-03-23T08:20:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432010dcc0 con 0x7f4320102e00 2024-03-23T08:20:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.266+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432010e280 con 0x7f4320102e00 2024-03-23T08:20:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.267+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43200fe550 con 0x7f4320102e00 2024-03-23T08:20:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.267+0000 7f43177fe700 1 -- 172.21.15.70:0/548506961 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4310013970 con 0x7f4320102e00 2024-03-23T08:20:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.268+0000 7f43177fe700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f430c0692a0 0x7f430c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.268+0000 7f43177fe700 1 -- 172.21.15.70:0/548506961 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4310087fe0 con 0x7f4320102e00 2024-03-23T08:20:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.268+0000 7f4317fff700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f430c0692a0 0x7f430c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:15.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.268+0000 7f4317fff700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f430c0692a0 0x7f430c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4308009fd0 tx=0x7f430800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:15.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.272+0000 7f43177fe700 1 -- 172.21.15.70:0/548506961 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4310058160 con 0x7f4320102e00 2024-03-23T08:20:15.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.495+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f432004bed0 con 0x7f4320102e00 2024-03-23T08:20:15.496 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.495+0000 7f43177fe700 1 -- 172.21.15.70:0/548506961 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4310057cf0 con 0x7f4320102e00 2024-03-23T08:20:15.496 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:15.496 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T08:20:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.497+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f430c0692a0 msgr2=0x7f430c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.497+0000 7f4325a5b700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f430c0692a0 0x7f430c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4308009fd0 tx=0x7f430800b040 comp rx=0 tx=0).stop 2024-03-23T08:20:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.497+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 msgr2=0x7f4320113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.497+0000 7f4325a5b700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 0x7f4320113ac0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f431000b3b0 tx=0x7f4310009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.497+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 shutdown_connections 2024-03-23T08:20:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.497+0000 7f4325a5b700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f430c0692a0 0x7f430c06b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.497+0000 7f4325a5b700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4320102e00 0x7f4320113ac0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.497+0000 7f4325a5b700 1 --2- 172.21.15.70:0/548506961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43201037a0 0x7f4320114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.497+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 >> 172.21.15.70:0/548506961 conn(0x7f43200f89b0 msgr2=0x7f43200f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.498+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 shutdown_connections 2024-03-23T08:20:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:15.498+0000 7f4325a5b700 1 -- 172.21.15.70:0/548506961 wait complete. 2024-03-23T08:20:15.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:15 smithi070 ceph-mon[39699]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:15.909 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:15 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/548506961' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T08:20:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:15 smithi193 ceph-mon[48370]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:15 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/548506961' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-23T08:20:16.208 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-23T08:20:16.209 INFO:tasks.cephadm:Setup complete, yielding 2024-03-23T08:20:16.209 INFO:teuthology.run_tasks:Running task print... 2024-03-23T08:20:16.219 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-03-23T08:20:16.219 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T08:20:16.227 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:20:16.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-23T08:20:16.493 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:18.038 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:17 smithi070 ceph-mon[39699]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:17 smithi193 ceph-mon[48370]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.270+0000 7f853c37e700 1 -- 172.21.15.70:0/3775508657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 msgr2=0x7f85340fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.270+0000 7f853c37e700 1 --2- 172.21.15.70:0/3775508657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 0x7f85340fe4b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8524004940 tx=0x7f8524009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.270+0000 7f853c37e700 1 -- 172.21.15.70:0/3775508657 shutdown_connections 2024-03-23T08:20:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.270+0000 7f853c37e700 1 --2- 172.21.15.70:0/3775508657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 0x7f85340fe4b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.270+0000 7f853c37e700 1 --2- 172.21.15.70:0/3775508657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85340fce40 0x7f85340fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.270+0000 7f853c37e700 1 -- 172.21.15.70:0/3775508657 >> 172.21.15.70:0/3775508657 conn(0x7f85340f8570 msgr2=0x7f85340fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.270+0000 7f853c37e700 1 -- 172.21.15.70:0/3775508657 shutdown_connections 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f853c37e700 1 -- 172.21.15.70:0/3775508657 wait complete. 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f853c37e700 1 Processor -- start 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f853c37e700 1 -- start start 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f853c37e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85340fce40 0x7f8534071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f853c37e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 0x7f8534070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f853c37e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85340708c0 con 0x7f85340fe040 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f853c37e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8534070a30 con 0x7f85340fce40 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f8539919700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 0x7f8534070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f8539919700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 0x7f8534070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50526/0 (socket says 172.21.15.70:50526) 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f8539919700 1 -- 172.21.15.70:0/285480117 learned_addr learned my addr 172.21.15.70:0/285480117 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f853a11a700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85340fce40 0x7f8534071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f8539919700 1 -- 172.21.15.70:0/285480117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85340fce40 msgr2=0x7f8534071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.271+0000 7f8539919700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85340fce40 0x7f8534071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.272+0000 7f8539919700 1 -- 172.21.15.70:0/285480117 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8524005040 con 0x7f85340fe040 2024-03-23T08:20:18.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.272+0000 7f8539919700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 0x7f8534070310 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f852400b3b0 tx=0x7f8524009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.272+0000 7f852affd700 1 -- 172.21.15.70:0/285480117 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f852400e040 con 0x7f85340fe040 2024-03-23T08:20:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.272+0000 7f852affd700 1 -- 172.21.15.70:0/285480117 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f852400bd20 con 0x7f85340fe040 2024-03-23T08:20:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.272+0000 7f852affd700 1 -- 172.21.15.70:0/285480117 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f852401c630 con 0x7f85340fe040 2024-03-23T08:20:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.272+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8534070cb0 con 0x7f85340fe040 2024-03-23T08:20:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.272+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8534115990 con 0x7f85340fe040 2024-03-23T08:20:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.273+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853405f5f0 con 0x7f85340fe040 2024-03-23T08:20:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.273+0000 7f852affd700 1 -- 172.21.15.70:0/285480117 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8524013420 con 0x7f85340fe040 2024-03-23T08:20:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.273+0000 7f852affd700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f85200692a0 0x7f852006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.274+0000 7f852affd700 1 -- 172.21.15.70:0/285480117 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8524087b10 con 0x7f85340fe040 2024-03-23T08:20:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.274+0000 7f853a11a700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f85200692a0 0x7f852006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:18.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.274+0000 7f853a11a700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f85200692a0 0x7f852006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8530007fd0 tx=0x7f8530009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:18.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.278+0000 7f852affd700 1 -- 172.21.15.70:0/285480117 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8524054180 con 0x7f85340fe040 2024-03-23T08:20:18.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.482+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f8534115b80 con 0x7f85340fe040 2024-03-23T08:20:18.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.487+0000 7f852affd700 1 -- 172.21.15.70:0/285480117 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f85240577a0 con 0x7f85340fe040 2024-03-23T08:20:18.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f85200692a0 msgr2=0x7f852006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:18.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f85200692a0 0x7f852006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8530007fd0 tx=0x7f8530009040 comp rx=0 tx=0).stop 2024-03-23T08:20:18.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 msgr2=0x7f8534070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:18.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 0x7f8534070310 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f852400b3b0 tx=0x7f8524009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:18.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 shutdown_connections 2024-03-23T08:20:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85340fce40 0x7f8534071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f85200692a0 0x7f852006b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 --2- 172.21.15.70:0/285480117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85340fe040 0x7f8534070310 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 >> 172.21.15.70:0/285480117 conn(0x7f85340f8570 msgr2=0x7f85340f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 shutdown_connections 2024-03-23T08:20:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:18.489+0000 7f853c37e700 1 -- 172.21.15.70:0/285480117 wait complete. 2024-03-23T08:20:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:18 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:20:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:18 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/285480117' entity='client.admin' 2024-03-23T08:20:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:18 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:20:19.223 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:18 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:20:19.223 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:18 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/285480117' entity='client.admin' 2024-03-23T08:20:19.224 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:18 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:20:19.242 INFO:teuthology.run_tasks:Running task print... 2024-03-23T08:20:19.252 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-23T08:20:19.252 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T08:20:19.260 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:20:19.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-23T08:20:19.526 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:19 smithi193 ceph-mon[48370]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:20:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:19 smithi070 ceph-mon[39699]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:20:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:21.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 -- 172.21.15.70:0/3566090651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fcf80 msgr2=0x7fe4cc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:21.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 --2- 172.21.15.70:0/3566090651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fcf80 0x7fe4cc0fd3f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc004940 tx=0x7fe4bc009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 -- 172.21.15.70:0/3566090651 shutdown_connections 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 --2- 172.21.15.70:0/3566090651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fcf80 0x7fe4cc0fd3f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 --2- 172.21.15.70:0/3566090651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc0fbcd0 0x7fe4cc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 -- 172.21.15.70:0/3566090651 >> 172.21.15.70:0/3566090651 conn(0x7fe4cc0f7430 msgr2=0x7fe4cc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 -- 172.21.15.70:0/3566090651 shutdown_connections 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 -- 172.21.15.70:0/3566090651 wait complete. 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 Processor -- start 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 -- start start 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fbcd0 0x7fe4cc10d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc0fcf80 0x7fe4cc10dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc10e570 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.190+0000 7fe4d4542700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc10ed10 con 0x7fe4cc0fcf80 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d22de700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fbcd0 0x7fe4cc10d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d22de700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fbcd0 0x7fe4cc10d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35622/0 (socket says 172.21.15.70:35622) 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d22de700 1 -- 172.21.15.70:0/2702150607 learned_addr learned my addr 172.21.15.70:0/2702150607 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d1add700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc0fcf80 0x7fe4cc10dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d22de700 1 -- 172.21.15.70:0/2702150607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc0fcf80 msgr2=0x7fe4cc10dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d22de700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc0fcf80 0x7fe4cc10dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d22de700 1 -- 172.21.15.70:0/2702150607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc005040 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d22de700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fbcd0 0x7fe4cc10d9a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800b790 tx=0x7fe4c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4c2ffd700 1 -- 172.21.15.70:0/2702150607 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c800f6f0 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4c2ffd700 1 -- 172.21.15.70:0/2702150607 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe4c800fd00 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4c2ffd700 1 -- 172.21.15.70:0/2702150607 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c80103a0 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4cc10eff0 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.191+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc10f5b0 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.192+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc101b10 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.192+0000 7fe4c2ffd700 1 -- 172.21.15.70:0/2702150607 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe4c800f850 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.193+0000 7fe4c2ffd700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe4b80692a0 0x7fe4b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:21.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.193+0000 7fe4c2ffd700 1 -- 172.21.15.70:0/2702150607 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe4c8086b40 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.193+0000 7fe4d1add700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe4b80692a0 0x7fe4b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:21.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.193+0000 7fe4d1add700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe4b80692a0 0x7fe4b806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00b3b0 tx=0x7fe4bc009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:21.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.197+0000 7fe4c2ffd700 1 -- 172.21.15.70:0/2702150607 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe4c8056ce0 con 0x7fe4cc0fbcd0 2024-03-23T08:20:21.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.397+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4cc104870 con 0x7fe4b80692a0 2024-03-23T08:20:21.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.398+0000 7fe4c2ffd700 1 -- 172.21.15.70:0/2702150607 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe4cc104870 con 0x7fe4b80692a0 2024-03-23T08:20:21.399 INFO:teuthology.orchestra.run.smithi070.stdout:Backend: cephadm 2024-03-23T08:20:21.399 INFO:teuthology.orchestra.run.smithi070.stdout:Available: Yes 2024-03-23T08:20:21.399 INFO:teuthology.orchestra.run.smithi070.stdout:Paused: No 2024-03-23T08:20:21.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.400+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe4b80692a0 msgr2=0x7fe4b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.400+0000 7fe4d4542700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe4b80692a0 0x7fe4b806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00b3b0 tx=0x7fe4bc009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.400+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fbcd0 msgr2=0x7fe4cc10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.400+0000 7fe4d4542700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fbcd0 0x7fe4cc10d9a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800b790 tx=0x7fe4c8009250 comp rx=0 tx=0).stop 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.400+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 shutdown_connections 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.400+0000 7fe4d4542700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe4b80692a0 0x7fe4b806b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.400+0000 7fe4d4542700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4cc0fbcd0 0x7fe4cc10d9a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.400+0000 7fe4d4542700 1 --2- 172.21.15.70:0/2702150607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc0fcf80 0x7fe4cc10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.400+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 >> 172.21.15.70:0/2702150607 conn(0x7fe4cc0f7430 msgr2=0x7fe4cc065cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.401+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 shutdown_connections 2024-03-23T08:20:21.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:21.401+0000 7fe4d4542700 1 -- 172.21.15.70:0/2702150607 wait complete. 2024-03-23T08:20:22.060 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:21 smithi070 ceph-mon[39699]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:21 smithi193 ceph-mon[48370]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:22.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-23T08:20:22.345 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:22.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:22 smithi070 ceph-mon[39699]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:22 smithi193 ceph-mon[48370]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:23.883 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:23 smithi070 ceph-mon[39699]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:23 smithi193 ceph-mon[48370]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:24.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.195+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/3149062070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 msgr2=0x7f8fe00fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:24.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.195+0000 7f8fe7e4a700 1 --2- 172.21.15.70:0/3149062070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 0x7f8fe00fc6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc004940 tx=0x7f8fdc009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:24.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.195+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/3149062070 shutdown_connections 2024-03-23T08:20:24.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.195+0000 7f8fe7e4a700 1 --2- 172.21.15.70:0/3149062070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe00fcc00 0x7f8fe00fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:24.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.195+0000 7f8fe7e4a700 1 --2- 172.21.15.70:0/3149062070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 0x7f8fe00fc6c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:24.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.195+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/3149062070 >> 172.21.15.70:0/3149062070 conn(0x7f8fe00f6ba0 msgr2=0x7f8fe00f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:24.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.195+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/3149062070 shutdown_connections 2024-03-23T08:20:24.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.195+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/3149062070 wait complete. 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe7e4a700 1 Processor -- start 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe7e4a700 1 -- start start 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe7e4a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 0x7f8fe010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe7e4a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe00fcc00 0x7f8fe01100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe7e4a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe01106e0 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe7e4a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe0110f30 con 0x7f8fe00fcc00 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe5be6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 0x7f8fe010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe5be6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 0x7f8fe010fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35646/0 (socket says 172.21.15.70:35646) 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe5be6700 1 -- 172.21.15.70:0/453568197 learned_addr learned my addr 172.21.15.70:0/453568197 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:24.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe53e5700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe00fcc00 0x7f8fe01100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:24.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe5be6700 1 -- 172.21.15.70:0/453568197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe00fcc00 msgr2=0x7f8fe01100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:24.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe5be6700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe00fcc00 0x7f8fe01100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:24.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe5be6700 1 -- 172.21.15.70:0/453568197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc005040 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe5be6700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 0x7f8fe010fba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00b3b0 tx=0x7f8fdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:24.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fd67fc700 1 -- 172.21.15.70:0/453568197 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fdc00e040 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe01111b0 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fd67fc700 1 -- 172.21.15.70:0/453568197 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8fdc00bd20 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.196+0000 7f8fd67fc700 1 -- 172.21.15.70:0/453568197 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fdc01c630 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.197+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe01116f0 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.198+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe005f5f0 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.198+0000 7f8fd67fc700 1 -- 172.21.15.70:0/453568197 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8fdc013420 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.198+0000 7f8fd67fc700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8fcc0692a0 0x7f8fcc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:24.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.198+0000 7f8fd67fc700 1 -- 172.21.15.70:0/453568197 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8fdc087c00 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.198+0000 7f8fe53e5700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8fcc0692a0 0x7f8fcc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:24.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.199+0000 7f8fe53e5700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8fcc0692a0 0x7f8fcc06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0007fd0 tx=0x7f8fd0009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:24.202 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.202+0000 7f8fd67fc700 1 -- 172.21.15.70:0/453568197 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8fdc08d020 con 0x7f8fe00fc2b0 2024-03-23T08:20:24.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.401+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fe0111a70 con 0x7f8fcc0692a0 2024-03-23T08:20:24.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.409+0000 7f8fd67fc700 1 -- 172.21.15.70:0/453568197 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2700 (secure 0 0 0) 0x7f8fe0111a70 con 0x7f8fcc0692a0 2024-03-23T08:20:24.409 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:20:24.409 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (3m) 2m ago 5m 26.3M - 0.25.0 c8568f914cd2 700168bef09e 2024-03-23T08:20:24.409 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (5m) 2m ago 5m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:20:24.409 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (4m) 68s ago 4m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:20:24.409 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (3m) 2m ago 4m 78.4M - 9.4.7 954c08fa6188 87fc95a6ce8f 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:9283 running (6m) 2m ago 6m 470M - 17.2.7-726-g9c4e0c34 837124da804f 85d0a92e59bc 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283 running (4m) 68s ago 4m 434M - 17.2.7-726-g9c4e0c34 837124da804f a023705720c4 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (6m) 2m ago 6m 43.7M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (4m) 68s ago 4m 38.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (5m) 2m ago 5m 18.1M - 1.5.0 0da6a335fe13 a14314a29bca 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (4m) 68s ago 4m 18.7M - 1.5.0 0da6a335fe13 4197d838b05a 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (3m) 2m ago 3m 35.8M 4299M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (2m) 2m ago 2m 36.2M 4299M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (2m) 2m ago 2m 38.0M 4299M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (2m) 68s ago 2m 41.5M 5323M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (100s) 68s ago 99s 35.8M 5323M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (73s) 68s ago 73s 13.5M 5323M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:20:24.410 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (3m) 2m ago 4m 35.9M - 2.43.0 a07b618ecd1d e842f03864f0 2024-03-23T08:20:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8fcc0692a0 msgr2=0x7f8fcc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8fcc0692a0 0x7f8fcc06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0007fd0 tx=0x7f8fd0009040 comp rx=0 tx=0).stop 2024-03-23T08:20:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 msgr2=0x7f8fe010fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 0x7f8fe010fba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00b3b0 tx=0x7f8fdc009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 shutdown_connections 2024-03-23T08:20:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f8fcc0692a0 0x7f8fcc06b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe00fc2b0 0x7f8fe010fba0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 --2- 172.21.15.70:0/453568197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fe00fcc00 0x7f8fe01100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 >> 172.21.15.70:0/453568197 conn(0x7f8fe00f6ba0 msgr2=0x7f8fe00f83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 shutdown_connections 2024-03-23T08:20:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:24.411+0000 7f8fe7e4a700 1 -- 172.21.15.70:0/453568197 wait complete. 2024-03-23T08:20:25.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-23T08:20:25.334 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:25.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:25 smithi070 ceph-mon[39699]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:25.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:25 smithi070 ceph-mon[39699]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:25 smithi193 ceph-mon[48370]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:25 smithi193 ceph-mon[48370]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:27.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.246+0000 7fa16861a700 1 -- 172.21.15.70:0/51661637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1600fd2d0 msgr2=0x7fa1600fd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:27.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.246+0000 7fa16861a700 1 --2- 172.21.15.70:0/51661637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1600fd2d0 0x7fa1600fd6e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa15c004940 tx=0x7fa15c009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:27.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.246+0000 7fa16861a700 1 -- 172.21.15.70:0/51661637 shutdown_connections 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.246+0000 7fa16861a700 1 --2- 172.21.15.70:0/51661637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1600fdc20 0x7fa1600fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.246+0000 7fa16861a700 1 --2- 172.21.15.70:0/51661637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1600fd2d0 0x7fa1600fd6e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.246+0000 7fa16861a700 1 -- 172.21.15.70:0/51661637 >> 172.21.15.70:0/51661637 conn(0x7fa1600f6bc0 msgr2=0x7fa1600f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.246+0000 7fa16861a700 1 -- 172.21.15.70:0/51661637 shutdown_connections 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.246+0000 7fa16861a700 1 -- 172.21.15.70:0/51661637 wait complete. 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa16861a700 1 Processor -- start 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa16861a700 1 -- start start 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa16861a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1600fd2d0 0x7fa16010c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa16861a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa16010cce0 0x7fa160111d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa16861a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16010d230 con 0x7fa16010cce0 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa16861a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16010d370 con 0x7fa1600fd2d0 2024-03-23T08:20:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa165bb5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa16010cce0 0x7fa160111d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa165bb5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa16010cce0 0x7fa160111d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35666/0 (socket says 172.21.15.70:35666) 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa165bb5700 1 -- 172.21.15.70:0/2249786896 learned_addr learned my addr 172.21.15.70:0/2249786896 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.247+0000 7fa1663b6700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1600fd2d0 0x7fa16010c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.248+0000 7fa165bb5700 1 -- 172.21.15.70:0/2249786896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1600fd2d0 msgr2=0x7fa16010c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.248+0000 7fa165bb5700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1600fd2d0 0x7fa16010c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.248+0000 7fa165bb5700 1 -- 172.21.15.70:0/2249786896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c005040 con 0x7fa16010cce0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.248+0000 7fa165bb5700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa16010cce0 0x7fa160111d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa15000b790 tx=0x7fa150009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.248+0000 7fa156ffd700 1 -- 172.21.15.70:0/2249786896 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa15000f6f0 con 0x7fa16010cce0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.248+0000 7fa156ffd700 1 -- 172.21.15.70:0/2249786896 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa15000fd00 con 0x7fa16010cce0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.248+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa160112310 con 0x7fa16010cce0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.248+0000 7fa156ffd700 1 -- 172.21.15.70:0/2249786896 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1500103a0 con 0x7fa16010cce0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.248+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1601128d0 con 0x7fa16010cce0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.249+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16004c710 con 0x7fa16010cce0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.249+0000 7fa156ffd700 1 -- 172.21.15.70:0/2249786896 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa15000f850 con 0x7fa16010cce0 2024-03-23T08:20:27.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.249+0000 7fa156ffd700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa14c0692a0 0x7fa14c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:27.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.250+0000 7fa156ffd700 1 -- 172.21.15.70:0/2249786896 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa150086c40 con 0x7fa16010cce0 2024-03-23T08:20:27.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.250+0000 7fa1663b6700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa14c0692a0 0x7fa14c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:27.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.250+0000 7fa1663b6700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa14c0692a0 0x7fa14c06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00b900 tx=0x7fa15c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:27.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.254+0000 7fa156ffd700 1 -- 172.21.15.70:0/2249786896 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa150056d40 con 0x7fa16010cce0 2024-03-23T08:20:27.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.444+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa160101370 con 0x7fa14c0692a0 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.449+0000 7fa156ffd700 1 -- 172.21.15.70:0/2249786896 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7fa160101370 con 0x7fa14c0692a0 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stdout:crash 2/2 2m ago 6m * 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stdout:mgr 2/2 2m ago 6m count:2 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stdout:mon 2/2 2m ago 5m smithi070:172.21.15.70=smithi070;smithi193:172.21.15.193=smithi193;count:2 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stdout:osd 6 2m ago - 2024-03-23T08:20:27.449 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa14c0692a0 msgr2=0x7fa14c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa14c0692a0 0x7fa14c06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00b900 tx=0x7fa15c009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa16010cce0 msgr2=0x7fa160111d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa16010cce0 0x7fa160111d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa15000b790 tx=0x7fa150009250 comp rx=0 tx=0).stop 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 shutdown_connections 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1600fd2d0 0x7fa16010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa14c0692a0 0x7fa14c06b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 --2- 172.21.15.70:0/2249786896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa16010cce0 0x7fa160111d10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 >> 172.21.15.70:0/2249786896 conn(0x7fa1600f6bc0 msgr2=0x7fa1600f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 shutdown_connections 2024-03-23T08:20:27.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:27.451+0000 7fa16861a700 1 -- 172.21.15.70:0/2249786896 wait complete. 2024-03-23T08:20:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:27 smithi193 ceph-mon[48370]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:27 smithi070 ceph-mon[39699]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:28.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-23T08:20:28.540 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:29.053 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:28 smithi070 ceph-mon[39699]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:29.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:28 smithi193 ceph-mon[48370]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:30.018 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:29 smithi070 ceph-mon[39699]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:29 smithi193 ceph-mon[48370]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.272+0000 7fd74081f700 1 -- 172.21.15.70:0/2151888517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 msgr2=0x7fd73c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.272+0000 7fd74081f700 1 --2- 172.21.15.70:0/2151888517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 0x7fd73c0fc0e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd730004940 tx=0x7fd730009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.272+0000 7fd74081f700 1 -- 172.21.15.70:0/2151888517 shutdown_connections 2024-03-23T08:20:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.272+0000 7fd74081f700 1 --2- 172.21.15.70:0/2151888517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73c0fcf80 0x7fd73c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.272+0000 7fd74081f700 1 --2- 172.21.15.70:0/2151888517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 0x7fd73c0fc0e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.272+0000 7fd74081f700 1 -- 172.21.15.70:0/2151888517 >> 172.21.15.70:0/2151888517 conn(0x7fd73c0f7430 msgr2=0x7fd73c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.272+0000 7fd74081f700 1 -- 172.21.15.70:0/2151888517 shutdown_connections 2024-03-23T08:20:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.272+0000 7fd74081f700 1 -- 172.21.15.70:0/2151888517 wait complete. 2024-03-23T08:20:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.272+0000 7fd74081f700 1 Processor -- start 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd74081f700 1 -- start start 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd74081f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 0x7fd73c066d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd74081f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73c0fcf80 0x7fd73c112610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd74081f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c112b50 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd74081f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c112cc0 con 0x7fd73c0fcf80 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd73a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 0x7fd73c066d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd73a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 0x7fd73c066d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42554/0 (socket says 172.21.15.70:42554) 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd73a59c700 1 -- 172.21.15.70:0/763484711 learned_addr learned my addr 172.21.15.70:0/763484711 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd73a59c700 1 -- 172.21.15.70:0/763484711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73c0fcf80 msgr2=0x7fd73c112610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd739d9b700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73c0fcf80 0x7fd73c112610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd73a59c700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73c0fcf80 0x7fd73c112610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd73a59c700 1 -- 172.21.15.70:0/763484711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd730005040 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd739d9b700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73c0fcf80 0x7fd73c112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:20:30.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd73a59c700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 0x7fd73c066d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b3b0 tx=0x7fd730009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:30.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd72affd700 1 -- 172.21.15.70:0/763484711 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd73000e040 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd72affd700 1 -- 172.21.15.70:0/763484711 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd73000bd20 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c112f40 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd72affd700 1 -- 172.21.15.70:0/763484711 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd73001c630 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.273+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c113470 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.274+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73c04c710 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.275+0000 7fd72affd700 1 -- 172.21.15.70:0/763484711 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd730013420 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.275+0000 7fd72affd700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd7240692a0 0x7fd72406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:30.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.275+0000 7fd72affd700 1 -- 172.21.15.70:0/763484711 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd730087b10 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.275+0000 7fd739d9b700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd7240692a0 0x7fd72406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:30.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.276+0000 7fd739d9b700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd7240692a0 0x7fd72406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd72c006fd0 tx=0x7fd72c008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:30.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.280+0000 7fd72affd700 1 -- 172.21.15.70:0/763484711 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd730057c10 con 0x7fd73c0fbcd0 2024-03-23T08:20:30.470 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.470+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd73c1136e0 con 0x7fd7240692a0 2024-03-23T08:20:30.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.471+0000 7fd72affd700 1 -- 172.21.15.70:0/763484711 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fd73c1136e0 con 0x7fd7240692a0 2024-03-23T08:20:30.471 INFO:teuthology.orchestra.run.smithi070.stdout:HOST ADDR LABELS STATUS 2024-03-23T08:20:30.471 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070 172.21.15.70 2024-03-23T08:20:30.471 INFO:teuthology.orchestra.run.smithi070.stdout:smithi193 172.21.15.193 2024-03-23T08:20:30.471 INFO:teuthology.orchestra.run.smithi070.stdout:2 hosts in cluster 2024-03-23T08:20:30.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.473+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd7240692a0 msgr2=0x7fd72406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.473+0000 7fd74081f700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd7240692a0 0x7fd72406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd72c006fd0 tx=0x7fd72c008040 comp rx=0 tx=0).stop 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.473+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 msgr2=0x7fd73c066d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.473+0000 7fd74081f700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 0x7fd73c066d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b3b0 tx=0x7fd730009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.473+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 shutdown_connections 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.473+0000 7fd74081f700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fd7240692a0 0x7fd72406b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.473+0000 7fd74081f700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd73c0fbcd0 0x7fd73c066d10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.473+0000 7fd74081f700 1 --2- 172.21.15.70:0/763484711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73c0fcf80 0x7fd73c112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.473+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 >> 172.21.15.70:0/763484711 conn(0x7fd73c0f7430 msgr2=0x7fd73c0f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.474+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 shutdown_connections 2024-03-23T08:20:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:30.474+0000 7fd74081f700 1 -- 172.21.15.70:0/763484711 wait complete. 2024-03-23T08:20:31.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-23T08:20:31.402 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:31.911 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:31 smithi070 ceph-mon[39699]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:31.912 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:31 smithi070 ceph-mon[39699]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:31 smithi193 ceph-mon[48370]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:31 smithi193 ceph-mon[48370]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.057+0000 7f10041a7700 1 -- 172.21.15.70:0/199234761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fe060 msgr2=0x7f0ffc0fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.057+0000 7f10041a7700 1 --2- 172.21.15.70:0/199234761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fe060 0x7f0ffc0fe4d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8004940 tx=0x7f0ff8009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.057+0000 7f10041a7700 1 -- 172.21.15.70:0/199234761 shutdown_connections 2024-03-23T08:20:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.057+0000 7f10041a7700 1 --2- 172.21.15.70:0/199234761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fe060 0x7f0ffc0fe4d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.057+0000 7f10041a7700 1 --2- 172.21.15.70:0/199234761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ffc0fcef0 0x7f0ffc0fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.057+0000 7f10041a7700 1 -- 172.21.15.70:0/199234761 >> 172.21.15.70:0/199234761 conn(0x7f0ffc0f85b0 msgr2=0x7f0ffc0fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.057+0000 7f10041a7700 1 -- 172.21.15.70:0/199234761 shutdown_connections 2024-03-23T08:20:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.057+0000 7f10041a7700 1 -- 172.21.15.70:0/199234761 wait complete. 2024-03-23T08:20:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f10041a7700 1 Processor -- start 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f10041a7700 1 -- start start 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f10041a7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fcef0 0x7f0ffc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f10041a7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ffc0fe060 0x7f0ffc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f10041a7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc10e540 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f10041a7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc10ed30 con 0x7f0ffc0fe060 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f1001f43700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fcef0 0x7f0ffc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f1001f43700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fcef0 0x7f0ffc10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42564/0 (socket says 172.21.15.70:42564) 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f1001f43700 1 -- 172.21.15.70:0/2098966715 learned_addr learned my addr 172.21.15.70:0/2098966715 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f1001742700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ffc0fe060 0x7f0ffc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f1001f43700 1 -- 172.21.15.70:0/2098966715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ffc0fe060 msgr2=0x7f0ffc10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f1001f43700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ffc0fe060 0x7f0ffc10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f1001f43700 1 -- 172.21.15.70:0/2098966715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8005040 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f1001742700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ffc0fe060 0x7f0ffc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.058+0000 7f1001f43700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fcef0 0x7f0ffc10d970 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00b790 tx=0x7f0fec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.059+0000 7f0ff27fc700 1 -- 172.21.15.70:0/2098966715 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fec00f770 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.059+0000 7f0ff27fc700 1 -- 172.21.15.70:0/2098966715 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0fec00fd80 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.059+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ffc10f010 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.059+0000 7f0ff27fc700 1 -- 172.21.15.70:0/2098966715 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fec00d310 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.059+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ffc10f5d0 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.059+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ffc05f5f0 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.060+0000 7f0ff27fc700 1 -- 172.21.15.70:0/2098966715 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0fec010420 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.060+0000 7f0ff27fc700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0fe80692b0 0x7f0fe806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.061+0000 7f1001742700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0fe80692b0 0x7f0fe806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.061+0000 7f0ff27fc700 1 -- 172.21.15.70:0/2098966715 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0fec086e90 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.061+0000 7f1001742700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0fe80692b0 0x7f0fe806b760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8004f90 tx=0x7f0ff8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.065+0000 7f0ff27fc700 1 -- 172.21.15.70:0/2098966715 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0fec056ef0 con 0x7f0ffc0fcef0 2024-03-23T08:20:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.265+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ffc10f8f0 con 0x7f0fe80692b0 2024-03-23T08:20:33.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.267+0000 7f0ff27fc700 1 -- 172.21.15.70:0/2098966715 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f0ffc10f8f0 con 0x7f0fe80692b0 2024-03-23T08:20:33.267 INFO:teuthology.orchestra.run.smithi070.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-23T08:20:33.267 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002P480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-23T08:20:33.267 INFO:teuthology.orchestra.run.smithi070.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 73s ago Has a FileSystem, LVM detected 2024-03-23T08:20:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0fe80692b0 msgr2=0x7f0fe806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0fe80692b0 0x7f0fe806b760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8004f90 tx=0x7f0ff8009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fcef0 msgr2=0x7f0ffc10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fcef0 0x7f0ffc10d970 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00b790 tx=0x7f0fec009250 comp rx=0 tx=0).stop 2024-03-23T08:20:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 shutdown_connections 2024-03-23T08:20:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f0fe80692b0 0x7f0fe806b760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc0fcef0 0x7f0ffc10d970 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 --2- 172.21.15.70:0/2098966715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ffc0fe060 0x7f0ffc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 >> 172.21.15.70:0/2098966715 conn(0x7f0ffc0f85b0 msgr2=0x7f0ffc0f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 shutdown_connections 2024-03-23T08:20:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:33.269+0000 7f10041a7700 1 -- 172.21.15.70:0/2098966715 wait complete. 2024-03-23T08:20:33.970 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:33 smithi070 ceph-mon[39699]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:33.971 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:20:33.990 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T08:20:34.000 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:20:34.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-23T08:20:34.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:33 smithi193 ceph-mon[48370]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:34.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:20:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:35.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:34 smithi193 ceph-mon[48370]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:35.214 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:34 smithi070 ceph-mon[39699]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 -- 172.21.15.70:0/3555155382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 msgr2=0x7f92480fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 --2- 172.21.15.70:0/3555155382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 0x7f92480fe4e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9238004940 tx=0x7f9238009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 -- 172.21.15.70:0/3555155382 shutdown_connections 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 --2- 172.21.15.70:0/3555155382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 0x7f92480fe4e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 --2- 172.21.15.70:0/3555155382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92480fcf00 0x7f92480fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 -- 172.21.15.70:0/3555155382 >> 172.21.15.70:0/3555155382 conn(0x7f92480f85a0 msgr2=0x7f92480fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 -- 172.21.15.70:0/3555155382 shutdown_connections 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 -- 172.21.15.70:0/3555155382 wait complete. 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 Processor -- start 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.066+0000 7f924f7dd700 1 -- start start 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924f7dd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92480fcf00 0x7f924810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924f7dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 0x7f924810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924f7dd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924810e550 con 0x7f92480fe070 2024-03-23T08:20:36.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924f7dd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924810ed40 con 0x7f92480fcf00 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924cd78700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 0x7f924810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924cd78700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 0x7f924810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42580/0 (socket says 172.21.15.70:42580) 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924cd78700 1 -- 172.21.15.70:0/1197990701 learned_addr learned my addr 172.21.15.70:0/1197990701 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924d579700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92480fcf00 0x7f924810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924cd78700 1 -- 172.21.15.70:0/1197990701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92480fcf00 msgr2=0x7f924810d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924cd78700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92480fcf00 0x7f924810d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924cd78700 1 -- 172.21.15.70:0/1197990701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9238005040 con 0x7f92480fe070 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924cd78700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 0x7f924810dec0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f923800b3b0 tx=0x7f9238009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f923dffb700 1 -- 172.21.15.70:0/1197990701 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923800e040 con 0x7f92480fe070 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f923dffb700 1 -- 172.21.15.70:0/1197990701 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f923800bd20 con 0x7f92480fe070 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924810efc0 con 0x7f92480fe070 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f923dffb700 1 -- 172.21.15.70:0/1197990701 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923801c630 con 0x7f92480fe070 2024-03-23T08:20:36.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.067+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924810f580 con 0x7f92480fe070 2024-03-23T08:20:36.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.068+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924805f5f0 con 0x7f92480fe070 2024-03-23T08:20:36.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.069+0000 7f923dffb700 1 -- 172.21.15.70:0/1197990701 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9238013420 con 0x7f92480fe070 2024-03-23T08:20:36.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.069+0000 7f923dffb700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f92340692a0 0x7f923406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:36.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.069+0000 7f923dffb700 1 -- 172.21.15.70:0/1197990701 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9238087cd0 con 0x7f92480fe070 2024-03-23T08:20:36.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.069+0000 7f924d579700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f92340692a0 0x7f923406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:36.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.070+0000 7f924d579700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f92340692a0 0x7f923406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9244006fd0 tx=0x7f9244008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:36.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.074+0000 7f923dffb700 1 -- 172.21.15.70:0/1197990701 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9238057cb0 con 0x7f92480fe070 2024-03-23T08:20:36.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:35 smithi193 ceph-mon[48370]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:36.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:35 smithi070 ceph-mon[39699]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:36.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:36.287+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f924810f810 con 0x7f92340692a0 2024-03-23T08:20:37.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:36 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-23T08:20:37.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:36 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-23T08:20:37.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.841+0000 7f923dffb700 1 -- 172.21.15.70:0/1197990701 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f924810f810 con 0x7f92340692a0 2024-03-23T08:20:37.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.843+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f92340692a0 msgr2=0x7f923406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.843+0000 7f924f7dd700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f92340692a0 0x7f923406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9244006fd0 tx=0x7f9244008040 comp rx=0 tx=0).stop 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.843+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 msgr2=0x7f924810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.843+0000 7f924f7dd700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 0x7f924810dec0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f923800b3b0 tx=0x7f9238009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.843+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 shutdown_connections 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.843+0000 7f924f7dd700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92480fcf00 0x7f924810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.843+0000 7f924f7dd700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f92340692a0 0x7f923406b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.843+0000 7f924f7dd700 1 --2- 172.21.15.70:0/1197990701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92480fe070 0x7f924810dec0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.843+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 >> 172.21.15.70:0/1197990701 conn(0x7f92480f85a0 msgr2=0x7f92480f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.844+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 shutdown_connections 2024-03-23T08:20:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:37.844+0000 7f924f7dd700 1 -- 172.21.15.70:0/1197990701 wait complete. 2024-03-23T08:20:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:37 smithi193 ceph-mon[48370]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:37 smithi193 ceph-mon[48370]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-23T08:20:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:37 smithi193 ceph-mon[48370]: osdmap e43: 6 total, 6 up, 6 in 2024-03-23T08:20:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:37 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-23T08:20:38.221 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:37 smithi070 ceph-mon[39699]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:20:38.221 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:37 smithi070 ceph-mon[39699]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:38.221 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-23T08:20:38.221 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:37 smithi070 ceph-mon[39699]: osdmap e43: 6 total, 6 up, 6 in 2024-03-23T08:20:38.221 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:37 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-23T08:20:38.221 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:37 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070[39676]: 2024-03-23T08:20:37.818+0000 7f5ad5a6c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T08:20:38.619 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-23T08:20:38.931 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-23T08:20:38.931 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: osdmap e44: 6 total, 6 up, 6 in 2024-03-23T08:20:38.931 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: osdmap e45: 6 total, 6 up, 6 in 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: fsmap cephfs:0 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: Saving service mds.cephfs spec with placement count:4 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.iziydh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.iziydh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:38.932 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:38 smithi070 ceph-mon[39699]: Deploying daemon mds.cephfs.smithi070.iziydh on smithi070 2024-03-23T08:20:39.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-23T08:20:39.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: osdmap e44: 6 total, 6 up, 6 in 2024-03-23T08:20:39.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-23T08:20:39.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T08:20:39.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: osdmap e45: 6 total, 6 up, 6 in 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: fsmap cephfs:0 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: Saving service mds.cephfs spec with placement count:4 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.iziydh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.iziydh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:39.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:38 smithi193 ceph-mon[48370]: Deploying daemon mds.cephfs.smithi070.iziydh on smithi070 2024-03-23T08:20:39.843 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:40.111 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:39 smithi070 ceph-mon[39699]: pgmap v178: 65 pgs: 5 creating+activating, 8 active+clean, 52 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:40.111 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:39 smithi070 ceph-mon[39699]: osdmap e46: 6 total, 6 up, 6 in 2024-03-23T08:20:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:39 smithi193 ceph-mon[48370]: pgmap v178: 65 pgs: 5 creating+activating, 8 active+clean, 52 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:39 smithi193 ceph-mon[48370]: osdmap e46: 6 total, 6 up, 6 in 2024-03-23T08:20:41.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:40 smithi070 ceph-mon[39699]: osdmap e47: 6 total, 6 up, 6 in 2024-03-23T08:20:41.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:40 smithi193 ceph-mon[48370]: osdmap e47: 6 total, 6 up, 6 in 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 -- 172.21.15.70:0/2561763180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fce40 msgr2=0x7ff5200fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 --2- 172.21.15.70:0/2561763180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fce40 0x7ff5200fd250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff514004a90 tx=0x7ff514009dd0 comp rx=0 tx=0).stop 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 -- 172.21.15.70:0/2561763180 shutdown_connections 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 --2- 172.21.15.70:0/2561763180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5200fe040 0x7ff5200fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 --2- 172.21.15.70:0/2561763180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fce40 0x7ff5200fd250 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 -- 172.21.15.70:0/2561763180 >> 172.21.15.70:0/2561763180 conn(0x7ff5200f8570 msgr2=0x7ff5200fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 -- 172.21.15.70:0/2561763180 shutdown_connections 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 -- 172.21.15.70:0/2561763180 wait complete. 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 Processor -- start 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.965+0000 7ff525f3a700 1 -- start start 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff525f3a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5200fce40 0x7ff520071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff525f3a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fe040 0x7ff520070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff525f3a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200708c0 con 0x7ff5200fe040 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff525f3a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520070a30 con 0x7ff5200fce40 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff51effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fe040 0x7ff520070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff51effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fe040 0x7ff520070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43094/0 (socket says 172.21.15.70:43094) 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff51effd700 1 -- 172.21.15.70:0/1598709105 learned_addr learned my addr 172.21.15.70:0/1598709105 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff51f7fe700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5200fce40 0x7ff520071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff51effd700 1 -- 172.21.15.70:0/1598709105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5200fce40 msgr2=0x7ff520071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff51effd700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5200fce40 0x7ff520071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff51effd700 1 -- 172.21.15.70:0/1598709105 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514004650 con 0x7ff5200fe040 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff51effd700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fe040 0x7ff520070310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff510002a50 tx=0x7ff5100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff507fff700 1 -- 172.21.15.70:0/1598709105 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff510007cc0 con 0x7ff5200fe040 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff520070d10 con 0x7ff5200fe040 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff507fff700 1 -- 172.21.15.70:0/1598709105 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff51000f040 con 0x7ff5200fe040 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff507fff700 1 -- 172.21.15.70:0/1598709105 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff51000d9c0 con 0x7ff5200fe040 2024-03-23T08:20:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.966+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5201158e0 con 0x7ff5200fe040 2024-03-23T08:20:41.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.967+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52005f5f0 con 0x7ff5200fe040 2024-03-23T08:20:41.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.968+0000 7ff507fff700 1 -- 172.21.15.70:0/1598709105 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff510009070 con 0x7ff5200fe040 2024-03-23T08:20:41.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.968+0000 7ff507fff700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff5080692a0 0x7ff50806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:41.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.968+0000 7ff507fff700 1 -- 172.21.15.70:0/1598709105 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff510086f10 con 0x7ff5200fe040 2024-03-23T08:20:41.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.968+0000 7ff51f7fe700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff5080692a0 0x7ff50806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:41.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.969+0000 7ff51f7fe700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff5080692a0 0x7ff50806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff514009fd0 tx=0x7ff514016040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:41.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:41.972+0000 7ff507fff700 1 -- 172.21.15.70:0/1598709105 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff5100531c0 con 0x7ff5200fe040 2024-03-23T08:20:41.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:41 smithi070 ceph-mon[39699]: pgmap v181: 65 pgs: 8 creating+peering, 5 creating+activating, 37 active+clean, 15 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:41 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:41 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:41 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:41 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sdlbdd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:20:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:41 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sdlbdd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T08:20:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:41 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:42.110 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:41 smithi193 ceph-mon[48370]: pgmap v181: 65 pgs: 8 creating+peering, 5 creating+activating, 37 active+clean, 15 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:42.110 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:41 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:42.110 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:41 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:42.110 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:41 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:42.111 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:41 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sdlbdd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:20:42.111 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:41 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sdlbdd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T08:20:42.111 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:41 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:42.202 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.203+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff52004bed0 con 0x7ff5200fe040 2024-03-23T08:20:42.202 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.203+0000 7ff507fff700 1 -- 172.21.15.70:0/1598709105 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1247 (secure 0 0 0) 0x7ff5100567e0 con 0x7ff5200fe040 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:e4 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 4 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:41.840348+0000 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:20:42.203 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14450} 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:20:42.204 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:20:42.205 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 0 2024-03-23T08:20:42.205 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{0:14450} state up:creating seq 1 addr [v2:172.21.15.70:6826/1612292110,v1:172.21.15.70:6827/1612292110] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:20:42.205 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:42.205 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:42.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.205+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff5080692a0 msgr2=0x7ff50806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:42.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.205+0000 7ff525f3a700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff5080692a0 0x7ff50806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff514009fd0 tx=0x7ff514016040 comp rx=0 tx=0).stop 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.205+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fe040 msgr2=0x7ff520070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.205+0000 7ff525f3a700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fe040 0x7ff520070310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff510002a50 tx=0x7ff5100046b0 comp rx=0 tx=0).stop 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.206+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 shutdown_connections 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.206+0000 7ff525f3a700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5200fce40 0x7ff520071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.206+0000 7ff525f3a700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff5080692a0 0x7ff50806b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.206+0000 7ff525f3a700 1 --2- 172.21.15.70:0/1598709105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200fe040 0x7ff520070310 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.206+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 >> 172.21.15.70:0/1598709105 conn(0x7ff5200f8570 msgr2=0x7ff5200f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.206+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 shutdown_connections 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:42.206+0000 7ff525f3a700 1 -- 172.21.15.70:0/1598709105 wait complete. 2024-03-23T08:20:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 4 2024-03-23T08:20:42.848 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T08:20:42.857 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:20:42.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-03-23T08:20:42.969 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: Deploying daemon mds.cephfs.smithi193.sdlbdd on smithi193 2024-03-23T08:20:42.969 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.70:6826/1612292110,v1:172.21.15.70:6827/1612292110] up:boot 2024-03-23T08:20:42.969 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: daemon mds.cephfs.smithi070.iziydh assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-23T08:20:42.969 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T08:20:42.969 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T08:20:42.969 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: Cluster is now healthy 2024-03-23T08:20:42.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: fsmap cephfs:0 1 up:standby 2024-03-23T08:20:42.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:20:42.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:creating} 2024-03-23T08:20:42.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: daemon mds.cephfs.smithi070.iziydh is now active in filesystem cephfs as rank 0 2024-03-23T08:20:42.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:42 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1598709105' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:20:43.128 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: Deploying daemon mds.cephfs.smithi193.sdlbdd on smithi193 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.70:6826/1612292110,v1:172.21.15.70:6827/1612292110] up:boot 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: daemon mds.cephfs.smithi070.iziydh assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: Cluster is now healthy 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: fsmap cephfs:0 1 up:standby 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:creating} 2024-03-23T08:20:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: daemon mds.cephfs.smithi070.iziydh is now active in filesystem cephfs as rank 0 2024-03-23T08:20:43.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:42 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1598709105' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:20:43.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:43 smithi070 ceph-mon[39699]: pgmap v182: 65 pgs: 8 creating+peering, 5 creating+activating, 52 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:43.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:43 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.70:6826/1612292110,v1:172.21.15.70:6827/1612292110] up:active 2024-03-23T08:20:43.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:43 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 2024-03-23T08:20:44.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:43 smithi193 ceph-mon[48370]: pgmap v182: 65 pgs: 8 creating+peering, 5 creating+activating, 52 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-23T08:20:44.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:43 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.70:6826/1612292110,v1:172.21.15.70:6827/1612292110] up:active 2024-03-23T08:20:44.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:43 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 2024-03-23T08:20:45.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.818+0000 7efec52b9700 1 -- 172.21.15.70:0/3029869502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec00fff20 msgr2=0x7efec0102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:45.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.818+0000 7efec52b9700 1 --2- 172.21.15.70:0/3029869502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec00fff20 0x7efec0102340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efeb0004940 tx=0x7efeb0009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:45.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.818+0000 7efec52b9700 1 -- 172.21.15.70:0/3029869502 shutdown_connections 2024-03-23T08:20:45.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.818+0000 7efec52b9700 1 --2- 172.21.15.70:0/3029869502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec00fff20 0x7efec0102340 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:45.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.818+0000 7efec52b9700 1 --2- 172.21.15.70:0/3029869502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec0066b70 0x7efec00ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:45.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.818+0000 7efec52b9700 1 -- 172.21.15.70:0/3029869502 >> 172.21.15.70:0/3029869502 conn(0x7efec00f7450 msgr2=0x7efec00f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:45.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.818+0000 7efec52b9700 1 -- 172.21.15.70:0/3029869502 shutdown_connections 2024-03-23T08:20:45.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.818+0000 7efec52b9700 1 -- 172.21.15.70:0/3029869502 wait complete. 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efec52b9700 1 Processor -- start 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efec52b9700 1 -- start start 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efec52b9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0066b70 0x7efec010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efec52b9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec00fff20 0x7efec010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efec52b9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec010e550 con 0x7efec0066b70 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efec52b9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec010ed40 con 0x7efec00fff20 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efebeffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0066b70 0x7efec010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efebeffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0066b70 0x7efec010d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43118/0 (socket says 172.21.15.70:43118) 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efebeffd700 1 -- 172.21.15.70:0/3675550655 learned_addr learned my addr 172.21.15.70:0/3675550655 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efebe7fc700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec00fff20 0x7efec010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efebeffd700 1 -- 172.21.15.70:0/3675550655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec00fff20 msgr2=0x7efec010dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efebeffd700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec00fff20 0x7efec010dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efebeffd700 1 -- 172.21.15.70:0/3675550655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0005040 con 0x7efec0066b70 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.819+0000 7efebeffd700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0066b70 0x7efec010d980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efeb400b790 tx=0x7efeb4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.820+0000 7efea77fe700 1 -- 172.21.15.70:0/3675550655 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeb400f6f0 con 0x7efec0066b70 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.820+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec010f020 con 0x7efec0066b70 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.820+0000 7efea77fe700 1 -- 172.21.15.70:0/3675550655 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efeb400fd00 con 0x7efec0066b70 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.820+0000 7efea77fe700 1 -- 172.21.15.70:0/3675550655 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeb40103a0 con 0x7efec0066b70 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.820+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec010f5e0 con 0x7efec0066b70 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.821+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec005f5f0 con 0x7efec0066b70 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.821+0000 7efea77fe700 1 -- 172.21.15.70:0/3675550655 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7efeb400f850 con 0x7efec0066b70 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.821+0000 7efea77fe700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efea80692a0 0x7efea806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.822+0000 7efea77fe700 1 -- 172.21.15.70:0/3675550655 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5464+0+0 (secure 0 0 0) 0x7efeb4087000 con 0x7efec0066b70 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.822+0000 7efebe7fc700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efea80692a0 0x7efea806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.822+0000 7efebe7fc700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efea80692a0 0x7efea806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efeb000b3b0 tx=0x7efeb0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:44.826+0000 7efea77fe700 1 -- 172.21.15.70:0/3675550655 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efeb4056d40 con 0x7efec0066b70 2024-03-23T08:20:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.070+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7efec004bed0 con 0x7efec0066b70 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:boot 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.aymzqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.aymzqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T08:20:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:45.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:44 smithi070 ceph-mon[39699]: Deploying daemon mds.cephfs.smithi070.aymzqb on smithi070 2024-03-23T08:20:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:boot 2024-03-23T08:20:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby 2024-03-23T08:20:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:20:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby 2024-03-23T08:20:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:45.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.aymzqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:20:45.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.aymzqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T08:20:45.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:45.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:44 smithi193 ceph-mon[48370]: Deploying daemon mds.cephfs.smithi070.aymzqb on smithi070 2024-03-23T08:20:45.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.921+0000 7efea77fe700 1 -- 172.21.15.70:0/3675550655 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7efeb40568d0 con 0x7efec0066b70 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.923+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efea80692a0 msgr2=0x7efea806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.923+0000 7efec52b9700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efea80692a0 0x7efea806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efeb000b3b0 tx=0x7efeb0009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.923+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0066b70 msgr2=0x7efec010d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.923+0000 7efec52b9700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0066b70 0x7efec010d980 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efeb400b790 tx=0x7efeb4009220 comp rx=0 tx=0).stop 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.923+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 shutdown_connections 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.923+0000 7efec52b9700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efea80692a0 0x7efea806b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.923+0000 7efec52b9700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0066b70 0x7efec010d980 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.923+0000 7efec52b9700 1 --2- 172.21.15.70:0/3675550655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efec00fff20 0x7efec010dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.923+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 >> 172.21.15.70:0/3675550655 conn(0x7efec00f7450 msgr2=0x7efec00f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.924+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 shutdown_connections 2024-03-23T08:20:45.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:45.924+0000 7efec52b9700 1 -- 172.21.15.70:0/3675550655 wait complete. 2024-03-23T08:20:46.199 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:45 smithi070 ceph-mon[39699]: pgmap v183: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 1 op/s 2024-03-23T08:20:46.199 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:45 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3675550655' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-23T08:20:46.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:45 smithi193 ceph-mon[48370]: pgmap v183: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 1 op/s 2024-03-23T08:20:46.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:45 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3675550655' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-23T08:20:46.856 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T08:20:46.866 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:20:46.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-03-23T08:20:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:46 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3675550655' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-23T08:20:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:46 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby 2024-03-23T08:20:47.221 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:46 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3675550655' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-23T08:20:47.221 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:46 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby 2024-03-23T08:20:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:48.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:47 smithi070 ceph-mon[39699]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-23T08:20:48.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:48.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:48.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:48.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.nucaub", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:20:48.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.nucaub", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T08:20:48.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:48.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:47 smithi193 ceph-mon[48370]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-23T08:20:48.317 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:48.318 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:48.318 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:48.318 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.nucaub", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:20:48.318 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.nucaub", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-23T08:20:48.318 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:49.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:48 smithi193 ceph-mon[48370]: Deploying daemon mds.cephfs.smithi193.nucaub on smithi193 2024-03-23T08:20:49.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:48 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:boot 2024-03-23T08:20:49.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:48 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 2 up:standby 2024-03-23T08:20:49.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:20:49.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:20:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:48 smithi070 ceph-mon[39699]: Deploying daemon mds.cephfs.smithi193.nucaub on smithi193 2024-03-23T08:20:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:48 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:boot 2024-03-23T08:20:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:48 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 2 up:standby 2024-03-23T08:20:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:20:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:20:49.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.259+0000 7efc3f45d700 1 -- 172.21.15.70:0/3957002032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 msgr2=0x7efc380fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:49.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.259+0000 7efc3f45d700 1 --2- 172.21.15.70:0/3957002032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 0x7efc380fe4b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc2c004940 tx=0x7efc2c009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:49.261 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.259+0000 7efc3f45d700 1 -- 172.21.15.70:0/3957002032 shutdown_connections 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.259+0000 7efc3f45d700 1 --2- 172.21.15.70:0/3957002032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 0x7efc380fe4b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.259+0000 7efc3f45d700 1 --2- 172.21.15.70:0/3957002032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc380fce40 0x7efc380fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.259+0000 7efc3f45d700 1 -- 172.21.15.70:0/3957002032 >> 172.21.15.70:0/3957002032 conn(0x7efc380f8510 msgr2=0x7efc380fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.259+0000 7efc3f45d700 1 -- 172.21.15.70:0/3957002032 shutdown_connections 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.259+0000 7efc3f45d700 1 -- 172.21.15.70:0/3957002032 wait complete. 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3f45d700 1 Processor -- start 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3f45d700 1 -- start start 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3f45d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc380fce40 0x7efc3810d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3f45d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 0x7efc3810de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3f45d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3810e420 con 0x7efc380fe040 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3f45d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3810ec70 con 0x7efc380fce40 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3c9f8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 0x7efc3810de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3c9f8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 0x7efc3810de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55064/0 (socket says 172.21.15.70:55064) 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3c9f8700 1 -- 172.21.15.70:0/581793275 learned_addr learned my addr 172.21.15.70:0/581793275 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3d1f9700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc380fce40 0x7efc3810d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3c9f8700 1 -- 172.21.15.70:0/581793275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc380fce40 msgr2=0x7efc3810d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3c9f8700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc380fce40 0x7efc3810d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.260+0000 7efc3c9f8700 1 -- 172.21.15.70:0/581793275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c005040 con 0x7efc380fe040 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.261+0000 7efc3c9f8700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 0x7efc3810de20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009380 tx=0x7efc2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.261+0000 7efc35ffb700 1 -- 172.21.15.70:0/581793275 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2c00e040 con 0x7efc380fe040 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.261+0000 7efc35ffb700 1 -- 172.21.15.70:0/581793275 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efc2c00bd20 con 0x7efc380fe040 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.261+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3810eef0 con 0x7efc380fe040 2024-03-23T08:20:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.261+0000 7efc35ffb700 1 -- 172.21.15.70:0/581793275 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2c01c630 con 0x7efc380fe040 2024-03-23T08:20:49.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.261+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3810f430 con 0x7efc380fe040 2024-03-23T08:20:49.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.262+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3805f5f0 con 0x7efc380fe040 2024-03-23T08:20:49.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.262+0000 7efc35ffb700 1 -- 172.21.15.70:0/581793275 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7efc2c013420 con 0x7efc380fe040 2024-03-23T08:20:49.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.263+0000 7efc35ffb700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efc200692a0 0x7efc2006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:49.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.263+0000 7efc35ffb700 1 -- 172.21.15.70:0/581793275 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5464+0+0 (secure 0 0 0) 0x7efc2c087da0 con 0x7efc380fe040 2024-03-23T08:20:49.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.263+0000 7efc3d1f9700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efc200692a0 0x7efc2006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:49.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.263+0000 7efc3d1f9700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efc200692a0 0x7efc2006b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc30009fd0 tx=0x7efc3000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:49.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.267+0000 7efc35ffb700 1 -- 172.21.15.70:0/581793275 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efc2c057c00 con 0x7efc380fe040 2024-03-23T08:20:49.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.493+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7efc3804bed0 con 0x7efc380fe040 2024-03-23T08:20:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.938+0000 7efc35ffb700 1 -- 172.21.15.70:0/581793275 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v10) v1 ==== 121+0+0 (secure 0 0 0) 0x7efc2c057790 con 0x7efc380fe040 2024-03-23T08:20:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efc200692a0 msgr2=0x7efc2006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efc200692a0 0x7efc2006b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efc30009fd0 tx=0x7efc3000b040 comp rx=0 tx=0).stop 2024-03-23T08:20:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 msgr2=0x7efc3810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 0x7efc3810de20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009380 tx=0x7efc2c009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 shutdown_connections 2024-03-23T08:20:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc380fce40 0x7efc3810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7efc200692a0 0x7efc2006b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 --2- 172.21.15.70:0/581793275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc380fe040 0x7efc3810de20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 >> 172.21.15.70:0/581793275 conn(0x7efc380f8510 msgr2=0x7efc380f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 shutdown_connections 2024-03-23T08:20:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:49.941+0000 7efc3f45d700 1 -- 172.21.15.70:0/581793275 wait complete. 2024-03-23T08:20:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:49 smithi070 ceph-mon[39699]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-23T08:20:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:49 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/581793275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T08:20:50.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:49 smithi193 ceph-mon[48370]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-23T08:20:50.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:49 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/581793275' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-23T08:20:50.646 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T08:20:50.657 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:20:50.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-23T08:20:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/581793275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T08:20:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.193:6826/2060413299,v1:172.21.15.193:6827/2060413299] up:boot 2024-03-23T08:20:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:20:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:20:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:20:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:50 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:20:51.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/581793275' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-23T08:20:51.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.193:6826/2060413299,v1:172.21.15.193:6827/2060413299] up:boot 2024-03-23T08:20:51.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:20:51.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:20:51.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:20:51.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:51.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:50 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:20:51.517 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:52.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:52 smithi070 ceph-mon[39699]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-03-23T08:20:52.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:52 smithi193 ceph-mon[48370]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-03-23T08:20:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:standby 2024-03-23T08:20:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:standby 2024-03-23T08:20:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: Dropping low affinity standby-replay daemon mds.cephfs.smithi193.nucaub in favor of higher affinity standby. 2024-03-23T08:20:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:20:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: Dropping low affinity active daemon mds.cephfs.smithi070.iziydh in favor of higher affinity standby. 2024-03-23T08:20:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: Replacing daemon mds.cephfs.smithi070.iziydh as rank 0 with standby daemon mds.cephfs.smithi070.aymzqb 2024-03-23T08:20:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T08:20:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby-replay 1 up:standby 2024-03-23T08:20:53.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: osdmap e48: 6 total, 6 up, 6 in 2024-03-23T08:20:53.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: fsmap cephfs:1/1 {0=cephfs.smithi070.aymzqb=up:replay} 1 up:standby 2024-03-23T08:20:53.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:53.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:53 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:53.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:standby 2024-03-23T08:20:53.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:standby 2024-03-23T08:20:53.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: Dropping low affinity standby-replay daemon mds.cephfs.smithi193.nucaub in favor of higher affinity standby. 2024-03-23T08:20:53.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:20:53.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: Dropping low affinity active daemon mds.cephfs.smithi070.iziydh in favor of higher affinity standby. 2024-03-23T08:20:53.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: Replacing daemon mds.cephfs.smithi070.iziydh as rank 0 with standby daemon mds.cephfs.smithi070.aymzqb 2024-03-23T08:20:53.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T08:20:53.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby-replay 1 up:standby 2024-03-23T08:20:53.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: osdmap e48: 6 total, 6 up, 6 in 2024-03-23T08:20:53.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: fsmap cephfs:1/1 {0=cephfs.smithi070.aymzqb=up:replay} 1 up:standby 2024-03-23T08:20:53.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:53.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:53 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:54.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 -- 172.21.15.70:0/2737661395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 msgr2=0x7ff2f41023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:54.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 --2- 172.21.15.70:0/2737661395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 0x7ff2f41023a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0004940 tx=0x7ff2e0009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:54.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 -- 172.21.15.70:0/2737661395 shutdown_connections 2024-03-23T08:20:54.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 --2- 172.21.15.70:0/2737661395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 0x7ff2f41023a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:54.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 --2- 172.21.15.70:0/2737661395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f4066220 0x7ff2f40ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:54.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 -- 172.21.15.70:0/2737661395 >> 172.21.15.70:0/2737661395 conn(0x7ff2f40f7430 msgr2=0x7ff2f40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:54.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 -- 172.21.15.70:0/2737661395 shutdown_connections 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 -- 172.21.15.70:0/2737661395 wait complete. 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 Processor -- start 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.930+0000 7ff2faf9d700 1 -- start start 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2faf9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f4066220 0x7ff2f410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2faf9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 0x7ff2f410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2faf9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f410e560 con 0x7ff2f40fffc0 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2faf9d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f410ed50 con 0x7ff2f4066220 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 0x7ff2f410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 0x7ff2f410ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55090/0 (socket says 172.21.15.70:55090) 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f3fff700 1 -- 172.21.15.70:0/3553494273 learned_addr learned my addr 172.21.15.70:0/3553494273 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f8d39700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f4066220 0x7ff2f410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f3fff700 1 -- 172.21.15.70:0/3553494273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f4066220 msgr2=0x7ff2f410d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f3fff700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f4066220 0x7ff2f410d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f3fff700 1 -- 172.21.15.70:0/3553494273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e0005040 con 0x7ff2f40fffc0 2024-03-23T08:20:54.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f8d39700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f4066220 0x7ff2f410d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f3fff700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 0x7ff2f410ded0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000b3b0 tx=0x7ff2e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f17fa700 1 -- 172.21.15.70:0/3553494273 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e000e040 con 0x7ff2f40fffc0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f17fa700 1 -- 172.21.15.70:0/3553494273 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2e000bd20 con 0x7ff2f40fffc0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2f17fa700 1 -- 172.21.15.70:0/3553494273 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e001c630 con 0x7ff2f40fffc0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.931+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f410efd0 con 0x7ff2f40fffc0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.932+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f410f590 con 0x7ff2f40fffc0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.932+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f405f5f0 con 0x7ff2f40fffc0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.933+0000 7ff2f17fa700 1 -- 172.21.15.70:0/3553494273 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff2e0013960 con 0x7ff2f40fffc0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.933+0000 7ff2f17fa700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff2e40692a0 0x7ff2e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.933+0000 7ff2f17fa700 1 -- 172.21.15.70:0/3553494273 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff2e0087f50 con 0x7ff2f40fffc0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.933+0000 7ff2f8d39700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff2e40692a0 0x7ff2e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.934+0000 7ff2f8d39700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff2e40692a0 0x7ff2e406b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4065f60 tx=0x7ff2e8008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:54.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:53.938+0000 7ff2f17fa700 1 -- 172.21.15.70:0/3553494273 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff2e00ba190 con 0x7ff2f40fffc0 2024-03-23T08:20:54.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.171+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7ff2f404bed0 con 0x7ff2f40fffc0 2024-03-23T08:20:54.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.406+0000 7ff2f17fa700 1 -- 172.21.15.70:0/3553494273 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v17) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff2e0057cc0 con 0x7ff2f40fffc0 2024-03-23T08:20:54.408 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff2e40692a0 msgr2=0x7ff2e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:54.408 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff2e40692a0 0x7ff2e406b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4065f60 tx=0x7ff2e8008040 comp rx=0 tx=0).stop 2024-03-23T08:20:54.408 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 msgr2=0x7ff2f410ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:54.408 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 0x7ff2f410ded0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000b3b0 tx=0x7ff2e0009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:54.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 shutdown_connections 2024-03-23T08:20:54.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f4066220 0x7ff2f410d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:54.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff2e40692a0 0x7ff2e406b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:54.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 --2- 172.21.15.70:0/3553494273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f40fffc0 0x7ff2f410ded0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:54.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 >> 172.21.15.70:0/3553494273 conn(0x7ff2f40f7430 msgr2=0x7ff2f40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:54.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 shutdown_connections 2024-03-23T08:20:54.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:54.408+0000 7ff2faf9d700 1 -- 172.21.15.70:0/3553494273 wait complete. 2024-03-23T08:20:54.409 INFO:teuthology.orchestra.run.smithi070.stderr:inline data disabled 2024-03-23T08:20:54.431 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:54 smithi070 ceph-mon[39699]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-23T08:20:54.432 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:54 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:reconnect 2024-03-23T08:20:54.432 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:54 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] up:boot 2024-03-23T08:20:54.432 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:54 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] up:boot 2024-03-23T08:20:54.432 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:54 smithi070 ceph-mon[39699]: fsmap cephfs:1/1 {0=cephfs.smithi070.aymzqb=up:reconnect} 3 up:standby 2024-03-23T08:20:54.432 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:20:54.432 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:20:54.432 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:54 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:54 smithi193 ceph-mon[48370]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-23T08:20:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:54 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:reconnect 2024-03-23T08:20:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:54 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] up:boot 2024-03-23T08:20:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:54 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] up:boot 2024-03-23T08:20:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:54 smithi193 ceph-mon[48370]: fsmap cephfs:1/1 {0=cephfs.smithi070.aymzqb=up:reconnect} 3 up:standby 2024-03-23T08:20:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:20:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:20:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:54 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:55.481 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:55 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:rejoin 2024-03-23T08:20:55.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:55 smithi070 ceph-mon[39699]: fsmap cephfs:1/1 {0=cephfs.smithi070.aymzqb=up:rejoin} 3 up:standby 2024-03-23T08:20:55.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:55 smithi070 ceph-mon[39699]: daemon mds.cephfs.smithi070.aymzqb is now active in filesystem cephfs as rank 0 2024-03-23T08:20:55.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:55 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3553494273' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-23T08:20:55.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:55 smithi070 ceph-mon[39699]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T08:20:55.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:55 smithi070 ceph-mon[39699]: Cluster is now healthy 2024-03-23T08:20:55.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:55 smithi070 ceph-mon[39699]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:active 2024-03-23T08:20:55.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:55 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3553494273' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-23T08:20:55.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:55 smithi070 ceph-mon[39699]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:20:55.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:55 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:rejoin 2024-03-23T08:20:55.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:55 smithi193 ceph-mon[48370]: fsmap cephfs:1/1 {0=cephfs.smithi070.aymzqb=up:rejoin} 3 up:standby 2024-03-23T08:20:55.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:55 smithi193 ceph-mon[48370]: daemon mds.cephfs.smithi070.aymzqb is now active in filesystem cephfs as rank 0 2024-03-23T08:20:55.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:55 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3553494273' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-23T08:20:55.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:55 smithi193 ceph-mon[48370]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T08:20:55.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:55 smithi193 ceph-mon[48370]: Cluster is now healthy 2024-03-23T08:20:55.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:55 smithi193 ceph-mon[48370]: mds.? [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] up:active 2024-03-23T08:20:55.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:55 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3553494273' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-23T08:20:55.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:55 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:20:55.975 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T08:20:55.987 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:20:55.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-23T08:20:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:56 smithi070 ceph-mon[39699]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 409 B/s wr, 4 op/s 2024-03-23T08:20:56.496 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:20:56.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:56 smithi193 ceph-mon[48370]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 409 B/s wr, 4 op/s 2024-03-23T08:20:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:57 smithi193 ceph-mon[48370]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-23T08:20:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:20:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:57 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:20:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:57 smithi070 ceph-mon[39699]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-23T08:20:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:58.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:20:58.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:20:58.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:20:58.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:57 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.456+0000 7f17ddbc4700 1 -- 172.21.15.70:0/2905311890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fbcd0 msgr2=0x7f17d80fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.456+0000 7f17ddbc4700 1 --2- 172.21.15.70:0/2905311890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fbcd0 0x7f17d80fc0e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f17c8004940 tx=0x7f17c8009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.456+0000 7f17ddbc4700 1 -- 172.21.15.70:0/2905311890 shutdown_connections 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.456+0000 7f17ddbc4700 1 --2- 172.21.15.70:0/2905311890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d80fcf80 0x7f17d80fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.456+0000 7f17ddbc4700 1 --2- 172.21.15.70:0/2905311890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fbcd0 0x7f17d80fc0e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.456+0000 7f17ddbc4700 1 -- 172.21.15.70:0/2905311890 >> 172.21.15.70:0/2905311890 conn(0x7f17d80f7430 msgr2=0x7f17d80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.456+0000 7f17ddbc4700 1 -- 172.21.15.70:0/2905311890 shutdown_connections 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.456+0000 7f17ddbc4700 1 -- 172.21.15.70:0/2905311890 wait complete. 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17ddbc4700 1 Processor -- start 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17ddbc4700 1 -- start start 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17ddbc4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d80fbcd0 0x7f17d810c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17ddbc4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fcf80 0x7f17d810cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:58.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17ddbc4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d810d390 con 0x7f17d80fcf80 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17ddbc4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d810dbe0 con 0x7f17d80fbcd0 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17d6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fcf80 0x7f17d810cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17d6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fcf80 0x7f17d810cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41952/0 (socket says 172.21.15.70:41952) 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17d6ffd700 1 -- 172.21.15.70:0/3073636831 learned_addr learned my addr 172.21.15.70:0/3073636831 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17d77fe700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d80fbcd0 0x7f17d810c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17d6ffd700 1 -- 172.21.15.70:0/3073636831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d80fbcd0 msgr2=0x7f17d810c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17d6ffd700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d80fbcd0 0x7f17d810c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.457+0000 7f17d6ffd700 1 -- 172.21.15.70:0/3073636831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8005040 con 0x7f17d80fcf80 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.458+0000 7f17d6ffd700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fcf80 0x7f17d810cd90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f17cc002a50 tx=0x7f17cc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.458+0000 7f17bffff700 1 -- 172.21.15.70:0/3073636831 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17cc007cc0 con 0x7f17d80fcf80 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.458+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d810dec0 con 0x7f17d80fcf80 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.458+0000 7f17bffff700 1 -- 172.21.15.70:0/3073636831 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17cc00f040 con 0x7f17d80fcf80 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.458+0000 7f17bffff700 1 -- 172.21.15.70:0/3073636831 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17cc00d9c0 con 0x7f17d80fcf80 2024-03-23T08:20:58.473 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.458+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d810e400 con 0x7f17d80fcf80 2024-03-23T08:20:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.459+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d805f5f0 con 0x7f17d80fcf80 2024-03-23T08:20:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.459+0000 7f17bffff700 1 -- 172.21.15.70:0/3073636831 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f17cc00db60 con 0x7f17d80fcf80 2024-03-23T08:20:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.460+0000 7f17bffff700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f17c00692a0 0x7f17c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:20:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.460+0000 7f17bffff700 1 -- 172.21.15.70:0/3073636831 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f17cc086900 con 0x7f17d80fcf80 2024-03-23T08:20:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.460+0000 7f17d77fe700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f17c00692a0 0x7f17c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:20:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.460+0000 7f17d77fe700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f17c00692a0 0x7f17c006b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f17c800b900 tx=0x7f17c8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:20:58.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.464+0000 7f17bffff700 1 -- 172.21.15.70:0/3073636831 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f17cc04d190 con 0x7f17d80fcf80 2024-03-23T08:20:58.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.694+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f17d804bed0 con 0x7f17d80fcf80 2024-03-23T08:20:58.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.695+0000 7f17bffff700 1 -- 172.21.15.70:0/3073636831 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1839 (secure 0 0 0) 0x7f17cc017740 con 0x7f17d80fcf80 2024-03-23T08:20:58.695 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:20:58.695 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:20:58.695 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:20:58.695 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:20:58.695 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:58.695 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:20:58.695 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:20:58.696 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:20:58.697 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:20:58.698 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:20:58.698 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:58.698 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:58.698 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:20:58.698 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:20:58.698 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:20:58.698 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.696+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f17c00692a0 msgr2=0x7f17c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.696+0000 7f17ddbc4700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f17c00692a0 0x7f17c006b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f17c800b900 tx=0x7f17c8009e30 comp rx=0 tx=0).stop 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.696+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fcf80 msgr2=0x7f17d810cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.696+0000 7f17ddbc4700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fcf80 0x7f17d810cd90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f17cc002a50 tx=0x7f17cc0046b0 comp rx=0 tx=0).stop 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.696+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 shutdown_connections 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.696+0000 7f17ddbc4700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17d80fbcd0 0x7f17d810c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.696+0000 7f17ddbc4700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f17c00692a0 0x7f17c006b750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.696+0000 7f17ddbc4700 1 --2- 172.21.15.70:0/3073636831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17d80fcf80 0x7f17d810cd90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.696+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 >> 172.21.15.70:0/3073636831 conn(0x7f17d80f7430 msgr2=0x7f17d80f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:20:58.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.697+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 shutdown_connections 2024-03-23T08:20:58.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:20:58.697+0000 7f17ddbc4700 1 -- 172.21.15.70:0/3073636831 wait complete. 2024-03-23T08:20:58.700 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:20:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:20:58 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3073636831' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:20:59.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:20:58 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3073636831' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:20:59.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-23T08:21:00.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:00 smithi070 ceph-mon[39699]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.1 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-23T08:21:00.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:00 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:00.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:00 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:00.495 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:21:00.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:00 smithi193 ceph-mon[48370]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.1 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-23T08:21:00.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:00 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:00.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:00 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:02.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:02 smithi070 ceph-mon[39699]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-23T08:21:02.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:02 smithi193 ceph-mon[48370]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-23T08:21:02.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.761+0000 7f523a1aa700 1 -- 172.21.15.70:0/2939560922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fcc60 msgr2=0x7f52340fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:02.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.761+0000 7f523a1aa700 1 --2- 172.21.15.70:0/2939560922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fcc60 0x7f52340fd0d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5224004940 tx=0x7f5224009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:02.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.761+0000 7f523a1aa700 1 -- 172.21.15.70:0/2939560922 shutdown_connections 2024-03-23T08:21:02.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.761+0000 7f523a1aa700 1 --2- 172.21.15.70:0/2939560922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fcc60 0x7f52340fd0d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:02.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.761+0000 7f523a1aa700 1 --2- 172.21.15.70:0/2939560922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52340fc310 0x7f52340fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:02.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.761+0000 7f523a1aa700 1 -- 172.21.15.70:0/2939560922 >> 172.21.15.70:0/2939560922 conn(0x7f52340f6ba0 msgr2=0x7f52340f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:02.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.761+0000 7f523a1aa700 1 -- 172.21.15.70:0/2939560922 shutdown_connections 2024-03-23T08:21:02.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.761+0000 7f523a1aa700 1 -- 172.21.15.70:0/2939560922 wait complete. 2024-03-23T08:21:02.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f523a1aa700 1 Processor -- start 2024-03-23T08:21:02.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f523a1aa700 1 -- start start 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f523a1aa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fc310 0x7f523410fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f523a1aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52340fcc60 0x7f5234110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f523a1aa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234110720 con 0x7f52340fc310 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f523a1aa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234110f10 con 0x7f52340fcc60 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f52337fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fc310 0x7f523410fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f52337fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fc310 0x7f523410fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41968/0 (socket says 172.21.15.70:41968) 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f52337fe700 1 -- 172.21.15.70:0/2723181913 learned_addr learned my addr 172.21.15.70:0/2723181913 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f52337fe700 1 -- 172.21.15.70:0/2723181913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52340fcc60 msgr2=0x7f5234110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f52337fe700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52340fcc60 0x7f5234110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f52337fe700 1 -- 172.21.15.70:0/2723181913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224005040 con 0x7f52340fc310 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.762+0000 7f52337fe700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fc310 0x7f523410fb50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f521c00b790 tx=0x7f521c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.763+0000 7f522bfff700 1 -- 172.21.15.70:0/2723181913 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521c00f770 con 0x7f52340fc310 2024-03-23T08:21:02.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.763+0000 7f522bfff700 1 -- 172.21.15.70:0/2723181913 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f521c00fd80 con 0x7f52340fc310 2024-03-23T08:21:02.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.763+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52341111f0 con 0x7f52340fc310 2024-03-23T08:21:02.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.763+0000 7f522bfff700 1 -- 172.21.15.70:0/2723181913 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521c00d310 con 0x7f52340fc310 2024-03-23T08:21:02.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.763+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52341117b0 con 0x7f52340fc310 2024-03-23T08:21:02.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.764+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5234101a60 con 0x7f52340fc310 2024-03-23T08:21:02.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.764+0000 7f522bfff700 1 -- 172.21.15.70:0/2723181913 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f521c00f8d0 con 0x7f52340fc310 2024-03-23T08:21:02.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.764+0000 7f522bfff700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f52200692a0 0x7f522006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:02.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.764+0000 7f5232ffd700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f52200692a0 0x7f522006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:02.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.764+0000 7f522bfff700 1 -- 172.21.15.70:0/2723181913 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f521c0871c0 con 0x7f52340fc310 2024-03-23T08:21:02.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.765+0000 7f5232ffd700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f52200692a0 0x7f522006b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f522400b900 tx=0x7f5224009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:02.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:02.769+0000 7f522bfff700 1 -- 172.21.15.70:0/2723181913 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f521c056f30 con 0x7f52340fc310 2024-03-23T08:21:03.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.003+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f523405f5f0 con 0x7f52340fc310 2024-03-23T08:21:03.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.004+0000 7f522bfff700 1 -- 172.21.15.70:0/2723181913 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4702 (secure 0 0 0) 0x7f521c056ac0 con 0x7f52340fc310 2024-03-23T08:21:03.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f52200692a0 msgr2=0x7f522006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:03.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f52200692a0 0x7f522006b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f522400b900 tx=0x7f5224009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:03.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fc310 msgr2=0x7f523410fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:03.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fc310 0x7f523410fb50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f521c00b790 tx=0x7f521c009250 comp rx=0 tx=0).stop 2024-03-23T08:21:03.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 shutdown_connections 2024-03-23T08:21:03.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f52200692a0 0x7f522006b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:03.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52340fc310 0x7f523410fb50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:03.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 --2- 172.21.15.70:0/2723181913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52340fcc60 0x7f5234110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:03.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 >> 172.21.15.70:0/2723181913 conn(0x7f52340f6ba0 msgr2=0x7f52340fb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:03.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 shutdown_connections 2024-03-23T08:21:03.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:03.006+0000 7f523a1aa700 1 -- 172.21.15.70:0/2723181913 wait complete. 2024-03-23T08:21:03.007 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:21:03.017 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:21:03.311 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:03 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2723181913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T08:21:03.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:03 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2723181913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T08:21:04.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-23T08:21:04.436 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:04 smithi070 ceph-mon[39699]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-23T08:21:04.436 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:04 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:21:04.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:04 smithi193 ceph-mon[48370]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-23T08:21:04.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:04 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:21:04.906 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:21:06.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:06 smithi070 ceph-mon[39699]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T08:21:06.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:06 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:06.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:06 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:06.305 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:06 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:21:06.305 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:06 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:21:06.305 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:06 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:06 smithi193 ceph-mon[48370]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T08:21:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:06 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:06 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:06 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:21:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:06 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:21:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:06 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:21:06.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 -- 172.21.15.70:0/383035664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6948066280 msgr2=0x7f69480ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 --2- 172.21.15.70:0/383035664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6948066280 0x7f69480ffa20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6944004940 tx=0x7f6944009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 -- 172.21.15.70:0/383035664 shutdown_connections 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 --2- 172.21.15.70:0/383035664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69480fff60 0x7f6948102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 --2- 172.21.15.70:0/383035664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6948066280 0x7f69480ffa20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 -- 172.21.15.70:0/383035664 >> 172.21.15.70:0/383035664 conn(0x7f69480f7430 msgr2=0x7f69480f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 -- 172.21.15.70:0/383035664 shutdown_connections 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 -- 172.21.15.70:0/383035664 wait complete. 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 Processor -- start 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 -- start start 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6948066280 0x7f694810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480fff60 0x7f694810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:06.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694810e550 con 0x7f69480fff60 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694f713700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694810ed40 con 0x7f6948066280 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694ccae700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480fff60 0x7f694810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694ccae700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480fff60 0x7f694810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41992/0 (socket says 172.21.15.70:41992) 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.576+0000 7f694ccae700 1 -- 172.21.15.70:0/2162190996 learned_addr learned my addr 172.21.15.70:0/2162190996 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f694d4af700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6948066280 0x7f694810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f694ccae700 1 -- 172.21.15.70:0/2162190996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6948066280 msgr2=0x7f694810d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f694ccae700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6948066280 0x7f694810d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f694ccae700 1 -- 172.21.15.70:0/2162190996 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6944005040 con 0x7f69480fff60 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f694d4af700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6948066280 0x7f694810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f694ccae700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480fff60 0x7f694810dec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f693800b790 tx=0x7f6938009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f693dffb700 1 -- 172.21.15.70:0/2162190996 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f693800f770 con 0x7f69480fff60 2024-03-23T08:21:06.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f693dffb700 1 -- 172.21.15.70:0/2162190996 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f693800fd80 con 0x7f69480fff60 2024-03-23T08:21:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f693dffb700 1 -- 172.21.15.70:0/2162190996 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f693800d310 con 0x7f69480fff60 2024-03-23T08:21:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694810f020 con 0x7f69480fff60 2024-03-23T08:21:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.577+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694810f590 con 0x7f69480fff60 2024-03-23T08:21:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.578+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694805f5f0 con 0x7f69480fff60 2024-03-23T08:21:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.578+0000 7f693dffb700 1 -- 172.21.15.70:0/2162190996 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6938010420 con 0x7f69480fff60 2024-03-23T08:21:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.579+0000 7f693dffb700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f69340692a0 0x7f693406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.579+0000 7f693dffb700 1 -- 172.21.15.70:0/2162190996 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6938087180 con 0x7f69480fff60 2024-03-23T08:21:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.579+0000 7f694d4af700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f69340692a0 0x7f693406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:06.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.580+0000 7f694d4af700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f69340692a0 0x7f693406b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f694400b3b0 tx=0x7f6944009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:06.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.584+0000 7f693dffb700 1 -- 172.21.15.70:0/2162190996 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6938056ef0 con 0x7f69480fff60 2024-03-23T08:21:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.848+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f694810f860 con 0x7f69480fff60 2024-03-23T08:21:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.848+0000 7f693dffb700 1 -- 172.21.15.70:0/2162190996 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+99 (secure 0 0 0) 0x7f6938056a80 con 0x7f69480fff60 2024-03-23T08:21:06.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f69340692a0 msgr2=0x7f693406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:06.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f69340692a0 0x7f693406b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f694400b3b0 tx=0x7f6944009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:06.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480fff60 msgr2=0x7f694810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:06.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480fff60 0x7f694810dec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f693800b790 tx=0x7f6938009250 comp rx=0 tx=0).stop 2024-03-23T08:21:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 shutdown_connections 2024-03-23T08:21:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6948066280 0x7f694810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f69340692a0 0x7f693406b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 --2- 172.21.15.70:0/2162190996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69480fff60 0x7f694810dec0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 >> 172.21.15.70:0/2162190996 conn(0x7f69480f7430 msgr2=0x7f69480f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 shutdown_connections 2024-03-23T08:21:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:06.850+0000 7f694f713700 1 -- 172.21.15.70:0/2162190996 wait complete. 2024-03-23T08:21:06.860 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:21:07.207 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:07 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2162190996' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-23T08:21:07.516 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-23T08:21:07.529 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T08:21:07.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:07 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2162190996' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-23T08:21:07.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.666+0000 7f6c0dbad700 1 -- 172.21.15.70:0/3914434894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fccd0 msgr2=0x7f6c080fd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:07.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.666+0000 7f6c0dbad700 1 --2- 172.21.15.70:0/3914434894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fccd0 0x7f6c080fd0e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc004940 tx=0x7f6bfc009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:07.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.666+0000 7f6c0dbad700 1 -- 172.21.15.70:0/3914434894 shutdown_connections 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.666+0000 7f6c0dbad700 1 --2- 172.21.15.70:0/3914434894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c080fdf80 0x7f6c080fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.666+0000 7f6c0dbad700 1 --2- 172.21.15.70:0/3914434894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fccd0 0x7f6c080fd0e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.666+0000 7f6c0dbad700 1 -- 172.21.15.70:0/3914434894 >> 172.21.15.70:0/3914434894 conn(0x7f6c080f8430 msgr2=0x7f6c080fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.666+0000 7f6c0dbad700 1 -- 172.21.15.70:0/3914434894 shutdown_connections 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.666+0000 7f6c0dbad700 1 -- 172.21.15.70:0/3914434894 wait complete. 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c0dbad700 1 Processor -- start 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c0dbad700 1 -- start start 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c0dbad700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c080fccd0 0x7f6c0810d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c0dbad700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fdf80 0x7f6c0810dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c0dbad700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0810e280 con 0x7f6c080fdf80 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c0dbad700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0810e3f0 con 0x7f6c080fccd0 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c06ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fdf80 0x7f6c0810dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c06ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fdf80 0x7f6c0810dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42010/0 (socket says 172.21.15.70:42010) 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c06ffd700 1 -- 172.21.15.70:0/2202319166 learned_addr learned my addr 172.21.15.70:0/2202319166 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.667+0000 7f6c077fe700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c080fccd0 0x7f6c0810d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.668+0000 7f6c06ffd700 1 -- 172.21.15.70:0/2202319166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c080fccd0 msgr2=0x7f6c0810d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.668+0000 7f6c06ffd700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c080fccd0 0x7f6c0810d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.668+0000 7f6c06ffd700 1 -- 172.21.15.70:0/2202319166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc005040 con 0x7f6c080fdf80 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.668+0000 7f6c06ffd700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fdf80 0x7f6c0810dc80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800c830 tx=0x7f6bf8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.668+0000 7f6beffff700 1 -- 172.21.15.70:0/2202319166 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf800a760 con 0x7f6c080fdf80 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.668+0000 7f6beffff700 1 -- 172.21.15.70:0/2202319166 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6bf800ad70 con 0x7f6c080fdf80 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.668+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c08112e50 con 0x7f6c080fdf80 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.668+0000 7f6beffff700 1 -- 172.21.15.70:0/2202319166 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf80113a0 con 0x7f6c080fdf80 2024-03-23T08:21:07.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.668+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c081133e0 con 0x7f6c080fdf80 2024-03-23T08:21:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.669+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c08064970 con 0x7f6c080fdf80 2024-03-23T08:21:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.669+0000 7f6beffff700 1 -- 172.21.15.70:0/2202319166 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6bf8012400 con 0x7f6c080fdf80 2024-03-23T08:21:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.670+0000 7f6beffff700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bf00692a0 0x7f6bf006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.670+0000 7f6c077fe700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bf00692a0 0x7f6bf006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.670+0000 7f6beffff700 1 -- 172.21.15.70:0/2202319166 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6bf8087270 con 0x7f6c080fdf80 2024-03-23T08:21:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.670+0000 7f6c077fe700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bf00692a0 0x7f6bf006b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc000c00 tx=0x7f6bfc009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.674+0000 7f6beffff700 1 -- 172.21.15.70:0/2202319166 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6bf8056fe0 con 0x7f6c080fdf80 2024-03-23T08:21:07.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.863+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c08105680 con 0x7f6c080fdf80 2024-03-23T08:21:07.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.864+0000 7f6beffff700 1 -- 172.21.15.70:0/2202319166 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+14398 (secure 0 0 0) 0x7f6bf8056b70 con 0x7f6c080fdf80 2024-03-23T08:21:07.865 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:21:07.865 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":48,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","created":"2024-03-23T08:13:36.916195+0000","modified":"2024-03-23T08:20:52.080973+0000","last_up_change":"2024-03-23T08:19:17.604967+0000","last_in_change":"2024-03-23T08:18:58.641957+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T08:18:02.704319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-23T08:20:36.515704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-23T08:20:37.038000+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a71163c1-e6c6-490d-ade2-f345e4b22ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6803","nonce":2404324767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6805","nonce":2404324767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6809","nonce":2404324767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2404324767},{"type":"v1","addr":"172.21.15.70:6807","nonce":2404324767}]},"public_addr":"172.21.15.70:6803/2404324767","cluster_addr":"172.21.15.70:6805/2404324767","heartbeat_back_addr":"172.21.15.70:6809/2404324767","heartbeat_front_addr":"172.21.15.70:6807/2404324767","state":["exists","up"]},{"osd":1,"uuid":"0054cfce-6cdc-46f2-b434-ecd02d1ad193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6811","nonce":3805865717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6813","nonce":3805865717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6817","nonce":3805865717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3805865717},{"type":"v1","addr":"172.21.15.70:6815","nonce":3805865717}]},"public_addr":"172.21.15.70:6811/3805865717","cluster_addr":"172.21.15.70:6813/3805865717","heartbeat_back_addr":"172.21.15.70:6817/3805865717","heartbeat_front_addr":"172.21.15.70:6815/3805865717","state":["exists","up"]},{"osd":2,"uuid":"24c96238-b75f-47c4-b8ef-3794e45e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6819","nonce":3317266675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6821","nonce":3317266675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6825","nonce":3317266675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3317266675},{"type":"v1","addr":"172.21.15.70:6823","nonce":3317266675}]},"public_addr":"172.21.15.70:6819/3317266675","cluster_addr":"172.21.15.70:6821/3317266675","heartbeat_back_addr":"172.21.15.70:6825/3317266675","heartbeat_front_addr":"172.21.15.70:6823/3317266675","state":["exists","up"]},{"osd":3,"uuid":"64e802c4-a037-4a42-8f0f-14756b62251b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6801","nonce":1644042787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6803","nonce":1644042787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6807","nonce":1644042787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1644042787},{"type":"v1","addr":"172.21.15.193:6805","nonce":1644042787}]},"public_addr":"172.21.15.193:6801/1644042787","cluster_addr":"172.21.15.193:6803/1644042787","heartbeat_back_addr":"172.21.15.193:6807/1644042787","heartbeat_front_addr":"172.21.15.193:6805/1644042787","state":["exists","up"]},{"osd":4,"uuid":"3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859463992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859463992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859463992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859463992},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859463992}]},"public_addr":"172.21.15.193:6809/2859463992","cluster_addr":"172.21.15.193:6811/2859463992","heartbeat_back_addr":"172.21.15.193:6815/2859463992","heartbeat_front_addr":"172.21.15.193:6813/2859463992","state":["exists","up"]},{"osd":5,"uuid":"57e8b767-4e62-41b1-8d96-caeaad28c929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6817","nonce":863428472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6819","nonce":863428472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6823","nonce":863428472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":863428472},{"type":"v1","addr":"172.21.15.193:6821","nonce":863428472}]},"public_addr":"172.21.15.193:6817/863428472","cluster_addr":"172.21.15.193:6819/863428472","heartbeat_back_addr":"172.21.15.193:6823/863428472","heartbeat_front_addr":"172.21.15.193:6821/863428472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:01.224711+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:29.865012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:58.894993+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:23.061992+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:49.665476+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:19:16.008523+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6827/1612292110":"2024-03-24T08:20:52.080956+0000","172.21.15.70:6800/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/370857700":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/3894371824":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6800/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:6800/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/3991559747":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/3004659310":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/3484360896":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/562426678":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6801/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/3721968086":"2024-03-24T08:15:48.390242+0000","172.21.15.70:6826/1612292110":"2024-03-24T08:20:52.080956+0000","172.21.15.70:0/1608386260":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6801/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/2209675146":"2024-03-24T08:15:48.390242+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.866+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bf00692a0 msgr2=0x7f6bf006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.866+0000 7f6c0dbad700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bf00692a0 0x7f6bf006b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc000c00 tx=0x7f6bfc009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.867+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fdf80 msgr2=0x7f6c0810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.867+0000 7f6c0dbad700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fdf80 0x7f6c0810dc80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800c830 tx=0x7f6bf8004470 comp rx=0 tx=0).stop 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.867+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 shutdown_connections 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.867+0000 7f6c0dbad700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c080fccd0 0x7f6c0810d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.867+0000 7f6c0dbad700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bf00692a0 0x7f6bf006b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.867+0000 7f6c0dbad700 1 --2- 172.21.15.70:0/2202319166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c080fdf80 0x7f6c0810dc80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.867+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 >> 172.21.15.70:0/2202319166 conn(0x7f6c080f8430 msgr2=0x7f6c080f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.867+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 shutdown_connections 2024-03-23T08:21:07.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:07.867+0000 7f6c0dbad700 1 -- 172.21.15.70:0/2202319166 wait complete. 2024-03-23T08:21:07.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T08:18:02.704319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-23T08:20:36.515704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-23T08:20:37.038000+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-23T08:21:07.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T08:21:08.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.000+0000 7fba250db700 1 -- 172.21.15.70:0/3992037107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 msgr2=0x7fba200fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.000+0000 7fba250db700 1 --2- 172.21.15.70:0/3992037107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 0x7fba200fe700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba08004940 tx=0x7fba08009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:08.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.000+0000 7fba250db700 1 -- 172.21.15.70:0/3992037107 shutdown_connections 2024-03-23T08:21:08.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.000+0000 7fba250db700 1 --2- 172.21.15.70:0/3992037107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba200fec40 0x7fba200fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.000+0000 7fba250db700 1 --2- 172.21.15.70:0/3992037107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 0x7fba200fe700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.000+0000 7fba250db700 1 -- 172.21.15.70:0/3992037107 >> 172.21.15.70:0/3992037107 conn(0x7fba200f6a90 msgr2=0x7fba200f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.000+0000 7fba250db700 1 -- 172.21.15.70:0/3992037107 shutdown_connections 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.000+0000 7fba250db700 1 -- 172.21.15.70:0/3992037107 wait complete. 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba250db700 1 Processor -- start 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba250db700 1 -- start start 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba250db700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 0x7fba2010d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba250db700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba200fec40 0x7fba2010dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba250db700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2010e270 con 0x7fba200fe2f0 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba250db700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2010e3e0 con 0x7fba200fec40 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba1ed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 0x7fba2010d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba1ed9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 0x7fba2010d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42036/0 (socket says 172.21.15.70:42036) 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba1ed9d700 1 -- 172.21.15.70:0/3317768765 learned_addr learned my addr 172.21.15.70:0/3317768765 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba1e59c700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba200fec40 0x7fba2010dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba1ed9d700 1 -- 172.21.15.70:0/3317768765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba200fec40 msgr2=0x7fba2010dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba1ed9d700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba200fec40 0x7fba2010dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba1ed9d700 1 -- 172.21.15.70:0/3317768765 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba08005040 con 0x7fba200fe2f0 2024-03-23T08:21:08.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba1e59c700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba200fec40 0x7fba2010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:21:08.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba1ed9d700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 0x7fba2010d730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fba0800b3b0 tx=0x7fba08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:08.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba177fe700 1 -- 172.21.15.70:0/3317768765 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0800e040 con 0x7fba200fe2f0 2024-03-23T08:21:08.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba177fe700 1 -- 172.21.15.70:0/3317768765 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba0800bd20 con 0x7fba200fe2f0 2024-03-23T08:21:08.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.001+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba20112de0 con 0x7fba200fe2f0 2024-03-23T08:21:08.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.002+0000 7fba177fe700 1 -- 172.21.15.70:0/3317768765 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0801c630 con 0x7fba200fe2f0 2024-03-23T08:21:08.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.002+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba20113320 con 0x7fba200fe2f0 2024-03-23T08:21:08.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.002+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba200640e0 con 0x7fba200fe2f0 2024-03-23T08:21:08.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.003+0000 7fba177fe700 1 -- 172.21.15.70:0/3317768765 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fba08013420 con 0x7fba200fe2f0 2024-03-23T08:21:08.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.003+0000 7fba177fe700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fba0c0692a0 0x7fba0c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.003+0000 7fba1e59c700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fba0c0692a0 0x7fba0c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.003+0000 7fba177fe700 1 -- 172.21.15.70:0/3317768765 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fba08087fc0 con 0x7fba200fe2f0 2024-03-23T08:21:08.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.004+0000 7fba1e59c700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fba0c0692a0 0x7fba0c06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba100041d0 tx=0x7fba10009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:08.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.008+0000 7fba177fe700 1 -- 172.21.15.70:0/3317768765 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fba08057cb0 con 0x7fba200fe2f0 2024-03-23T08:21:08.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.195+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba20113540 con 0x7fba200fe2f0 2024-03-23T08:21:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.196+0000 7fba177fe700 1 -- 172.21.15.70:0/3317768765 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba08057840 con 0x7fba200fe2f0 2024-03-23T08:21:08.196 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-03-23T08:21:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fba0c0692a0 msgr2=0x7fba0c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fba0c0692a0 0x7fba0c06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba100041d0 tx=0x7fba10009040 comp rx=0 tx=0).stop 2024-03-23T08:21:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 msgr2=0x7fba2010d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 0x7fba2010d730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fba0800b3b0 tx=0x7fba08009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 shutdown_connections 2024-03-23T08:21:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fba0c0692a0 0x7fba0c06b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba200fe2f0 0x7fba2010d730 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 --2- 172.21.15.70:0/3317768765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba200fec40 0x7fba2010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 >> 172.21.15.70:0/3317768765 conn(0x7fba200f6a90 msgr2=0x7fba200f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 shutdown_connections 2024-03-23T08:21:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.198+0000 7fba250db700 1 -- 172.21.15.70:0/3317768765 wait complete. 2024-03-23T08:21:08.208 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:08 smithi070 ceph-mon[39699]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.0 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T08:21:08.208 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:08 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2202319166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:21:08.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-23T08:21:08.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 -- 172.21.15.70:0/918696481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fbba0 msgr2=0x7f4de80fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 --2- 172.21.15.70:0/918696481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fbba0 0x7f4de80fbfb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc004940 tx=0x7f4ddc009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 -- 172.21.15.70:0/918696481 shutdown_connections 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 --2- 172.21.15.70:0/918696481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4de80fce50 0x7f4de80fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 --2- 172.21.15.70:0/918696481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fbba0 0x7f4de80fbfb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 -- 172.21.15.70:0/918696481 >> 172.21.15.70:0/918696481 conn(0x7f4de80f7300 msgr2=0x7f4de80f9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 -- 172.21.15.70:0/918696481 shutdown_connections 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 -- 172.21.15.70:0/918696481 wait complete. 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 Processor -- start 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 -- start start 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.330+0000 7f4dee6af700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4de80fbba0 0x7f4de80ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4dee6af700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fce50 0x7f4de80fe190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4dee6af700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8100040 con 0x7f4de80fce50 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4dee6af700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80fe6d0 con 0x7f4de80fbba0 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fce50 0x7f4de80fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de77fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fce50 0x7f4de80fe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42058/0 (socket says 172.21.15.70:42058) 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de77fe700 1 -- 172.21.15.70:0/1297831093 learned_addr learned my addr 172.21.15.70:0/1297831093 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de7fff700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4de80fbba0 0x7f4de80ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de77fe700 1 -- 172.21.15.70:0/1297831093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4de80fbba0 msgr2=0x7f4de80ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de77fe700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4de80fbba0 0x7f4de80ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de77fe700 1 -- 172.21.15.70:0/1297831093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc005040 con 0x7f4de80fce50 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de77fe700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fce50 0x7f4de80fe190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800eb40 tx=0x7f4dd800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de4ff9700 1 -- 172.21.15.70:0/1297831093 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dd8007cc0 con 0x7f4de80fce50 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de80fe950 con 0x7f4de80fce50 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de4ff9700 1 -- 172.21.15.70:0/1297831093 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4dd800f040 con 0x7f4de80fce50 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4de4ff9700 1 -- 172.21.15.70:0/1297831093 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dd8009570 con 0x7f4de80fce50 2024-03-23T08:21:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.331+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de80fef10 con 0x7f4de80fce50 2024-03-23T08:21:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.332+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de8064970 con 0x7f4de80fce50 2024-03-23T08:21:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.333+0000 7f4de4ff9700 1 -- 172.21.15.70:0/1297831093 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4dd80096d0 con 0x7f4de80fce50 2024-03-23T08:21:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.333+0000 7f4de4ff9700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4dd00692a0 0x7f4dd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.333+0000 7f4de7fff700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4dd00692a0 0x7f4dd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.333+0000 7f4de4ff9700 1 -- 172.21.15.70:0/1297831093 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4dd8086e80 con 0x7f4de80fce50 2024-03-23T08:21:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.333+0000 7f4de7fff700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4dd00692a0 0x7f4dd006b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0137b0 tx=0x7f4ddc009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.337+0000 7f4de4ff9700 1 -- 172.21.15.70:0/1297831093 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4dd804e190 con 0x7f4de80fce50 2024-03-23T08:21:08.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.523+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f4de8104660 con 0x7f4de80fce50 2024-03-23T08:21:08.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.523+0000 7f4de4ff9700 1 -- 172.21.15.70:0/1297831093 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v48) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4dd8056c20 con 0x7f4de80fce50 2024-03-23T08:21:08.523 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-03-23T08:21:08.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4dd00692a0 msgr2=0x7f4dd006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4dd00692a0 0x7f4dd006b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0137b0 tx=0x7f4ddc009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:08.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fce50 msgr2=0x7f4de80fe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fce50 0x7f4de80fe190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800eb40 tx=0x7f4dd800b040 comp rx=0 tx=0).stop 2024-03-23T08:21:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 shutdown_connections 2024-03-23T08:21:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4de80fbba0 0x7f4de80ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f4dd00692a0 0x7f4dd006b750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 --2- 172.21.15.70:0/1297831093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4de80fce50 0x7f4de80fe190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 >> 172.21.15.70:0/1297831093 conn(0x7f4de80f7300 msgr2=0x7f4de8065450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 shutdown_connections 2024-03-23T08:21:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.525+0000 7f4dee6af700 1 -- 172.21.15.70:0/1297831093 wait complete. 2024-03-23T08:21:08.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-23T08:21:08.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:08 smithi193 ceph-mon[48370]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.0 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-23T08:21:08.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:08 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2202319166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 -- 172.21.15.70:0/608403435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fbba0 msgr2=0x7f88640fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 --2- 172.21.15.70:0/608403435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fbba0 0x7f88640fbfb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8854004940 tx=0x7f8854009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 -- 172.21.15.70:0/608403435 shutdown_connections 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 --2- 172.21.15.70:0/608403435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88640fce50 0x7f88640fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 --2- 172.21.15.70:0/608403435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fbba0 0x7f88640fbfb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 -- 172.21.15.70:0/608403435 >> 172.21.15.70:0/608403435 conn(0x7f88640f7300 msgr2=0x7f88640f9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 -- 172.21.15.70:0/608403435 shutdown_connections 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 -- 172.21.15.70:0/608403435 wait complete. 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 Processor -- start 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.658+0000 7f886a52e700 1 -- start start 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f886a52e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88640fbba0 0x7f886410c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f886a52e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fce50 0x7f886410cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f886a52e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886410d180 con 0x7f88640fce50 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f886a52e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886410d2f0 con 0x7f88640fbba0 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f88637fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fce50 0x7f886410cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f88637fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fce50 0x7f886410cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43890/0 (socket says 172.21.15.70:43890) 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f88637fe700 1 -- 172.21.15.70:0/2644604189 learned_addr learned my addr 172.21.15.70:0/2644604189 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f8863fff700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88640fbba0 0x7f886410c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f88637fe700 1 -- 172.21.15.70:0/2644604189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88640fbba0 msgr2=0x7f886410c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f88637fe700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88640fbba0 0x7f886410c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f88637fe700 1 -- 172.21.15.70:0/2644604189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8854005040 con 0x7f88640fce50 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f88637fe700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fce50 0x7f886410cb80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8858002a50 tx=0x7f88580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f8860ff9700 1 -- 172.21.15.70:0/2644604189 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8858007cc0 con 0x7f88640fce50 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8864111d50 con 0x7f88640fce50 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f8860ff9700 1 -- 172.21.15.70:0/2644604189 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f885800f040 con 0x7f88640fce50 2024-03-23T08:21:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f8860ff9700 1 -- 172.21.15.70:0/2644604189 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f885800d9c0 con 0x7f88640fce50 2024-03-23T08:21:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.659+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8864112310 con 0x7f88640fce50 2024-03-23T08:21:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.660+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8864064970 con 0x7f88640fce50 2024-03-23T08:21:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.661+0000 7f8860ff9700 1 -- 172.21.15.70:0/2644604189 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f885800db20 con 0x7f88640fce50 2024-03-23T08:21:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.661+0000 7f8860ff9700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f884c0692a0 0x7f884c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.661+0000 7f8860ff9700 1 -- 172.21.15.70:0/2644604189 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8858086ed0 con 0x7f88640fce50 2024-03-23T08:21:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.661+0000 7f8863fff700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f884c0692a0 0x7f884c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.661+0000 7f8863fff700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f884c0692a0 0x7f884c06b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f885400b900 tx=0x7f8854009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.665+0000 7f8860ff9700 1 -- 172.21.15.70:0/2644604189 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8858056bc0 con 0x7f88640fce50 2024-03-23T08:21:08.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.851+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f8864104550 con 0x7f88640fce50 2024-03-23T08:21:08.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.851+0000 7f8860ff9700 1 -- 172.21.15.70:0/2644604189 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v48) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8858056750 con 0x7f88640fce50 2024-03-23T08:21:08.851 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-03-23T08:21:08.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f884c0692a0 msgr2=0x7f884c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f884c0692a0 0x7f884c06b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f885400b900 tx=0x7f8854009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fce50 msgr2=0x7f886410cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fce50 0x7f886410cb80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8858002a50 tx=0x7f88580046b0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 shutdown_connections 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88640fbba0 0x7f886410c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f884c0692a0 0x7f884c06b750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 --2- 172.21.15.70:0/2644604189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88640fce50 0x7f886410cb80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 >> 172.21.15.70:0/2644604189 conn(0x7f88640f7300 msgr2=0x7f88640f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 shutdown_connections 2024-03-23T08:21:08.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.853+0000 7f886a52e700 1 -- 172.21.15.70:0/2644604189 wait complete. 2024-03-23T08:21:08.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 -- 172.21.15.70:0/3645289957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 msgr2=0x7f08600fd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 --2- 172.21.15.70:0/3645289957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 0x7f08600fd0f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0850004940 tx=0x7f0850009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 -- 172.21.15.70:0/3645289957 shutdown_connections 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 --2- 172.21.15.70:0/3645289957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08600fdf90 0x7f08600fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 --2- 172.21.15.70:0/3645289957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 0x7f08600fd0f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 -- 172.21.15.70:0/3645289957 >> 172.21.15.70:0/3645289957 conn(0x7f08600f8440 msgr2=0x7f08600fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 -- 172.21.15.70:0/3645289957 shutdown_connections 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 -- 172.21.15.70:0/3645289957 wait complete. 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 Processor -- start 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.982+0000 7f0867a5f700 1 -- start start 2024-03-23T08:21:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f0867a5f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 0x7f086010d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f0867a5f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08600fdf90 0x7f086010dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f0867a5f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086010e2c0 con 0x7f08600fcce0 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f0867a5f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086010e430 con 0x7f08600fdf90 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f08657fb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 0x7f086010d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f08657fb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 0x7f086010d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43912/0 (socket says 172.21.15.70:43912) 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f08657fb700 1 -- 172.21.15.70:0/2499813918 learned_addr learned my addr 172.21.15.70:0/2499813918 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f08657fb700 1 -- 172.21.15.70:0/2499813918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08600fdf90 msgr2=0x7f086010dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f08657fb700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08600fdf90 0x7f086010dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f08657fb700 1 -- 172.21.15.70:0/2499813918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0850005040 con 0x7f08600fcce0 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f08657fb700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 0x7f086010d780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0850009fd0 tx=0x7f0850009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f0855ffb700 1 -- 172.21.15.70:0/2499813918 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f085000e040 con 0x7f08600fcce0 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0860112de0 con 0x7f08600fcce0 2024-03-23T08:21:08.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f0855ffb700 1 -- 172.21.15.70:0/2499813918 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0850004290 con 0x7f08600fcce0 2024-03-23T08:21:08.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f0855ffb700 1 -- 172.21.15.70:0/2499813918 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f085001c3f0 con 0x7f08600fcce0 2024-03-23T08:21:08.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.983+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0860113350 con 0x7f08600fcce0 2024-03-23T08:21:08.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.984+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0860064970 con 0x7f08600fcce0 2024-03-23T08:21:08.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.985+0000 7f0855ffb700 1 -- 172.21.15.70:0/2499813918 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f085000bc50 con 0x7f08600fcce0 2024-03-23T08:21:08.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.985+0000 7f0855ffb700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f084c0692a0 0x7f084c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:08.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.985+0000 7f0864ffa700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f084c0692a0 0x7f084c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:08.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.985+0000 7f0855ffb700 1 -- 172.21.15.70:0/2499813918 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f08500886b0 con 0x7f08600fcce0 2024-03-23T08:21:08.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.986+0000 7f0864ffa700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f084c0692a0 0x7f084c06b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f085c004ad0 tx=0x7f085c007d40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:08.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:08.990+0000 7f0855ffb700 1 -- 172.21.15.70:0/2499813918 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0850058420 con 0x7f08600fcce0 2024-03-23T08:21:09.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.223+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0860105690 con 0x7f08600fcce0 2024-03-23T08:21:09.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.223+0000 7f0855ffb700 1 -- 172.21.15.70:0/2499813918 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4702 (secure 0 0 0) 0x7f0850057fb0 con 0x7f08600fcce0 2024-03-23T08:21:09.223 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:21:09.223 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1586743298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1586743298},{"type":"v1","addr":"172.21.15.70:6827","nonce":1586743298}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:20:54.403320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14462":{"gid":14462,"name":"cephfs.smithi070.aymzqb","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6829/2658800706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2658800706},{"type":"v1","addr":"172.21.15.70:6829","nonce":2658800706}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14474":{"gid":14474,"name":"cephfs.smithi193.nucaub","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6827/4066762341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":4066762341},{"type":"v1","addr":"172.21.15.193:6827","nonce":4066762341}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-03-23T08:21:09.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f084c0692a0 msgr2=0x7f084c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:09.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f084c0692a0 0x7f084c06b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f085c004ad0 tx=0x7f085c007d40 comp rx=0 tx=0).stop 2024-03-23T08:21:09.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 msgr2=0x7f086010d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 0x7f086010d780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0850009fd0 tx=0x7f0850009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 shutdown_connections 2024-03-23T08:21:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f084c0692a0 0x7f084c06b750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08600fcce0 0x7f086010d780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 --2- 172.21.15.70:0/2499813918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08600fdf90 0x7f086010dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 >> 172.21.15.70:0/2499813918 conn(0x7f08600f8440 msgr2=0x7f08600f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 shutdown_connections 2024-03-23T08:21:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:09.225+0000 7f0867a5f700 1 -- 172.21.15.70:0/2499813918 wait complete. 2024-03-23T08:21:09.226 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:21:09.237 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 1, 'flags': 50} 2024-03-23T08:21:09.237 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-23T08:21:09.260 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-23T08:21:09.260 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-03-23T08:21:09.260 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-23T08:21:09.260 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T08:21:09.260 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:09.261 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-23T08:21:09.261 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T08:21:09.261 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:09.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:09.261 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-03-23T08:21:09.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:09.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2024-03-23T08:21:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:Cannot find device "ceph-brx" 2024-03-23T08:21:09.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T08:21:09.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:09.350 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-03-23T08:21:09.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:09.373 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link delete ceph-brx 2024-03-23T08:21:09.445 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:09 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3317768765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:21:09.445 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:09 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1297831093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-23T08:21:09.445 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:09 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2644604189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-23T08:21:09.452 INFO:teuthology.orchestra.run.smithi193.stderr:Cannot find device "ceph-brx" 2024-03-23T08:21:09.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T08:21:09.453 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-23T08:21:09.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-23T08:21:09.477 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:09 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3317768765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:21:09.478 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:09 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1297831093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-23T08:21:09.478 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:09 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2644604189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-23T08:21:09.719 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:21:10.203 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:10 smithi070 ceph-mon[39699]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-23T08:21:10.203 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:10 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2499813918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T08:21:10.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:10 smithi193 ceph-mon[48370]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-23T08:21:10.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:10 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2499813918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 -- 172.21.15.70:0/963840941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc102710 msgr2=0x7f00fc0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 --2- 172.21.15.70:0/963840941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc102710 0x7f00fc0fbd80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f00e8004940 tx=0x7f00e8009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 -- 172.21.15.70:0/963840941 shutdown_connections 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 --2- 172.21.15.70:0/963840941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc102710 0x7f00fc0fbd80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 --2- 172.21.15.70:0/963840941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00fc101d70 0x7f00fc102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 -- 172.21.15.70:0/963840941 >> 172.21.15.70:0/963840941 conn(0x7f00fc072bc0 msgr2=0x7f00fc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 -- 172.21.15.70:0/963840941 shutdown_connections 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 -- 172.21.15.70:0/963840941 wait complete. 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 Processor -- start 2024-03-23T08:21:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.451+0000 7f01031b1700 1 -- start start 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f01031b1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc101d70 0x7f00fc10ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f01031b1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00fc102710 0x7f00fc10efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f01031b1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc10f5f0 con 0x7f00fc101d70 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f01031b1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc10f730 con 0x7f00fc102710 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f0100f4d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc101d70 0x7f00fc10ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f0100f4d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc101d70 0x7f00fc10ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43932/0 (socket says 172.21.15.70:43932) 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f0100f4d700 1 -- 172.21.15.70:0/2556782234 learned_addr learned my addr 172.21.15.70:0/2556782234 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f00fbfff700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00fc102710 0x7f00fc10efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f0100f4d700 1 -- 172.21.15.70:0/2556782234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00fc102710 msgr2=0x7f00fc10efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f0100f4d700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00fc102710 0x7f00fc10efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f0100f4d700 1 -- 172.21.15.70:0/2556782234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e8005040 con 0x7f00fc101d70 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f0100f4d700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc101d70 0x7f00fc10ea60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f00f000c8c0 tx=0x7f00f00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f00f97fa700 1 -- 172.21.15.70:0/2556782234 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f000a980 con 0x7f00fc101d70 2024-03-23T08:21:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f00f97fa700 1 -- 172.21.15.70:0/2556782234 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f00f000ce60 con 0x7f00fc101d70 2024-03-23T08:21:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc117ad0 con 0x7f00fc101d70 2024-03-23T08:21:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f00f97fa700 1 -- 172.21.15.70:0/2556782234 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f00113a0 con 0x7f00fc101d70 2024-03-23T08:21:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.452+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc118030 con 0x7f00fc101d70 2024-03-23T08:21:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.453+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc0ff080 con 0x7f00fc101d70 2024-03-23T08:21:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.454+0000 7f00f97fa700 1 -- 172.21.15.70:0/2556782234 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f00f0012400 con 0x7f00fc101d70 2024-03-23T08:21:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.454+0000 7f00f97fa700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00ec0692a0 0x7f00ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.455+0000 7f00f97fa700 1 -- 172.21.15.70:0/2556782234 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f00f0087c70 con 0x7f00fc101d70 2024-03-23T08:21:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.455+0000 7f00fbfff700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00ec0692a0 0x7f00ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.455+0000 7f00fbfff700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00ec0692a0 0x7f00ec06b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00e800b3b0 tx=0x7f00e8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.458+0000 7f00f97fa700 1 -- 172.21.15.70:0/2556782234 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f00f0053ff0 con 0x7f00fc101d70 2024-03-23T08:21:11.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.685+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f00fc04bed0 con 0x7f00fc101d70 2024-03-23T08:21:11.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.686+0000 7f00f97fa700 1 -- 172.21.15.70:0/2556782234 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f00f0057610 con 0x7f00fc101d70 2024-03-23T08:21:11.686 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-23T08:21:11.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00ec0692a0 msgr2=0x7f00ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:11.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00ec0692a0 0x7f00ec06b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00e800b3b0 tx=0x7f00e8009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:11.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc101d70 msgr2=0x7f00fc10ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:11.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc101d70 0x7f00fc10ea60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f00f000c8c0 tx=0x7f00f00044c0 comp rx=0 tx=0).stop 2024-03-23T08:21:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 shutdown_connections 2024-03-23T08:21:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f00ec0692a0 0x7f00ec06b750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc101d70 0x7f00fc10ea60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 --2- 172.21.15.70:0/2556782234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00fc102710 0x7f00fc10efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 >> 172.21.15.70:0/2556782234 conn(0x7f00fc072bc0 msgr2=0x7f00fc0fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 shutdown_connections 2024-03-23T08:21:11.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:11.688+0000 7f01031b1700 1 -- 172.21.15.70:0/2556782234 wait complete. 2024-03-23T08:21:12.359 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:12 smithi070 ceph-mon[39699]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-23T08:21:12.359 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:12 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2556782234' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T08:21:12.379 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T08:21:12.379 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T08:21:12.379 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2024-03-23T08:21:12.379 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T08:21:12.379 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:12.379 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T08:21:12.379 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T08:21:12.379 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T08:21:12.379 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T08:21:12.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:12.380 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2024-03-23T08:21:12.439 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2024-03-23T08:21:12.440 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T08:21:12.441 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2024-03-23T08:21:12.441 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T08:21:12.441 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42001sec preferred_lft 42001sec 2024-03-23T08:21:12.441 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2024-03-23T08:21:12.441 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-03-23T08:21:12.441 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T08:21:12.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:12.441 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2024-03-23T08:21:12.441 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-03-23T08:21:12.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add name ceph-brx type bridge 2024-03-23T08:21:12.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr flush dev ceph-brx 2024-03-23T08:21:12.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set ceph-brx up 2024-03-23T08:21:12.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T08:21:12.442 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-03-23T08:21:12.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:12 smithi193 ceph-mon[48370]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-23T08:21:12.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:12 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2556782234' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T08:21:12.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:12.607 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T08:21:12.634 INFO:teuthology.orchestra.run.smithi070.stdout:1 2024-03-23T08:21:12.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:12.635 DEBUG:teuthology.orchestra.run.smithi070:> route 2024-03-23T08:21:12.711 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2024-03-23T08:21:12.711 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T08:21:12.711 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T08:21:12.711 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T08:21:12.711 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T08:21:12.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:12.712 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2024-03-23T08:21:12.712 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-03-23T08:21:12.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T08:21:12.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T08:21:12.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T08:21:12.712 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-03-23T08:21:12.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:12.974 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-03-23T08:21:12.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:12.992 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2024-03-23T08:21:13.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:13.049 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2024-03-23T08:21:13.049 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-03-23T08:21:13.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T08:21:13.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-23T08:21:13.049 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-03-23T08:21:13.143 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-23T08:21:13.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:13.144 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2024-03-23T08:21:13.144 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-03-23T08:21:13.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-23T08:21:13.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T08:21:13.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-23T08:21:13.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-23T08:21:13.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-23T08:21:13.144 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-03-23T08:21:13.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:13.344 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2024-03-23T08:21:13.344 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-03-23T08:21:13.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2024-03-23T08:21:13.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T08:21:13.344 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-03-23T08:21:13.441 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T08:21:13.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T08:21:13.441 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:13.459 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T08:21:13.459 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T08:21:13.459 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:13.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2024-03-23T08:21:13.583 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T08:21:13.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T08:21:13.650 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T08:21:13.651 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T08:21:13.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:13.652 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2024-03-23T08:21:13.709 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T08:21:13.709 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T08:21:13.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2024-03-23T08:21:13.779 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T08:21:13.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T08:21:13.846 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T08:21:13.847 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T08:21:13.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:13.847 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2024-03-23T08:21:13.860 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2024-03-23T08:21:13.861+0000 7feff1e6f540 -1 init, newargv = 0x557ab0714d80 newargc=15 2024-03-23T08:21:13.860 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[100427]: starting ceph client 2024-03-23T08:21:13.865 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[100427]: starting fuse 2024-03-23T08:21:13.879 INFO:teuthology.orchestra.run.smithi070.stdout:161 2024-03-23T08:21:13.880 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [161] 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> import re 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> import os 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T08:21:13.880 DEBUG:teuthology.orchestra.run.smithi070:> 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> return f 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2024-03-23T08:21:13.881 DEBUG:teuthology.orchestra.run.smithi070:> ' 2024-03-23T08:21:14.002 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.100427.asok 2024-03-23T08:21:14.184 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.100427.asok 2024-03-23T08:21:14.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:14.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.100427.asok status 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45", 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "100427", 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2024-03-23T08:21:14.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 14522, 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 14522 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1817993757 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2024-03-23T08:21:14.339 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1817993757 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.14522 192.168.0.1:0/1817993757", 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/1817993757", 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 17, 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 48, 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2024-03-23T08:21:14.340 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:21:14.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-23T08:21:14.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:14 smithi070 ceph-mon[39699]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:14.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:14 smithi193 ceph-mon[48370]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:14.615 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:21:16.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:16 smithi070 ceph-mon[39699]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:16.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:16 smithi193 ceph-mon[48370]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.323+0000 7f65e075f700 1 -- 172.21.15.70:0/635450205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 msgr2=0x7f65d80fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.323+0000 7f65e075f700 1 --2- 172.21.15.70:0/635450205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 0x7f65d80fbd10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f65d4004940 tx=0x7f65d4009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.323+0000 7f65e075f700 1 -- 172.21.15.70:0/635450205 shutdown_connections 2024-03-23T08:21:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.323+0000 7f65e075f700 1 --2- 172.21.15.70:0/635450205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 0x7f65d80fbd10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.323+0000 7f65e075f700 1 --2- 172.21.15.70:0/635450205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d8101d00 0x7f65d81020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.323+0000 7f65e075f700 1 -- 172.21.15.70:0/635450205 >> 172.21.15.70:0/635450205 conn(0x7f65d80f7830 msgr2=0x7f65d80f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:16.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.323+0000 7f65e075f700 1 -- 172.21.15.70:0/635450205 shutdown_connections 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.323+0000 7f65e075f700 1 -- 172.21.15.70:0/635450205 wait complete. 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65e075f700 1 Processor -- start 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65e075f700 1 -- start start 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65e075f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d8101d00 0x7f65d8113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65e075f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 0x7f65d8113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65e075f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d81144e0 con 0x7f65d81026a0 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65e075f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d810dae0 con 0x7f65d8101d00 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65ddcfa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 0x7f65d8113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65ddcfa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 0x7f65d8113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43950/0 (socket says 172.21.15.70:43950) 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65ddcfa700 1 -- 172.21.15.70:0/712689041 learned_addr learned my addr 172.21.15.70:0/712689041 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65de4fb700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d8101d00 0x7f65d8113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65ddcfa700 1 -- 172.21.15.70:0/712689041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d8101d00 msgr2=0x7f65d8113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:16.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65ddcfa700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d8101d00 0x7f65d8113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65ddcfa700 1 -- 172.21.15.70:0/712689041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4005040 con 0x7f65d81026a0 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.324+0000 7f65ddcfa700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 0x7f65d8113e90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009390 tx=0x7f65d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.325+0000 7f65ceffd700 1 -- 172.21.15.70:0/712689041 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d400e040 con 0x7f65d81026a0 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.325+0000 7f65ceffd700 1 -- 172.21.15.70:0/712689041 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f65d4013420 con 0x7f65d81026a0 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.325+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d810dd60 con 0x7f65d81026a0 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.325+0000 7f65ceffd700 1 -- 172.21.15.70:0/712689041 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d401c630 con 0x7f65d81026a0 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.325+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d810e2a0 con 0x7f65d81026a0 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.326+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d80fd450 con 0x7f65d81026a0 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.326+0000 7f65ceffd700 1 -- 172.21.15.70:0/712689041 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f65d40139e0 con 0x7f65d81026a0 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.326+0000 7f65ceffd700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65c40692a0 0x7f65c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.327+0000 7f65ceffd700 1 -- 172.21.15.70:0/712689041 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f65d40883e0 con 0x7f65d81026a0 2024-03-23T08:21:16.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.327+0000 7f65de4fb700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65c40692a0 0x7f65c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:16.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.327+0000 7f65de4fb700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65c40692a0 0x7f65c406b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65c8007fd0 tx=0x7f65c8009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:16.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.331+0000 7f65ceffd700 1 -- 172.21.15.70:0/712689041 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f65d40581f0 con 0x7f65d81026a0 2024-03-23T08:21:16.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.562+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f65d8108ca0 con 0x7f65d81026a0 2024-03-23T08:21:16.752 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-23T08:21:16.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.563+0000 7f65ceffd700 1 -- 172.21.15.70:0/712689041 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f65d4057d80 con 0x7f65d81026a0 2024-03-23T08:21:16.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65c40692a0 msgr2=0x7f65c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65c40692a0 0x7f65c406b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65c8007fd0 tx=0x7f65c8009040 comp rx=0 tx=0).stop 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 msgr2=0x7f65d8113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 0x7f65d8113e90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009390 tx=0x7f65d4009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 shutdown_connections 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d8101d00 0x7f65d8113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65c40692a0 0x7f65c406b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 --2- 172.21.15.70:0/712689041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65d81026a0 0x7f65d8113e90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 >> 172.21.15.70:0/712689041 conn(0x7f65d80f7830 msgr2=0x7f65d80f8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 shutdown_connections 2024-03-23T08:21:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:16.565+0000 7f65e075f700 1 -- 172.21.15.70:0/712689041 wait complete. 2024-03-23T08:21:17.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:17 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/712689041' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T08:21:17.353 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T08:21:17.353 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T08:21:17.353 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi193.front.sepia.ceph.com 2024-03-23T08:21:17.353 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-23T08:21:17.353 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:17.353 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T08:21:17.354 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T08:21:17.354 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T08:21:17.354 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-23T08:21:17.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:17.354 DEBUG:teuthology.orchestra.run.smithi193:> ip addr 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 ::1/128 scope host 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b6 brd ff:ff:ff:ff:ff:ff 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f0 2024-03-23T08:21:17.377 INFO:teuthology.orchestra.run.smithi193.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b7 brd ff:ff:ff:ff:ff:ff 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f1 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:52 brd ff:ff:ff:ff:ff:ff 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:53 brd ff:ff:ff:ff:ff:ff 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout: inet 172.21.15.193/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft 42001sec preferred_lft 42001sec 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 fe80::ec4:7aff:fe88:7253/64 scope link 2024-03-23T08:21:17.378 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-03-23T08:21:17.378 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T08:21:17.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:17.379 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-03-23T08:21:17.379 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-03-23T08:21:17.379 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link add name ceph-brx type bridge 2024-03-23T08:21:17.379 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip addr flush dev ceph-brx 2024-03-23T08:21:17.379 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set ceph-brx up 2024-03-23T08:21:17.379 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T08:21:17.379 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-03-23T08:21:17.462 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:17 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/712689041' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-23T08:21:17.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:17.551 DEBUG:teuthology.orchestra.run.smithi193:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T08:21:17.580 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-03-23T08:21:17.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:17.581 DEBUG:teuthology.orchestra.run.smithi193:> route 2024-03-23T08:21:17.663 INFO:teuthology.orchestra.run.smithi193.stdout:Kernel IP routing table 2024-03-23T08:21:17.663 INFO:teuthology.orchestra.run.smithi193.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T08:21:17.663 INFO:teuthology.orchestra.run.smithi193.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T08:21:17.663 INFO:teuthology.orchestra.run.smithi193.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T08:21:17.663 INFO:teuthology.orchestra.run.smithi193.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T08:21:17.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:17.664 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-03-23T08:21:17.664 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-03-23T08:21:17.664 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T08:21:17.664 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T08:21:17.664 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T08:21:17.664 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-03-23T08:21:17.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:17.935 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-03-23T08:21:17.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:17.953 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list-id 2024-03-23T08:21:18.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:18.011 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-03-23T08:21:18.011 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-03-23T08:21:18.011 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-23T08:21:18.011 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-23T08:21:18.011 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-03-23T08:21:18.105 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-23T08:21:18.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:18.105 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-03-23T08:21:18.105 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-03-23T08:21:18.105 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-23T08:21:18.105 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T08:21:18.106 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-23T08:21:18.106 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-23T08:21:18.106 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-23T08:21:18.106 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-03-23T08:21:18.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:18.312 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-03-23T08:21:18.312 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-03-23T08:21:18.312 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set brx.0 up 2024-03-23T08:21:18.312 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T08:21:18.312 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-03-23T08:21:18.351 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:18 smithi193 ceph-mon[48370]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-23T08:21:18.408 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-23T08:21:18.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T08:21:18.408 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:18.467 INFO:teuthology.orchestra.run.smithi193.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-23T08:21:18.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T08:21:18.467 DEBUG:teuthology.orchestra.run.smithi193:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:18.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:18 smithi070 ceph-mon[39699]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-23T08:21:18.524 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-03-23T08:21:18.592 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T08:21:18.592 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T08:21:18.658 INFO:teuthology.orchestra.run.smithi193.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T08:21:18.659 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T08:21:18.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:18.660 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-03-23T08:21:18.718 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T08:21:18.718 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-23T08:21:18.760 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-03-23T08:21:18.786 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-23T08:21:18.786 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-23T08:21:18.853 INFO:teuthology.orchestra.run.smithi193.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-23T08:21:18.853 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-23T08:21:18.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:18.854 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-03-23T08:21:18.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:2024-03-23T08:21:18.886+0000 7f0e7d07b540 -1 init, newargv = 0x564f24f85f50 newargc=15 2024-03-23T08:21:18.887 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:ceph-fuse[79396]: starting ceph client 2024-03-23T08:21:18.890 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:ceph-fuse[79396]: starting fuse 2024-03-23T08:21:18.904 INFO:teuthology.orchestra.run.smithi193.stdout:125 2024-03-23T08:21:18.904 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [125] 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> import glob 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> import re 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> import os 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> import subprocess 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> def _find_admin_socket(client_name): 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> files = glob.glob(asok_path) 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> 2024-03-23T08:21:18.905 DEBUG:teuthology.orchestra.run.smithi193:> # Given a non-glob path, it better be there 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> if "*" not in asok_path: 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> assert(len(files) == 1) 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> return files[0] 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> for f in files: 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> contents = proc_f.read() 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> if mountpoint in contents: 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> return f 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> print(_find_admin_socket("client.1")) 2024-03-23T08:21:18.906 DEBUG:teuthology.orchestra.run.smithi193:> ' 2024-03-23T08:21:19.027 INFO:teuthology.orchestra.run.smithi193.stdout:/var/run/ceph/ceph-client.1.79396.asok 2024-03-23T08:21:19.211 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.79396.asok 2024-03-23T08:21:19.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:19.211 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.79396.asok status 2024-03-23T08:21:19.275 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:19 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:21:19.275 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:19 smithi193 ceph-mon[48370]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-03-23T08:21:19.363 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-03-23T08:21:19.363 INFO:teuthology.orchestra.run.smithi193.stdout: "metadata": { 2024-03-23T08:21:19.363 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_sha1": "9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45", 2024-03-23T08:21:19.363 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-23T08:21:19.363 INFO:teuthology.orchestra.run.smithi193.stdout: "entity_id": "1", 2024-03-23T08:21:19.363 INFO:teuthology.orchestra.run.smithi193.stdout: "hostname": "smithi193", 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "pid": "79396", 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "root": "/" 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_count": 0, 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_pinned_count": 0, 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "id": 24291, 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "inst": { 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "name": { 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "client", 2024-03-23T08:21:19.364 INFO:teuthology.orchestra.run.smithi193.stdout: "num": 24291 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.1:0", 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 2176118012 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.1:0", 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 2176118012 2024-03-23T08:21:19.365 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-23T08:21:19.366 INFO:teuthology.orchestra.run.smithi193.stdout: "inst_str": "client.24291 192.168.0.1:0/2176118012", 2024-03-23T08:21:19.366 INFO:teuthology.orchestra.run.smithi193.stdout: "addr_str": "192.168.0.1:0/2176118012", 2024-03-23T08:21:19.366 INFO:teuthology.orchestra.run.smithi193.stdout: "inode_count": 1, 2024-03-23T08:21:19.366 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_epoch": 17, 2024-03-23T08:21:19.366 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch": 48, 2024-03-23T08:21:19.366 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch_barrier": 0, 2024-03-23T08:21:19.366 INFO:teuthology.orchestra.run.smithi193.stdout: "blocklisted": false, 2024-03-23T08:21:19.366 INFO:teuthology.orchestra.run.smithi193.stdout: "fs_name": "cephfs" 2024-03-23T08:21:19.366 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-03-23T08:21:19.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:19.373 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2024-03-23T08:21:19.373 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:19.397 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2024-03-23T08:21:19.398 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:19.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:19.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:19.468 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:19 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:21:19.468 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:19 smithi070 ceph-mon[39699]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-03-23T08:21:19.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:19.474 DEBUG:teuthology.orchestra.run.smithi193:> stat --file-system '--printf=%T 2024-03-23T08:21:19.474 DEBUG:teuthology.orchestra.run.smithi193:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:19.492 INFO:teuthology.orchestra.run.smithi193.stdout:fuseblk 2024-03-23T08:21:19.492 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:19.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:21:19.493 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:19.562 INFO:teuthology.run_tasks:Running task print... 2024-03-23T08:21:19.572 INFO:teuthology.task.print:**** done client 2024-03-23T08:21:19.572 INFO:teuthology.run_tasks:Running task parallel... 2024-03-23T08:21:19.582 INFO:teuthology.task.parallel:starting parallel... 2024-03-23T08:21:19.583 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-23T08:21:19.583 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-23T08:21:19.583 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:21:19.584 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-23T08:21:19.584 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-23T08:21:19.584 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-23T08:21:19.587 INFO:tasks.workunit:Pulling workunits from ref 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:21:19.588 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-23T08:21:19.588 INFO:tasks.workunit:timeout=3h 2024-03-23T08:21:19.588 INFO:tasks.workunit:cleanup=True 2024-03-23T08:21:19.588 DEBUG:teuthology.orchestra.run.smithi070:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:19.610 INFO:teuthology.orchestra.run.smithi070.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:19.610 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-23T08:21:19.610 INFO:teuthology.orchestra.run.smithi070.stdout:Device: a1h/161d Inode: 1 Links: 2 2024-03-23T08:21:19.610 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:21:19.610 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-23T08:21:19.610 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-23T08:21:19.610 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-03-23 08:20:41.844907585 +0000 2024-03-23T08:21:19.611 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-03-23 08:21:19.472887233 +0000 2024-03-23T08:21:19.611 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2024-03-23T08:21:19.611 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-23T08:21:19.611 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-23T08:21:19.688 DEBUG:teuthology.orchestra.run.smithi193:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:19.707 INFO:teuthology.orchestra.run.smithi193.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:19.707 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-23T08:21:19.707 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-03-23T08:21:19.708 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T08:21:19.708 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-23T08:21:19.708 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-23T08:21:19.708 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-03-23 08:21:19.682907619 +0000 2024-03-23T08:21:19.708 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-03-23 08:21:19.682907619 +0000 2024-03-23T08:21:19.708 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2024-03-23T08:21:19.708 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-23T08:21:19.708 DEBUG:teuthology.orchestra.run.smithi193:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-23T08:21:19.780 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:21:19.781 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:21:19.798 INFO:tasks.workunit.client.0.smithi070.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-23T08:21:19.838 INFO:tasks.workunit.client.1.smithi193.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-23T08:21:19.867 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:21:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 -- 172.21.15.70:0/3388090989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fcf00 msgr2=0x7f18980fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 --2- 172.21.15.70:0/3388090989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fcf00 0x7f18980fd310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1894004940 tx=0x7f1894009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 -- 172.21.15.70:0/3388090989 shutdown_connections 2024-03-23T08:21:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 --2- 172.21.15.70:0/3388090989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980fe070 0x7f18980fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 --2- 172.21.15.70:0/3388090989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fcf00 0x7f18980fd310 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 -- 172.21.15.70:0/3388090989 >> 172.21.15.70:0/3388090989 conn(0x7f18980f85a0 msgr2=0x7f18980fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 -- 172.21.15.70:0/3388090989 shutdown_connections 2024-03-23T08:21:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 -- 172.21.15.70:0/3388090989 wait complete. 2024-03-23T08:21:21.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 Processor -- start 2024-03-23T08:21:21.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.772+0000 7f18a0395700 1 -- start start 2024-03-23T08:21:21.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f18a0395700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980fcf00 0x7f189810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:21.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f18a0395700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fe070 0x7f189810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f18a0395700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189810e4c0 con 0x7f18980fe070 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f18a0395700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189810ed10 con 0x7f18980fcf00 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f189d930700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fe070 0x7f189810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f189d930700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fe070 0x7f189810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56082/0 (socket says 172.21.15.70:56082) 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f189d930700 1 -- 172.21.15.70:0/1248168089 learned_addr learned my addr 172.21.15.70:0/1248168089 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f189e131700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980fcf00 0x7f189810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f189d930700 1 -- 172.21.15.70:0/1248168089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980fcf00 msgr2=0x7f189810d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f189d930700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980fcf00 0x7f189810d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f189d930700 1 -- 172.21.15.70:0/1248168089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894005040 con 0x7f18980fe070 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f189d930700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fe070 0x7f189810dec0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f188800c8c0 tx=0x7f18880044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f188effd700 1 -- 172.21.15.70:0/1248168089 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f188800a760 con 0x7f18980fe070 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189810eff0 con 0x7f18980fe070 2024-03-23T08:21:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f188effd700 1 -- 172.21.15.70:0/1248168089 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f188800ad70 con 0x7f18980fe070 2024-03-23T08:21:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f188effd700 1 -- 172.21.15.70:0/1248168089 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18880113a0 con 0x7f18980fe070 2024-03-23T08:21:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.773+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189810f530 con 0x7f18980fe070 2024-03-23T08:21:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.774+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189805f5f0 con 0x7f18980fe070 2024-03-23T08:21:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.775+0000 7f188effd700 1 -- 172.21.15.70:0/1248168089 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1888012400 con 0x7f18980fe070 2024-03-23T08:21:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.775+0000 7f188effd700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f18840692a0 0x7f188406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.775+0000 7f188effd700 1 -- 172.21.15.70:0/1248168089 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1888087240 con 0x7f18980fe070 2024-03-23T08:21:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.775+0000 7f189e131700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f18840692a0 0x7f188406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.776+0000 7f189e131700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f18840692a0 0x7f188406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f189400b900 tx=0x7f1894009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.779+0000 7f188effd700 1 -- 172.21.15.70:0/1248168089 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1888056fb0 con 0x7f18980fe070 2024-03-23T08:21:21.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:21 smithi193 ceph-mon[48370]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-23T08:21:21.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.963+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f189804bed0 con 0x7f18980fe070 2024-03-23T08:21:21.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.964+0000 7f188effd700 1 -- 172.21.15.70:0/1248168089 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f188805c020 con 0x7f18980fe070 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.965+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f18840692a0 msgr2=0x7f188406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.965+0000 7f18a0395700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f18840692a0 0x7f188406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f189400b900 tx=0x7f1894009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.965+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fe070 msgr2=0x7f189810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.965+0000 7f18a0395700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fe070 0x7f189810dec0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f188800c8c0 tx=0x7f18880044c0 comp rx=0 tx=0).stop 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.966+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 shutdown_connections 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.966+0000 7f18a0395700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980fcf00 0x7f189810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.966+0000 7f18a0395700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f18840692a0 0x7f188406b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.966+0000 7f18a0395700 1 --2- 172.21.15.70:0/1248168089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18980fe070 0x7f189810dec0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.966+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 >> 172.21.15.70:0/1248168089 conn(0x7f18980f85a0 msgr2=0x7f18980f9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.966+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 shutdown_connections 2024-03-23T08:21:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:21.966+0000 7f18a0395700 1 -- 172.21.15.70:0/1248168089 wait complete. 2024-03-23T08:21:21.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:21 smithi070 ceph-mon[39699]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-23T08:21:22.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-23T08:21:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:21:23.545 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:23 smithi070 ceph-mon[39699]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 682 B/s wr, 2 op/s 2024-03-23T08:21:23.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:23 smithi193 ceph-mon[48370]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 682 B/s wr, 2 op/s 2024-03-23T08:21:24.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.824+0000 7facc2edc700 1 -- 172.21.15.70:0/3616861884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 msgr2=0x7facbc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:24.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.824+0000 7facc2edc700 1 --2- 172.21.15.70:0/3616861884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 0x7facbc0fd3f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faca8004940 tx=0x7faca8009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:24.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.824+0000 7facc2edc700 1 -- 172.21.15.70:0/3616861884 shutdown_connections 2024-03-23T08:21:24.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.824+0000 7facc2edc700 1 --2- 172.21.15.70:0/3616861884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 0x7facbc0fd3f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:24.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.824+0000 7facc2edc700 1 --2- 172.21.15.70:0/3616861884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facbc0fbcd0 0x7facbc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:24.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.824+0000 7facc2edc700 1 -- 172.21.15.70:0/3616861884 >> 172.21.15.70:0/3616861884 conn(0x7facbc0f7430 msgr2=0x7facbc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:24.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.824+0000 7facc2edc700 1 -- 172.21.15.70:0/3616861884 shutdown_connections 2024-03-23T08:21:24.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.824+0000 7facc2edc700 1 -- 172.21.15.70:0/3616861884 wait complete. 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facc2edc700 1 Processor -- start 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facc2edc700 1 -- start start 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facc2edc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facbc0fbcd0 0x7facbc1105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facc2edc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 0x7facbc112b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facc2edc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc066350 con 0x7facbc0fcf80 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facc2edc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc066490 con 0x7facbc0fbcd0 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facbbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 0x7facbc112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facbbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 0x7facbc112b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56096/0 (socket says 172.21.15.70:56096) 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facbbfff700 1 -- 172.21.15.70:0/2251733442 learned_addr learned my addr 172.21.15.70:0/2251733442 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:24.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facc0c78700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facbc0fbcd0 0x7facbc1105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facbbfff700 1 -- 172.21.15.70:0/2251733442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facbc0fbcd0 msgr2=0x7facbc1105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facbbfff700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facbc0fbcd0 0x7facbc1105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facbbfff700 1 -- 172.21.15.70:0/2251733442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8005040 con 0x7facbc0fcf80 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facc0c78700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facbc0fbcd0 0x7facbc1105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.825+0000 7facbbfff700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 0x7facbc112b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faca800b3b0 tx=0x7faca8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.826+0000 7facb97fa700 1 -- 172.21.15.70:0/2251733442 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faca800e040 con 0x7facbc0fcf80 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.826+0000 7facb97fa700 1 -- 172.21.15.70:0/2251733442 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faca8004290 con 0x7facbc0fcf80 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.826+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facbc113080 con 0x7facbc0fcf80 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.826+0000 7facb97fa700 1 -- 172.21.15.70:0/2251733442 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faca801c3f0 con 0x7facbc0fcf80 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.826+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facbc113640 con 0x7facbc0fcf80 2024-03-23T08:21:24.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.826+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facbc05f5f0 con 0x7facbc0fcf80 2024-03-23T08:21:24.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.827+0000 7facb97fa700 1 -- 172.21.15.70:0/2251733442 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7faca8013420 con 0x7facbc0fcf80 2024-03-23T08:21:24.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.827+0000 7facb97fa700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7facac0692b0 0x7facac06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:24.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.827+0000 7facb97fa700 1 -- 172.21.15.70:0/2251733442 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7faca8088570 con 0x7facbc0fcf80 2024-03-23T08:21:24.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.828+0000 7facc0c78700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7facac0692b0 0x7facac06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:24.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.828+0000 7facc0c78700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7facac0692b0 0x7facac06b760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7facb0009b80 tx=0x7facb000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:24.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:24.832+0000 7facb97fa700 1 -- 172.21.15.70:0/2251733442 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7faca8054850 con 0x7facbc0fcf80 2024-03-23T08:21:25.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.016+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7facbc04bed0 con 0x7facbc0fcf80 2024-03-23T08:21:25.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.016+0000 7facb97fa700 1 -- 172.21.15.70:0/2251733442 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7faca8057e70 con 0x7facbc0fcf80 2024-03-23T08:21:25.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7facac0692b0 msgr2=0x7facac06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:25.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7facac0692b0 0x7facac06b760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7facb0009b80 tx=0x7facb000b040 comp rx=0 tx=0).stop 2024-03-23T08:21:25.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 msgr2=0x7facbc112b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:25.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 0x7facbc112b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faca800b3b0 tx=0x7faca8009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 shutdown_connections 2024-03-23T08:21:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facbc0fbcd0 0x7facbc1105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7facac0692b0 0x7facac06b760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 --2- 172.21.15.70:0/2251733442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facbc0fcf80 0x7facbc112b40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 >> 172.21.15.70:0/2251733442 conn(0x7facbc0f7430 msgr2=0x7facbc0f8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 shutdown_connections 2024-03-23T08:21:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:25.018+0000 7facc2edc700 1 -- 172.21.15.70:0/2251733442 wait complete. 2024-03-23T08:21:25.712 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:25 smithi070 ceph-mon[39699]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 767 B/s wr, 2 op/s 2024-03-23T08:21:25.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-23T08:21:25.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:25 smithi193 ceph-mon[48370]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 767 B/s wr, 2 op/s 2024-03-23T08:21:26.003 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:21:27.811 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:27 smithi070 ceph-mon[39699]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-23T08:21:27.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:27 smithi193 ceph-mon[48370]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-23T08:21:28.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 -- 172.21.15.70:0/447385983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 msgr2=0x7ff6d80fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:28.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 --2- 172.21.15.70:0/447385983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 0x7ff6d80fc0e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc004940 tx=0x7ff6cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 -- 172.21.15.70:0/447385983 shutdown_connections 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 --2- 172.21.15.70:0/447385983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6d80fcf80 0x7ff6d80fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 --2- 172.21.15.70:0/447385983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 0x7ff6d80fc0e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 -- 172.21.15.70:0/447385983 >> 172.21.15.70:0/447385983 conn(0x7ff6d80f7430 msgr2=0x7ff6d80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 -- 172.21.15.70:0/447385983 shutdown_connections 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 -- 172.21.15.70:0/447385983 wait complete. 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 Processor -- start 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.097+0000 7ff6df19e700 1 -- start start 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6df19e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 0x7ff6d8110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6df19e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6d80fcf80 0x7ff6d8112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6df19e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8066350 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6df19e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8066490 con 0x7ff6d80fcf80 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6dcf3a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 0x7ff6d8110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6dcf3a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 0x7ff6d8110690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56104/0 (socket says 172.21.15.70:56104) 2024-03-23T08:21:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6dcf3a700 1 -- 172.21.15.70:0/636996757 learned_addr learned my addr 172.21.15.70:0/636996757 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:28.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6d7fff700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6d80fcf80 0x7ff6d8112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:28.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6dcf3a700 1 -- 172.21.15.70:0/636996757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6d80fcf80 msgr2=0x7ff6d8112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:28.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6dcf3a700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6d80fcf80 0x7ff6d8112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:28.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6dcf3a700 1 -- 172.21.15.70:0/636996757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc005040 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6dcf3a700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 0x7ff6d8110690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00b900 tx=0x7ff6cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:28.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6d57fa700 1 -- 172.21.15.70:0/636996757 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6cc00e040 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6d57fa700 1 -- 172.21.15.70:0/636996757 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6cc00bc50 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6d57fa700 1 -- 172.21.15.70:0/636996757 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6cc01c630 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.098+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d8113120 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.099+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d81136e0 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.100+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d805f5f0 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.100+0000 7ff6d57fa700 1 -- 172.21.15.70:0/636996757 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff6cc013420 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.100+0000 7ff6d57fa700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff6c80692a0 0x7ff6c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:28.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.101+0000 7ff6d7fff700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff6c80692a0 0x7ff6c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.101+0000 7ff6d57fa700 1 -- 172.21.15.70:0/636996757 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff6cc0885f0 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.101+0000 7ff6d7fff700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff6c80692a0 0x7ff6c806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4004f40 tx=0x7ff6c400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:28.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.105+0000 7ff6d57fa700 1 -- 172.21.15.70:0/636996757 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff6cc054980 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.300+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7ff6d804bed0 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.300+0000 7ff6d57fa700 1 -- 172.21.15.70:0/636996757 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7ff6cc00bdc0 con 0x7ff6d80fbcd0 2024-03-23T08:21:28.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.302+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff6c80692a0 msgr2=0x7ff6c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.302+0000 7ff6df19e700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff6c80692a0 0x7ff6c806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4004f40 tx=0x7ff6c400b040 comp rx=0 tx=0).stop 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.302+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 msgr2=0x7ff6d8110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.302+0000 7ff6df19e700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 0x7ff6d8110690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00b900 tx=0x7ff6cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.302+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 shutdown_connections 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.302+0000 7ff6df19e700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7ff6c80692a0 0x7ff6c806b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.302+0000 7ff6df19e700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d80fbcd0 0x7ff6d8110690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.302+0000 7ff6df19e700 1 --2- 172.21.15.70:0/636996757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6d80fcf80 0x7ff6d8112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.302+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 >> 172.21.15.70:0/636996757 conn(0x7ff6d80f7430 msgr2=0x7ff6d80f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.303+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 shutdown_connections 2024-03-23T08:21:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:28.303+0000 7ff6df19e700 1 -- 172.21.15.70:0/636996757 wait complete. 2024-03-23T08:21:29.140 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-23T08:21:29.421 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:21:29.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:29 smithi193 ceph-mon[48370]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-03-23T08:21:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:29 smithi070 ceph-mon[39699]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.412+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/2812392829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0070870 msgr2=0x7f6bc0070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.412+0000 7f6bc7eb5700 1 --2- 172.21.15.70:0/2812392829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0070870 0x7f6bc0070ce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0004940 tx=0x7f6bb0009e30 comp rx=0 tx=0).stop 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.412+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/2812392829 shutdown_connections 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.412+0000 7f6bc7eb5700 1 --2- 172.21.15.70:0/2812392829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0070870 0x7f6bc0070ce0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.412+0000 7f6bc7eb5700 1 --2- 172.21.15.70:0/2812392829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0071e20 0x7f6bc0070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.412+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/2812392829 >> 172.21.15.70:0/2812392829 conn(0x7f6bc00f8570 msgr2=0x7f6bc00fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.412+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/2812392829 shutdown_connections 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.412+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/2812392829 wait complete. 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc7eb5700 1 Processor -- start 2024-03-23T08:21:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc7eb5700 1 -- start start 2024-03-23T08:21:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc7eb5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0070870 0x7f6bc010fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc7eb5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0071e20 0x7f6bc0110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc7eb5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0110600 con 0x7f6bc0071e20 2024-03-23T08:21:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc7eb5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0110e50 con 0x7f6bc0070870 2024-03-23T08:21:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc5450700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0071e20 0x7f6bc0110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc5c51700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0070870 0x7f6bc010fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc5c51700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0070870 0x7f6bc010fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:49312/0 (socket says 172.21.15.70:49312) 2024-03-23T08:21:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc5450700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0071e20 0x7f6bc0110000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47374/0 (socket says 172.21.15.70:47374) 2024-03-23T08:21:31.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc5c51700 1 -- 172.21.15.70:0/3020025008 learned_addr learned my addr 172.21.15.70:0/3020025008 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc5450700 1 -- 172.21.15.70:0/3020025008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0070870 msgr2=0x7f6bc010fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc5450700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0070870 0x7f6bc010fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.413+0000 7f6bc5450700 1 -- 172.21.15.70:0/3020025008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb0005040 con 0x7f6bc0071e20 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.414+0000 7f6bc5450700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0071e20 0x7f6bc0110000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000b7d0 tx=0x7f6bb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.414+0000 7f6bb67fc700 1 -- 172.21.15.70:0/3020025008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bb000e040 con 0x7f6bc0071e20 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.414+0000 7f6bb67fc700 1 -- 172.21.15.70:0/3020025008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6bb0004290 con 0x7f6bc0071e20 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.414+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc01110d0 con 0x7f6bc0071e20 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.414+0000 7f6bb67fc700 1 -- 172.21.15.70:0/3020025008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bb001c3f0 con 0x7f6bc0071e20 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.414+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc0111610 con 0x7f6bc0071e20 2024-03-23T08:21:31.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.415+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc005f5f0 con 0x7f6bc0071e20 2024-03-23T08:21:31.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.416+0000 7f6bb67fc700 1 -- 172.21.15.70:0/3020025008 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6bb0013420 con 0x7f6bc0071e20 2024-03-23T08:21:31.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.416+0000 7f6bb67fc700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bac0692b0 0x7f6bac06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:21:31.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.416+0000 7f6bb67fc700 1 -- 172.21.15.70:0/3020025008 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6bb00884e0 con 0x7f6bc0071e20 2024-03-23T08:21:31.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.416+0000 7f6bc5c51700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bac0692b0 0x7f6bac06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:21:31.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.417+0000 7f6bc5c51700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bac0692b0 0x7f6bac06b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc010210 tx=0x7f6bbc00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:21:31.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.420+0000 7f6bb67fc700 1 -- 172.21.15.70:0/3020025008 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6bb0058250 con 0x7f6bc0071e20 2024-03-23T08:21:31.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:21:31.633+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bc0111940 con 0x7f6bac0692b0 2024-03-23T08:21:31.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:31 smithi193 ceph-mon[48370]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-23T08:21:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:31 smithi070 ceph-mon[39699]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-23T08:21:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:32 smithi193 ceph-mon[48370]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:21:32.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:32 smithi070 ceph-mon[39699]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:21:33.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:33 smithi070 ceph-mon[39699]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-23T08:21:33.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:21:33.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:33 smithi193 ceph-mon[48370]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-23T08:21:33.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:21:35.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:35 smithi193 ceph-mon[48370]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-23T08:21:35.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:35 smithi070 ceph-mon[39699]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-23T08:21:37.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:37 smithi193 ceph-mon[48370]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-23T08:21:37.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:37 smithi070 ceph-mon[39699]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 2 op/s 2024-03-23T08:21:39.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:39 smithi193 ceph-mon[48370]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-03-23T08:21:39.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:39 smithi070 ceph-mon[39699]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-03-23T08:21:41.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:41 smithi193 ceph-mon[48370]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 341 B/s wr, 2 op/s 2024-03-23T08:21:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:41 smithi070 ceph-mon[39699]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 341 B/s wr, 2 op/s 2024-03-23T08:21:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:43 smithi070 ceph-mon[39699]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:43.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:43 smithi193 ceph-mon[48370]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:45.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:45 smithi193 ceph-mon[48370]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:45.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:45 smithi070 ceph-mon[39699]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:47.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:47 smithi193 ceph-mon[48370]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:21:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:47 smithi070 ceph-mon[39699]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:21:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:48 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:21:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:48 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:21:49.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:49 smithi193 ceph-mon[48370]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:49 smithi070 ceph-mon[39699]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:51.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:51 smithi193 ceph-mon[48370]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:21:51.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:51 smithi070 ceph-mon[39699]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:21:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:53 smithi070 ceph-mon[39699]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:53.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:53 smithi193 ceph-mon[48370]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:55.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:55 smithi193 ceph-mon[48370]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:55 smithi070 ceph-mon[39699]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:57.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:57 smithi193 ceph-mon[48370]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:21:57.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:57 smithi070 ceph-mon[39699]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:21:59.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:21:59 smithi193 ceph-mon[48370]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:21:59.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:21:59 smithi070 ceph-mon[39699]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:01.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.289+0000 7f6bb67fc700 1 -- 172.21.15.70:0/3020025008 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f6bc0111940 con 0x7f6bac0692b0 2024-03-23T08:22:01.289 INFO:teuthology.orchestra.run.smithi070.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.291+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bac0692b0 msgr2=0x7f6bac06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.291+0000 7f6bc7eb5700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bac0692b0 0x7f6bac06b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc010210 tx=0x7f6bbc00b040 comp rx=0 tx=0).stop 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.291+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0071e20 msgr2=0x7f6bc0110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.291+0000 7f6bc7eb5700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0071e20 0x7f6bc0110000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000b7d0 tx=0x7f6bb0009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.292+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 shutdown_connections 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.292+0000 7f6bc7eb5700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0070870 0x7f6bc010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.292+0000 7f6bc7eb5700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f6bac0692b0 0x7f6bac06b760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.292+0000 7f6bc7eb5700 1 --2- 172.21.15.70:0/3020025008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6bc0071e20 0x7f6bc0110000 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.292+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 >> 172.21.15.70:0/3020025008 conn(0x7f6bc00f8570 msgr2=0x7f6bc00f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:01.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.292+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 shutdown_connections 2024-03-23T08:22:01.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:01.292+0000 7f6bc7eb5700 1 -- 172.21.15.70:0/3020025008 wait complete. 2024-03-23T08:22:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:01 smithi193 ceph-mon[48370]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:22:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:01 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:01 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:01 smithi070 ceph-mon[39699]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:22:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:01 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:01 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:02.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-23T08:22:02.444 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:22:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:02 smithi193 ceph-mon[48370]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:22:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:02 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:22:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:02 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:22:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:02 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:02 smithi193 ceph-mon[48370]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:22:02.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:02 smithi070 ceph-mon[39699]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:22:02.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:02 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:22:02.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:02 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:22:02.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:02 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:02.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:02 smithi070 ceph-mon[39699]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:22:04.166 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:03 smithi070 ceph-mon[39699]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:04.166 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:03 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:22:04.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:03 smithi193 ceph-mon[48370]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:04.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:03 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:22:04.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.756+0000 7f6546492700 1 -- 172.21.15.70:0/946374789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65400fcfb0 msgr2=0x7f65400fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:04.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.756+0000 7f6546492700 1 --2- 172.21.15.70:0/946374789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65400fcfb0 0x7f65400fd420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6534004940 tx=0x7f6534009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:04.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.756+0000 7f6546492700 1 -- 172.21.15.70:0/946374789 shutdown_connections 2024-03-23T08:22:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.756+0000 7f6546492700 1 --2- 172.21.15.70:0/946374789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65400fcfb0 0x7f65400fd420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.756+0000 7f6546492700 1 --2- 172.21.15.70:0/946374789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65400fbd00 0x7f65400fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.756+0000 7f6546492700 1 -- 172.21.15.70:0/946374789 >> 172.21.15.70:0/946374789 conn(0x7f65400f7440 msgr2=0x7f65400f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.757+0000 7f6546492700 1 -- 172.21.15.70:0/946374789 shutdown_connections 2024-03-23T08:22:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.757+0000 7f6546492700 1 -- 172.21.15.70:0/946374789 wait complete. 2024-03-23T08:22:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.757+0000 7f6546492700 1 Processor -- start 2024-03-23T08:22:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.757+0000 7f6546492700 1 -- start start 2024-03-23T08:22:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f6546492700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65400fbd00 0x7f6540066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f6546492700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65400fcfb0 0x7f6540112580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f6546492700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6540112ac0 con 0x7f65400fcfb0 2024-03-23T08:22:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f6546492700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6540112c00 con 0x7f65400fbd00 2024-03-23T08:22:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f653ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65400fbd00 0x7f6540066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f653ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65400fbd00 0x7f6540066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:40578/0 (socket says 172.21.15.70:40578) 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f653ffff700 1 -- 172.21.15.70:0/1248922757 learned_addr learned my addr 172.21.15.70:0/1248922757 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f653ffff700 1 -- 172.21.15.70:0/1248922757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65400fcfb0 msgr2=0x7f6540112580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f653ffff700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65400fcfb0 0x7f6540112580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.758+0000 7f653ffff700 1 -- 172.21.15.70:0/1248922757 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6534005040 con 0x7f65400fbd00 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.759+0000 7f653ffff700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65400fbd00 0x7f6540066cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f653000eb40 tx=0x7f653000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.759+0000 7f653cff9700 1 -- 172.21.15.70:0/1248922757 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6530007cc0 con 0x7f65400fbd00 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.759+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6540112e80 con 0x7f65400fbd00 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.759+0000 7f653cff9700 1 -- 172.21.15.70:0/1248922757 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f653000f040 con 0x7f65400fbd00 2024-03-23T08:22:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.759+0000 7f653cff9700 1 -- 172.21.15.70:0/1248922757 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6530009570 con 0x7f65400fbd00 2024-03-23T08:22:04.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.760+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65401133e0 con 0x7f65400fbd00 2024-03-23T08:22:04.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.761+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654004c710 con 0x7f65400fbd00 2024-03-23T08:22:04.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.761+0000 7f653cff9700 1 -- 172.21.15.70:0/1248922757 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f65300096d0 con 0x7f65400fbd00 2024-03-23T08:22:04.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.761+0000 7f653cff9700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65280692a0 0x7f652806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.761+0000 7f653cff9700 1 -- 172.21.15.70:0/1248922757 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6530086d10 con 0x7f65400fbd00 2024-03-23T08:22:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.761+0000 7f653f7fe700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65280692a0 0x7f652806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.762+0000 7f653f7fe700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65280692a0 0x7f652806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65340137b0 tx=0x7f6534009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:04.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:04.769+0000 7f653cff9700 1 -- 172.21.15.70:0/1248922757 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6530056a80 con 0x7f65400fbd00 2024-03-23T08:22:05.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.121+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6540101310 con 0x7f65280692a0 2024-03-23T08:22:05.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.122+0000 7f653cff9700 1 -- 172.21.15.70:0/1248922757 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f6540101310 con 0x7f65280692a0 2024-03-23T08:22:05.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65280692a0 msgr2=0x7f652806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65280692a0 0x7f652806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65340137b0 tx=0x7f6534009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:05.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65400fbd00 msgr2=0x7f6540066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65400fbd00 0x7f6540066cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f653000eb40 tx=0x7f653000b040 comp rx=0 tx=0).stop 2024-03-23T08:22:05.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 shutdown_connections 2024-03-23T08:22:05.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65400fbd00 0x7f6540066cd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f65280692a0 0x7f652806b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 --2- 172.21.15.70:0/1248922757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65400fcfb0 0x7f6540112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 >> 172.21.15.70:0/1248922757 conn(0x7f65400f7440 msgr2=0x7f65400f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:05.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 shutdown_connections 2024-03-23T08:22:05.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.124+0000 7f6546492700 1 -- 172.21.15.70:0/1248922757 wait complete. 2024-03-23T08:22:05.138 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:22:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.280+0000 7f5d62bd5700 1 -- 172.21.15.70:0/1222020485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fcf80 msgr2=0x7f5d5c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.280+0000 7f5d62bd5700 1 --2- 172.21.15.70:0/1222020485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fcf80 0x7f5d5c0fd3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5d48004940 tx=0x7f5d48009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.280+0000 7f5d62bd5700 1 -- 172.21.15.70:0/1222020485 shutdown_connections 2024-03-23T08:22:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.280+0000 7f5d62bd5700 1 --2- 172.21.15.70:0/1222020485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fcf80 0x7f5d5c0fd3f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.280+0000 7f5d62bd5700 1 --2- 172.21.15.70:0/1222020485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d5c0fbcd0 0x7f5d5c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.280+0000 7f5d62bd5700 1 -- 172.21.15.70:0/1222020485 >> 172.21.15.70:0/1222020485 conn(0x7f5d5c0f7430 msgr2=0x7f5d5c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.280+0000 7f5d62bd5700 1 -- 172.21.15.70:0/1222020485 shutdown_connections 2024-03-23T08:22:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.280+0000 7f5d62bd5700 1 -- 172.21.15.70:0/1222020485 wait complete. 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d62bd5700 1 Processor -- start 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d62bd5700 1 -- start start 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d62bd5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fbcd0 0x7f5d5c110610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d62bd5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d5c0fcf80 0x7f5d5c112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d62bd5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c066350 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d62bd5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c066490 con 0x7f5d5c0fcf80 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d60971700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fbcd0 0x7f5d5c110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d60971700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fbcd0 0x7f5d5c110610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60836/0 (socket says 172.21.15.70:60836) 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d60971700 1 -- 172.21.15.70:0/2116768673 learned_addr learned my addr 172.21.15.70:0/2116768673 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d5bfff700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d5c0fcf80 0x7f5d5c112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d60971700 1 -- 172.21.15.70:0/2116768673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d5c0fcf80 msgr2=0x7f5d5c112b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d60971700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d5c0fcf80 0x7f5d5c112b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.281+0000 7f5d60971700 1 -- 172.21.15.70:0/2116768673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48005040 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.282+0000 7f5d60971700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fbcd0 0x7f5d5c110610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000eb70 tx=0x7f5d5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.282+0000 7f5d597fa700 1 -- 172.21.15.70:0/2116768673 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d50007cc0 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.282+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1130a0 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.282+0000 7f5d597fa700 1 -- 172.21.15.70:0/2116768673 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d5000f040 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.282+0000 7f5d597fa700 1 -- 172.21.15.70:0/2116768673 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d5000c9f0 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.282+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c113660 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.283+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c05f5f0 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.284+0000 7f5d597fa700 1 -- 172.21.15.70:0/2116768673 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5d5000cb50 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.285+0000 7f5d597fa700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5d4c0692a0 0x7f5d4c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.285+0000 7f5d597fa700 1 -- 172.21.15.70:0/2116768673 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5d50087110 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.285+0000 7f5d5bfff700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5d4c0692a0 0x7f5d4c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:05.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.286+0000 7f5d5bfff700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5d4c0692a0 0x7f5d4c06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5d48004890 tx=0x7f5d48009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:05.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.292+0000 7f5d597fa700 1 -- 172.21.15.70:0/2116768673 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5d500533f0 con 0x7f5d5c0fbcd0 2024-03-23T08:22:05.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.571+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d5c101390 con 0x7f5d4c0692a0 2024-03-23T08:22:05.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.572+0000 7f5d597fa700 1 -- 172.21.15.70:0/2116768673 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f5d5c101390 con 0x7f5d4c0692a0 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.574+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5d4c0692a0 msgr2=0x7f5d4c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.574+0000 7f5d62bd5700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5d4c0692a0 0x7f5d4c06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5d48004890 tx=0x7f5d48009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.574+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fbcd0 msgr2=0x7f5d5c110610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.574+0000 7f5d62bd5700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fbcd0 0x7f5d5c110610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000eb70 tx=0x7f5d5000b040 comp rx=0 tx=0).stop 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.575+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 shutdown_connections 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.575+0000 7f5d62bd5700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f5d4c0692a0 0x7f5d4c06b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.575+0000 7f5d62bd5700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5c0fbcd0 0x7f5d5c110610 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.575+0000 7f5d62bd5700 1 --2- 172.21.15.70:0/2116768673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d5c0fcf80 0x7f5d5c112b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.575+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 >> 172.21.15.70:0/2116768673 conn(0x7f5d5c0f7430 msgr2=0x7f5d5c0f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.575+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 shutdown_connections 2024-03-23T08:22:05.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.575+0000 7f5d62bd5700 1 -- 172.21.15.70:0/2116768673 wait complete. 2024-03-23T08:22:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.698+0000 7fae9f493700 1 -- 172.21.15.70:0/130292911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 msgr2=0x7fae98100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.698+0000 7fae9f493700 1 --2- 172.21.15.70:0/130292911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 0x7fae98100190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fae94004940 tx=0x7fae94009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.698+0000 7fae9f493700 1 -- 172.21.15.70:0/130292911 shutdown_connections 2024-03-23T08:22:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.698+0000 7fae9f493700 1 --2- 172.21.15.70:0/130292911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 0x7fae98100190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.698+0000 7fae9f493700 1 --2- 172.21.15.70:0/130292911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae98066b70 0x7fae980fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.698+0000 7fae9f493700 1 -- 172.21.15.70:0/130292911 >> 172.21.15.70:0/130292911 conn(0x7fae980f7450 msgr2=0x7fae980f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.698+0000 7fae9f493700 1 -- 172.21.15.70:0/130292911 shutdown_connections 2024-03-23T08:22:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.698+0000 7fae9f493700 1 -- 172.21.15.70:0/130292911 wait complete. 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9f493700 1 Processor -- start 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9f493700 1 -- start start 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9f493700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae98066b70 0x7fae98113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9f493700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 0x7fae98114420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9f493700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae98114a20 con 0x7fae980fdd10 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9f493700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae98115270 con 0x7fae98066b70 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9ca2e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 0x7fae98114420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9ca2e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 0x7fae98114420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60858/0 (socket says 172.21.15.70:60858) 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9d22f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae98066b70 0x7fae98113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9ca2e700 1 -- 172.21.15.70:0/371546028 learned_addr learned my addr 172.21.15.70:0/371546028 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9ca2e700 1 -- 172.21.15.70:0/371546028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae98066b70 msgr2=0x7fae98113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9ca2e700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae98066b70 0x7fae98113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.699+0000 7fae9ca2e700 1 -- 172.21.15.70:0/371546028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94005040 con 0x7fae980fdd10 2024-03-23T08:22:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.700+0000 7fae9ca2e700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 0x7fae98114420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae940137b0 tx=0x7fae94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.700+0000 7fae8dffb700 1 -- 172.21.15.70:0/371546028 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae94021070 con 0x7fae980fdd10 2024-03-23T08:22:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.700+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae981154f0 con 0x7fae980fdd10 2024-03-23T08:22:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.700+0000 7fae8dffb700 1 -- 172.21.15.70:0/371546028 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae9401c400 con 0x7fae980fdd10 2024-03-23T08:22:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.700+0000 7fae8dffb700 1 -- 172.21.15.70:0/371546028 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae9400e070 con 0x7fae980fdd10 2024-03-23T08:22:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.700+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae981159d0 con 0x7fae980fdd10 2024-03-23T08:22:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.701+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9805f5f0 con 0x7fae980fdd10 2024-03-23T08:22:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.702+0000 7fae8dffb700 1 -- 172.21.15.70:0/371546028 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fae9401c570 con 0x7fae980fdd10 2024-03-23T08:22:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.702+0000 7fae8dffb700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae840692b0 0x7fae8406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.702+0000 7fae8dffb700 1 -- 172.21.15.70:0/371546028 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fae94088370 con 0x7fae980fdd10 2024-03-23T08:22:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.704+0000 7fae9d22f700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae840692b0 0x7fae8406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.704+0000 7fae9d22f700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae840692b0 0x7fae8406b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae88009b80 tx=0x7fae8800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:05.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.707+0000 7fae8dffb700 1 -- 172.21.15.70:0/371546028 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fae940580e0 con 0x7fae980fdd10 2024-03-23T08:22:05.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.915+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fae98115d00 con 0x7fae840692b0 2024-03-23T08:22:05.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.925+0000 7fae8dffb700 1 -- 172.21.15.70:0/371546028 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fae98115d00 con 0x7fae840692b0 2024-03-23T08:22:05.925 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:22:05.925 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (5m) 60s ago 6m 26.3M - 0.25.0 c8568f914cd2 700168bef09e 2024-03-23T08:22:05.925 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (7m) 60s ago 7m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (6m) 66s ago 6m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (5m) 60s ago 6m 86.0M - 9.4.7 954c08fa6188 87fc95a6ce8f 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (78s) 60s ago 78s 16.3M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (85s) 60s ago 84s 10.4M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (76s) 66s ago 75s 11.2M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (82s) 66s ago 82s 12.1M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:9283 running (8m) 60s ago 8m 480M - 17.2.7-726-g9c4e0c34 837124da804f 85d0a92e59bc 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283 running (5m) 66s ago 5m 434M - 17.2.7-726-g9c4e0c34 837124da804f a023705720c4 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (8m) 60s ago 8m 48.7M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (5m) 66s ago 5m 35.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 60s ago 7m 18.6M - 1.5.0 0da6a335fe13 a14314a29bca 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (6m) 66s ago 6m 18.7M - 1.5.0 0da6a335fe13 4197d838b05a 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (5m) 60s ago 5m 45.9M 4299M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:22:05.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (4m) 60s ago 4m 45.7M 4299M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:22:05.927 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (4m) 60s ago 4m 46.4M 4299M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:22:05.927 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (3m) 66s ago 3m 47.1M 5323M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:22:05.927 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (3m) 66s ago 3m 40.7M 5323M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:22:05.927 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (2m) 66s ago 2m 44.1M 5323M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:22:05.927 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (5m) 60s ago 6m 53.8M - 2.43.0 a07b618ecd1d e842f03864f0 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.927+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae840692b0 msgr2=0x7fae8406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.927+0000 7fae9f493700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae840692b0 0x7fae8406b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae88009b80 tx=0x7fae8800b040 comp rx=0 tx=0).stop 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.927+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 msgr2=0x7fae98114420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.927+0000 7fae9f493700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 0x7fae98114420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae940137b0 tx=0x7fae94009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.927+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 shutdown_connections 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.927+0000 7fae9f493700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae98066b70 0x7fae98113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.927+0000 7fae9f493700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae840692b0 0x7fae8406b760 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.927+0000 7fae9f493700 1 --2- 172.21.15.70:0/371546028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae980fdd10 0x7fae98114420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.927+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 >> 172.21.15.70:0/371546028 conn(0x7fae980f7450 msgr2=0x7fae980f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.928+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 shutdown_connections 2024-03-23T08:22:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:05.928+0000 7fae9f493700 1 -- 172.21.15.70:0/371546028 wait complete. 2024-03-23T08:22:05.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:05 smithi070 ceph-mon[39699]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:05.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:05 smithi070 ceph-mon[39699]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:05.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:05 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:05.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:05 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:05.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:05 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.051+0000 7fa73147e700 1 -- 172.21.15.70:0/780660543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fbcd0 msgr2=0x7fa72c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.051+0000 7fa73147e700 1 --2- 172.21.15.70:0/780660543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fbcd0 0x7fa72c0fc0e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa71c004940 tx=0x7fa71c009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.051+0000 7fa73147e700 1 -- 172.21.15.70:0/780660543 shutdown_connections 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.051+0000 7fa73147e700 1 --2- 172.21.15.70:0/780660543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa72c0fcf80 0x7fa72c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.051+0000 7fa73147e700 1 --2- 172.21.15.70:0/780660543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fbcd0 0x7fa72c0fc0e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.051+0000 7fa73147e700 1 -- 172.21.15.70:0/780660543 >> 172.21.15.70:0/780660543 conn(0x7fa72c0f7450 msgr2=0x7fa72c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.051+0000 7fa73147e700 1 -- 172.21.15.70:0/780660543 shutdown_connections 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.051+0000 7fa73147e700 1 -- 172.21.15.70:0/780660543 wait complete. 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.052+0000 7fa73147e700 1 Processor -- start 2024-03-23T08:22:06.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.052+0000 7fa73147e700 1 -- start start 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.052+0000 7fa73147e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa72c0fbcd0 0x7fa72c10c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.052+0000 7fa73147e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fcf80 0x7fa72c10cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.052+0000 7fa73147e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c10d1f0 con 0x7fa72c0fcf80 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.052+0000 7fa73147e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c10d360 con 0x7fa72c0fbcd0 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.052+0000 7fa72a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fcf80 0x7fa72c10cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa72a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fcf80 0x7fa72c10cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60866/0 (socket says 172.21.15.70:60866) 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa72a7fc700 1 -- 172.21.15.70:0/1780752152 learned_addr learned my addr 172.21.15.70:0/1780752152 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa72a7fc700 1 -- 172.21.15.70:0/1780752152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa72c0fbcd0 msgr2=0x7fa72c10c6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa72a7fc700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa72c0fbcd0 0x7fa72c10c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa72a7fc700 1 -- 172.21.15.70:0/1780752152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c005040 con 0x7fa72c0fcf80 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa72a7fc700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fcf80 0x7fa72c10cbf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa72000eb40 tx=0x7fa72000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa7137fe700 1 -- 172.21.15.70:0/1780752152 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa720007cc0 con 0x7fa72c0fcf80 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa7137fe700 1 -- 172.21.15.70:0/1780752152 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa72000f040 con 0x7fa72c0fcf80 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c111d60 con 0x7fa72c0fcf80 2024-03-23T08:22:06.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c112320 con 0x7fa72c0fcf80 2024-03-23T08:22:06.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.053+0000 7fa7137fe700 1 -- 172.21.15.70:0/1780752152 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa720009570 con 0x7fa72c0fcf80 2024-03-23T08:22:06.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.055+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7180052e0 con 0x7fa72c0fcf80 2024-03-23T08:22:06.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.055+0000 7fa7137fe700 1 -- 172.21.15.70:0/1780752152 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa7200096d0 con 0x7fa72c0fcf80 2024-03-23T08:22:06.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.056+0000 7fa7137fe700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa7140692a0 0x7fa71406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:06.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.056+0000 7fa7137fe700 1 -- 172.21.15.70:0/1780752152 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa720086e00 con 0x7fa72c0fcf80 2024-03-23T08:22:06.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.056+0000 7fa72affd700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa7140692a0 0x7fa71406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:06.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.057+0000 7fa72affd700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa7140692a0 0x7fa71406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0137b0 tx=0x7fa71c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:06.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.061+0000 7fa7137fe700 1 -- 172.21.15.70:0/1780752152 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa72004e190 con 0x7fa72c0fcf80 2024-03-23T08:22:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:05 smithi193 ceph-mon[48370]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:05 smithi193 ceph-mon[48370]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:05 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:05 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:05 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:06.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.340+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa718005da0 con 0x7fa72c0fcf80 2024-03-23T08:22:06.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.340+0000 7fa7137fe700 1 -- 172.21.15.70:0/1780752152 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fa720056b80 con 0x7fa72c0fcf80 2024-03-23T08:22:06.342 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:22:06.342 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:22:06.342 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:22:06.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:22:06.342 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:22:06.343 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.342+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa7140692a0 msgr2=0x7fa71406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.342+0000 7fa73147e700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa7140692a0 0x7fa71406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0137b0 tx=0x7fa71c009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.343+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fcf80 msgr2=0x7fa72c10cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.343+0000 7fa73147e700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fcf80 0x7fa72c10cbf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa72000eb40 tx=0x7fa72000b040 comp rx=0 tx=0).stop 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.343+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 shutdown_connections 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.343+0000 7fa73147e700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa72c0fbcd0 0x7fa72c10c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.343+0000 7fa73147e700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fa7140692a0 0x7fa71406b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.343+0000 7fa73147e700 1 --2- 172.21.15.70:0/1780752152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa72c0fcf80 0x7fa72c10cbf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.343+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 >> 172.21.15.70:0/1780752152 conn(0x7fa72c0f7450 msgr2=0x7fa72c0f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.343+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 shutdown_connections 2024-03-23T08:22:06.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.343+0000 7fa73147e700 1 -- 172.21.15.70:0/1780752152 wait complete. 2024-03-23T08:22:06.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.450+0000 7faea60c0700 1 -- 172.21.15.70:0/3542488131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 msgr2=0x7faea00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:06.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.450+0000 7faea60c0700 1 --2- 172.21.15.70:0/3542488131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 0x7faea00fd3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae94004940 tx=0x7fae94009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:06.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.450+0000 7faea60c0700 1 -- 172.21.15.70:0/3542488131 shutdown_connections 2024-03-23T08:22:06.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.450+0000 7faea60c0700 1 --2- 172.21.15.70:0/3542488131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 0x7faea00fd3f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.450+0000 7faea60c0700 1 --2- 172.21.15.70:0/3542488131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00fbcd0 0x7faea00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.450+0000 7faea60c0700 1 -- 172.21.15.70:0/3542488131 >> 172.21.15.70:0/3542488131 conn(0x7faea00f7430 msgr2=0x7faea00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:06.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.450+0000 7faea60c0700 1 -- 172.21.15.70:0/3542488131 shutdown_connections 2024-03-23T08:22:06.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.450+0000 7faea60c0700 1 -- 172.21.15.70:0/3542488131 wait complete. 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.451+0000 7faea60c0700 1 Processor -- start 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.451+0000 7faea60c0700 1 -- start start 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.451+0000 7faea60c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00fbcd0 0x7faea010b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.451+0000 7faea60c0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 0x7faea010bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.451+0000 7faea60c0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea010c1c0 con 0x7faea00fcf80 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.451+0000 7faea60c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea010ca30 con 0x7faea00fbcd0 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae9effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 0x7faea010bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae9f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00fbcd0 0x7faea010b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae9f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00fbcd0 0x7faea010b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:40672/0 (socket says 172.21.15.70:40672) 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae9f7fe700 1 -- 172.21.15.70:0/3235508213 learned_addr learned my addr 172.21.15.70:0/3235508213 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:06.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae9effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 0x7faea010bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60882/0 (socket says 172.21.15.70:60882) 2024-03-23T08:22:06.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae9effd700 1 -- 172.21.15.70:0/3235508213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00fbcd0 msgr2=0x7faea010b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:06.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae9effd700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00fbcd0 0x7faea010b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae9effd700 1 -- 172.21.15.70:0/3235508213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94005040 con 0x7faea00fcf80 2024-03-23T08:22:06.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae9effd700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 0x7faea010bb70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fae9400b0d0 tx=0x7fae94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:06.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae87fff700 1 -- 172.21.15.70:0/3235508213 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae9400e070 con 0x7faea00fcf80 2024-03-23T08:22:06.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae87fff700 1 -- 172.21.15.70:0/3235508213 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae9401c400 con 0x7faea00fcf80 2024-03-23T08:22:06.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea010ccb0 con 0x7faea00fcf80 2024-03-23T08:22:06.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.452+0000 7fae87fff700 1 -- 172.21.15.70:0/3235508213 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae9400b670 con 0x7faea00fcf80 2024-03-23T08:22:06.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.453+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea010d210 con 0x7faea00fcf80 2024-03-23T08:22:06.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.454+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea0101910 con 0x7faea00fcf80 2024-03-23T08:22:06.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.455+0000 7fae87fff700 1 -- 172.21.15.70:0/3235508213 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fae9401ca00 con 0x7faea00fcf80 2024-03-23T08:22:06.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.455+0000 7fae87fff700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae880692a0 0x7fae8806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:06.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.455+0000 7fae87fff700 1 -- 172.21.15.70:0/3235508213 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fae94088410 con 0x7faea00fcf80 2024-03-23T08:22:06.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.455+0000 7fae9f7fe700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae880692a0 0x7fae8806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:06.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.456+0000 7fae9f7fe700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae880692a0 0x7fae8806b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae90008fd0 tx=0x7fae9000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:06.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.460+0000 7fae87fff700 1 -- 172.21.15.70:0/3235508213 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fae940546f0 con 0x7faea00fcf80 2024-03-23T08:22:06.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.792+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faea00008d0 con 0x7fae880692a0 2024-03-23T08:22:06.793 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.793+0000 7fae87fff700 1 -- 172.21.15.70:0/3235508213 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7faea00008d0 con 0x7fae880692a0 2024-03-23T08:22:06.794 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:22:06.794 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:22:06.794 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:22:06.794 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-23T08:22:06.794 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-03-23T08:22:06.794 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "0/2 daemons upgraded", 2024-03-23T08:22:06.794 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f image on host smithi193", 2024-03-23T08:22:06.794 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:22:06.794 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:22:06.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.796+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae880692a0 msgr2=0x7fae8806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:06.796 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.796+0000 7faea60c0700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae880692a0 0x7fae8806b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae90008fd0 tx=0x7fae9000b040 comp rx=0 tx=0).stop 2024-03-23T08:22:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.796+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 msgr2=0x7faea010bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.796+0000 7faea60c0700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 0x7faea010bb70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fae9400b0d0 tx=0x7fae94009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.796+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 shutdown_connections 2024-03-23T08:22:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.796+0000 7faea60c0700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea00fbcd0 0x7faea010b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.796+0000 7faea60c0700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fae880692a0 0x7fae8806b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.796+0000 7faea60c0700 1 --2- 172.21.15.70:0/3235508213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faea00fcf80 0x7faea010bb70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.796+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 >> 172.21.15.70:0/3235508213 conn(0x7faea00f7430 msgr2=0x7faea0065b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.797+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 shutdown_connections 2024-03-23T08:22:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:06.797+0000 7faea60c0700 1 -- 172.21.15.70:0/3235508213 wait complete. 2024-03-23T08:22:06.969 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:06 smithi193 ceph-mon[48370]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:06.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:06 smithi193 ceph-mon[48370]: Upgrade: Target is version 18.2.2-954-g82316b04 (unknown) 2024-03-23T08:22:06.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:06 smithi193 ceph-mon[48370]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f, digests ['quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f'] 2024-03-23T08:22:06.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:06 smithi193 ceph-mon[48370]: Upgrade: Need to upgrade myself (mgr.smithi070.jhsbch) 2024-03-23T08:22:06.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:06 smithi193 ceph-mon[48370]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:06.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:06 smithi193 ceph-mon[48370]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f on smithi193 2024-03-23T08:22:06.970 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:06 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1780752152' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:22:07.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:06 smithi070 ceph-mon[39699]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:07.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:06 smithi070 ceph-mon[39699]: Upgrade: Target is version 18.2.2-954-g82316b04 (unknown) 2024-03-23T08:22:07.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:06 smithi070 ceph-mon[39699]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f, digests ['quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f'] 2024-03-23T08:22:07.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:06 smithi070 ceph-mon[39699]: Upgrade: Need to upgrade myself (mgr.smithi070.jhsbch) 2024-03-23T08:22:07.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:06 smithi070 ceph-mon[39699]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:07.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:06 smithi070 ceph-mon[39699]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f on smithi193 2024-03-23T08:22:07.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:06 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1780752152' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:22:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:07 smithi070 ceph-mon[39699]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:22:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:07 smithi070 ceph-mon[39699]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:07 smithi193 ceph-mon[48370]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:22:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:07 smithi193 ceph-mon[48370]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:09 smithi070 ceph-mon[39699]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:09 smithi193 ceph-mon[48370]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:11 smithi070 ceph-mon[39699]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:22:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:11 smithi193 ceph-mon[48370]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:22:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:13 smithi070 ceph-mon[39699]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:13 smithi193 ceph-mon[48370]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:15.919 INFO:tasks.workunit.client.1.smithi193.stderr:Updating files: 95% (11490/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-03-23T08:22:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:15 smithi070 ceph-mon[39699]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:15 smithi193 ceph-mon[48370]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:16.786 INFO:tasks.workunit.client.1.smithi193.stderr:Note: switching to '82316b04821a6c2c1d458a881920454bbadb8837'. 2024-03-23T08:22:16.786 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-03-23T08:22:16.786 INFO:tasks.workunit.client.1.smithi193.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-23T08:22:16.786 INFO:tasks.workunit.client.1.smithi193.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-23T08:22:16.786 INFO:tasks.workunit.client.1.smithi193.stderr:state without impacting any branches by switching back to a branch. 2024-03-23T08:22:16.786 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-03-23T08:22:16.786 INFO:tasks.workunit.client.1.smithi193.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-23T08:22:16.786 INFO:tasks.workunit.client.1.smithi193.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr: git switch -c 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr:Or undo this operation with: 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr: git switch - 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-03-23T08:22:16.787 INFO:tasks.workunit.client.1.smithi193.stderr:HEAD is now at 82316b04821 Merge branch 'wip-64881-reef' of https://github.com/sseshasa/ceph into wip-yuri11-testing-2024-03-21-1324-reef 2024-03-23T08:22:16.791 DEBUG:teuthology.orchestra.run.smithi193:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-23T08:22:16.859 INFO:tasks.workunit.client.1.smithi193.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-23T08:22:16.861 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-23T08:22:16.861 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-23T08:22:17.419 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-23T08:22:17.476 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-23T08:22:17.514 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-23T08:22:17.516 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-23T08:22:17.516 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-23T08:22:17.553 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-23T08:22:17.556 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-23T08:22:17.556 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-23T08:22:17.615 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-23T08:22:17.617 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-23T08:22:17.617 DEBUG:teuthology.orchestra.run.smithi193:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=82316b04821a6c2c1d458a881920454bbadb8837 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-23T08:22:17.684 INFO:tasks.workunit.client.1.smithi193.stderr:+ mkdir -p fsstress 2024-03-23T08:22:17.686 INFO:tasks.workunit.client.1.smithi193.stderr:+ pushd fsstress 2024-03-23T08:22:17.687 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-23T08:22:17.687 INFO:tasks.workunit.client.1.smithi193.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-23T08:22:17.710 INFO:tasks.workunit.client.0.smithi070.stderr:Note: switching to '82316b04821a6c2c1d458a881920454bbadb8837'. 2024-03-23T08:22:17.710 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-03-23T08:22:17.710 INFO:tasks.workunit.client.0.smithi070.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-23T08:22:17.710 INFO:tasks.workunit.client.0.smithi070.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-23T08:22:17.710 INFO:tasks.workunit.client.0.smithi070.stderr:state without impacting any branches by switching back to a branch. 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr: git switch -c 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr:Or undo this operation with: 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr: git switch - 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-03-23T08:22:17.711 INFO:tasks.workunit.client.0.smithi070.stderr:HEAD is now at 82316b04821 Merge branch 'wip-64881-reef' of https://github.com/sseshasa/ceph into wip-yuri11-testing-2024-03-21-1324-reef 2024-03-23T08:22:17.715 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-23T08:22:17.745 INFO:tasks.workunit.client.0.smithi070.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-23T08:22:17.747 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T08:22:17.747 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-23T08:22:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:17 smithi070 ceph-mon[39699]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:22:18.286 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-23T08:22:18.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:17 smithi193 ceph-mon[48370]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:22:18.341 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-23T08:22:18.378 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-23T08:22:18.380 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T08:22:18.380 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-23T08:22:18.416 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-23T08:22:18.419 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:22:18.419 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-23T08:22:18.478 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-23T08:22:18.479 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-23T08:22:18.479 DEBUG:teuthology.orchestra.run.smithi070:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=82316b04821a6c2c1d458a881920454bbadb8837 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-23T08:22:18.544 INFO:tasks.workunit.client.0.smithi070.stderr:+ mkdir -p fsstress 2024-03-23T08:22:18.546 INFO:tasks.workunit.client.0.smithi070.stderr:+ pushd fsstress 2024-03-23T08:22:18.547 INFO:tasks.workunit.client.0.smithi070.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-23T08:22:18.548 INFO:tasks.workunit.client.0.smithi070.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-23T08:22:19.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:18 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:22:19.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:18 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:22:20.002 INFO:tasks.workunit.client.1.smithi193.stderr:+ tar xzf ltp-full.tgz 2024-03-23T08:22:20.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:19 smithi070 ceph-mon[39699]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:19 smithi193 ceph-mon[48370]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:22:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:21 smithi193 ceph-mon[48370]: pgmap v232: 65 pgs: 65 active+clean; 609 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 2 op/s 2024-03-23T08:22:22.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:21 smithi070 ceph-mon[39699]: pgmap v232: 65 pgs: 65 active+clean; 609 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 2 op/s 2024-03-23T08:22:23.514 INFO:tasks.workunit.client.0.smithi070.stderr:+ tar xzf ltp-full.tgz 2024-03-23T08:22:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:23 smithi193 ceph-mon[48370]: pgmap v233: 65 pgs: 65 active+clean; 1.2 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 63 KiB/s wr, 12 op/s 2024-03-23T08:22:24.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:23 smithi070 ceph-mon[39699]: pgmap v233: 65 pgs: 65 active+clean; 1.2 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 63 KiB/s wr, 12 op/s 2024-03-23T08:22:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:25 smithi193 ceph-mon[48370]: pgmap v234: 65 pgs: 65 active+clean; 22 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 1.8 MiB/s wr, 69 op/s 2024-03-23T08:22:26.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:25 smithi070 ceph-mon[39699]: pgmap v234: 65 pgs: 65 active+clean; 22 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 1.8 MiB/s wr, 69 op/s 2024-03-23T08:22:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:27 smithi193 ceph-mon[48370]: pgmap v235: 65 pgs: 65 active+clean; 30 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-03-23T08:22:28.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:27 smithi070 ceph-mon[39699]: pgmap v235: 65 pgs: 65 active+clean; 30 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 2.5 MiB/s wr, 121 op/s 2024-03-23T08:22:30.333 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:29 smithi193 ceph-mon[48370]: pgmap v236: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 581 KiB/s rd, 3.1 MiB/s wr, 176 op/s 2024-03-23T08:22:30.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:29 smithi070 ceph-mon[39699]: pgmap v236: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 581 KiB/s rd, 3.1 MiB/s wr, 176 op/s 2024-03-23T08:22:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:31 smithi193 ceph-mon[48370]: pgmap v237: 65 pgs: 65 active+clean; 46 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 685 KiB/s rd, 4.0 MiB/s wr, 314 op/s 2024-03-23T08:22:32.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:31 smithi070 ceph-mon[39699]: pgmap v237: 65 pgs: 65 active+clean; 46 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 685 KiB/s rd, 4.0 MiB/s wr, 314 op/s 2024-03-23T08:22:34.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:33 smithi193 ceph-mon[48370]: pgmap v238: 65 pgs: 65 active+clean; 48 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-03-23T08:22:34.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:34.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:22:34.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:22:34.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:22:34.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:33 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:22:34.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:33 smithi070 ceph-mon[39699]: pgmap v238: 65 pgs: 65 active+clean; 48 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-03-23T08:22:34.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:34.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:22:34.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:22:34.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:22:34.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:33 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:22:35.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:35 smithi193 ceph-mon[48370]: Upgrade: Updating mgr.smithi193.zgpdbu 2024-03-23T08:22:35.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:35 smithi193 ceph-mon[48370]: Deploying daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:22:35.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:35 smithi070 ceph-mon[39699]: Upgrade: Updating mgr.smithi193.zgpdbu 2024-03-23T08:22:35.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:35 smithi070 ceph-mon[39699]: Deploying daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:22:36.250 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:36 smithi193 ceph-mon[48370]: pgmap v239: 65 pgs: 65 active+clean; 59 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 447 op/s 2024-03-23T08:22:36.487 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:36 smithi070 ceph-mon[39699]: pgmap v239: 65 pgs: 65 active+clean; 59 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 447 op/s 2024-03-23T08:22:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.939+0000 7fc1217dd700 1 -- 172.21.15.70:0/2900088539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 msgr2=0x7fc11c0fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.939+0000 7fc1217dd700 1 --2- 172.21.15.70:0/2900088539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 0x7fc11c0fe4b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc110004940 tx=0x7fc110009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.940+0000 7fc1217dd700 1 -- 172.21.15.70:0/2900088539 shutdown_connections 2024-03-23T08:22:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.940+0000 7fc1217dd700 1 --2- 172.21.15.70:0/2900088539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 0x7fc11c0fe4b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.940+0000 7fc1217dd700 1 --2- 172.21.15.70:0/2900088539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc11c0fce40 0x7fc11c0fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.940+0000 7fc1217dd700 1 -- 172.21.15.70:0/2900088539 >> 172.21.15.70:0/2900088539 conn(0x7fc11c0f8570 msgr2=0x7fc11c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.940+0000 7fc1217dd700 1 -- 172.21.15.70:0/2900088539 shutdown_connections 2024-03-23T08:22:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.940+0000 7fc1217dd700 1 -- 172.21.15.70:0/2900088539 wait complete. 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc1217dd700 1 Processor -- start 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc1217dd700 1 -- start start 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc1217dd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc11c0fce40 0x7fc11c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc1217dd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 0x7fc11c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc1217dd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0708c0 con 0x7fc11c0fe040 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc1217dd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c070a30 con 0x7fc11c0fce40 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc11a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 0x7fc11c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc11affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc11c0fce40 0x7fc11c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc11a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 0x7fc11c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52014/0 (socket says 172.21.15.70:52014) 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.941+0000 7fc11a7fc700 1 -- 172.21.15.70:0/3373993920 learned_addr learned my addr 172.21.15.70:0/3373993920 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.942+0000 7fc11a7fc700 1 -- 172.21.15.70:0/3373993920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc11c0fce40 msgr2=0x7fc11c071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.942+0000 7fc11a7fc700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc11c0fce40 0x7fc11c071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:36.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.942+0000 7fc11a7fc700 1 -- 172.21.15.70:0/3373993920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110005040 con 0x7fc11c0fe040 2024-03-23T08:22:36.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.942+0000 7fc11affd700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc11c0fce40 0x7fc11c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T08:22:36.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.942+0000 7fc11a7fc700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 0x7fc11c070310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc1100137b0 tx=0x7fc110009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:36.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.942+0000 7fc1037fe700 1 -- 172.21.15.70:0/3373993920 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11000e070 con 0x7fc11c0fe040 2024-03-23T08:22:36.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.942+0000 7fc1037fe700 1 -- 172.21.15.70:0/3373993920 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc11001c400 con 0x7fc11c0fe040 2024-03-23T08:22:36.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.942+0000 7fc1037fe700 1 -- 172.21.15.70:0/3373993920 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11000b650 con 0x7fc11c0fe040 2024-03-23T08:22:36.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.942+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c070cb0 con 0x7fc11c0fe040 2024-03-23T08:22:36.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.943+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c1158f0 con 0x7fc11c0fe040 2024-03-23T08:22:36.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.944+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c05f5f0 con 0x7fc11c0fe040 2024-03-23T08:22:36.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.944+0000 7fc1037fe700 1 -- 172.21.15.70:0/3373993920 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc11001c9e0 con 0x7fc11c0fe040 2024-03-23T08:22:36.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.945+0000 7fc1037fe700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fc1040692a0 0x7fc10406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:36.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.945+0000 7fc1037fe700 1 -- 172.21.15.70:0/3373993920 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc110088720 con 0x7fc11c0fe040 2024-03-23T08:22:36.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.945+0000 7fc11affd700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fc1040692a0 0x7fc10406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:36.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.945+0000 7fc11affd700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fc1040692a0 0x7fc10406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc11c071bd0 tx=0x7fc10c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:36.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:36.951+0000 7fc1037fe700 1 -- 172.21.15.70:0/3373993920 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc110054aa0 con 0x7fc11c0fe040 2024-03-23T08:22:37.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.230+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc11c1023a0 con 0x7fc1040692a0 2024-03-23T08:22:37.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.231+0000 7fc1037fe700 1 -- 172.21.15.70:0/3373993920 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fc11c1023a0 con 0x7fc1040692a0 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.234+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fc1040692a0 msgr2=0x7fc10406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.234+0000 7fc1217dd700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fc1040692a0 0x7fc10406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc11c071bd0 tx=0x7fc10c00b040 comp rx=0 tx=0).stop 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.234+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 msgr2=0x7fc11c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.234+0000 7fc1217dd700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 0x7fc11c070310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc1100137b0 tx=0x7fc110009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.235+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 shutdown_connections 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.235+0000 7fc1217dd700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc11c0fce40 0x7fc11c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.235+0000 7fc1217dd700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fc1040692a0 0x7fc10406b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.235+0000 7fc1217dd700 1 --2- 172.21.15.70:0/3373993920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc11c0fe040 0x7fc11c070310 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.235+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 >> 172.21.15.70:0/3373993920 conn(0x7fc11c0f8570 msgr2=0x7fc11c0f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.235+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 shutdown_connections 2024-03-23T08:22:37.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.235+0000 7fc1217dd700 1 -- 172.21.15.70:0/3373993920 wait complete. 2024-03-23T08:22:37.249 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:22:37.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.378+0000 7fe8ec942700 1 -- 172.21.15.70:0/2733120661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e80fcc00 msgr2=0x7fe8e80fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:37.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.378+0000 7fe8ec942700 1 --2- 172.21.15.70:0/2733120661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e80fcc00 0x7fe8e80fd070 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc004940 tx=0x7fe8dc009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:37.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.378+0000 7fe8ec942700 1 -- 172.21.15.70:0/2733120661 shutdown_connections 2024-03-23T08:22:37.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.378+0000 7fe8ec942700 1 --2- 172.21.15.70:0/2733120661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e80fcc00 0x7fe8e80fd070 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.378+0000 7fe8ec942700 1 --2- 172.21.15.70:0/2733120661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8e80fc2b0 0x7fe8e80fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.378+0000 7fe8ec942700 1 -- 172.21.15.70:0/2733120661 >> 172.21.15.70:0/2733120661 conn(0x7fe8e80f6ba0 msgr2=0x7fe8e80f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:37.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.379+0000 7fe8ec942700 1 -- 172.21.15.70:0/2733120661 shutdown_connections 2024-03-23T08:22:37.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.379+0000 7fe8ec942700 1 -- 172.21.15.70:0/2733120661 wait complete. 2024-03-23T08:22:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.379+0000 7fe8ec942700 1 Processor -- start 2024-03-23T08:22:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.379+0000 7fe8ec942700 1 -- start start 2024-03-23T08:22:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8ec942700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e80fc2b0 0x7fe8e810d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8ec942700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8e80fcc00 0x7fe8e810de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8ec942700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e810e400 con 0x7fe8e80fc2b0 2024-03-23T08:22:37.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8ec942700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e810ec50 con 0x7fe8e80fcc00 2024-03-23T08:22:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8e5d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8e80fcc00 0x7fe8e810de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8e5d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8e80fcc00 0x7fe8e810de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:51748/0 (socket says 172.21.15.70:51748) 2024-03-23T08:22:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8e5d9b700 1 -- 172.21.15.70:0/304076262 learned_addr learned my addr 172.21.15.70:0/304076262 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8e659c700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e80fc2b0 0x7fe8e810d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8e5d9b700 1 -- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e80fc2b0 msgr2=0x7fe8e810d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8e5d9b700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e80fc2b0 0x7fe8e810d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8e5d9b700 1 -- 172.21.15.70:0/304076262 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc005040 con 0x7fe8e80fcc00 2024-03-23T08:22:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.380+0000 7fe8e5d9b700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8e80fcc00 0x7fe8e810de00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0137b0 tx=0x7fe8dc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.381+0000 7fe8d6ffd700 1 -- 172.21.15.70:0/304076262 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8dc00e030 con 0x7fe8e80fcc00 2024-03-23T08:22:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.381+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e810eed0 con 0x7fe8e80fcc00 2024-03-23T08:22:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.381+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e810f410 con 0x7fe8e80fcc00 2024-03-23T08:22:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.381+0000 7fe8d6ffd700 1 -- 172.21.15.70:0/304076262 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe8dc01c400 con 0x7fe8e80fcc00 2024-03-23T08:22:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.381+0000 7fe8d6ffd700 1 -- 172.21.15.70:0/304076262 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8dc012600 con 0x7fe8e80fcc00 2024-03-23T08:22:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.382+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c80052e0 con 0x7fe8e80fcc00 2024-03-23T08:22:37.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.384+0000 7fe8d6ffd700 1 -- 172.21.15.70:0/304076262 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe8dc013a30 con 0x7fe8e80fcc00 2024-03-23T08:22:37.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.384+0000 7fe8d6ffd700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe8d0069380 0x7fe8d006b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:37.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.384+0000 7fe8d6ffd700 1 -- 172.21.15.70:0/304076262 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe8dc088b70 con 0x7fe8e80fcc00 2024-03-23T08:22:37.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.384+0000 7fe8e659c700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe8d0069380 0x7fe8d006b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:37.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.386+0000 7fe8e659c700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe8d0069380 0x7fe8d006b830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8004600 tx=0x7fe8d800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:37.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.390+0000 7fe8d6ffd700 1 -- 172.21.15.70:0/304076262 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe8dc0588c0 con 0x7fe8e80fcc00 2024-03-23T08:22:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.732+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8c8000bf0 con 0x7fe8d0069380 2024-03-23T08:22:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.733+0000 7fe8d6ffd700 1 -- 172.21.15.70:0/304076262 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe8c8000bf0 con 0x7fe8d0069380 2024-03-23T08:22:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.736+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe8d0069380 msgr2=0x7fe8d006b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.736+0000 7fe8ec942700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe8d0069380 0x7fe8d006b830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8004600 tx=0x7fe8d800b040 comp rx=0 tx=0).stop 2024-03-23T08:22:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.736+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8e80fcc00 msgr2=0x7fe8e810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.736+0000 7fe8ec942700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8e80fcc00 0x7fe8e810de00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0137b0 tx=0x7fe8dc009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.737+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 shutdown_connections 2024-03-23T08:22:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.737+0000 7fe8ec942700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7fe8d0069380 0x7fe8d006b830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.737+0000 7fe8ec942700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e80fc2b0 0x7fe8e810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.737+0000 7fe8ec942700 1 --2- 172.21.15.70:0/304076262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8e80fcc00 0x7fe8e810de00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.737+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 >> 172.21.15.70:0/304076262 conn(0x7fe8e80f6ba0 msgr2=0x7fe8e80f8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.737+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 shutdown_connections 2024-03-23T08:22:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.737+0000 7fe8ec942700 1 -- 172.21.15.70:0/304076262 wait complete. 2024-03-23T08:22:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.885+0000 7f58ac38f700 1 -- 172.21.15.70:0/3828072525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a40fcf80 msgr2=0x7f58a40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.885+0000 7f58ac38f700 1 --2- 172.21.15.70:0/3828072525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a40fcf80 0x7f58a40fd3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5894004940 tx=0x7f5894009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.886+0000 7f58ac38f700 1 -- 172.21.15.70:0/3828072525 shutdown_connections 2024-03-23T08:22:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.886+0000 7f58ac38f700 1 --2- 172.21.15.70:0/3828072525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a40fcf80 0x7f58a40fd3f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.886+0000 7f58ac38f700 1 --2- 172.21.15.70:0/3828072525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a40fbcd0 0x7f58a40fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.886+0000 7f58ac38f700 1 -- 172.21.15.70:0/3828072525 >> 172.21.15.70:0/3828072525 conn(0x7f58a40f7450 msgr2=0x7f58a40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.886+0000 7f58ac38f700 1 -- 172.21.15.70:0/3828072525 shutdown_connections 2024-03-23T08:22:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.886+0000 7f58ac38f700 1 -- 172.21.15.70:0/3828072525 wait complete. 2024-03-23T08:22:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.886+0000 7f58ac38f700 1 Processor -- start 2024-03-23T08:22:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.887+0000 7f58ac38f700 1 -- start start 2024-03-23T08:22:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.887+0000 7f58ac38f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a40fbcd0 0x7f58a4113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.887+0000 7f58ac38f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a40fcf80 0x7f58a41143b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.887+0000 7f58ac38f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a41149b0 con 0x7f58a40fcf80 2024-03-23T08:22:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.887+0000 7f58ac38f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4114b20 con 0x7f58a40fbcd0 2024-03-23T08:22:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.887+0000 7f58a992a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a40fcf80 0x7f58a41143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.887+0000 7f58a992a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a40fcf80 0x7f58a41143b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52066/0 (socket says 172.21.15.70:52066) 2024-03-23T08:22:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.887+0000 7f58a992a700 1 -- 172.21.15.70:0/2897690171 learned_addr learned my addr 172.21.15.70:0/2897690171 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.887+0000 7f58aa12b700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a40fbcd0 0x7f58a4113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.888+0000 7f58aa12b700 1 -- 172.21.15.70:0/2897690171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a40fcf80 msgr2=0x7f58a41143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.888+0000 7f58aa12b700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a40fcf80 0x7f58a41143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.888+0000 7f58aa12b700 1 -- 172.21.15.70:0/2897690171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5894005040 con 0x7f58a40fbcd0 2024-03-23T08:22:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.888+0000 7f58aa12b700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a40fbcd0 0x7f58a4113e70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009fd0 tx=0x7f58a000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.888+0000 7f589affd700 1 -- 172.21.15.70:0/2897690171 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a0004e90 con 0x7f58a40fbcd0 2024-03-23T08:22:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.888+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a41194d0 con 0x7f58a40fbcd0 2024-03-23T08:22:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.888+0000 7f589affd700 1 -- 172.21.15.70:0/2897690171 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58a000f040 con 0x7f58a40fbcd0 2024-03-23T08:22:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.888+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a4119a90 con 0x7f58a40fbcd0 2024-03-23T08:22:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.888+0000 7f589affd700 1 -- 172.21.15.70:0/2897690171 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a000c910 con 0x7f58a40fbcd0 2024-03-23T08:22:37.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.890+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58880052b0 con 0x7f58a40fbcd0 2024-03-23T08:22:37.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.890+0000 7f589affd700 1 -- 172.21.15.70:0/2897690171 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f58a0007cc0 con 0x7f58a40fbcd0 2024-03-23T08:22:37.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.891+0000 7f589affd700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f58900692a0 0x7f589006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:37.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.891+0000 7f589affd700 1 -- 172.21.15.70:0/2897690171 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f58a0086d70 con 0x7f58a40fbcd0 2024-03-23T08:22:37.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.891+0000 7f58a992a700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f58900692a0 0x7f589006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:37.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.892+0000 7f58a992a700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f58900692a0 0x7f589006b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5894013680 tx=0x7f5894009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:37.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:37.897+0000 7f589affd700 1 -- 172.21.15.70:0/2897690171 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f58a00569c0 con 0x7f58a40fbcd0 2024-03-23T08:22:38.003 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:38 smithi193 ceph-mon[48370]: pgmap v240: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-03-23T08:22:38.004 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:38.004 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:38.004 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:38 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.220+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5888000bc0 con 0x7f58900692a0 2024-03-23T08:22:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.232+0000 7f589affd700 1 -- 172.21.15.70:0/2897690171 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f5888000bc0 con 0x7f58900692a0 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (6m) 92s ago 7m 26.3M - 0.25.0 c8568f914cd2 700168bef09e 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (7m) 92s ago 7m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (6m) 99s ago 6m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (6m) 92s ago 7m 86.0M - 9.4.7 954c08fa6188 87fc95a6ce8f 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (111s) 92s ago 110s 16.3M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (117s) 92s ago 116s 10.4M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (108s) 99s ago 108s 11.2M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (114s) 99s ago 114s 12.1M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:9283 running (8m) 92s ago 8m 480M - 17.2.7-726-g9c4e0c34 837124da804f 85d0a92e59bc 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283 starting - - - - 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (8m) 92s ago 9m 48.7M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (6m) 99s ago 6m 35.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:22:38.233 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 92s ago 7m 18.6M - 1.5.0 0da6a335fe13 a14314a29bca 2024-03-23T08:22:38.234 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (6m) 99s ago 6m 18.7M - 1.5.0 0da6a335fe13 4197d838b05a 2024-03-23T08:22:38.234 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (5m) 92s ago 5m 45.9M 4299M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:22:38.234 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (5m) 92s ago 5m 45.7M 4299M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:22:38.234 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (4m) 92s ago 4m 46.4M 4299M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:22:38.234 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (4m) 99s ago 4m 47.1M 5323M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:22:38.234 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (3m) 99s ago 3m 40.7M 5323M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:22:38.234 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (3m) 99s ago 3m 44.1M 5323M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:22:38.234 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (5m) 92s ago 6m 53.8M - 2.43.0 a07b618ecd1d e842f03864f0 2024-03-23T08:22:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.235+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f58900692a0 msgr2=0x7f589006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.235+0000 7f58ac38f700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f58900692a0 0x7f589006b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5894013680 tx=0x7f5894009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.235+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a40fbcd0 msgr2=0x7f58a4113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.235+0000 7f58ac38f700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a40fbcd0 0x7f58a4113e70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009fd0 tx=0x7f58a000b040 comp rx=0 tx=0).stop 2024-03-23T08:22:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.235+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 shutdown_connections 2024-03-23T08:22:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.235+0000 7f58ac38f700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a40fbcd0 0x7f58a4113e70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.235+0000 7f58ac38f700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f58900692a0 0x7f589006b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.235+0000 7f58ac38f700 1 --2- 172.21.15.70:0/2897690171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a40fcf80 0x7f58a41143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.235+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 >> 172.21.15.70:0/2897690171 conn(0x7f58a40f7450 msgr2=0x7f58a40f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.236+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 shutdown_connections 2024-03-23T08:22:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.236+0000 7f58ac38f700 1 -- 172.21.15.70:0/2897690171 wait complete. 2024-03-23T08:22:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.372+0000 7f925819b700 1 -- 172.21.15.70:0/1575874755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92500fbce0 msgr2=0x7f92500fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.372+0000 7f925819b700 1 --2- 172.21.15.70:0/1575874755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92500fbce0 0x7f92500fc0f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f924c004ef0 tx=0x7f924c004990 comp rx=0 tx=0).stop 2024-03-23T08:22:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 -- 172.21.15.70:0/1575874755 shutdown_connections 2024-03-23T08:22:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 --2- 172.21.15.70:0/1575874755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92500fcf90 0x7f92500fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 --2- 172.21.15.70:0/1575874755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92500fbce0 0x7f92500fc0f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 -- 172.21.15.70:0/1575874755 >> 172.21.15.70:0/1575874755 conn(0x7f92500f7480 msgr2=0x7f92500f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 -- 172.21.15.70:0/1575874755 shutdown_connections 2024-03-23T08:22:38.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 -- 172.21.15.70:0/1575874755 wait complete. 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 Processor -- start 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 -- start start 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92500fbce0 0x7f9250067d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92500fcf90 0x7f92500682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9250068920 con 0x7f92500fcf90 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.373+0000 7f925819b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9250069170 con 0x7f92500fbce0 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f9255736700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92500fcf90 0x7f92500682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f9255736700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92500fcf90 0x7f92500682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60662/0 (socket says 172.21.15.70:60662) 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f9255736700 1 -- 172.21.15.70:0/885449467 learned_addr learned my addr 172.21.15.70:0/885449467 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f9255f37700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92500fbce0 0x7f9250067d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f9255736700 1 -- 172.21.15.70:0/885449467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92500fbce0 msgr2=0x7f9250067d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f9255736700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92500fbce0 0x7f9250067d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f9255736700 1 -- 172.21.15.70:0/885449467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c005040 con 0x7f92500fcf90 2024-03-23T08:22:38.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f9255736700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92500fcf90 0x7f92500682d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9240002a80 tx=0x7f92400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f92467fc700 1 -- 172.21.15.70:0/885449467 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9240007cc0 con 0x7f92500fcf90 2024-03-23T08:22:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9250069450 con 0x7f92500fcf90 2024-03-23T08:22:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f92467fc700 1 -- 172.21.15.70:0/885449467 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f924000f040 con 0x7f92500fcf90 2024-03-23T08:22:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f92467fc700 1 -- 172.21.15.70:0/885449467 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f924000da60 con 0x7f92500fcf90 2024-03-23T08:22:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.374+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9250069a10 con 0x7f92500fcf90 2024-03-23T08:22:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.376+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925005f5f0 con 0x7f92500fcf90 2024-03-23T08:22:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.377+0000 7f92467fc700 1 -- 172.21.15.70:0/885449467 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f924000dbc0 con 0x7f92500fcf90 2024-03-23T08:22:38.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.377+0000 7f92467fc700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f923c069300 0x7f923c06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:38.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.377+0000 7f92467fc700 1 -- 172.21.15.70:0/885449467 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9240086820 con 0x7f92500fcf90 2024-03-23T08:22:38.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.377+0000 7f9255f37700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f923c069300 0x7f923c06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:38.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.378+0000 7f9255f37700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f923c069300 0x7f923c06b7b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f924c000c00 tx=0x7f924c004990 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:38.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.382+0000 7f92467fc700 1 -- 172.21.15.70:0/885449467 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f92400529f0 con 0x7f92500fcf90 2024-03-23T08:22:38.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:38 smithi070 ceph-mon[39699]: pgmap v240: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-03-23T08:22:38.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:38.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:38.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:38 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:38.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.828+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9250103580 con 0x7f92500fcf90 2024-03-23T08:22:38.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.829+0000 7f92467fc700 1 -- 172.21.15.70:0/885449467 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f9240056010 con 0x7f92500fcf90 2024-03-23T08:22:38.829 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:22:38.829 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:22:38.829 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:22:38.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:22:38.829 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:22:38.830 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:22:38.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f923c069300 msgr2=0x7f923c06b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:38.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f923c069300 0x7f923c06b7b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f924c000c00 tx=0x7f924c004990 comp rx=0 tx=0).stop 2024-03-23T08:22:38.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92500fcf90 msgr2=0x7f92500682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:38.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92500fcf90 0x7f92500682d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9240002a80 tx=0x7f92400046b0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 shutdown_connections 2024-03-23T08:22:38.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92500fbce0 0x7f9250067d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f923c069300 0x7f923c06b7b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 --2- 172.21.15.70:0/885449467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92500fcf90 0x7f92500682d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 >> 172.21.15.70:0/885449467 conn(0x7f92500f7480 msgr2=0x7f92500f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:38.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 shutdown_connections 2024-03-23T08:22:38.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.832+0000 7f925819b700 1 -- 172.21.15.70:0/885449467 wait complete. 2024-03-23T08:22:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.981+0000 7f88fc940700 1 -- 172.21.15.70:0/4201317279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f80fcaf0 msgr2=0x7f88f80fac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.981+0000 7f88fc940700 1 --2- 172.21.15.70:0/4201317279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f80fcaf0 0x7f88f80fac20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f88ec004940 tx=0x7f88ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.981+0000 7f88fc940700 1 -- 172.21.15.70:0/4201317279 shutdown_connections 2024-03-23T08:22:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.981+0000 7f88fc940700 1 --2- 172.21.15.70:0/4201317279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f80fcaf0 0x7f88f80fac20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.981+0000 7f88fc940700 1 --2- 172.21.15.70:0/4201317279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88f80fc110 0x7f88f80fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.981+0000 7f88fc940700 1 -- 172.21.15.70:0/4201317279 >> 172.21.15.70:0/4201317279 conn(0x7f88f80f67a0 msgr2=0x7f88f80f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.982+0000 7f88fc940700 1 -- 172.21.15.70:0/4201317279 shutdown_connections 2024-03-23T08:22:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.982+0000 7f88fc940700 1 -- 172.21.15.70:0/4201317279 wait complete. 2024-03-23T08:22:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.982+0000 7f88fc940700 1 Processor -- start 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.982+0000 7f88fc940700 1 -- start start 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.982+0000 7f88fc940700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88f80fc110 0x7f88f8113e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.982+0000 7f88fc940700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f8114340 0x7f88f8119370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.982+0000 7f88fc940700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f8114800 con 0x7f88f8114340 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.982+0000 7f88fc940700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f8114940 con 0x7f88f80fc110 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.983+0000 7f88f5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f8114340 0x7f88f8119370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.983+0000 7f88f5d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f8114340 0x7f88f8119370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60674/0 (socket says 172.21.15.70:60674) 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.983+0000 7f88f5d9b700 1 -- 172.21.15.70:0/4194289089 learned_addr learned my addr 172.21.15.70:0/4194289089 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.983+0000 7f88f659c700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88f80fc110 0x7f88f8113e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.983+0000 7f88f5d9b700 1 -- 172.21.15.70:0/4194289089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88f80fc110 msgr2=0x7f88f8113e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.983+0000 7f88f5d9b700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88f80fc110 0x7f88f8113e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.983+0000 7f88f5d9b700 1 -- 172.21.15.70:0/4194289089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec005040 con 0x7f88f8114340 2024-03-23T08:22:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.983+0000 7f88f5d9b700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f8114340 0x7f88f8119370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f88ec000c00 tx=0x7f88ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.984+0000 7f88e6ffd700 1 -- 172.21.15.70:0/4194289089 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88ec00e070 con 0x7f88f8114340 2024-03-23T08:22:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.984+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f80679e0 con 0x7f88f8114340 2024-03-23T08:22:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.984+0000 7f88e6ffd700 1 -- 172.21.15.70:0/4194289089 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f88ec01c400 con 0x7f88f8114340 2024-03-23T08:22:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.984+0000 7f88e6ffd700 1 -- 172.21.15.70:0/4194289089 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88ec00b650 con 0x7f88f8114340 2024-03-23T08:22:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.984+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f8067ef0 con 0x7f88f8114340 2024-03-23T08:22:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.985+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f804c710 con 0x7f88f8114340 2024-03-23T08:22:38.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.986+0000 7f88e6ffd700 1 -- 172.21.15.70:0/4194289089 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f88ec01c570 con 0x7f88f8114340 2024-03-23T08:22:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.987+0000 7f88e6ffd700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f88e00692a0 0x7f88e006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:22:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.987+0000 7f88e6ffd700 1 -- 172.21.15.70:0/4194289089 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f88ec088790 con 0x7f88f8114340 2024-03-23T08:22:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.987+0000 7f88f659c700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f88e00692a0 0x7f88e006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:22:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.987+0000 7f88f659c700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f88e00692a0 0x7f88e006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009b80 tx=0x7f88e800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:22:38.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:38.993+0000 7f88e6ffd700 1 -- 172.21.15.70:0/4194289089 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f88ec0584e0 con 0x7f88f8114340 2024-03-23T08:22:39.232 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:39 smithi193 ceph-mon[48370]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:39.232 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:39 smithi193 ceph-mon[48370]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:39.232 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:39 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/885449467' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:22:39.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.286+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 --> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f88f8100280 con 0x7f88e00692a0 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.288+0000 7f88e6ffd700 1 -- 172.21.15.70:0/4194289089 <== mgr.14206 v2:172.21.15.70:6800/3694824274 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f88f8100280 con 0x7f88e00692a0 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "", 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mgr daemons", 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:22:39.288 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f88e00692a0 msgr2=0x7f88e006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f88e00692a0 0x7f88e006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009b80 tx=0x7f88e800b040 comp rx=0 tx=0).stop 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f8114340 msgr2=0x7f88f8119370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f8114340 0x7f88f8119370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f88ec000c00 tx=0x7f88ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 shutdown_connections 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88f80fc110 0x7f88f8113e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:6800/3694824274,v1:172.21.15.70:6801/3694824274] conn(0x7f88e00692a0 0x7f88e006b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 --2- 172.21.15.70:0/4194289089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88f8114340 0x7f88f8119370 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 >> 172.21.15.70:0/4194289089 conn(0x7f88f80f67a0 msgr2=0x7f88f80f7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 shutdown_connections 2024-03-23T08:22:39.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:22:39.290+0000 7f88fc940700 1 -- 172.21.15.70:0/4194289089 wait complete. 2024-03-23T08:22:39.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:39 smithi070 ceph-mon[39699]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:39.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:39 smithi070 ceph-mon[39699]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:39.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:39 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/885449467' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:22:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:40 smithi193 ceph-mon[48370]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:40 smithi193 ceph-mon[48370]: pgmap v241: 65 pgs: 65 active+clean; 77 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 455 op/s 2024-03-23T08:22:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:40 smithi070 ceph-mon[39699]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:40.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:40 smithi070 ceph-mon[39699]: pgmap v241: 65 pgs: 65 active+clean; 77 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 455 op/s 2024-03-23T08:22:41.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:41 smithi193 ceph-mon[48370]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:41.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:41 smithi070 ceph-mon[39699]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:22:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:42 smithi193 ceph-mon[48370]: pgmap v242: 65 pgs: 65 active+clean; 93 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2024-03-23T08:22:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:42 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:42 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:42.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:42 smithi070 ceph-mon[39699]: pgmap v242: 65 pgs: 65 active+clean; 93 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2024-03-23T08:22:42.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:42 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:42.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:42 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:43.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:43 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:43.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:43 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:43.482 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:43 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:43.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:43 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:44.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:44 smithi070 ceph-mon[39699]: pgmap v243: 65 pgs: 65 active+clean; 95 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-03-23T08:22:44.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:44 smithi193 ceph-mon[48370]: pgmap v243: 65 pgs: 65 active+clean; 95 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-03-23T08:22:45.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:45 smithi070 ceph-mon[39699]: pgmap v244: 65 pgs: 65 active+clean; 103 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 505 op/s 2024-03-23T08:22:45.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:45 smithi193 ceph-mon[48370]: pgmap v244: 65 pgs: 65 active+clean; 103 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 505 op/s 2024-03-23T08:22:47.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: Standby manager daemon smithi193.zgpdbu restarted 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:22:47.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr fail", "who": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: osdmap e49: 6 total, 6 up, 6 in 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "mgr fail", "who": "smithi070.jhsbch"}]': finished 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: mgrmap e20: smithi193.zgpdbu(active, starting, since 0.00627126s) 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr metadata", "who": "smithi193.zgpdbu", "id": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:22:47.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: Manager daemon smithi193.zgpdbu is now available 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.zgpdbu/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.zgpdbu/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.zgpdbu/trash_purge_schedule"}]: dispatch 2024-03-23T08:22:47.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:47 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.zgpdbu/trash_purge_schedule"}]: dispatch 2024-03-23T08:22:47.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:47.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:47.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:22:47.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: Standby manager daemon smithi193.zgpdbu restarted 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr fail", "who": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: osdmap e49: 6 total, 6 up, 6 in 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.14206 172.21.15.70:0/2938809469' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "mgr fail", "who": "smithi070.jhsbch"}]': finished 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: mgrmap e20: smithi193.zgpdbu(active, starting, since 0.00627126s) 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:22:47.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr metadata", "who": "smithi193.zgpdbu", "id": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:22:47.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: Manager daemon smithi193.zgpdbu is now available 2024-03-23T08:22:47.578 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:47.578 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:22:47.578 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:22:47.578 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.zgpdbu/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:22:47.578 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.zgpdbu/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:22:47.578 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.zgpdbu/trash_purge_schedule"}]: dispatch 2024-03-23T08:22:47.578 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:47 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.zgpdbu/trash_purge_schedule"}]: dispatch 2024-03-23T08:22:48.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:48 smithi070 ceph-mon[39699]: Deploying cephadm binary to smithi070 2024-03-23T08:22:48.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:48 smithi070 ceph-mon[39699]: Deploying cephadm binary to smithi193 2024-03-23T08:22:48.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:48 smithi070 ceph-mon[39699]: mgrmap e21: smithi193.zgpdbu(active, since 1.00849s) 2024-03-23T08:22:48.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:48 smithi070 ceph-mon[39699]: pgmap v3: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:22:48.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:48 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:48.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:48 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:48.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:48 smithi193 ceph-mon[48370]: Deploying cephadm binary to smithi070 2024-03-23T08:22:48.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:48 smithi193 ceph-mon[48370]: Deploying cephadm binary to smithi193 2024-03-23T08:22:48.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:48 smithi193 ceph-mon[48370]: mgrmap e21: smithi193.zgpdbu(active, since 1.00849s) 2024-03-23T08:22:48.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:48 smithi193 ceph-mon[48370]: pgmap v3: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:22:48.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:48 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:48.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:48 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:49.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:49 smithi193 ceph-mon[48370]: pgmap v4: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:22:49.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:49 smithi193 ceph-mon[48370]: mgrmap e22: smithi193.zgpdbu(active, since 2s) 2024-03-23T08:22:49.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:49 smithi193 ceph-mon[48370]: [23/Mar/2024:08:22:48] ENGINE Bus STARTING 2024-03-23T08:22:49.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:49 smithi193 ceph-mon[48370]: [23/Mar/2024:08:22:48] ENGINE Serving on http://172.21.15.193:8765 2024-03-23T08:22:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:49 smithi070 ceph-mon[39699]: pgmap v4: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:22:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:49 smithi070 ceph-mon[39699]: mgrmap e22: smithi193.zgpdbu(active, since 2s) 2024-03-23T08:22:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:49 smithi070 ceph-mon[39699]: [23/Mar/2024:08:22:48] ENGINE Bus STARTING 2024-03-23T08:22:49.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:49 smithi070 ceph-mon[39699]: [23/Mar/2024:08:22:48] ENGINE Serving on http://172.21.15.193:8765 2024-03-23T08:22:50.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:50 smithi070 ceph-mon[39699]: [23/Mar/2024:08:22:48] ENGINE Serving on https://172.21.15.193:7150 2024-03-23T08:22:50.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:50 smithi070 ceph-mon[39699]: [23/Mar/2024:08:22:48] ENGINE Bus STARTED 2024-03-23T08:22:50.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:50 smithi193 ceph-mon[48370]: [23/Mar/2024:08:22:48] ENGINE Serving on https://172.21.15.193:7150 2024-03-23T08:22:50.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:50 smithi193 ceph-mon[48370]: [23/Mar/2024:08:22:48] ENGINE Bus STARTED 2024-03-23T08:22:51.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:51 smithi193 ceph-mon[48370]: pgmap v5: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:22:51.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:51 smithi193 ceph-mon[48370]: mgrmap e23: smithi193.zgpdbu(active, since 4s) 2024-03-23T08:22:51.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:51 smithi070 ceph-mon[39699]: pgmap v5: 65 pgs: 65 active+clean; 117 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:22:51.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:51 smithi070 ceph-mon[39699]: mgrmap e23: smithi193.zgpdbu(active, since 4s) 2024-03-23T08:22:53.626 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:53 smithi193 ceph-mon[48370]: pgmap v6: 65 pgs: 65 active+clean; 124 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 829 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-03-23T08:22:53.626 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:53 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:53.626 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:53 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:53.626 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:53 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:53.626 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:53 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:53 smithi070 ceph-mon[39699]: pgmap v6: 65 pgs: 65 active+clean; 124 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 829 KiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-03-23T08:22:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:53 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:53 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:53 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:53 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:22:55.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:55 smithi070 ceph-mon[39699]: pgmap v7: 65 pgs: 65 active+clean; 133 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 293 op/s 2024-03-23T08:22:55.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:55 smithi070 ceph-mon[39699]: Standby manager daemon smithi070.jhsbch started 2024-03-23T08:22:55.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:55 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.70:0/1118832584' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.jhsbch/crt"}]: dispatch 2024-03-23T08:22:55.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:55 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.70:0/1118832584' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:22:55.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:55 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.70:0/1118832584' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.jhsbch/key"}]: dispatch 2024-03-23T08:22:55.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:55 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.70:0/1118832584' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:22:55.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:55 smithi193 ceph-mon[48370]: pgmap v7: 65 pgs: 65 active+clean; 133 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 293 op/s 2024-03-23T08:22:55.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:55 smithi193 ceph-mon[48370]: Standby manager daemon smithi070.jhsbch started 2024-03-23T08:22:55.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:55 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.70:0/1118832584' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.jhsbch/crt"}]: dispatch 2024-03-23T08:22:55.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:55 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.70:0/1118832584' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:22:55.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:55 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.70:0/1118832584' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.jhsbch/key"}]: dispatch 2024-03-23T08:22:55.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:55 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.70:0/1118832584' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:22:56.716 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:56 smithi193 ceph-mon[48370]: mgrmap e24: smithi193.zgpdbu(active, since 9s), standbys: smithi070.jhsbch 2024-03-23T08:22:56.717 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:56 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:22:56.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:56 smithi070 ceph-mon[39699]: mgrmap e24: smithi193.zgpdbu(active, since 9s), standbys: smithi070.jhsbch 2024-03-23T08:22:56.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:56 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:22:57.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:57 smithi193 ceph-mon[48370]: pgmap v8: 65 pgs: 65 active+clean; 133 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-03-23T08:22:57.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:57 smithi070 ceph-mon[39699]: pgmap v8: 65 pgs: 65 active+clean; 133 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-03-23T08:22:59.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:22:59 smithi070 ceph-mon[39699]: pgmap v9: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-03-23T08:22:59.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:22:59 smithi193 ceph-mon[48370]: pgmap v9: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-03-23T08:23:01.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:01 smithi070 ceph-mon[39699]: pgmap v10: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-03-23T08:23:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:01 smithi193 ceph-mon[48370]: pgmap v10: 65 pgs: 65 active+clean; 153 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 313 op/s 2024-03-23T08:23:02.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:02 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:23:02.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:02 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:02.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:02 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:02 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:23:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:02 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:02 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: pgmap v11: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 393 op/s 2024-03-23T08:23:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: Adjusting osd_memory_target on smithi070 to 1568M 2024-03-23T08:23:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:03 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: pgmap v11: 65 pgs: 65 active+clean; 161 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 393 op/s 2024-03-23T08:23:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: Adjusting osd_memory_target on smithi070 to 1568M 2024-03-23T08:23:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:03 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:05.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:05 smithi070 ceph-mon[39699]: pgmap v12: 65 pgs: 65 active+clean; 169 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 408 op/s 2024-03-23T08:23:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:05 smithi193 ceph-mon[48370]: pgmap v12: 65 pgs: 65 active+clean; 169 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 408 op/s 2024-03-23T08:23:07.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:07 smithi070 ceph-mon[39699]: pgmap v13: 65 pgs: 65 active+clean; 169 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-03-23T08:23:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:07 smithi193 ceph-mon[48370]: pgmap v13: 65 pgs: 65 active+clean; 169 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-03-23T08:23:09.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.442+0000 7fc1e7015700 1 -- 172.21.15.70:0/1626181809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e00fcf80 msgr2=0x7fc1e00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:09.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.442+0000 7fc1e7015700 1 --2- 172.21.15.70:0/1626181809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e00fcf80 0x7fc1e00fd3f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc004880 tx=0x7fc1cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:09.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.442+0000 7fc1e7015700 1 -- 172.21.15.70:0/1626181809 shutdown_connections 2024-03-23T08:23:09.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.442+0000 7fc1e7015700 1 --2- 172.21.15.70:0/1626181809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e00fcf80 0x7fc1e00fd3f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:09.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.442+0000 7fc1e7015700 1 --2- 172.21.15.70:0/1626181809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e00fbcd0 0x7fc1e00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:09.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.442+0000 7fc1e7015700 1 -- 172.21.15.70:0/1626181809 >> 172.21.15.70:0/1626181809 conn(0x7fc1e00f7430 msgr2=0x7fc1e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:09.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.443+0000 7fc1e7015700 1 -- 172.21.15.70:0/1626181809 shutdown_connections 2024-03-23T08:23:09.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.443+0000 7fc1e7015700 1 -- 172.21.15.70:0/1626181809 wait complete. 2024-03-23T08:23:09.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.444+0000 7fc1e7015700 1 Processor -- start 2024-03-23T08:23:09.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.444+0000 7fc1e7015700 1 -- start start 2024-03-23T08:23:09.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.444+0000 7fc1e7015700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e00fbcd0 0x7fc1e0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:09.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.444+0000 7fc1e7015700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e00fcf80 0x7fc1e0114400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:09.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.444+0000 7fc1e7015700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e0114a00 con 0x7fc1e00fcf80 2024-03-23T08:23:09.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.444+0000 7fc1e7015700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e0115250 con 0x7fc1e00fbcd0 2024-03-23T08:23:09.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.444+0000 7fc1dffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e00fcf80 0x7fc1e0114400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:09.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1dffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e00fcf80 0x7fc1e0114400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55946/0 (socket says 172.21.15.70:55946) 2024-03-23T08:23:09.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1dffff700 1 -- 172.21.15.70:0/865580125 learned_addr learned my addr 172.21.15.70:0/865580125 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:09.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1e4db1700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e00fbcd0 0x7fc1e0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:09.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1dffff700 1 -- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e00fbcd0 msgr2=0x7fc1e0113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:09.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1dffff700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e00fbcd0 0x7fc1e0113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:09.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1dffff700 1 -- 172.21.15.70:0/865580125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc005040 con 0x7fc1e00fcf80 2024-03-23T08:23:09.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1dffff700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e00fcf80 0x7fc1e0114400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009c90 tx=0x7fc1cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:09.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1dd7fa700 1 -- 172.21.15.70:0/865580125 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1cc00e030 con 0x7fc1e00fcf80 2024-03-23T08:23:09.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1dd7fa700 1 -- 172.21.15.70:0/865580125 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1cc01c400 con 0x7fc1e00fcf80 2024-03-23T08:23:09.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1dd7fa700 1 -- 172.21.15.70:0/865580125 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1cc012620 con 0x7fc1e00fcf80 2024-03-23T08:23:09.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e01154d0 con 0x7fc1e00fcf80 2024-03-23T08:23:09.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.445+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e01159c0 con 0x7fc1e00fcf80 2024-03-23T08:23:09.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.447+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e0101910 con 0x7fc1e00fcf80 2024-03-23T08:23:09.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.453+0000 7fc1dd7fa700 1 -- 172.21.15.70:0/865580125 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fc1cc013c30 con 0x7fc1e00fcf80 2024-03-23T08:23:09.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.453+0000 7fc1dd7fa700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc1d006aec0 0x7fc1d006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:09.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.453+0000 7fc1dd7fa700 1 -- 172.21.15.70:0/865580125 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fc1cc08b790 con 0x7fc1e00fcf80 2024-03-23T08:23:09.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.453+0000 7fc1e4db1700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc1d006aec0 0x7fc1d006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:09.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.454+0000 7fc1e4db1700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc1d006aec0 0x7fc1d006d370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009b80 tx=0x7fc1d400b040 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:09.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.455+0000 7fc1dd7fa700 1 -- 172.21.15.70:0/865580125 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fc1cc059850 con 0x7fc1e00fcf80 2024-03-23T08:23:09.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.780+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1e0115ca0 con 0x7fc1d006aec0 2024-03-23T08:23:09.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.782+0000 7fc1dd7fa700 1 -- 172.21.15.70:0/865580125 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fc1e0115ca0 con 0x7fc1d006aec0 2024-03-23T08:23:09.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc1d006aec0 msgr2=0x7fc1d006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:09.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc1d006aec0 0x7fc1d006d370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009b80 tx=0x7fc1d400b040 comp rx=0 tx=0).stop 2024-03-23T08:23:09.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e00fcf80 msgr2=0x7fc1e0114400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:09.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e00fcf80 0x7fc1e0114400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009c90 tx=0x7fc1cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:09.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 shutdown_connections 2024-03-23T08:23:09.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc1d006aec0 0x7fc1d006d370 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:09.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1e00fbcd0 0x7fc1e0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:09.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 --2- 172.21.15.70:0/865580125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e00fcf80 0x7fc1e0114400 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:09.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 >> 172.21.15.70:0/865580125 conn(0x7fc1e00f7430 msgr2=0x7fc1e0065ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:09.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 shutdown_connections 2024-03-23T08:23:09.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.784+0000 7fc1e7015700 1 -- 172.21.15.70:0/865580125 wait complete. 2024-03-23T08:23:09.798 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:23:09.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.912+0000 7f96fda2f700 1 -- 172.21.15.70:0/3295928565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fe4a0 msgr2=0x7f96f80fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:09.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.912+0000 7f96fda2f700 1 --2- 172.21.15.70:0/3295928565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fe4a0 0x7f96f80fe8b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f96e8004940 tx=0x7f96e8009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:09.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.912+0000 7f96fda2f700 1 -- 172.21.15.70:0/3295928565 shutdown_connections 2024-03-23T08:23:09.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.912+0000 7f96fda2f700 1 --2- 172.21.15.70:0/3295928565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80fedf0 0x7f96f80faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:09.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.912+0000 7f96fda2f700 1 --2- 172.21.15.70:0/3295928565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fe4a0 0x7f96f80fe8b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:09.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.912+0000 7f96fda2f700 1 -- 172.21.15.70:0/3295928565 >> 172.21.15.70:0/3295928565 conn(0x7f96f80f6ba0 msgr2=0x7f96f80f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:09.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.912+0000 7f96fda2f700 1 -- 172.21.15.70:0/3295928565 shutdown_connections 2024-03-23T08:23:09.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.912+0000 7f96fda2f700 1 -- 172.21.15.70:0/3295928565 wait complete. 2024-03-23T08:23:09.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96fda2f700 1 Processor -- start 2024-03-23T08:23:09.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96fda2f700 1 -- start start 2024-03-23T08:23:09.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96fda2f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80fe4a0 0x7f96f810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96fda2f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fedf0 0x7f96f810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96fda2f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f810e460 con 0x7f96f80fedf0 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96fda2f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f810ecb0 con 0x7f96f80fe4a0 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96f67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fedf0 0x7f96f810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96f67fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fedf0 0x7f96f810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55954/0 (socket says 172.21.15.70:55954) 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96f67fc700 1 -- 172.21.15.70:0/3336190058 learned_addr learned my addr 172.21.15.70:0/3336190058 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96f6ffd700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80fe4a0 0x7f96f810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96f67fc700 1 -- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80fe4a0 msgr2=0x7f96f810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96f67fc700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80fe4a0 0x7f96f810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.913+0000 7f96f67fc700 1 -- 172.21.15.70:0/3336190058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e8005040 con 0x7f96f80fedf0 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.914+0000 7f96f67fc700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fedf0 0x7f96f810de60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f96ec002a50 tx=0x7f96ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:09.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.914+0000 7f96df7fe700 1 -- 172.21.15.70:0/3336190058 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96ec007cc0 con 0x7f96f80fedf0 2024-03-23T08:23:09.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.914+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f810ef90 con 0x7f96f80fedf0 2024-03-23T08:23:09.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.914+0000 7f96df7fe700 1 -- 172.21.15.70:0/3336190058 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f96ec00f040 con 0x7f96f80fedf0 2024-03-23T08:23:09.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.914+0000 7f96df7fe700 1 -- 172.21.15.70:0/3336190058 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96ec00d9c0 con 0x7f96f80fedf0 2024-03-23T08:23:09.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.914+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f810f4d0 con 0x7f96f80fedf0 2024-03-23T08:23:09.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.915+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f805f5f0 con 0x7f96f80fedf0 2024-03-23T08:23:09.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.916+0000 7f96df7fe700 1 -- 172.21.15.70:0/3336190058 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f96ec00db20 con 0x7f96f80fedf0 2024-03-23T08:23:09.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.916+0000 7f96df7fe700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f96e006aec0 0x7f96e006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:09.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.916+0000 7f96df7fe700 1 -- 172.21.15.70:0/3336190058 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f96ec089300 con 0x7f96f80fedf0 2024-03-23T08:23:09.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.917+0000 7f96f6ffd700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f96e006aec0 0x7f96e006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:09.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.917+0000 7f96f6ffd700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f96e006aec0 0x7f96e006d370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009fd0 tx=0x7f96e8009e30 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:09.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:09.920+0000 7f96df7fe700 1 -- 172.21.15.70:0/3336190058 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f96ec0572a0 con 0x7f96f80fedf0 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: pgmap v14: 65 pgs: 65 active+clean; 191 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 482 op/s 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:23:09.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:09 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:23:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: pgmap v14: 65 pgs: 65 active+clean; 191 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 482 op/s 2024-03-23T08:23:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:10.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-23T08:23:10.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:10.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:23:10.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:09 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:23:10.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.265+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96f810f8c0 con 0x7f96e006aec0 2024-03-23T08:23:10.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.267+0000 7f96df7fe700 1 -- 172.21.15.70:0/3336190058 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f96f810f8c0 con 0x7f96e006aec0 2024-03-23T08:23:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.269+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f96e006aec0 msgr2=0x7f96e006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.269+0000 7f96fda2f700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f96e006aec0 0x7f96e006d370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009fd0 tx=0x7f96e8009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.269+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fedf0 msgr2=0x7f96f810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.269+0000 7f96fda2f700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fedf0 0x7f96f810de60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f96ec002a50 tx=0x7f96ec0046b0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.269+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 shutdown_connections 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.269+0000 7f96fda2f700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f96e006aec0 0x7f96e006d370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.269+0000 7f96fda2f700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80fe4a0 0x7f96f810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.269+0000 7f96fda2f700 1 --2- 172.21.15.70:0/3336190058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f80fedf0 0x7f96f810de60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.269+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 >> 172.21.15.70:0/3336190058 conn(0x7f96f80f6ba0 msgr2=0x7f96f80f8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.270+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 shutdown_connections 2024-03-23T08:23:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.270+0000 7f96fda2f700 1 -- 172.21.15.70:0/3336190058 wait complete. 2024-03-23T08:23:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.408+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1336865381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3d40662e0 msgr2=0x7fc3d40ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.408+0000 7fc3dc2ae700 1 --2- 172.21.15.70:0/1336865381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3d40662e0 0x7fc3d40ff9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0004940 tx=0x7fc3d0009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.409+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1336865381 shutdown_connections 2024-03-23T08:23:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.409+0000 7fc3dc2ae700 1 --2- 172.21.15.70:0/1336865381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d40fff00 0x7fc3d4102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.409+0000 7fc3dc2ae700 1 --2- 172.21.15.70:0/1336865381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3d40662e0 0x7fc3d40ff9c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.409+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1336865381 >> 172.21.15.70:0/1336865381 conn(0x7fc3d40f7430 msgr2=0x7fc3d40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.409+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1336865381 shutdown_connections 2024-03-23T08:23:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.409+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1336865381 wait complete. 2024-03-23T08:23:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.409+0000 7fc3dc2ae700 1 Processor -- start 2024-03-23T08:23:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.409+0000 7fc3dc2ae700 1 -- start start 2024-03-23T08:23:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3dc2ae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d40662e0 0x7fc3d40fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3dc2ae700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3d40fff00 0x7fc3d40fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3dc2ae700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40fbe70 con 0x7fc3d40fff00 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3dc2ae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40fbfe0 con 0x7fc3d40662e0 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3da04a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d40662e0 0x7fc3d40fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3da04a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d40662e0 0x7fc3d40fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:58708/0 (socket says 172.21.15.70:58708) 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3da04a700 1 -- 172.21.15.70:0/1353413580 learned_addr learned my addr 172.21.15.70:0/1353413580 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3d9849700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3d40fff00 0x7fc3d40fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3da04a700 1 -- 172.21.15.70:0/1353413580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3d40fff00 msgr2=0x7fc3d40fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3da04a700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3d40fff00 0x7fc3d40fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.410+0000 7fc3da04a700 1 -- 172.21.15.70:0/1353413580 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0005040 con 0x7fc3d40662e0 2024-03-23T08:23:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.411+0000 7fc3da04a700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d40662e0 0x7fc3d40fd3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.411+0000 7fc3caffd700 1 -- 172.21.15.70:0/1353413580 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3d000e070 con 0x7fc3d40662e0 2024-03-23T08:23:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.411+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d40fc260 con 0x7fc3d40662e0 2024-03-23T08:23:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.411+0000 7fc3caffd700 1 -- 172.21.15.70:0/1353413580 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc3d00093b0 con 0x7fc3d40662e0 2024-03-23T08:23:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.411+0000 7fc3caffd700 1 -- 172.21.15.70:0/1353413580 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3d001cd70 con 0x7fc3d40662e0 2024-03-23T08:23:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.411+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d4115920 con 0x7fc3d40662e0 2024-03-23T08:23:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.412+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d405f5f0 con 0x7fc3d40662e0 2024-03-23T08:23:10.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.414+0000 7fc3caffd700 1 -- 172.21.15.70:0/1353413580 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fc3d001c600 con 0x7fc3d40662e0 2024-03-23T08:23:10.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.414+0000 7fc3caffd700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc3c006aec0 0x7fc3c006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:10.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.414+0000 7fc3caffd700 1 -- 172.21.15.70:0/1353413580 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fc3d008b950 con 0x7fc3d40662e0 2024-03-23T08:23:10.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.414+0000 7fc3d9849700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc3c006aec0 0x7fc3c006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:10.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.415+0000 7fc3d9849700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc3c006aec0 0x7fc3c006d370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4004fd0 tx=0x7fc3c400b040 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.419+0000 7fc3caffd700 1 -- 172.21.15.70:0/1353413580 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fc3d0055eb0 con 0x7fc3d40662e0 2024-03-23T08:23:10.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.746+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3d40f8f70 con 0x7fc3c006aec0 2024-03-23T08:23:10.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.756+0000 7fc3caffd700 1 -- 172.21.15.70:0/1353413580 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fc3d40f8f70 con 0x7fc3c006aec0 2024-03-23T08:23:10.756 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:23:10.756 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (6m) 8s ago 7m 26.3M - 0.25.0 c8568f914cd2 700168bef09e 2024-03-23T08:23:10.756 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (8m) 8s ago 8m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (7m) 18s ago 7m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (6m) 8s ago 7m 92.4M - 9.4.7 954c08fa6188 87fc95a6ce8f 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (2m) 8s ago 2m 107M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (2m) 8s ago 2m 11.2M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (2m) 18s ago 2m 69.7M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (2m) 18s ago 2m 12.9M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:9283 running (9m) 8s ago 9m 178M - 17.2.7-726-g9c4e0c34 837124da804f 85d0a92e59bc 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283 running (34s) 18s ago 7m 481M - 18.2.2-954-g82316b04 b24e1baba637 d2827fb230ee 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (9m) 8s ago 9m 51.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (6m) 18s ago 6m 39.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (8m) 8s ago 8m 18.7M - 1.5.0 0da6a335fe13 a14314a29bca 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (7m) 18s ago 7m 18.6M - 1.5.0 0da6a335fe13 4197d838b05a 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (6m) 8s ago 6m 94.5M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (5m) 8s ago 5m 102M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:23:10.757 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (5m) 8s ago 5m 89.0M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:23:10.758 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (4m) 18s ago 4m 111M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:23:10.758 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (4m) 18s ago 4m 95.2M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:23:10.758 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (4m) 18s ago 3m 94.1M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:23:10.758 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (6m) 8s ago 7m 58.2M - 2.43.0 a07b618ecd1d e842f03864f0 2024-03-23T08:23:10.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc3c006aec0 msgr2=0x7fc3c006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc3c006aec0 0x7fc3c006d370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4004fd0 tx=0x7fc3c400b040 comp rx=0 tx=0).stop 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d40662e0 msgr2=0x7fc3d40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d40662e0 0x7fc3d40fd3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d0009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 shutdown_connections 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fc3c006aec0 0x7fc3c006d370 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3d40662e0 0x7fc3d40fd3f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 --2- 172.21.15.70:0/1353413580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3d40fff00 0x7fc3d40fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 >> 172.21.15.70:0/1353413580 conn(0x7fc3d40f7430 msgr2=0x7fc3d40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 shutdown_connections 2024-03-23T08:23:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.758+0000 7fc3dc2ae700 1 -- 172.21.15.70:0/1353413580 wait complete. 2024-03-23T08:23:10.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.879+0000 7f859bc1e700 1 -- 172.21.15.70:0/1924643550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fcf80 msgr2=0x7f85940fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:10.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.879+0000 7f859bc1e700 1 --2- 172.21.15.70:0/1924643550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fcf80 0x7f85940fd3f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8584004940 tx=0x7f8584009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:10.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.879+0000 7f859bc1e700 1 -- 172.21.15.70:0/1924643550 shutdown_connections 2024-03-23T08:23:10.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.879+0000 7f859bc1e700 1 --2- 172.21.15.70:0/1924643550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fcf80 0x7f85940fd3f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.879+0000 7f859bc1e700 1 --2- 172.21.15.70:0/1924643550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85940fbcd0 0x7f85940fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.879+0000 7f859bc1e700 1 -- 172.21.15.70:0/1924643550 >> 172.21.15.70:0/1924643550 conn(0x7f85940f7430 msgr2=0x7f85940f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:10.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.879+0000 7f859bc1e700 1 -- 172.21.15.70:0/1924643550 shutdown_connections 2024-03-23T08:23:10.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.879+0000 7f859bc1e700 1 -- 172.21.15.70:0/1924643550 wait complete. 2024-03-23T08:23:10.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f859bc1e700 1 Processor -- start 2024-03-23T08:23:10.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f859bc1e700 1 -- start start 2024-03-23T08:23:10.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f859bc1e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fbcd0 0x7f85941105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:10.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f859bc1e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85940fcf80 0x7f8594112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f859bc1e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8594066350 con 0x7f85940fbcd0 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f859bc1e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8594066490 con 0x7f85940fcf80 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f85999ba700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fbcd0 0x7f85941105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f85999ba700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fbcd0 0x7f85941105b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55988/0 (socket says 172.21.15.70:55988) 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f85999ba700 1 -- 172.21.15.70:0/2474522488 learned_addr learned my addr 172.21.15.70:0/2474522488 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.880+0000 7f85991b9700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85940fcf80 0x7f8594112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.881+0000 7f85999ba700 1 -- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85940fcf80 msgr2=0x7f8594112b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.881+0000 7f85999ba700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85940fcf80 0x7f8594112b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.881+0000 7f85999ba700 1 -- 172.21.15.70:0/2474522488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8584005040 con 0x7f85940fbcd0 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.881+0000 7f85999ba700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fbcd0 0x7f85941105b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f859000eb40 tx=0x7f859000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.881+0000 7f858a7fc700 1 -- 172.21.15.70:0/2474522488 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8590007cc0 con 0x7f85940fbcd0 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.881+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8594113040 con 0x7f85940fbcd0 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.881+0000 7f858a7fc700 1 -- 172.21.15.70:0/2474522488 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f859000f040 con 0x7f85940fbcd0 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.881+0000 7f858a7fc700 1 -- 172.21.15.70:0/2474522488 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f859000c990 con 0x7f85940fbcd0 2024-03-23T08:23:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.881+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8594113600 con 0x7f85940fbcd0 2024-03-23T08:23:10.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.882+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859405f5f0 con 0x7f85940fbcd0 2024-03-23T08:23:10.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.883+0000 7f858a7fc700 1 -- 172.21.15.70:0/2474522488 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f859000cc00 con 0x7f85940fbcd0 2024-03-23T08:23:10.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.883+0000 7f858a7fc700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f858006aec0 0x7f858006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:10.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.883+0000 7f858a7fc700 1 -- 172.21.15.70:0/2474522488 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8590089870 con 0x7f85940fbcd0 2024-03-23T08:23:10.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.884+0000 7f85991b9700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f858006aec0 0x7f858006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:10.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.884+0000 7f85991b9700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f858006aec0 0x7f858006d370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8584004890 tx=0x7f8584009e30 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:10.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:10.890+0000 7f858a7fc700 1 -- 172.21.15.70:0/2474522488 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f8590053cf0 con 0x7f85940fbcd0 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: Adjusting osd_memory_target on smithi193 to 2592M 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:10 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: Adjusting osd_memory_target on smithi193 to 2592M 2024-03-23T08:23:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:23:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:23:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:23:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:23:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:23:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:10 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.249 INFO:tasks.workunit.client.1.smithi193.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-23T08:23:11.256 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-23T08:23:11.256 INFO:tasks.workunit.client.1.smithi193.stderr:+ make 2024-03-23T08:23:11.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.324+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8594113880 con 0x7f85940fbcd0 2024-03-23T08:23:11.326 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:23:11.326 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-23T08:23:11.327 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:23:11.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:23:11.328 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:23:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.324+0000 7f858a7fc700 1 -- 172.21.15.70:0/2474522488 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+848 (secure 0 0 0) 0x7f8590057310 con 0x7f85940fbcd0 2024-03-23T08:23:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.327+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f858006aec0 msgr2=0x7f858006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.327+0000 7f859bc1e700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f858006aec0 0x7f858006d370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8584004890 tx=0x7f8584009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.327+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fbcd0 msgr2=0x7f85941105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.327+0000 7f859bc1e700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fbcd0 0x7f85941105b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f859000eb40 tx=0x7f859000b040 comp rx=0 tx=0).stop 2024-03-23T08:23:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.327+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 shutdown_connections 2024-03-23T08:23:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.327+0000 7f859bc1e700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f858006aec0 0x7f858006d370 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.327+0000 7f859bc1e700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85940fbcd0 0x7f85941105b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.327+0000 7f859bc1e700 1 --2- 172.21.15.70:0/2474522488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85940fcf80 0x7f8594112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.327+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 >> 172.21.15.70:0/2474522488 conn(0x7f85940f7430 msgr2=0x7f85940f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.328+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 shutdown_connections 2024-03-23T08:23:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.328+0000 7f859bc1e700 1 -- 172.21.15.70:0/2474522488 wait complete. 2024-03-23T08:23:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.484+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1373684864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 msgr2=0x7f81b40fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.484+0000 7f81b8ff8700 1 --2- 172.21.15.70:0/1373684864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 0x7f81b40fae90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f81a8004940 tx=0x7f81a8009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.484+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1373684864 shutdown_connections 2024-03-23T08:23:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.484+0000 7f81b8ff8700 1 --2- 172.21.15.70:0/1373684864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 0x7f81b40fae90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.484+0000 7f81b8ff8700 1 --2- 172.21.15.70:0/1373684864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b40fe400 0x7f81b40fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.484+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1373684864 >> 172.21.15.70:0/1373684864 conn(0x7f81b40f6ba0 msgr2=0x7f81b40f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.484+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1373684864 shutdown_connections 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.485+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1373684864 wait complete. 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.485+0000 7f81b8ff8700 1 Processor -- start 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.485+0000 7f81b8ff8700 1 -- start start 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.485+0000 7f81b8ff8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b40fe400 0x7f81b406e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.485+0000 7f81b8ff8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 0x7f81b406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.485+0000 7f81b8ff8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b406ef00 con 0x7f81b40fed50 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.485+0000 7f81b8ff8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b406f040 con 0x7f81b40fe400 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b1d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 0x7f81b406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b1d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 0x7f81b406e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56010/0 (socket says 172.21.15.70:56010) 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b40fe400 0x7f81b406e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b1d9b700 1 -- 172.21.15.70:0/1457466860 learned_addr learned my addr 172.21.15.70:0/1457466860 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:11.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b259c700 1 -- 172.21.15.70:0/1457466860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 msgr2=0x7f81b406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:11.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b259c700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 0x7f81b406e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:11.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b259c700 1 -- 172.21.15.70:0/1457466860 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a8005040 con 0x7f81b40fe400 2024-03-23T08:23:11.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b1d9b700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 0x7f81b406e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T08:23:11.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b259c700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b40fe400 0x7f81b406e480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009fd0 tx=0x7f81a400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:11.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81a2ffd700 1 -- 172.21.15.70:0/1457466860 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81a4004e90 con 0x7f81b40fe400 2024-03-23T08:23:11.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81a2ffd700 1 -- 172.21.15.70:0/1457466860 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f81a400f040 con 0x7f81b40fe400 2024-03-23T08:23:11.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b406f2c0 con 0x7f81b40fe400 2024-03-23T08:23:11.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.486+0000 7f81a2ffd700 1 -- 172.21.15.70:0/1457466860 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81a400c9b0 con 0x7f81b40fe400 2024-03-23T08:23:11.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.487+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b411be70 con 0x7f81b40fe400 2024-03-23T08:23:11.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.488+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b405f5f0 con 0x7f81b40fe400 2024-03-23T08:23:11.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.488+0000 7f81a2ffd700 1 -- 172.21.15.70:0/1457466860 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f81a400cb10 con 0x7f81b40fe400 2024-03-23T08:23:11.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.488+0000 7f81a2ffd700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f819c06aec0 0x7f819c06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:11.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.489+0000 7f81a2ffd700 1 -- 172.21.15.70:0/1457466860 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f81a40893a0 con 0x7f81b40fe400 2024-03-23T08:23:11.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.489+0000 7f81b1d9b700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f819c06aec0 0x7f819c06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:11.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.489+0000 7f81b1d9b700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f819c06aec0 0x7f819c06d370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f81a8013600 tx=0x7f81a8009e30 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:11.494 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.494+0000 7f81a2ffd700 1 -- 172.21.15.70:0/1457466860 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f81a404f560 con 0x7f81b40fe400 2024-03-23T08:23:11.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.800+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81b4102560 con 0x7f819c06aec0 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.802+0000 7f81a2ffd700 1 -- 172.21.15.70:0/1457466860 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f81b4102560 con 0x7f819c06aec0 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "1/2 daemons upgraded", 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:23:11.802 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:23:11.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f819c06aec0 msgr2=0x7f819c06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:11.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f819c06aec0 0x7f819c06d370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f81a8013600 tx=0x7f81a8009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:11.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b40fe400 msgr2=0x7f81b406e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:11.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b40fe400 0x7f81b406e480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009fd0 tx=0x7f81a400b040 comp rx=0 tx=0).stop 2024-03-23T08:23:11.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 shutdown_connections 2024-03-23T08:23:11.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f819c06aec0 0x7f819c06d370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:11.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b40fe400 0x7f81b406e480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:11.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 --2- 172.21.15.70:0/1457466860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81b40fed50 0x7f81b406e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:11.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 >> 172.21.15.70:0/1457466860 conn(0x7f81b40f6ba0 msgr2=0x7f81b40f8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:11.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 shutdown_connections 2024-03-23T08:23:11.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:11.804+0000 7f81b8ff8700 1 -- 172.21.15.70:0/1457466860 wait complete. 2024-03-23T08:23:11.854 INFO:tasks.workunit.client.1.smithi193.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-23T08:23:11.944 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:23:11.944 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: pgmap v15: 65 pgs: 65 active+clean; 191 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 339 op/s 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: Reconfiguring node-exporter.smithi070 (dependencies changed)... 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: Reconfiguring daemon node-exporter.smithi070 on smithi070 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:11.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:11 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2474522488' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:23:12.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:23:12.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:12.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:23:12.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: pgmap v15: 65 pgs: 65 active+clean; 191 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 339 op/s 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: Reconfiguring node-exporter.smithi070 (dependencies changed)... 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: Reconfiguring daemon node-exporter.smithi070 on smithi070 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:12.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:11 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2474522488' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:23:12.212 INFO:tasks.workunit.client.1.smithi193.stderr:++ readlink -f fsstress 2024-03-23T08:23:12.213 INFO:tasks.workunit.client.1.smithi193.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-23T08:23:12.213 INFO:tasks.workunit.client.1.smithi193.stderr:+ popd 2024-03-23T08:23:12.214 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-23T08:23:12.215 INFO:tasks.workunit.client.1.smithi193.stderr:+ popd 2024-03-23T08:23:12.215 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-23T08:23:12.215 INFO:tasks.workunit.client.1.smithi193.stderr:++ mktemp -d -p . 2024-03-23T08:23:12.218 INFO:tasks.workunit.client.1.smithi193.stderr:+ T=./tmp.pjPqDdIjg0 2024-03-23T08:23:12.218 INFO:tasks.workunit.client.1.smithi193.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.pjPqDdIjg0 -l 1 -n 1000 -p 10 -v 2024-03-23T08:23:12.222 INFO:tasks.workunit.client.1.smithi193.stdout:seed = 1711142937 2024-03-23T08:23:12.226 INFO:tasks.workunit.client.1.smithi193.stdout:0/0: write - no filename 2024-03-23T08:23:12.229 INFO:tasks.workunit.client.1.smithi193.stdout:0/1: creat f0 x:0 0 0 2024-03-23T08:23:12.230 INFO:tasks.workunit.client.1.smithi193.stdout:0/2: write f0 [961964,120823] 0 2024-03-23T08:23:12.231 INFO:tasks.workunit.client.1.smithi193.stdout:2/0: write - no filename 2024-03-23T08:23:12.231 INFO:tasks.workunit.client.1.smithi193.stdout:2/1: chown . 1922 1 2024-03-23T08:23:12.231 INFO:tasks.workunit.client.1.smithi193.stdout:2/2: unlink - no file 2024-03-23T08:23:12.233 INFO:tasks.workunit.client.1.smithi193.stdout:0/3: creat f1 x:0 0 0 2024-03-23T08:23:12.233 INFO:tasks.workunit.client.1.smithi193.stdout:3/0: write - no filename 2024-03-23T08:23:12.233 INFO:tasks.workunit.client.1.smithi193.stdout:1/0: creat f0 x:0 0 0 2024-03-23T08:23:12.236 INFO:tasks.workunit.client.1.smithi193.stdout:4/0: rename - no filename 2024-03-23T08:23:12.236 INFO:tasks.workunit.client.1.smithi193.stdout:4/1: fdatasync - no filename 2024-03-23T08:23:12.236 INFO:tasks.workunit.client.1.smithi193.stdout:4/2: dread - no filename 2024-03-23T08:23:12.236 INFO:tasks.workunit.client.1.smithi193.stdout:4/3: dwrite - no filename 2024-03-23T08:23:12.236 INFO:tasks.workunit.client.1.smithi193.stdout:0/4: creat f2 x:0 0 0 2024-03-23T08:23:12.237 INFO:tasks.workunit.client.1.smithi193.stdout:0/5: write f2 [473359,67904] 0 2024-03-23T08:23:12.238 INFO:tasks.workunit.client.1.smithi193.stdout:2/3: getdents . 0 2024-03-23T08:23:12.238 INFO:tasks.workunit.client.1.smithi193.stdout:0/6: chown f1 330365 1 2024-03-23T08:23:12.239 INFO:tasks.workunit.client.1.smithi193.stdout:1/1: rename f0 to f1 0 2024-03-23T08:23:12.239 INFO:tasks.workunit.client.1.smithi193.stdout:1/2: read - f1 zero size 2024-03-23T08:23:12.240 INFO:tasks.workunit.client.1.smithi193.stdout:3/1: creat f0 x:0 0 0 2024-03-23T08:23:12.242 INFO:tasks.workunit.client.1.smithi193.stdout:0/7: unlink f1 0 2024-03-23T08:23:12.243 INFO:tasks.workunit.client.1.smithi193.stdout:1/3: mknod c2 0 2024-03-23T08:23:12.243 INFO:tasks.workunit.client.1.smithi193.stdout:1/4: chown c2 2 1 2024-03-23T08:23:12.244 INFO:tasks.workunit.client.1.smithi193.stdout:3/2: mkdir d1 0 2024-03-23T08:23:12.244 INFO:tasks.workunit.client.1.smithi193.stdout:3/3: dread - f0 zero size 2024-03-23T08:23:12.246 INFO:tasks.workunit.client.1.smithi193.stdout:6/0: truncate - no filename 2024-03-23T08:23:12.246 INFO:tasks.workunit.client.1.smithi193.stdout:6/1: link - no file 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:4/4: mknod c0 0 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:4/5: dwrite - no filename 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/4: getdents . 0 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/5: write - no filename 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/6: dwrite - no filename 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/7: write - no filename 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/8: write - no filename 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/9: write - no filename 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/10: chown . 2 1 2024-03-23T08:23:12.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/11: link - no file 2024-03-23T08:23:12.248 INFO:tasks.workunit.client.1.smithi193.stdout:2/12: dwrite - no filename 2024-03-23T08:23:12.248 INFO:tasks.workunit.client.1.smithi193.stdout:2/13: write - no filename 2024-03-23T08:23:12.248 INFO:tasks.workunit.client.1.smithi193.stdout:2/14: unlink - no file 2024-03-23T08:23:12.248 INFO:tasks.workunit.client.1.smithi193.stdout:2/15: dwrite - no filename 2024-03-23T08:23:12.248 INFO:tasks.workunit.client.1.smithi193.stdout:4/6: chown c0 986 1 2024-03-23T08:23:12.248 INFO:tasks.workunit.client.1.smithi193.stdout:0/8: symlink l3 0 2024-03-23T08:23:12.248 INFO:tasks.workunit.client.1.smithi193.stdout:1/5: write f1 [358552,49174] 0 2024-03-23T08:23:12.249 INFO:tasks.workunit.client.1.smithi193.stdout:5/0: mkdir d0 0 2024-03-23T08:23:12.250 INFO:tasks.workunit.client.1.smithi193.stdout:7/0: unlink - no file 2024-03-23T08:23:12.253 INFO:tasks.workunit.client.1.smithi193.stdout:3/4: creat d1/f2 x:0 0 0 2024-03-23T08:23:12.254 INFO:tasks.workunit.client.1.smithi193.stdout:1/6: dread f1 [0,4194304] 0 2024-03-23T08:23:12.254 INFO:tasks.workunit.client.1.smithi193.stdout:6/2: mkdir d0 0 2024-03-23T08:23:12.256 INFO:tasks.workunit.client.1.smithi193.stdout:8/0: fsync - no filename 2024-03-23T08:23:12.256 INFO:tasks.workunit.client.1.smithi193.stdout:2/16: mkdir d0 0 2024-03-23T08:23:12.256 INFO:tasks.workunit.client.1.smithi193.stdout:2/17: dwrite - no filename 2024-03-23T08:23:12.258 INFO:tasks.workunit.client.1.smithi193.stdout:3/5: symlink d1/l3 0 2024-03-23T08:23:12.259 INFO:tasks.workunit.client.1.smithi193.stdout:1/7: write f1 [311081,64153] 0 2024-03-23T08:23:12.261 INFO:tasks.workunit.client.1.smithi193.stdout:7/1: symlink l0 0 2024-03-23T08:23:12.262 INFO:tasks.workunit.client.1.smithi193.stdout:7/2: rmdir - no directory 2024-03-23T08:23:12.263 INFO:tasks.workunit.client.1.smithi193.stdout:9/0: fdatasync - no filename 2024-03-23T08:23:12.263 INFO:tasks.workunit.client.1.smithi193.stdout:9/1: fdatasync - no filename 2024-03-23T08:23:12.263 INFO:tasks.workunit.client.1.smithi193.stdout:9/2: rmdir - no directory 2024-03-23T08:23:12.263 INFO:tasks.workunit.client.1.smithi193.stdout:3/6: rename f0 to d1/f4 0 2024-03-23T08:23:12.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/3: symlink d0/l1 0 2024-03-23T08:23:12.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/4: truncate - no filename 2024-03-23T08:23:12.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/5: dread - no filename 2024-03-23T08:23:12.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/6: dwrite - no filename 2024-03-23T08:23:12.265 INFO:tasks.workunit.client.1.smithi193.stdout:6/7: chown d0 118836714 1 2024-03-23T08:23:12.265 INFO:tasks.workunit.client.1.smithi193.stdout:6/8: chown d0 415 1 2024-03-23T08:23:12.265 INFO:tasks.workunit.client.1.smithi193.stdout:6/9: dread - no filename 2024-03-23T08:23:12.265 INFO:tasks.workunit.client.1.smithi193.stdout:6/10: write - no filename 2024-03-23T08:23:12.265 INFO:tasks.workunit.client.1.smithi193.stdout:6/11: write - no filename 2024-03-23T08:23:12.266 INFO:tasks.workunit.client.1.smithi193.stdout:8/1: creat f0 x:0 0 0 2024-03-23T08:23:12.267 INFO:tasks.workunit.client.1.smithi193.stdout:2/18: mknod d0/c1 0 2024-03-23T08:23:12.267 INFO:tasks.workunit.client.1.smithi193.stdout:2/19: truncate - no filename 2024-03-23T08:23:12.267 INFO:tasks.workunit.client.1.smithi193.stdout:2/20: write - no filename 2024-03-23T08:23:12.267 INFO:tasks.workunit.client.1.smithi193.stdout:2/21: truncate - no filename 2024-03-23T08:23:12.267 INFO:tasks.workunit.client.1.smithi193.stdout:2/22: write - no filename 2024-03-23T08:23:12.267 INFO:tasks.workunit.client.1.smithi193.stdout:2/23: fdatasync - no filename 2024-03-23T08:23:12.268 INFO:tasks.workunit.client.1.smithi193.stdout:1/8: dwrite f1 [0,4194304] 0 2024-03-23T08:23:12.268 INFO:tasks.workunit.client.1.smithi193.stdout:8/2: chown f0 260097380 1 2024-03-23T08:23:12.268 INFO:tasks.workunit.client.1.smithi193.stdout:1/9: chown c2 2402 1 2024-03-23T08:23:12.269 INFO:tasks.workunit.client.1.smithi193.stdout:8/3: write f0 [223902,104991] 0 2024-03-23T08:23:12.269 INFO:tasks.workunit.client.1.smithi193.stdout:7/3: symlink l1 0 2024-03-23T08:23:12.269 INFO:tasks.workunit.client.1.smithi193.stdout:7/4: write - no filename 2024-03-23T08:23:12.272 INFO:tasks.workunit.client.1.smithi193.stdout:8/4: dread f0 [0,4194304] 0 2024-03-23T08:23:12.273 INFO:tasks.workunit.client.1.smithi193.stdout:6/12: creat d0/f2 x:0 0 0 2024-03-23T08:23:12.274 INFO:tasks.workunit.client.1.smithi193.stdout:2/24: mkdir d0/d2 0 2024-03-23T08:23:12.274 INFO:tasks.workunit.client.1.smithi193.stdout:2/25: dread - no filename 2024-03-23T08:23:12.276 INFO:tasks.workunit.client.1.smithi193.stdout:1/10: unlink c2 0 2024-03-23T08:23:12.277 INFO:tasks.workunit.client.1.smithi193.stdout:1/11: stat f1 0 2024-03-23T08:23:12.279 INFO:tasks.workunit.client.1.smithi193.stdout:9/3: symlink l0 0 2024-03-23T08:23:12.280 INFO:tasks.workunit.client.1.smithi193.stdout:9/4: fdatasync - no filename 2024-03-23T08:23:12.280 INFO:tasks.workunit.client.1.smithi193.stdout:3/7: dwrite d1/f4 [0,4194304] 0 2024-03-23T08:23:12.281 INFO:tasks.workunit.client.1.smithi193.stdout:7/5: mknod c2 0 2024-03-23T08:23:12.281 INFO:tasks.workunit.client.1.smithi193.stdout:9/5: chown l0 1521 1 2024-03-23T08:23:12.283 INFO:tasks.workunit.client.1.smithi193.stdout:8/5: mknod c1 0 2024-03-23T08:23:12.286 INFO:tasks.workunit.client.1.smithi193.stdout:2/26: rename d0/d2 to d0/d3 0 2024-03-23T08:23:12.286 INFO:tasks.workunit.client.1.smithi193.stdout:1/12: fdatasync f1 0 2024-03-23T08:23:12.287 INFO:tasks.workunit.client.1.smithi193.stdout:6/13: dwrite d0/f2 [0,4194304] 0 2024-03-23T08:23:12.291 INFO:tasks.workunit.client.1.smithi193.stdout:7/6: creat f3 x:0 0 0 2024-03-23T08:23:12.293 INFO:tasks.workunit.client.1.smithi193.stdout:3/8: mknod d1/c5 0 2024-03-23T08:23:12.296 INFO:tasks.workunit.client.1.smithi193.stdout:9/6: symlink l1 0 2024-03-23T08:23:12.296 INFO:tasks.workunit.client.1.smithi193.stdout:9/7: dwrite - no filename 2024-03-23T08:23:12.298 INFO:tasks.workunit.client.1.smithi193.stdout:8/6: write f0 [1245802,99954] 0 2024-03-23T08:23:12.299 INFO:tasks.workunit.client.1.smithi193.stdout:2/27: symlink d0/l4 0 2024-03-23T08:23:12.301 INFO:tasks.workunit.client.1.smithi193.stdout:6/14: dwrite d0/f2 [0,4194304] 0 2024-03-23T08:23:12.303 INFO:tasks.workunit.client.1.smithi193.stdout:7/7: symlink l4 0 2024-03-23T08:23:12.304 INFO:tasks.workunit.client.1.smithi193.stdout:7/8: dread - f3 zero size 2024-03-23T08:23:12.307 INFO:tasks.workunit.client.1.smithi193.stdout:9/8: mkdir d2 0 2024-03-23T08:23:12.307 INFO:tasks.workunit.client.1.smithi193.stdout:9/9: chown l1 104530810 1 2024-03-23T08:23:12.307 INFO:tasks.workunit.client.1.smithi193.stdout:9/10: truncate - no filename 2024-03-23T08:23:12.308 INFO:tasks.workunit.client.1.smithi193.stdout:9/11: dread - no filename 2024-03-23T08:23:12.310 INFO:tasks.workunit.client.1.smithi193.stdout:8/7: mknod c2 0 2024-03-23T08:23:12.310 INFO:tasks.workunit.client.1.smithi193.stdout:2/28: symlink d0/l5 0 2024-03-23T08:23:12.318 INFO:tasks.workunit.client.1.smithi193.stdout:2/29: symlink d0/d3/l6 0 2024-03-23T08:23:12.318 INFO:tasks.workunit.client.1.smithi193.stdout:2/30: write - no filename 2024-03-23T08:23:12.322 INFO:tasks.workunit.client.1.smithi193.stdout:2/31: creat d0/f7 x:0 0 0 2024-03-23T08:23:12.322 INFO:tasks.workunit.client.1.smithi193.stdout:2/32: dread - d0/f7 zero size 2024-03-23T08:23:12.323 INFO:tasks.workunit.client.1.smithi193.stdout:6/15: link d0/l1 d0/l3 0 2024-03-23T08:23:12.329 INFO:tasks.workunit.client.1.smithi193.stdout:2/33: link d0/f7 d0/d3/f8 0 2024-03-23T08:23:12.330 INFO:tasks.workunit.client.1.smithi193.stdout:6/16: link d0/l3 d0/l4 0 2024-03-23T08:23:12.337 INFO:tasks.workunit.client.1.smithi193.stdout:6/17: dwrite d0/f2 [0,4194304] 0 2024-03-23T08:23:12.339 INFO:tasks.workunit.client.1.smithi193.stdout:6/18: write d0/f2 [3037594,71360] 0 2024-03-23T08:23:12.342 INFO:tasks.workunit.client.1.smithi193.stdout:2/34: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:12.349 INFO:tasks.workunit.client.1.smithi193.stdout:2/35: write d0/d3/f8 [1511883,77189] 0 2024-03-23T08:23:12.350 INFO:tasks.workunit.client.1.smithi193.stdout:2/36: stat d0/c1 0 2024-03-23T08:23:12.353 INFO:tasks.workunit.client.1.smithi193.stdout:6/19: dwrite d0/f2 [0,4194304] 0 2024-03-23T08:23:12.357 INFO:tasks.workunit.client.1.smithi193.stdout:6/20: creat d0/f5 x:0 0 0 2024-03-23T08:23:12.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/37: dwrite d0/d3/f8 [0,4194304] 0 2024-03-23T08:23:12.545 INFO:tasks.workunit.client.1.smithi193.stdout:0/9: fsync f0 0 2024-03-23T08:23:12.547 INFO:tasks.workunit.client.1.smithi193.stdout:0/10: creat f4 x:0 0 0 2024-03-23T08:23:12.550 INFO:tasks.workunit.client.1.smithi193.stdout:0/11: dread f0 [0,4194304] 0 2024-03-23T08:23:12.553 INFO:tasks.workunit.client.1.smithi193.stdout:0/12: dread f2 [0,4194304] 0 2024-03-23T08:23:12.555 INFO:tasks.workunit.client.1.smithi193.stdout:0/13: write f2 [1300471,44339] 0 2024-03-23T08:23:12.555 INFO:tasks.workunit.client.1.smithi193.stdout:0/14: chown f2 50 1 2024-03-23T08:23:12.557 INFO:tasks.workunit.client.1.smithi193.stdout:0/15: creat f5 x:0 0 0 2024-03-23T08:23:12.559 INFO:tasks.workunit.client.1.smithi193.stdout:0/16: creat f6 x:0 0 0 2024-03-23T08:23:12.560 INFO:tasks.workunit.client.1.smithi193.stdout:8/8: fsync f0 0 2024-03-23T08:23:12.560 INFO:tasks.workunit.client.1.smithi193.stdout:0/17: dread f0 [0,4194304] 0 2024-03-23T08:23:12.565 INFO:tasks.workunit.client.1.smithi193.stdout:1/13: truncate f1 1115318 0 2024-03-23T08:23:12.566 INFO:tasks.workunit.client.1.smithi193.stdout:0/18: dwrite f5 [0,4194304] 0 2024-03-23T08:23:12.567 INFO:tasks.workunit.client.1.smithi193.stdout:3/9: getdents d1 0 2024-03-23T08:23:12.568 INFO:tasks.workunit.client.1.smithi193.stdout:0/19: creat f7 x:0 0 0 2024-03-23T08:23:12.569 INFO:tasks.workunit.client.1.smithi193.stdout:0/20: truncate f6 124875 0 2024-03-23T08:23:12.569 INFO:tasks.workunit.client.1.smithi193.stdout:3/10: rmdir d1 39 2024-03-23T08:23:12.570 INFO:tasks.workunit.client.1.smithi193.stdout:3/11: chown d1/c5 15510 1 2024-03-23T08:23:12.571 INFO:tasks.workunit.client.1.smithi193.stdout:3/12: dread - d1/f2 zero size 2024-03-23T08:23:12.576 INFO:tasks.workunit.client.1.smithi193.stdout:0/21: dwrite f7 [0,4194304] 0 2024-03-23T08:23:12.579 INFO:tasks.workunit.client.1.smithi193.stdout:0/22: unlink f2 0 2024-03-23T08:23:12.579 INFO:tasks.workunit.client.1.smithi193.stdout:2/38: fsync d0/f7 0 2024-03-23T08:23:12.581 INFO:tasks.workunit.client.1.smithi193.stdout:0/23: mknod c8 0 2024-03-23T08:23:12.581 INFO:tasks.workunit.client.1.smithi193.stdout:2/39: rmdir d0/d3 39 2024-03-23T08:23:12.582 INFO:tasks.workunit.client.1.smithi193.stdout:0/24: write f4 [54493,11884] 0 2024-03-23T08:23:12.584 INFO:tasks.workunit.client.1.smithi193.stdout:0/25: mknod c9 0 2024-03-23T08:23:12.587 INFO:tasks.workunit.client.1.smithi193.stdout:0/26: creat fa x:0 0 0 2024-03-23T08:23:12.589 INFO:tasks.workunit.client.1.smithi193.stdout:0/27: mkdir db 0 2024-03-23T08:23:12.589 INFO:tasks.workunit.client.1.smithi193.stdout:2/40: link d0/c1 d0/d3/c9 0 2024-03-23T08:23:12.590 INFO:tasks.workunit.client.1.smithi193.stdout:2/41: read d0/d3/f8 [3756352,109051] 0 2024-03-23T08:23:12.591 INFO:tasks.workunit.client.1.smithi193.stdout:0/28: mknod db/cc 0 2024-03-23T08:23:12.594 INFO:tasks.workunit.client.1.smithi193.stdout:0/29: rename f6 to db/fd 0 2024-03-23T08:23:12.596 INFO:tasks.workunit.client.1.smithi193.stdout:2/42: dwrite d0/d3/f8 [0,4194304] 0 2024-03-23T08:23:12.603 INFO:tasks.workunit.client.1.smithi193.stdout:2/43: mknod d0/d3/ca 0 2024-03-23T08:23:12.612 INFO:tasks.workunit.client.1.smithi193.stdout:2/44: fsync d0/d3/f8 0 2024-03-23T08:23:12.614 INFO:tasks.workunit.client.1.smithi193.stdout:2/45: creat d0/d3/fb x:0 0 0 2024-03-23T08:23:12.616 INFO:tasks.workunit.client.1.smithi193.stdout:3/13: truncate d1/f4 202688 0 2024-03-23T08:23:12.616 INFO:tasks.workunit.client.1.smithi193.stdout:3/14: chown d1 3648773 1 2024-03-23T08:23:12.617 INFO:tasks.workunit.client.1.smithi193.stdout:3/15: write d1/f2 [416513,7412] 0 2024-03-23T08:23:12.618 INFO:tasks.workunit.client.1.smithi193.stdout:2/46: dread d0/d3/f8 [0,4194304] 0 2024-03-23T08:23:12.625 INFO:tasks.workunit.client.1.smithi193.stdout:2/47: truncate d0/f7 291761 0 2024-03-23T08:23:12.627 INFO:tasks.workunit.client.1.smithi193.stdout:1/14: write f1 [1391258,110880] 0 2024-03-23T08:23:12.630 INFO:tasks.workunit.client.1.smithi193.stdout:1/15: mknod c3 0 2024-03-23T08:23:12.637 INFO:tasks.workunit.client.1.smithi193.stdout:1/16: dwrite f1 [0,4194304] 0 2024-03-23T08:23:12.640 INFO:tasks.workunit.client.1.smithi193.stdout:1/17: creat f4 x:0 0 0 2024-03-23T08:23:12.642 INFO:tasks.workunit.client.1.smithi193.stdout:1/18: creat f5 x:0 0 0 2024-03-23T08:23:12.647 INFO:tasks.workunit.client.1.smithi193.stdout:2/48: dwrite d0/d3/f8 [0,4194304] 0 2024-03-23T08:23:12.648 INFO:tasks.workunit.client.1.smithi193.stdout:2/49: chown d0/l4 0 1 2024-03-23T08:23:12.658 INFO:tasks.workunit.client.1.smithi193.stdout:2/50: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:12.663 INFO:tasks.workunit.client.1.smithi193.stdout:2/51: dread d0/f7 [0,4194304] 0 2024-03-23T08:23:12.667 INFO:tasks.workunit.client.1.smithi193.stdout:2/52: write d0/f7 [760617,64704] 0 2024-03-23T08:23:12.669 INFO:tasks.workunit.client.1.smithi193.stdout:2/53: write d0/f7 [2660939,101049] 0 2024-03-23T08:23:12.674 INFO:tasks.workunit.client.1.smithi193.stdout:2/54: creat d0/fc x:0 0 0 2024-03-23T08:23:12.677 INFO:tasks.workunit.client.1.smithi193.stdout:2/55: dread d0/f7 [0,4194304] 0 2024-03-23T08:23:12.682 INFO:tasks.workunit.client.1.smithi193.stdout:2/56: rename d0/d3/fb to d0/fd 0 2024-03-23T08:23:12.694 INFO:tasks.workunit.client.1.smithi193.stdout:2/57: creat d0/fe x:0 0 0 2024-03-23T08:23:12.700 INFO:tasks.workunit.client.1.smithi193.stdout:2/58: dwrite d0/fc [0,4194304] 0 2024-03-23T08:23:12.706 INFO:tasks.workunit.client.1.smithi193.stdout:2/59: dwrite d0/fc [0,4194304] 0 2024-03-23T08:23:12.890 INFO:tasks.workunit.client.1.smithi193.stdout:2/60: symlink d0/d3/lf 0 2024-03-23T08:23:12.893 INFO:tasks.workunit.client.1.smithi193.stdout:2/61: symlink d0/l10 0 2024-03-23T08:23:12.897 INFO:tasks.workunit.client.1.smithi193.stdout:2/62: dread d0/d3/f8 [0,4194304] 0 2024-03-23T08:23:12.899 INFO:tasks.workunit.client.1.smithi193.stdout:2/63: write d0/d3/f8 [39442,95334] 0 2024-03-23T08:23:13.021 INFO:tasks.workunit.client.1.smithi193.stdout:2/64: truncate d0/f7 577248 0 2024-03-23T08:23:13.025 INFO:tasks.workunit.client.1.smithi193.stdout:2/65: link d0/l10 d0/l11 0 2024-03-23T08:23:13.027 INFO:tasks.workunit.client.1.smithi193.stdout:2/66: creat d0/d3/f12 x:0 0 0 2024-03-23T08:23:13.030 INFO:tasks.workunit.client.1.smithi193.stdout:2/67: dread d0/fc [0,4194304] 0 2024-03-23T08:23:13.032 INFO:tasks.workunit.client.1.smithi193.stdout:2/68: creat d0/d3/f13 x:0 0 0 2024-03-23T08:23:13.032 INFO:tasks.workunit.client.1.smithi193.stdout:2/69: dread - d0/d3/f12 zero size 2024-03-23T08:23:13.035 INFO:tasks.workunit.client.1.smithi193.stdout:2/70: mkdir d0/d3/d14 0 2024-03-23T08:23:13.035 INFO:tasks.workunit.client.1.smithi193.stdout:2/71: read d0/fc [50927,18878] 0 2024-03-23T08:23:13.042 INFO:tasks.workunit.client.1.smithi193.stdout:2/72: dwrite d0/fd [0,4194304] 0 2024-03-23T08:23:13.065 INFO:tasks.workunit.client.1.smithi193.stdout:2/73: write d0/f7 [1053197,35282] 0 2024-03-23T08:23:13.069 INFO:tasks.workunit.client.1.smithi193.stdout:2/74: symlink d0/l15 0 2024-03-23T08:23:13.070 INFO:tasks.workunit.client.1.smithi193.stdout:2/75: readlink d0/d3/l6 0 2024-03-23T08:23:13.074 INFO:tasks.workunit.client.1.smithi193.stdout:2/76: rmdir d0/d3 39 2024-03-23T08:23:13.075 INFO:tasks.workunit.client.1.smithi193.stdout:2/77: fdatasync d0/d3/f8 0 2024-03-23T08:23:13.077 INFO:tasks.workunit.client.1.smithi193.stdout:2/78: read d0/d3/f8 [960216,118293] 0 2024-03-23T08:23:13.079 INFO:tasks.workunit.client.1.smithi193.stdout:2/79: write d0/fc [539978,83221] 0 2024-03-23T08:23:13.085 INFO:tasks.workunit.client.1.smithi193.stdout:2/80: rmdir d0 39 2024-03-23T08:23:13.088 INFO:tasks.workunit.client.1.smithi193.stdout:2/81: mknod d0/d3/d14/c16 0 2024-03-23T08:23:13.095 INFO:tasks.workunit.client.1.smithi193.stdout:2/82: dwrite d0/fe [0,4194304] 0 2024-03-23T08:23:13.095 INFO:tasks.workunit.client.1.smithi193.stdout:2/83: readlink d0/l4 0 2024-03-23T08:23:13.461 INFO:tasks.workunit.client.0.smithi070.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-23T08:23:13.466 INFO:tasks.workunit.client.0.smithi070.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-23T08:23:13.466 INFO:tasks.workunit.client.0.smithi070.stderr:+ make 2024-03-23T08:23:13.534 INFO:tasks.workunit.client.0.smithi070.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-23T08:23:13.846 INFO:tasks.workunit.client.1.smithi193.stdout:6/21: sync 2024-03-23T08:23:13.846 INFO:tasks.workunit.client.1.smithi193.stdout:3/16: sync 2024-03-23T08:23:13.846 INFO:tasks.workunit.client.1.smithi193.stdout:0/30: sync 2024-03-23T08:23:13.846 INFO:tasks.workunit.client.1.smithi193.stdout:8/9: sync 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:5/1: sync 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:5/2: write - no filename 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:5/3: write - no filename 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:2/84: sync 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:1/19: sync 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:7/9: sync 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:4/7: sync 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:9/12: sync 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:4/8: read - no filename 2024-03-23T08:23:13.847 INFO:tasks.workunit.client.1.smithi193.stdout:4/9: write - no filename 2024-03-23T08:23:13.848 INFO:tasks.workunit.client.1.smithi193.stdout:1/20: chown c3 143 1 2024-03-23T08:23:13.848 INFO:tasks.workunit.client.1.smithi193.stdout:8/10: write f0 [334306,37210] 0 2024-03-23T08:23:13.848 INFO:tasks.workunit.client.1.smithi193.stdout:2/85: write d0/fd [2670977,93362] 0 2024-03-23T08:23:13.851 INFO:tasks.workunit.client.1.smithi193.stdout:2/86: chown d0/l15 3 1 2024-03-23T08:23:13.851 INFO:tasks.workunit.client.1.smithi193.stdout:8/11: fdatasync f0 0 2024-03-23T08:23:13.855 INFO:tasks.workunit.client.1.smithi193.stdout:0/31: dread f5 [0,4194304] 0 2024-03-23T08:23:13.856 INFO:tasks.workunit.client.1.smithi193.stdout:8/12: dread f0 [0,4194304] 0 2024-03-23T08:23:13.857 INFO:tasks.workunit.client.1.smithi193.stdout:7/10: dwrite f3 [0,4194304] 0 2024-03-23T08:23:13.861 INFO:tasks.workunit.client.1.smithi193.stdout:7/11: dread f3 [0,4194304] 0 2024-03-23T08:23:13.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:13 smithi070 ceph-mon[39699]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:13.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:13 smithi070 ceph-mon[39699]: pgmap v16: 65 pgs: 65 active+clean; 200 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 382 op/s 2024-03-23T08:23:13.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:13 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:13.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:13 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:13.976 INFO:tasks.workunit.client.1.smithi193.stdout:4/10: creat f1 x:0 0 0 2024-03-23T08:23:13.982 INFO:tasks.workunit.client.1.smithi193.stdout:5/4: sync 2024-03-23T08:23:13.982 INFO:tasks.workunit.client.1.smithi193.stdout:5/5: write - no filename 2024-03-23T08:23:13.989 INFO:tasks.workunit.client.1.smithi193.stdout:0/32: creat db/fe x:0 0 0 2024-03-23T08:23:13.989 INFO:tasks.workunit.client.1.smithi193.stdout:0/33: stat db/fe 0 2024-03-23T08:23:13.999 INFO:tasks.workunit.client.1.smithi193.stdout:9/13: creat d2/f3 x:0 0 0 2024-03-23T08:23:14.000 INFO:tasks.workunit.client.1.smithi193.stdout:9/14: dread - d2/f3 zero size 2024-03-23T08:23:14.000 INFO:tasks.workunit.client.1.smithi193.stdout:9/15: truncate d2/f3 787328 0 2024-03-23T08:23:14.008 INFO:tasks.workunit.client.1.smithi193.stdout:0/34: unlink db/fd 0 2024-03-23T08:23:14.008 INFO:tasks.workunit.client.1.smithi193.stdout:0/35: chown l3 2464 1 2024-03-23T08:23:14.014 INFO:tasks.workunit.client.1.smithi193.stdout:4/11: link f1 f2 0 2024-03-23T08:23:14.018 INFO:tasks.workunit.client.1.smithi193.stdout:9/16: creat d2/f4 x:0 0 0 2024-03-23T08:23:14.018 INFO:tasks.workunit.client.1.smithi193.stdout:3/17: link d1/c5 d1/c6 0 2024-03-23T08:23:14.019 INFO:tasks.workunit.client.1.smithi193.stdout:5/6: getdents d0 0 2024-03-23T08:23:14.019 INFO:tasks.workunit.client.1.smithi193.stdout:5/7: write - no filename 2024-03-23T08:23:14.020 INFO:tasks.workunit.client.1.smithi193.stdout:1/21: link c3 c6 0 2024-03-23T08:23:14.023 INFO:tasks.workunit.client.1.smithi193.stdout:2/87: rename d0/fe to d0/f17 0 2024-03-23T08:23:14.024 INFO:tasks.workunit.client.1.smithi193.stdout:0/36: mkdir db/df 0 2024-03-23T08:23:14.027 INFO:tasks.workunit.client.1.smithi193.stdout:4/12: mkdir d3 0 2024-03-23T08:23:14.027 INFO:tasks.workunit.client.1.smithi193.stdout:6/22: link d0/l4 d0/l6 0 2024-03-23T08:23:14.028 INFO:tasks.workunit.client.1.smithi193.stdout:9/17: mknod d2/c5 0 2024-03-23T08:23:14.030 INFO:tasks.workunit.client.1.smithi193.stdout:3/18: symlink d1/l7 0 2024-03-23T08:23:14.030 INFO:tasks.workunit.client.1.smithi193.stdout:3/19: write d1/f2 [433788,59693] 0 2024-03-23T08:23:14.031 INFO:tasks.workunit.client.1.smithi193.stdout:1/22: mknod c7 0 2024-03-23T08:23:14.035 INFO:tasks.workunit.client.1.smithi193.stdout:0/37: mknod db/c10 0 2024-03-23T08:23:14.035 INFO:tasks.workunit.client.1.smithi193.stdout:8/13: getdents . 0 2024-03-23T08:23:14.037 INFO:tasks.workunit.client.1.smithi193.stdout:6/23: mknod d0/c7 0 2024-03-23T08:23:14.038 INFO:tasks.workunit.client.1.smithi193.stdout:9/18: symlink d2/l6 0 2024-03-23T08:23:14.040 INFO:tasks.workunit.client.1.smithi193.stdout:1/23: creat f8 x:0 0 0 2024-03-23T08:23:14.040 INFO:tasks.workunit.client.1.smithi193.stdout:3/20: dread d1/f4 [0,4194304] 0 2024-03-23T08:23:14.040 INFO:tasks.workunit.client.1.smithi193.stdout:9/19: dread d2/f3 [0,4194304] 0 2024-03-23T08:23:14.041 INFO:tasks.workunit.client.1.smithi193.stdout:2/88: symlink d0/l18 0 2024-03-23T08:23:14.042 INFO:tasks.workunit.client.1.smithi193.stdout:0/38: mkdir db/d11 0 2024-03-23T08:23:14.042 INFO:tasks.workunit.client.1.smithi193.stdout:0/39: chown db/d11 372 1 2024-03-23T08:23:14.042 INFO:tasks.workunit.client.1.smithi193.stdout:2/89: truncate d0/d3/f12 300125 0 2024-03-23T08:23:14.043 INFO:tasks.workunit.client.1.smithi193.stdout:8/14: creat f3 x:0 0 0 2024-03-23T08:23:14.043 INFO:tasks.workunit.client.1.smithi193.stdout:8/15: fdatasync f0 0 2024-03-23T08:23:14.044 INFO:tasks.workunit.client.1.smithi193.stdout:0/40: dread f0 [0,4194304] 0 2024-03-23T08:23:14.046 INFO:tasks.workunit.client.1.smithi193.stdout:0/41: chown l3 973141 1 2024-03-23T08:23:14.046 INFO:tasks.workunit.client.1.smithi193.stdout:1/24: creat f9 x:0 0 0 2024-03-23T08:23:14.046 INFO:tasks.workunit.client.1.smithi193.stdout:9/20: mkdir d2/d7 0 2024-03-23T08:23:14.048 INFO:tasks.workunit.client.1.smithi193.stdout:3/21: symlink d1/l8 0 2024-03-23T08:23:14.048 INFO:tasks.workunit.client.1.smithi193.stdout:1/25: dread - f8 zero size 2024-03-23T08:23:14.049 INFO:tasks.workunit.client.1.smithi193.stdout:4/13: rmdir d3 0 2024-03-23T08:23:14.049 INFO:tasks.workunit.client.1.smithi193.stdout:4/14: rmdir - no directory 2024-03-23T08:23:14.053 INFO:tasks.workunit.client.1.smithi193.stdout:6/24: unlink d0/l1 0 2024-03-23T08:23:14.054 INFO:tasks.workunit.client.1.smithi193.stdout:1/26: write f9 [758712,71800] 0 2024-03-23T08:23:14.054 INFO:tasks.workunit.client.1.smithi193.stdout:1/27: write f9 [300313,86845] 0 2024-03-23T08:23:14.054 INFO:tasks.workunit.client.1.smithi193.stdout:2/90: dwrite d0/fc [0,4194304] 0 2024-03-23T08:23:14.055 INFO:tasks.workunit.client.1.smithi193.stdout:3/22: dread d1/f2 [0,4194304] 0 2024-03-23T08:23:14.056 INFO:tasks.workunit.client.1.smithi193.stdout:9/21: creat d2/d7/f8 x:0 0 0 2024-03-23T08:23:14.058 INFO:tasks.workunit.client.1.smithi193.stdout:6/25: mkdir d0/d8 0 2024-03-23T08:23:14.058 INFO:tasks.workunit.client.1.smithi193.stdout:8/16: dwrite f0 [0,4194304] 0 2024-03-23T08:23:14.061 INFO:tasks.workunit.client.1.smithi193.stdout:1/28: mkdir da 0 2024-03-23T08:23:14.062 INFO:tasks.workunit.client.1.smithi193.stdout:3/23: mknod d1/c9 0 2024-03-23T08:23:14.062 INFO:tasks.workunit.client.1.smithi193.stdout:8/17: chown c1 497455 1 2024-03-23T08:23:14.064 INFO:tasks.workunit.client.1.smithi193.stdout:9/22: mknod d2/d7/c9 0 2024-03-23T08:23:14.066 INFO:tasks.workunit.client.1.smithi193.stdout:1/29: write f1 [3397264,120045] 0 2024-03-23T08:23:14.067 INFO:tasks.workunit.client.1.smithi193.stdout:8/18: mknod c4 0 2024-03-23T08:23:14.071 INFO:tasks.workunit.client.1.smithi193.stdout:0/42: dread f5 [0,4194304] 0 2024-03-23T08:23:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:13 smithi193 ceph-mon[48370]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:13 smithi193 ceph-mon[48370]: pgmap v16: 65 pgs: 65 active+clean; 200 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 382 op/s 2024-03-23T08:23:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:13 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:13 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:23:14.075 INFO:tasks.workunit.client.1.smithi193.stdout:4/15: dwrite f2 [0,4194304] 0 2024-03-23T08:23:14.076 INFO:tasks.workunit.client.1.smithi193.stdout:4/16: readlink - no filename 2024-03-23T08:23:14.080 INFO:tasks.workunit.client.1.smithi193.stdout:3/24: link d1/f4 d1/fa 0 2024-03-23T08:23:14.080 INFO:tasks.workunit.client.1.smithi193.stdout:2/91: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:14.080 INFO:tasks.workunit.client.1.smithi193.stdout:0/43: write f0 [1905837,77542] 0 2024-03-23T08:23:14.081 INFO:tasks.workunit.client.1.smithi193.stdout:8/19: link c1 c5 0 2024-03-23T08:23:14.081 INFO:tasks.workunit.client.1.smithi193.stdout:8/20: rmdir - no directory 2024-03-23T08:23:14.085 INFO:tasks.workunit.client.1.smithi193.stdout:8/21: read - f3 zero size 2024-03-23T08:23:14.085 INFO:tasks.workunit.client.1.smithi193.stdout:6/26: dread d0/f2 [0,4194304] 0 2024-03-23T08:23:14.087 INFO:tasks.workunit.client.1.smithi193.stdout:4/17: write f2 [2429542,34696] 0 2024-03-23T08:23:14.088 INFO:tasks.workunit.client.1.smithi193.stdout:5/8: sync 2024-03-23T08:23:14.090 INFO:tasks.workunit.client.1.smithi193.stdout:8/22: dread - f3 zero size 2024-03-23T08:23:14.093 INFO:tasks.workunit.client.1.smithi193.stdout:1/30: dwrite f4 [0,4194304] 0 2024-03-23T08:23:14.093 INFO:tasks.workunit.client.1.smithi193.stdout:2/92: write d0/d3/f13 [955168,91131] 0 2024-03-23T08:23:14.098 INFO:tasks.workunit.client.1.smithi193.stdout:4/18: creat f4 x:0 0 0 2024-03-23T08:23:14.101 INFO:tasks.workunit.client.1.smithi193.stdout:9/23: dwrite d2/f3 [0,4194304] 0 2024-03-23T08:23:14.103 INFO:tasks.workunit.client.1.smithi193.stdout:2/93: rmdir d0/d3 39 2024-03-23T08:23:14.103 INFO:tasks.workunit.client.1.smithi193.stdout:5/9: symlink d0/l1 0 2024-03-23T08:23:14.107 INFO:tasks.workunit.client.1.smithi193.stdout:0/44: dwrite db/fe [0,4194304] 0 2024-03-23T08:23:14.107 INFO:tasks.workunit.client.1.smithi193.stdout:9/24: truncate d2/f4 533775 0 2024-03-23T08:23:14.107 INFO:tasks.workunit.client.1.smithi193.stdout:6/27: dread d0/f2 [0,4194304] 0 2024-03-23T08:23:14.109 INFO:tasks.workunit.client.1.smithi193.stdout:9/25: truncate d2/f4 702076 0 2024-03-23T08:23:14.111 INFO:tasks.workunit.client.1.smithi193.stdout:8/23: rename c5 to c6 0 2024-03-23T08:23:14.112 INFO:tasks.workunit.client.1.smithi193.stdout:3/25: dread d1/f4 [0,4194304] 0 2024-03-23T08:23:14.113 INFO:tasks.workunit.client.1.smithi193.stdout:5/10: mknod d0/c2 0 2024-03-23T08:23:14.113 INFO:tasks.workunit.client.1.smithi193.stdout:9/26: dread d2/f4 [0,4194304] 0 2024-03-23T08:23:14.113 INFO:tasks.workunit.client.1.smithi193.stdout:8/24: truncate f3 725172 0 2024-03-23T08:23:14.115 INFO:tasks.workunit.client.1.smithi193.stdout:9/27: readlink l0 0 2024-03-23T08:23:14.116 INFO:tasks.workunit.client.1.smithi193.stdout:2/94: read d0/d3/f13 [33573,111549] 0 2024-03-23T08:23:14.119 INFO:tasks.workunit.client.1.smithi193.stdout:5/11: creat d0/f3 x:0 0 0 2024-03-23T08:23:14.124 INFO:tasks.workunit.client.1.smithi193.stdout:0/45: rmdir db/df 0 2024-03-23T08:23:14.125 INFO:tasks.workunit.client.1.smithi193.stdout:5/12: symlink d0/l4 0 2024-03-23T08:23:14.129 INFO:tasks.workunit.client.1.smithi193.stdout:0/46: chown f7 33414459 1 2024-03-23T08:23:14.131 INFO:tasks.workunit.client.1.smithi193.stdout:6/28: dwrite d0/f2 [0,4194304] 0 2024-03-23T08:23:14.132 INFO:tasks.workunit.client.1.smithi193.stdout:9/28: dwrite d2/d7/f8 [0,4194304] 0 2024-03-23T08:23:14.132 INFO:tasks.workunit.client.1.smithi193.stdout:8/25: dwrite f3 [0,4194304] 0 2024-03-23T08:23:14.132 INFO:tasks.workunit.client.1.smithi193.stdout:8/26: rmdir - no directory 2024-03-23T08:23:14.137 INFO:tasks.workunit.client.1.smithi193.stdout:9/29: read d2/f3 [791652,37664] 0 2024-03-23T08:23:14.139 INFO:tasks.workunit.client.1.smithi193.stdout:8/27: symlink l7 0 2024-03-23T08:23:14.144 INFO:tasks.workunit.client.1.smithi193.stdout:8/28: readlink l7 0 2024-03-23T08:23:14.145 INFO:tasks.workunit.client.1.smithi193.stdout:0/47: dwrite f7 [0,4194304] 0 2024-03-23T08:23:14.148 INFO:tasks.workunit.client.1.smithi193.stdout:9/30: dread d2/f4 [0,4194304] 0 2024-03-23T08:23:14.149 INFO:tasks.workunit.client.1.smithi193.stdout:0/48: unlink db/c10 0 2024-03-23T08:23:14.191 INFO:tasks.workunit.client.1.smithi193.stdout:4/19: fsync f1 0 2024-03-23T08:23:14.192 INFO:tasks.workunit.client.0.smithi070.stderr:++ readlink -f fsstress 2024-03-23T08:23:14.193 INFO:tasks.workunit.client.1.smithi193.stdout:4/20: write f4 [176554,9656] 0 2024-03-23T08:23:14.195 INFO:tasks.workunit.client.0.smithi070.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-23T08:23:14.196 INFO:tasks.workunit.client.0.smithi070.stderr:+ popd 2024-03-23T08:23:14.198 INFO:tasks.workunit.client.0.smithi070.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-23T08:23:14.198 INFO:tasks.workunit.client.0.smithi070.stderr:+ popd 2024-03-23T08:23:14.199 INFO:tasks.workunit.client.1.smithi193.stdout:7/12: dwrite f3 [4194304,4194304] 0 2024-03-23T08:23:14.199 INFO:tasks.workunit.client.1.smithi193.stdout:3/26: getdents d1 0 2024-03-23T08:23:14.199 INFO:tasks.workunit.client.0.smithi070.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-23T08:23:14.199 INFO:tasks.workunit.client.0.smithi070.stderr:++ mktemp -d -p . 2024-03-23T08:23:14.200 INFO:tasks.workunit.client.1.smithi193.stdout:3/27: readlink d1/l7 0 2024-03-23T08:23:14.202 INFO:tasks.workunit.client.1.smithi193.stdout:0/49: getdents db/d11 0 2024-03-23T08:23:14.203 INFO:tasks.workunit.client.1.smithi193.stdout:0/50: readlink l3 0 2024-03-23T08:23:14.205 INFO:tasks.workunit.client.0.smithi070.stderr:+ T=./tmp.r0PBGTPYZv 2024-03-23T08:23:14.205 INFO:tasks.workunit.client.0.smithi070.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.r0PBGTPYZv -l 1 -n 1000 -p 10 -v 2024-03-23T08:23:14.206 INFO:tasks.workunit.client.1.smithi193.stdout:4/21: dwrite f4 [0,4194304] 0 2024-03-23T08:23:14.208 INFO:tasks.workunit.client.1.smithi193.stdout:4/22: chown f1 33 1 2024-03-23T08:23:14.209 INFO:tasks.workunit.client.1.smithi193.stdout:0/51: rename db/fe to db/d11/f12 0 2024-03-23T08:23:14.210 INFO:tasks.workunit.client.1.smithi193.stdout:7/13: dwrite f3 [8388608,4194304] 0 2024-03-23T08:23:14.210 INFO:tasks.workunit.client.1.smithi193.stdout:0/52: dread - fa zero size 2024-03-23T08:23:14.214 INFO:tasks.workunit.client.1.smithi193.stdout:5/13: fsync d0/f3 0 2024-03-23T08:23:14.216 INFO:tasks.workunit.client.0.smithi070.stdout:seed = 1711128712 2024-03-23T08:23:14.219 INFO:tasks.workunit.client.1.smithi193.stdout:7/14: creat f5 x:0 0 0 2024-03-23T08:23:14.219 INFO:tasks.workunit.client.1.smithi193.stdout:4/23: dread f4 [0,4194304] 0 2024-03-23T08:23:14.220 INFO:tasks.workunit.client.1.smithi193.stdout:0/53: creat db/d11/f13 x:0 0 0 2024-03-23T08:23:14.222 INFO:tasks.workunit.client.1.smithi193.stdout:2/95: truncate d0/d3/f8 1936067 0 2024-03-23T08:23:14.225 INFO:tasks.workunit.client.1.smithi193.stdout:5/14: creat d0/f5 x:0 0 0 2024-03-23T08:23:14.228 INFO:tasks.workunit.client.1.smithi193.stdout:7/15: unlink f5 0 2024-03-23T08:23:14.229 INFO:tasks.workunit.client.1.smithi193.stdout:0/54: dwrite fa [0,4194304] 0 2024-03-23T08:23:14.230 INFO:tasks.workunit.client.1.smithi193.stdout:1/31: sync 2024-03-23T08:23:14.232 INFO:tasks.workunit.client.1.smithi193.stdout:1/32: truncate f5 873803 0 2024-03-23T08:23:14.233 INFO:tasks.workunit.client.0.smithi070.stdout:2/0: fsync - no filename 2024-03-23T08:23:14.233 INFO:tasks.workunit.client.0.smithi070.stdout:2/1: stat - no entries 2024-03-23T08:23:14.233 INFO:tasks.workunit.client.0.smithi070.stdout:2/2: chown . 2849134 1 2024-03-23T08:23:14.237 INFO:tasks.workunit.client.1.smithi193.stdout:8/29: dwrite f0 [4194304,4194304] 0 2024-03-23T08:23:14.238 INFO:tasks.workunit.client.0.smithi070.stdout:0/0: symlink l0 0 2024-03-23T08:23:14.243 INFO:tasks.workunit.client.1.smithi193.stdout:4/24: link f4 f5 0 2024-03-23T08:23:14.244 INFO:tasks.workunit.client.1.smithi193.stdout:7/16: mkdir d6 0 2024-03-23T08:23:14.244 INFO:tasks.workunit.client.1.smithi193.stdout:3/28: write d1/f4 [799392,21130] 0 2024-03-23T08:23:14.245 INFO:tasks.workunit.client.0.smithi070.stdout:3/0: chown . 4140 1 2024-03-23T08:23:14.245 INFO:tasks.workunit.client.0.smithi070.stdout:3/1: dread - no filename 2024-03-23T08:23:14.246 INFO:tasks.workunit.client.1.smithi193.stdout:7/17: fdatasync f3 0 2024-03-23T08:23:14.246 INFO:tasks.workunit.client.1.smithi193.stdout:0/55: rmdir db 39 2024-03-23T08:23:14.246 INFO:tasks.workunit.client.1.smithi193.stdout:7/18: fsync f3 0 2024-03-23T08:23:14.247 INFO:tasks.workunit.client.0.smithi070.stdout:1/0: symlink l0 0 2024-03-23T08:23:14.247 INFO:tasks.workunit.client.0.smithi070.stdout:1/1: read - no filename 2024-03-23T08:23:14.247 INFO:tasks.workunit.client.0.smithi070.stdout:0/1: mknod c1 0 2024-03-23T08:23:14.247 INFO:tasks.workunit.client.0.smithi070.stdout:0/2: write - no filename 2024-03-23T08:23:14.248 INFO:tasks.workunit.client.0.smithi070.stdout:0/3: rmdir - no directory 2024-03-23T08:23:14.248 INFO:tasks.workunit.client.1.smithi193.stdout:0/56: dread f4 [0,4194304] 0 2024-03-23T08:23:14.249 INFO:tasks.workunit.client.1.smithi193.stdout:3/29: dread d1/f2 [0,4194304] 0 2024-03-23T08:23:14.250 INFO:tasks.workunit.client.1.smithi193.stdout:1/33: unlink c7 0 2024-03-23T08:23:14.251 INFO:tasks.workunit.client.0.smithi070.stdout:2/3: creat f0 x:0 0 0 2024-03-23T08:23:14.252 INFO:tasks.workunit.client.0.smithi070.stdout:4/0: fdatasync - no filename 2024-03-23T08:23:14.255 INFO:tasks.workunit.client.0.smithi070.stdout:1/2: creat f1 x:0 0 0 2024-03-23T08:23:14.256 INFO:tasks.workunit.client.1.smithi193.stdout:8/30: mknod c8 0 2024-03-23T08:23:14.256 INFO:tasks.workunit.client.1.smithi193.stdout:4/25: creat f6 x:0 0 0 2024-03-23T08:23:14.256 INFO:tasks.workunit.client.1.smithi193.stdout:4/26: readlink - no filename 2024-03-23T08:23:14.256 INFO:tasks.workunit.client.0.smithi070.stdout:1/3: fdatasync f1 0 2024-03-23T08:23:14.257 INFO:tasks.workunit.client.0.smithi070.stdout:1/4: chown f1 7713857 1 2024-03-23T08:23:14.258 INFO:tasks.workunit.client.1.smithi193.stdout:3/30: fsync d1/f4 0 2024-03-23T08:23:14.258 INFO:tasks.workunit.client.0.smithi070.stdout:2/4: mkdir d1 0 2024-03-23T08:23:14.258 INFO:tasks.workunit.client.0.smithi070.stdout:0/4: creat f2 x:0 0 0 2024-03-23T08:23:14.262 INFO:tasks.workunit.client.0.smithi070.stdout:2/5: write f0 [535527,7753] 0 2024-03-23T08:23:14.263 INFO:tasks.workunit.client.0.smithi070.stdout:3/2: symlink l0 0 2024-03-23T08:23:14.263 INFO:tasks.workunit.client.0.smithi070.stdout:3/3: dwrite - no filename 2024-03-23T08:23:14.263 INFO:tasks.workunit.client.0.smithi070.stdout:2/6: truncate f0 877698 0 2024-03-23T08:23:14.264 INFO:tasks.workunit.client.1.smithi193.stdout:1/34: chown c3 808 1 2024-03-23T08:23:14.265 INFO:tasks.workunit.client.1.smithi193.stdout:1/35: truncate f8 425140 0 2024-03-23T08:23:14.265 INFO:tasks.workunit.client.0.smithi070.stdout:1/5: creat f2 x:0 0 0 2024-03-23T08:23:14.266 INFO:tasks.workunit.client.1.smithi193.stdout:5/15: link d0/l1 d0/l6 0 2024-03-23T08:23:14.269 INFO:tasks.workunit.client.0.smithi070.stdout:0/5: creat f3 x:0 0 0 2024-03-23T08:23:14.269 INFO:tasks.workunit.client.0.smithi070.stdout:0/6: rmdir - no directory 2024-03-23T08:23:14.269 INFO:tasks.workunit.client.0.smithi070.stdout:0/7: chown f3 103 1 2024-03-23T08:23:14.270 INFO:tasks.workunit.client.0.smithi070.stdout:0/8: write f3 [80556,121608] 0 2024-03-23T08:23:14.271 INFO:tasks.workunit.client.0.smithi070.stdout:6/0: write - no filename 2024-03-23T08:23:14.273 INFO:tasks.workunit.client.0.smithi070.stdout:4/1: creat f0 x:0 0 0 2024-03-23T08:23:14.276 INFO:tasks.workunit.client.0.smithi070.stdout:3/4: creat f1 x:0 0 0 2024-03-23T08:23:14.277 INFO:tasks.workunit.client.0.smithi070.stdout:3/5: write f1 [210458,64806] 0 2024-03-23T08:23:14.280 INFO:tasks.workunit.client.0.smithi070.stdout:0/9: dwrite f2 [0,4194304] 0 2024-03-23T08:23:14.282 INFO:tasks.workunit.client.0.smithi070.stdout:0/10: dread f3 [0,4194304] 0 2024-03-23T08:23:14.282 INFO:tasks.workunit.client.0.smithi070.stdout:3/6: dread f1 [0,4194304] 0 2024-03-23T08:23:14.283 INFO:tasks.workunit.client.0.smithi070.stdout:0/11: write f3 [476125,42947] 0 2024-03-23T08:23:14.283 INFO:tasks.workunit.client.0.smithi070.stdout:3/7: chown l0 80676 1 2024-03-23T08:23:14.286 INFO:tasks.workunit.client.0.smithi070.stdout:5/0: getdents . 0 2024-03-23T08:23:14.286 INFO:tasks.workunit.client.0.smithi070.stdout:5/1: write - no filename 2024-03-23T08:23:14.286 INFO:tasks.workunit.client.0.smithi070.stdout:5/2: fsync - no filename 2024-03-23T08:23:14.287 INFO:tasks.workunit.client.0.smithi070.stdout:5/3: dwrite - no filename 2024-03-23T08:23:14.287 INFO:tasks.workunit.client.0.smithi070.stdout:5/4: stat - no entries 2024-03-23T08:23:14.287 INFO:tasks.workunit.client.0.smithi070.stdout:5/5: fdatasync - no filename 2024-03-23T08:23:14.287 INFO:tasks.workunit.client.0.smithi070.stdout:5/6: unlink - no file 2024-03-23T08:23:14.287 INFO:tasks.workunit.client.0.smithi070.stdout:5/7: dread - no filename 2024-03-23T08:23:14.287 INFO:tasks.workunit.client.0.smithi070.stdout:0/12: dread f3 [0,4194304] 0 2024-03-23T08:23:14.288 INFO:tasks.workunit.client.0.smithi070.stdout:0/13: readlink l0 0 2024-03-23T08:23:14.288 INFO:tasks.workunit.client.0.smithi070.stdout:0/14: readlink l0 0 2024-03-23T08:23:14.288 INFO:tasks.workunit.client.0.smithi070.stdout:4/2: creat f1 x:0 0 0 2024-03-23T08:23:14.289 INFO:tasks.workunit.client.0.smithi070.stdout:2/7: symlink d1/l2 0 2024-03-23T08:23:14.291 INFO:tasks.workunit.client.0.smithi070.stdout:2/8: write f0 [1023365,29991] 0 2024-03-23T08:23:14.291 INFO:tasks.workunit.client.1.smithi193.stdout:8/31: mknod c9 0 2024-03-23T08:23:14.292 INFO:tasks.workunit.client.0.smithi070.stdout:1/6: link f2 f3 0 2024-03-23T08:23:14.293 INFO:tasks.workunit.client.1.smithi193.stdout:8/32: readlink l7 0 2024-03-23T08:23:14.293 INFO:tasks.workunit.client.0.smithi070.stdout:2/9: write f0 [1969319,73002] 0 2024-03-23T08:23:14.294 INFO:tasks.workunit.client.1.smithi193.stdout:4/27: symlink l7 0 2024-03-23T08:23:14.294 INFO:tasks.workunit.client.1.smithi193.stdout:3/31: rename d1/l8 to d1/lb 0 2024-03-23T08:23:14.294 INFO:tasks.workunit.client.0.smithi070.stdout:3/8: creat f2 x:0 0 0 2024-03-23T08:23:14.295 INFO:tasks.workunit.client.1.smithi193.stdout:7/19: mknod d6/c7 0 2024-03-23T08:23:14.295 INFO:tasks.workunit.client.1.smithi193.stdout:4/28: write f2 [2702213,89296] 0 2024-03-23T08:23:14.296 INFO:tasks.workunit.client.0.smithi070.stdout:6/1: mkdir d0 0 2024-03-23T08:23:14.296 INFO:tasks.workunit.client.0.smithi070.stdout:6/2: dread - no filename 2024-03-23T08:23:14.296 INFO:tasks.workunit.client.0.smithi070.stdout:6/3: truncate - no filename 2024-03-23T08:23:14.296 INFO:tasks.workunit.client.0.smithi070.stdout:6/4: dread - no filename 2024-03-23T08:23:14.296 INFO:tasks.workunit.client.0.smithi070.stdout:6/5: write - no filename 2024-03-23T08:23:14.297 INFO:tasks.workunit.client.0.smithi070.stdout:6/6: fdatasync - no filename 2024-03-23T08:23:14.297 INFO:tasks.workunit.client.0.smithi070.stdout:6/7: truncate - no filename 2024-03-23T08:23:14.297 INFO:tasks.workunit.client.1.smithi193.stdout:1/36: rename f9 to da/fb 0 2024-03-23T08:23:14.297 INFO:tasks.workunit.client.1.smithi193.stdout:5/16: creat d0/f7 x:0 0 0 2024-03-23T08:23:14.298 INFO:tasks.workunit.client.1.smithi193.stdout:8/33: mknod ca 0 2024-03-23T08:23:14.299 INFO:tasks.workunit.client.0.smithi070.stdout:8/0: write - no filename 2024-03-23T08:23:14.299 INFO:tasks.workunit.client.1.smithi193.stdout:7/20: write f3 [1914218,97809] 0 2024-03-23T08:23:14.300 INFO:tasks.workunit.client.0.smithi070.stdout:4/3: dwrite f0 [0,4194304] 0 2024-03-23T08:23:14.300 INFO:tasks.workunit.client.0.smithi070.stdout:4/4: dread - f1 zero size 2024-03-23T08:23:14.301 INFO:tasks.workunit.client.1.smithi193.stdout:3/32: creat d1/fc x:0 0 0 2024-03-23T08:23:14.301 INFO:tasks.workunit.client.1.smithi193.stdout:2/96: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:14.301 INFO:tasks.workunit.client.0.smithi070.stdout:4/5: stat f1 0 2024-03-23T08:23:14.303 INFO:tasks.workunit.client.0.smithi070.stdout:0/15: mknod c4 0 2024-03-23T08:23:14.307 INFO:tasks.workunit.client.1.smithi193.stdout:2/97: chown d0/l18 6738431 1 2024-03-23T08:23:14.309 INFO:tasks.workunit.client.1.smithi193.stdout:2/98: truncate d0/d3/f12 1284653 0 2024-03-23T08:23:14.310 INFO:tasks.workunit.client.1.smithi193.stdout:2/99: chown d0/f17 4149342 1 2024-03-23T08:23:14.311 INFO:tasks.workunit.client.1.smithi193.stdout:5/17: unlink d0/f5 0 2024-03-23T08:23:14.312 INFO:tasks.workunit.client.1.smithi193.stdout:1/37: creat da/fc x:0 0 0 2024-03-23T08:23:14.312 INFO:tasks.workunit.client.0.smithi070.stdout:1/7: mknod c4 0 2024-03-23T08:23:14.313 INFO:tasks.workunit.client.1.smithi193.stdout:8/34: mknod cb 0 2024-03-23T08:23:14.315 INFO:tasks.workunit.client.0.smithi070.stdout:5/8: creat f0 x:0 0 0 2024-03-23T08:23:14.315 INFO:tasks.workunit.client.0.smithi070.stdout:5/9: rmdir - no directory 2024-03-23T08:23:14.315 INFO:tasks.workunit.client.1.smithi193.stdout:4/29: dwrite f5 [0,4194304] 0 2024-03-23T08:23:14.315 INFO:tasks.workunit.client.1.smithi193.stdout:3/33: write d1/f2 [1540733,130182] 0 2024-03-23T08:23:14.316 INFO:tasks.workunit.client.0.smithi070.stdout:3/9: rename f2 to f3 0 2024-03-23T08:23:14.319 INFO:tasks.workunit.client.1.smithi193.stdout:2/100: read d0/d3/f13 [456829,123728] 0 2024-03-23T08:23:14.319 INFO:tasks.workunit.client.0.smithi070.stdout:4/6: fsync f0 0 2024-03-23T08:23:14.320 INFO:tasks.workunit.client.1.smithi193.stdout:5/18: mknod d0/c8 0 2024-03-23T08:23:14.321 INFO:tasks.workunit.client.0.smithi070.stdout:4/7: write f0 [3056656,39201] 0 2024-03-23T08:23:14.323 INFO:tasks.workunit.client.0.smithi070.stdout:9/0: link - no file 2024-03-23T08:23:14.324 INFO:tasks.workunit.client.1.smithi193.stdout:1/38: mkdir da/dd 0 2024-03-23T08:23:14.325 INFO:tasks.workunit.client.1.smithi193.stdout:3/34: dwrite d1/fa [0,4194304] 0 2024-03-23T08:23:14.326 INFO:tasks.workunit.client.1.smithi193.stdout:1/39: fdatasync f1 0 2024-03-23T08:23:14.327 INFO:tasks.workunit.client.0.smithi070.stdout:7/0: creat f0 x:0 0 0 2024-03-23T08:23:14.327 INFO:tasks.workunit.client.1.smithi193.stdout:3/35: read - d1/fc zero size 2024-03-23T08:23:14.328 INFO:tasks.workunit.client.0.smithi070.stdout:7/1: write f0 [758194,86159] 0 2024-03-23T08:23:14.330 INFO:tasks.workunit.client.1.smithi193.stdout:4/30: unlink c0 0 2024-03-23T08:23:14.330 INFO:tasks.workunit.client.1.smithi193.stdout:4/31: rmdir - no directory 2024-03-23T08:23:14.331 INFO:tasks.workunit.client.0.smithi070.stdout:4/8: dwrite f1 [0,4194304] 0 2024-03-23T08:23:14.332 INFO:tasks.workunit.client.1.smithi193.stdout:1/40: write f4 [2072492,120099] 0 2024-03-23T08:23:14.333 INFO:tasks.workunit.client.0.smithi070.stdout:0/16: symlink l5 0 2024-03-23T08:23:14.334 INFO:tasks.workunit.client.1.smithi193.stdout:5/19: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:14.336 INFO:tasks.workunit.client.0.smithi070.stdout:1/8: fdatasync f2 0 2024-03-23T08:23:14.340 INFO:tasks.workunit.client.0.smithi070.stdout:4/9: write f0 [4253564,27861] 0 2024-03-23T08:23:14.341 INFO:tasks.workunit.client.0.smithi070.stdout:5/10: creat f1 x:0 0 0 2024-03-23T08:23:14.342 INFO:tasks.workunit.client.1.smithi193.stdout:2/101: dwrite d0/f17 [0,4194304] 0 2024-03-23T08:23:14.342 INFO:tasks.workunit.client.0.smithi070.stdout:5/11: fdatasync f1 0 2024-03-23T08:23:14.344 INFO:tasks.workunit.client.0.smithi070.stdout:5/12: fdatasync f1 0 2024-03-23T08:23:14.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/36: creat d1/fd x:0 0 0 2024-03-23T08:23:14.344 INFO:tasks.workunit.client.1.smithi193.stdout:5/20: dread d0/f7 [0,4194304] 0 2024-03-23T08:23:14.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/37: stat d1/fd 0 2024-03-23T08:23:14.345 INFO:tasks.workunit.client.0.smithi070.stdout:6/8: mknod d0/c1 0 2024-03-23T08:23:14.346 INFO:tasks.workunit.client.0.smithi070.stdout:8/1: mknod c0 0 2024-03-23T08:23:14.346 INFO:tasks.workunit.client.0.smithi070.stdout:8/2: fdatasync - no filename 2024-03-23T08:23:14.348 INFO:tasks.workunit.client.1.smithi193.stdout:7/21: getdents d6 0 2024-03-23T08:23:14.350 INFO:tasks.workunit.client.0.smithi070.stdout:7/2: rename f0 to f1 0 2024-03-23T08:23:14.350 INFO:tasks.workunit.client.1.smithi193.stdout:3/38: fsync d1/f4 0 2024-03-23T08:23:14.352 INFO:tasks.workunit.client.1.smithi193.stdout:7/22: dread f3 [8388608,4194304] 0 2024-03-23T08:23:14.354 INFO:tasks.workunit.client.0.smithi070.stdout:7/3: dread f1 [0,4194304] 0 2024-03-23T08:23:14.355 INFO:tasks.workunit.client.1.smithi193.stdout:2/102: mkdir d0/d3/d19 0 2024-03-23T08:23:14.357 INFO:tasks.workunit.client.1.smithi193.stdout:4/32: link f5 f8 0 2024-03-23T08:23:14.357 INFO:tasks.workunit.client.1.smithi193.stdout:4/33: rmdir - no directory 2024-03-23T08:23:14.358 INFO:tasks.workunit.client.1.smithi193.stdout:1/41: creat da/dd/fe x:0 0 0 2024-03-23T08:23:14.360 INFO:tasks.workunit.client.1.smithi193.stdout:5/21: unlink d0/l4 0 2024-03-23T08:23:14.361 INFO:tasks.workunit.client.1.smithi193.stdout:5/22: chown d0/f3 23 1 2024-03-23T08:23:14.363 INFO:tasks.workunit.client.0.smithi070.stdout:1/9: mknod c5 0 2024-03-23T08:23:14.364 INFO:tasks.workunit.client.0.smithi070.stdout:1/10: read - f3 zero size 2024-03-23T08:23:14.364 INFO:tasks.workunit.client.0.smithi070.stdout:1/11: read - f2 zero size 2024-03-23T08:23:14.365 INFO:tasks.workunit.client.0.smithi070.stdout:4/10: creat f2 x:0 0 0 2024-03-23T08:23:14.365 INFO:tasks.workunit.client.0.smithi070.stdout:4/11: dread - f2 zero size 2024-03-23T08:23:14.366 INFO:tasks.workunit.client.0.smithi070.stdout:2/10: link d1/l2 d1/l3 0 2024-03-23T08:23:14.368 INFO:tasks.workunit.client.1.smithi193.stdout:3/39: mkdir d1/de 0 2024-03-23T08:23:14.369 INFO:tasks.workunit.client.1.smithi193.stdout:3/40: write d1/fa [835692,34238] 0 2024-03-23T08:23:14.369 INFO:tasks.workunit.client.0.smithi070.stdout:5/13: symlink l2 0 2024-03-23T08:23:14.370 INFO:tasks.workunit.client.1.smithi193.stdout:7/23: rmdir d6 39 2024-03-23T08:23:14.372 INFO:tasks.workunit.client.0.smithi070.stdout:9/1: symlink l0 0 2024-03-23T08:23:14.372 INFO:tasks.workunit.client.0.smithi070.stdout:5/14: truncate f1 741206 0 2024-03-23T08:23:14.374 INFO:tasks.workunit.client.0.smithi070.stdout:1/12: dwrite f1 [0,4194304] 0 2024-03-23T08:23:14.374 INFO:tasks.workunit.client.0.smithi070.stdout:8/3: mknod c1 0 2024-03-23T08:23:14.374 INFO:tasks.workunit.client.1.smithi193.stdout:6/29: sync 2024-03-23T08:23:14.374 INFO:tasks.workunit.client.1.smithi193.stdout:9/31: sync 2024-03-23T08:23:14.375 INFO:tasks.workunit.client.0.smithi070.stdout:2/11: dread f0 [0,4194304] 0 2024-03-23T08:23:14.376 INFO:tasks.workunit.client.0.smithi070.stdout:6/9: mknod d0/c2 0 2024-03-23T08:23:14.378 INFO:tasks.workunit.client.1.smithi193.stdout:4/34: creat f9 x:0 0 0 2024-03-23T08:23:14.378 INFO:tasks.workunit.client.1.smithi193.stdout:4/35: readlink l7 0 2024-03-23T08:23:14.378 INFO:tasks.workunit.client.1.smithi193.stdout:4/36: fdatasync f1 0 2024-03-23T08:23:14.381 INFO:tasks.workunit.client.0.smithi070.stdout:3/10: rename f3 to f4 0 2024-03-23T08:23:14.381 INFO:tasks.workunit.client.0.smithi070.stdout:5/15: dwrite f1 [0,4194304] 0 2024-03-23T08:23:14.383 INFO:tasks.workunit.client.1.smithi193.stdout:5/23: creat d0/f9 x:0 0 0 2024-03-23T08:23:14.384 INFO:tasks.workunit.client.0.smithi070.stdout:7/4: mknod c2 0 2024-03-23T08:23:14.385 INFO:tasks.workunit.client.1.smithi193.stdout:5/24: write d0/f3 [641122,124256] 0 2024-03-23T08:23:14.386 INFO:tasks.workunit.client.0.smithi070.stdout:7/5: read f1 [318744,32805] 0 2024-03-23T08:23:14.387 INFO:tasks.workunit.client.1.smithi193.stdout:5/25: truncate d0/f7 5052007 0 2024-03-23T08:23:14.387 INFO:tasks.workunit.client.1.smithi193.stdout:1/42: dwrite da/fb [0,4194304] 0 2024-03-23T08:23:14.392 INFO:tasks.workunit.client.0.smithi070.stdout:4/12: mkdir d3 0 2024-03-23T08:23:14.393 INFO:tasks.workunit.client.1.smithi193.stdout:2/103: creat d0/d3/d19/f1a x:0 0 0 2024-03-23T08:23:14.393 INFO:tasks.workunit.client.0.smithi070.stdout:4/13: dread - f2 zero size 2024-03-23T08:23:14.395 INFO:tasks.workunit.client.1.smithi193.stdout:7/24: dread f3 [0,4194304] 0 2024-03-23T08:23:14.397 INFO:tasks.workunit.client.1.smithi193.stdout:4/37: rename f6 to fa 0 2024-03-23T08:23:14.399 INFO:tasks.workunit.client.1.smithi193.stdout:7/25: dread f3 [8388608,4194304] 0 2024-03-23T08:23:14.400 INFO:tasks.workunit.client.0.smithi070.stdout:5/16: dwrite f0 [0,4194304] 0 2024-03-23T08:23:14.400 INFO:tasks.workunit.client.0.smithi070.stdout:8/4: symlink l2 0 2024-03-23T08:23:14.400 INFO:tasks.workunit.client.0.smithi070.stdout:8/5: dwrite - no filename 2024-03-23T08:23:14.400 INFO:tasks.workunit.client.0.smithi070.stdout:8/6: dread - no filename 2024-03-23T08:23:14.400 INFO:tasks.workunit.client.0.smithi070.stdout:8/7: write - no filename 2024-03-23T08:23:14.401 INFO:tasks.workunit.client.0.smithi070.stdout:5/17: stat f0 0 2024-03-23T08:23:14.402 INFO:tasks.workunit.client.0.smithi070.stdout:1/13: rename f2 to f6 0 2024-03-23T08:23:14.405 INFO:tasks.workunit.client.0.smithi070.stdout:2/12: readlink d1/l3 0 2024-03-23T08:23:14.406 INFO:tasks.workunit.client.1.smithi193.stdout:1/43: mkdir da/dd/df 0 2024-03-23T08:23:14.407 INFO:tasks.workunit.client.0.smithi070.stdout:6/10: mkdir d0/d3 0 2024-03-23T08:23:14.407 INFO:tasks.workunit.client.0.smithi070.stdout:6/11: write - no filename 2024-03-23T08:23:14.407 INFO:tasks.workunit.client.0.smithi070.stdout:6/12: write - no filename 2024-03-23T08:23:14.410 INFO:tasks.workunit.client.0.smithi070.stdout:5/18: dwrite f1 [0,4194304] 0 2024-03-23T08:23:14.411 INFO:tasks.workunit.client.1.smithi193.stdout:3/41: rename d1/f4 to d1/de/ff 0 2024-03-23T08:23:14.412 INFO:tasks.workunit.client.1.smithi193.stdout:3/42: chown d1/l7 1781 1 2024-03-23T08:23:14.412 INFO:tasks.workunit.client.1.smithi193.stdout:3/43: truncate d1/fd 479294 0 2024-03-23T08:23:14.415 INFO:tasks.workunit.client.1.smithi193.stdout:6/30: fsync d0/f5 0 2024-03-23T08:23:14.419 INFO:tasks.workunit.client.1.smithi193.stdout:4/38: creat fb x:0 0 0 2024-03-23T08:23:14.420 INFO:tasks.workunit.client.0.smithi070.stdout:8/8: rename c1 to c3 0 2024-03-23T08:23:14.420 INFO:tasks.workunit.client.0.smithi070.stdout:8/9: dwrite - no filename 2024-03-23T08:23:14.420 INFO:tasks.workunit.client.0.smithi070.stdout:0/17: truncate f2 3666372 0 2024-03-23T08:23:14.421 INFO:tasks.workunit.client.1.smithi193.stdout:7/26: fdatasync f3 0 2024-03-23T08:23:14.428 INFO:tasks.workunit.client.1.smithi193.stdout:9/32: rename l0 to d2/la 0 2024-03-23T08:23:14.432 INFO:tasks.workunit.client.0.smithi070.stdout:2/13: dwrite f0 [0,4194304] 0 2024-03-23T08:23:14.432 INFO:tasks.workunit.client.1.smithi193.stdout:1/44: rmdir da/dd 39 2024-03-23T08:23:14.433 INFO:tasks.workunit.client.1.smithi193.stdout:6/31: stat d0/d8 0 2024-03-23T08:23:14.435 INFO:tasks.workunit.client.0.smithi070.stdout:5/19: write f1 [4988604,84667] 0 2024-03-23T08:23:14.436 INFO:tasks.workunit.client.1.smithi193.stdout:4/39: mknod cc 0 2024-03-23T08:23:14.437 INFO:tasks.workunit.client.0.smithi070.stdout:9/2: link l0 l1 0 2024-03-23T08:23:14.438 INFO:tasks.workunit.client.0.smithi070.stdout:9/3: dread - no filename 2024-03-23T08:23:14.440 INFO:tasks.workunit.client.0.smithi070.stdout:8/10: creat f4 x:0 0 0 2024-03-23T08:23:14.442 INFO:tasks.workunit.client.0.smithi070.stdout:0/18: creat f6 x:0 0 0 2024-03-23T08:23:14.443 INFO:tasks.workunit.client.0.smithi070.stdout:5/20: dwrite f1 [0,4194304] 0 2024-03-23T08:23:14.444 INFO:tasks.workunit.client.1.smithi193.stdout:4/40: dwrite f9 [0,4194304] 0 2024-03-23T08:23:14.448 INFO:tasks.workunit.client.1.smithi193.stdout:9/33: mkdir d2/d7/db 0 2024-03-23T08:23:14.449 INFO:tasks.workunit.client.0.smithi070.stdout:6/13: creat d0/d3/f4 x:0 0 0 2024-03-23T08:23:14.449 INFO:tasks.workunit.client.1.smithi193.stdout:4/41: dread f2 [0,4194304] 0 2024-03-23T08:23:14.452 INFO:tasks.workunit.client.1.smithi193.stdout:2/104: rename d0/d3 to d0/d1b 0 2024-03-23T08:23:14.455 INFO:tasks.workunit.client.0.smithi070.stdout:2/14: creat d1/f4 x:0 0 0 2024-03-23T08:23:14.455 INFO:tasks.workunit.client.0.smithi070.stdout:2/15: dread - d1/f4 zero size 2024-03-23T08:23:14.456 INFO:tasks.workunit.client.1.smithi193.stdout:2/105: dread d0/d1b/f8 [0,4194304] 0 2024-03-23T08:23:14.457 INFO:tasks.workunit.client.1.smithi193.stdout:6/32: write d0/f2 [4749260,31175] 0 2024-03-23T08:23:14.458 INFO:tasks.workunit.client.1.smithi193.stdout:3/44: dwrite d1/de/ff [0,4194304] 0 2024-03-23T08:23:14.458 INFO:tasks.workunit.client.0.smithi070.stdout:9/4: symlink l2 0 2024-03-23T08:23:14.459 INFO:tasks.workunit.client.0.smithi070.stdout:9/5: dwrite - no filename 2024-03-23T08:23:14.459 INFO:tasks.workunit.client.0.smithi070.stdout:9/6: truncate - no filename 2024-03-23T08:23:14.459 INFO:tasks.workunit.client.0.smithi070.stdout:9/7: dwrite - no filename 2024-03-23T08:23:14.459 INFO:tasks.workunit.client.0.smithi070.stdout:3/11: truncate f1 247423 0 2024-03-23T08:23:14.460 INFO:tasks.workunit.client.0.smithi070.stdout:8/11: mkdir d5 0 2024-03-23T08:23:14.462 INFO:tasks.workunit.client.1.smithi193.stdout:6/33: dread d0/f2 [0,4194304] 0 2024-03-23T08:23:14.463 INFO:tasks.workunit.client.1.smithi193.stdout:6/34: chown d0/d8 2796254 1 2024-03-23T08:23:14.463 INFO:tasks.workunit.client.1.smithi193.stdout:7/27: creat d6/f8 x:0 0 0 2024-03-23T08:23:14.464 INFO:tasks.workunit.client.0.smithi070.stdout:0/19: fsync f6 0 2024-03-23T08:23:14.464 INFO:tasks.workunit.client.1.smithi193.stdout:5/26: getdents d0 0 2024-03-23T08:23:14.465 INFO:tasks.workunit.client.0.smithi070.stdout:5/21: symlink l3 0 2024-03-23T08:23:14.467 INFO:tasks.workunit.client.1.smithi193.stdout:0/57: sync 2024-03-23T08:23:14.467 INFO:tasks.workunit.client.1.smithi193.stdout:9/34: write d2/f3 [2775911,31241] 0 2024-03-23T08:23:14.467 INFO:tasks.workunit.client.0.smithi070.stdout:1/14: link c5 c7 0 2024-03-23T08:23:14.468 INFO:tasks.workunit.client.1.smithi193.stdout:4/42: fdatasync fa 0 2024-03-23T08:23:14.468 INFO:tasks.workunit.client.0.smithi070.stdout:6/14: mknod d0/d3/c5 0 2024-03-23T08:23:14.469 INFO:tasks.workunit.client.1.smithi193.stdout:9/35: write d2/f3 [1705445,30762] 0 2024-03-23T08:23:14.470 INFO:tasks.workunit.client.1.smithi193.stdout:4/43: write f2 [4479903,71663] 0 2024-03-23T08:23:14.471 INFO:tasks.workunit.client.0.smithi070.stdout:9/8: chown l1 165924 1 2024-03-23T08:23:14.472 INFO:tasks.workunit.client.1.smithi193.stdout:1/45: mknod da/c10 0 2024-03-23T08:23:14.472 INFO:tasks.workunit.client.1.smithi193.stdout:6/35: dwrite d0/f2 [0,4194304] 0 2024-03-23T08:23:14.473 INFO:tasks.workunit.client.0.smithi070.stdout:3/12: mknod c5 0 2024-03-23T08:23:14.475 INFO:tasks.workunit.client.1.smithi193.stdout:2/106: rmdir d0/d1b 39 2024-03-23T08:23:14.476 INFO:tasks.workunit.client.0.smithi070.stdout:0/20: unlink f6 0 2024-03-23T08:23:14.480 INFO:tasks.workunit.client.1.smithi193.stdout:3/45: creat d1/de/f10 x:0 0 0 2024-03-23T08:23:14.480 INFO:tasks.workunit.client.1.smithi193.stdout:3/46: stat d1 0 2024-03-23T08:23:14.481 INFO:tasks.workunit.client.1.smithi193.stdout:3/47: write d1/fd [35748,63704] 0 2024-03-23T08:23:14.482 INFO:tasks.workunit.client.0.smithi070.stdout:5/22: rename f0 to f4 0 2024-03-23T08:23:14.482 INFO:tasks.workunit.client.1.smithi193.stdout:7/28: mkdir d6/d9 0 2024-03-23T08:23:14.485 INFO:tasks.workunit.client.1.smithi193.stdout:5/27: unlink d0/f9 0 2024-03-23T08:23:14.486 INFO:tasks.workunit.client.0.smithi070.stdout:6/15: creat d0/f6 x:0 0 0 2024-03-23T08:23:14.489 INFO:tasks.workunit.client.1.smithi193.stdout:5/28: dread d0/f3 [0,4194304] 0 2024-03-23T08:23:14.490 INFO:tasks.workunit.client.0.smithi070.stdout:5/23: dwrite f1 [0,4194304] 0 2024-03-23T08:23:14.490 INFO:tasks.workunit.client.0.smithi070.stdout:5/24: chown f1 29 1 2024-03-23T08:23:14.492 INFO:tasks.workunit.client.0.smithi070.stdout:8/12: mknod d5/c6 0 2024-03-23T08:23:14.493 INFO:tasks.workunit.client.1.smithi193.stdout:5/29: dread d0/f7 [0,4194304] 0 2024-03-23T08:23:14.498 INFO:tasks.workunit.client.1.smithi193.stdout:9/36: unlink d2/c5 0 2024-03-23T08:23:14.499 INFO:tasks.workunit.client.0.smithi070.stdout:5/25: dwrite f1 [4194304,4194304] 0 2024-03-23T08:23:14.500 INFO:tasks.workunit.client.0.smithi070.stdout:5/26: chown f4 1578865085 1 2024-03-23T08:23:14.504 INFO:tasks.workunit.client.1.smithi193.stdout:2/107: write d0/f7 [1440558,77681] 0 2024-03-23T08:23:14.506 INFO:tasks.workunit.client.0.smithi070.stdout:1/15: truncate f6 635906 0 2024-03-23T08:23:14.506 INFO:tasks.workunit.client.1.smithi193.stdout:4/44: write f8 [61525,91162] 0 2024-03-23T08:23:14.510 INFO:tasks.workunit.client.1.smithi193.stdout:4/45: dread f2 [0,4194304] 0 2024-03-23T08:23:14.510 INFO:tasks.workunit.client.0.smithi070.stdout:6/16: mknod d0/d3/c7 0 2024-03-23T08:23:14.511 INFO:tasks.workunit.client.0.smithi070.stdout:6/17: write d0/d3/f4 [836844,115059] 0 2024-03-23T08:23:14.513 INFO:tasks.workunit.client.0.smithi070.stdout:8/13: creat d5/f7 x:0 0 0 2024-03-23T08:23:14.514 INFO:tasks.workunit.client.1.smithi193.stdout:0/58: mkdir db/d11/d14 0 2024-03-23T08:23:14.515 INFO:tasks.workunit.client.0.smithi070.stdout:8/14: stat d5 0 2024-03-23T08:23:14.515 INFO:tasks.workunit.client.1.smithi193.stdout:0/59: dread - db/d11/f13 zero size 2024-03-23T08:23:14.515 INFO:tasks.workunit.client.0.smithi070.stdout:0/21: truncate f2 1003315 0 2024-03-23T08:23:14.516 INFO:tasks.workunit.client.1.smithi193.stdout:0/60: write fa [5147962,38100] 0 2024-03-23T08:23:14.516 INFO:tasks.workunit.client.0.smithi070.stdout:1/16: dwrite f1 [0,4194304] 0 2024-03-23T08:23:14.519 INFO:tasks.workunit.client.1.smithi193.stdout:5/30: creat d0/fa x:0 0 0 2024-03-23T08:23:14.521 INFO:tasks.workunit.client.1.smithi193.stdout:9/37: creat d2/fc x:0 0 0 2024-03-23T08:23:14.522 INFO:tasks.workunit.client.1.smithi193.stdout:9/38: read - d2/fc zero size 2024-03-23T08:23:14.525 INFO:tasks.workunit.client.1.smithi193.stdout:1/46: rename da/dd/fe to da/dd/df/f11 0 2024-03-23T08:23:14.526 INFO:tasks.workunit.client.1.smithi193.stdout:7/29: dwrite f3 [0,4194304] 0 2024-03-23T08:23:14.528 INFO:tasks.workunit.client.1.smithi193.stdout:6/36: mknod d0/d8/c9 0 2024-03-23T08:23:14.528 INFO:tasks.workunit.client.1.smithi193.stdout:7/30: dread - d6/f8 zero size 2024-03-23T08:23:14.528 INFO:tasks.workunit.client.1.smithi193.stdout:6/37: fsync d0/f5 0 2024-03-23T08:23:14.530 INFO:tasks.workunit.client.0.smithi070.stdout:8/15: mkdir d5/d8 0 2024-03-23T08:23:14.533 INFO:tasks.workunit.client.0.smithi070.stdout:9/9: getdents . 0 2024-03-23T08:23:14.533 INFO:tasks.workunit.client.0.smithi070.stdout:9/10: truncate - no filename 2024-03-23T08:23:14.533 INFO:tasks.workunit.client.0.smithi070.stdout:9/11: dwrite - no filename 2024-03-23T08:23:14.540 INFO:tasks.workunit.client.1.smithi193.stdout:5/31: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:14.541 INFO:tasks.workunit.client.0.smithi070.stdout:1/17: fdatasync f1 0 2024-03-23T08:23:14.541 INFO:tasks.workunit.client.1.smithi193.stdout:0/61: rename l3 to db/l15 0 2024-03-23T08:23:14.542 INFO:tasks.workunit.client.1.smithi193.stdout:0/62: write f7 [3128775,17066] 0 2024-03-23T08:23:14.542 INFO:tasks.workunit.client.0.smithi070.stdout:0/22: dread f2 [0,4194304] 0 2024-03-23T08:23:14.543 INFO:tasks.workunit.client.1.smithi193.stdout:0/63: chown db/d11/f12 47 1 2024-03-23T08:23:14.544 INFO:tasks.workunit.client.0.smithi070.stdout:1/18: mkdir d8 0 2024-03-23T08:23:14.545 INFO:tasks.workunit.client.0.smithi070.stdout:3/13: write f1 [468426,4865] 0 2024-03-23T08:23:14.545 INFO:tasks.workunit.client.0.smithi070.stdout:0/23: fdatasync f3 0 2024-03-23T08:23:14.546 INFO:tasks.workunit.client.0.smithi070.stdout:3/14: dread - f4 zero size 2024-03-23T08:23:14.546 INFO:tasks.workunit.client.0.smithi070.stdout:3/15: readlink l0 0 2024-03-23T08:23:14.547 INFO:tasks.workunit.client.0.smithi070.stdout:9/12: link l1 l3 0 2024-03-23T08:23:14.547 INFO:tasks.workunit.client.0.smithi070.stdout:9/13: fdatasync - no filename 2024-03-23T08:23:14.547 INFO:tasks.workunit.client.0.smithi070.stdout:9/14: fsync - no filename 2024-03-23T08:23:14.547 INFO:tasks.workunit.client.0.smithi070.stdout:3/16: truncate f1 1054641 0 2024-03-23T08:23:14.547 INFO:tasks.workunit.client.0.smithi070.stdout:3/17: stat f1 0 2024-03-23T08:23:14.548 INFO:tasks.workunit.client.0.smithi070.stdout:3/18: write f1 [181250,10136] 0 2024-03-23T08:23:14.573 INFO:tasks.workunit.client.1.smithi193.stdout:8/35: sync 2024-03-23T08:23:14.574 INFO:tasks.workunit.client.1.smithi193.stdout:8/36: write f3 [1426513,29560] 0 2024-03-23T08:23:14.579 INFO:tasks.workunit.client.1.smithi193.stdout:8/37: dwrite f0 [4194304,4194304] 0 2024-03-23T08:23:14.583 INFO:tasks.workunit.client.1.smithi193.stdout:8/38: dread f0 [4194304,4194304] 0 2024-03-23T08:23:14.592 INFO:tasks.workunit.client.1.smithi193.stdout:3/48: sync 2024-03-23T08:23:14.659 INFO:tasks.workunit.client.0.smithi070.stdout:6/18: truncate d0/d3/f4 63358 0 2024-03-23T08:23:14.660 INFO:tasks.workunit.client.0.smithi070.stdout:6/19: write d0/f6 [929728,6541] 0 2024-03-23T08:23:14.662 INFO:tasks.workunit.client.0.smithi070.stdout:0/24: rename c1 to c7 0 2024-03-23T08:23:14.663 INFO:tasks.workunit.client.1.smithi193.stdout:5/32: mkdir d0/db 0 2024-03-23T08:23:14.665 INFO:tasks.workunit.client.0.smithi070.stdout:9/15: creat f4 x:0 0 0 2024-03-23T08:23:14.668 INFO:tasks.workunit.client.0.smithi070.stdout:1/19: creat d8/f9 x:0 0 0 2024-03-23T08:23:14.670 INFO:tasks.workunit.client.0.smithi070.stdout:3/19: rename f4 to f6 0 2024-03-23T08:23:14.671 INFO:tasks.workunit.client.1.smithi193.stdout:9/39: truncate d2/f3 954052 0 2024-03-23T08:23:14.672 INFO:tasks.workunit.client.1.smithi193.stdout:0/64: mknod db/d11/c16 0 2024-03-23T08:23:14.672 INFO:tasks.workunit.client.0.smithi070.stdout:9/16: dwrite f4 [0,4194304] 0 2024-03-23T08:23:14.674 INFO:tasks.workunit.client.1.smithi193.stdout:7/31: creat d6/d9/fa x:0 0 0 2024-03-23T08:23:14.675 INFO:tasks.workunit.client.1.smithi193.stdout:7/32: chown d6/d9 0 1 2024-03-23T08:23:14.676 INFO:tasks.workunit.client.1.smithi193.stdout:0/65: dread db/d11/f12 [0,4194304] 0 2024-03-23T08:23:14.678 INFO:tasks.workunit.client.0.smithi070.stdout:3/20: creat f7 x:0 0 0 2024-03-23T08:23:14.680 INFO:tasks.workunit.client.0.smithi070.stdout:9/17: symlink l5 0 2024-03-23T08:23:14.681 INFO:tasks.workunit.client.1.smithi193.stdout:3/49: unlink d1/de/ff 0 2024-03-23T08:23:14.682 INFO:tasks.workunit.client.1.smithi193.stdout:3/50: write d1/fc [578860,95166] 0 2024-03-23T08:23:14.683 INFO:tasks.workunit.client.0.smithi070.stdout:0/25: dwrite f2 [0,4194304] 0 2024-03-23T08:23:14.683 INFO:tasks.workunit.client.1.smithi193.stdout:1/47: creat da/f12 x:0 0 0 2024-03-23T08:23:14.685 INFO:tasks.workunit.client.0.smithi070.stdout:3/21: rename l0 to l8 0 2024-03-23T08:23:14.687 INFO:tasks.workunit.client.1.smithi193.stdout:6/38: creat d0/fa x:0 0 0 2024-03-23T08:23:14.688 INFO:tasks.workunit.client.1.smithi193.stdout:6/39: chown d0/fa 28834059 1 2024-03-23T08:23:14.695 INFO:tasks.workunit.client.1.smithi193.stdout:2/108: getdents d0 0 2024-03-23T08:23:14.695 INFO:tasks.workunit.client.1.smithi193.stdout:4/46: getdents . 0 2024-03-23T08:23:14.696 INFO:tasks.workunit.client.0.smithi070.stdout:3/22: creat f9 x:0 0 0 2024-03-23T08:23:14.696 INFO:tasks.workunit.client.0.smithi070.stdout:9/18: dwrite f4 [0,4194304] 0 2024-03-23T08:23:14.696 INFO:tasks.workunit.client.1.smithi193.stdout:7/33: symlink d6/lb 0 2024-03-23T08:23:14.697 INFO:tasks.workunit.client.1.smithi193.stdout:0/66: mknod db/c17 0 2024-03-23T08:23:14.698 INFO:tasks.workunit.client.0.smithi070.stdout:9/19: read f4 [2676941,129219] 0 2024-03-23T08:23:14.699 INFO:tasks.workunit.client.1.smithi193.stdout:0/67: write f7 [5055674,85805] 0 2024-03-23T08:23:14.700 INFO:tasks.workunit.client.0.smithi070.stdout:0/26: dwrite f2 [0,4194304] 0 2024-03-23T08:23:14.701 INFO:tasks.workunit.client.0.smithi070.stdout:3/23: rename f7 to fa 0 2024-03-23T08:23:14.703 INFO:tasks.workunit.client.1.smithi193.stdout:4/47: dwrite fa [0,4194304] 0 2024-03-23T08:23:14.708 INFO:tasks.workunit.client.1.smithi193.stdout:3/51: creat d1/de/f11 x:0 0 0 2024-03-23T08:23:14.714 INFO:tasks.workunit.client.0.smithi070.stdout:3/24: mkdir db 0 2024-03-23T08:23:14.718 INFO:tasks.workunit.client.0.smithi070.stdout:0/27: creat f8 x:0 0 0 2024-03-23T08:23:14.721 INFO:tasks.workunit.client.0.smithi070.stdout:1/20: write f6 [1047042,101539] 0 2024-03-23T08:23:14.725 INFO:tasks.workunit.client.1.smithi193.stdout:9/40: mknod d2/d7/db/cd 0 2024-03-23T08:23:14.726 INFO:tasks.workunit.client.1.smithi193.stdout:2/109: unlink d0/f17 0 2024-03-23T08:23:14.727 INFO:tasks.workunit.client.1.smithi193.stdout:7/34: rename f3 to d6/fc 0 2024-03-23T08:23:14.728 INFO:tasks.workunit.client.1.smithi193.stdout:5/33: write d0/f3 [1691435,39342] 0 2024-03-23T08:23:14.728 INFO:tasks.workunit.client.1.smithi193.stdout:0/68: rmdir db 39 2024-03-23T08:23:14.730 INFO:tasks.workunit.client.1.smithi193.stdout:4/48: dwrite fb [0,4194304] 0 2024-03-23T08:23:14.730 INFO:tasks.workunit.client.1.smithi193.stdout:4/49: chown f9 2421 1 2024-03-23T08:23:14.734 INFO:tasks.workunit.client.0.smithi070.stdout:0/28: symlink l9 0 2024-03-23T08:23:14.734 INFO:tasks.workunit.client.0.smithi070.stdout:1/21: unlink l0 0 2024-03-23T08:23:14.734 INFO:tasks.workunit.client.0.smithi070.stdout:0/29: readlink l9 0 2024-03-23T08:23:14.734 INFO:tasks.workunit.client.0.smithi070.stdout:3/25: write fa [430058,108735] 0 2024-03-23T08:23:14.735 INFO:tasks.workunit.client.1.smithi193.stdout:3/52: mknod d1/de/c12 0 2024-03-23T08:23:14.736 INFO:tasks.workunit.client.0.smithi070.stdout:3/26: dread - f9 zero size 2024-03-23T08:23:14.881 INFO:tasks.workunit.client.0.smithi070.stdout:0/30: symlink la 0 2024-03-23T08:23:14.881 INFO:tasks.workunit.client.0.smithi070.stdout:6/20: dread d0/d3/f4 [0,4194304] 0 2024-03-23T08:23:14.882 INFO:tasks.workunit.client.0.smithi070.stdout:6/21: chown d0/d3 49 1 2024-03-23T08:23:14.883 INFO:tasks.workunit.client.0.smithi070.stdout:1/22: rename f6 to d8/fa 0 2024-03-23T08:23:14.884 INFO:tasks.workunit.client.0.smithi070.stdout:1/23: chown c5 168932828 1 2024-03-23T08:23:14.887 INFO:tasks.workunit.client.0.smithi070.stdout:0/31: creat fb x:0 0 0 2024-03-23T08:23:14.889 INFO:tasks.workunit.client.0.smithi070.stdout:6/22: dwrite d0/f6 [0,4194304] 0 2024-03-23T08:23:14.891 INFO:tasks.workunit.client.0.smithi070.stdout:1/24: mkdir d8/db 0 2024-03-23T08:23:14.895 INFO:tasks.workunit.client.0.smithi070.stdout:0/32: symlink lc 0 2024-03-23T08:23:14.897 INFO:tasks.workunit.client.0.smithi070.stdout:6/23: mkdir d0/d8 0 2024-03-23T08:23:14.902 INFO:tasks.workunit.client.0.smithi070.stdout:0/33: unlink c4 0 2024-03-23T08:23:14.903 INFO:tasks.workunit.client.0.smithi070.stdout:6/24: mknod d0/d3/c9 0 2024-03-23T08:23:14.906 INFO:tasks.workunit.client.0.smithi070.stdout:1/25: creat d8/db/fc x:0 0 0 2024-03-23T08:23:14.906 INFO:tasks.workunit.client.0.smithi070.stdout:0/34: dread f2 [0,4194304] 0 2024-03-23T08:23:14.910 INFO:tasks.workunit.client.0.smithi070.stdout:6/25: dwrite d0/f6 [0,4194304] 0 2024-03-23T08:23:14.911 INFO:tasks.workunit.client.0.smithi070.stdout:6/26: chown d0/d8 14717598 1 2024-03-23T08:23:14.912 INFO:tasks.workunit.client.0.smithi070.stdout:1/26: symlink d8/ld 0 2024-03-23T08:23:14.912 INFO:tasks.workunit.client.0.smithi070.stdout:6/27: chown d0/f6 1352057 1 2024-03-23T08:23:14.918 INFO:tasks.workunit.client.0.smithi070.stdout:1/27: unlink c5 0 2024-03-23T08:23:14.918 INFO:tasks.workunit.client.0.smithi070.stdout:6/28: dwrite d0/f6 [0,4194304] 0 2024-03-23T08:23:14.919 INFO:tasks.workunit.client.0.smithi070.stdout:6/29: chown d0/c2 379036290 1 2024-03-23T08:23:14.920 INFO:tasks.workunit.client.0.smithi070.stdout:0/35: dread f2 [0,4194304] 0 2024-03-23T08:23:14.922 INFO:tasks.workunit.client.0.smithi070.stdout:1/28: creat d8/fe x:0 0 0 2024-03-23T08:23:14.924 INFO:tasks.workunit.client.0.smithi070.stdout:6/30: creat d0/d3/fa x:0 0 0 2024-03-23T08:23:14.928 INFO:tasks.workunit.client.0.smithi070.stdout:0/36: dwrite f3 [0,4194304] 0 2024-03-23T08:23:14.934 INFO:tasks.workunit.client.0.smithi070.stdout:6/31: dwrite d0/d3/fa [0,4194304] 0 2024-03-23T08:23:14.934 INFO:tasks.workunit.client.0.smithi070.stdout:6/32: readlink - no filename 2024-03-23T08:23:14.939 INFO:tasks.workunit.client.0.smithi070.stdout:6/33: dread d0/d3/fa [0,4194304] 0 2024-03-23T08:23:14.940 INFO:tasks.workunit.client.0.smithi070.stdout:6/34: dread d0/d3/f4 [0,4194304] 0 2024-03-23T08:23:14.945 INFO:tasks.workunit.client.0.smithi070.stdout:6/35: creat d0/d3/fb x:0 0 0 2024-03-23T08:23:14.950 INFO:tasks.workunit.client.0.smithi070.stdout:6/36: mknod d0/d8/cc 0 2024-03-23T08:23:14.976 INFO:tasks.workunit.client.1.smithi193.stdout:1/48: mkdir da/d13 0 2024-03-23T08:23:14.976 INFO:tasks.workunit.client.1.smithi193.stdout:6/40: mkdir d0/db 0 2024-03-23T08:23:14.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:14 smithi070 ceph-mon[39699]: Reconfiguring alertmanager.smithi070 (dependencies changed)... 2024-03-23T08:23:14.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:14 smithi070 ceph-mon[39699]: Reconfiguring daemon alertmanager.smithi070 on smithi070 2024-03-23T08:23:14.983 INFO:tasks.workunit.client.1.smithi193.stdout:6/41: dwrite d0/f5 [0,4194304] 0 2024-03-23T08:23:14.984 INFO:tasks.workunit.client.1.smithi193.stdout:9/41: rename d2/d7/f8 to d2/d7/fe 0 2024-03-23T08:23:14.987 INFO:tasks.workunit.client.1.smithi193.stdout:8/39: getdents . 0 2024-03-23T08:23:14.988 INFO:tasks.workunit.client.1.smithi193.stdout:4/50: creat fd x:0 0 0 2024-03-23T08:23:14.991 INFO:tasks.workunit.client.1.smithi193.stdout:1/49: creat da/dd/f14 x:0 0 0 2024-03-23T08:23:14.991 INFO:tasks.workunit.client.1.smithi193.stdout:1/50: readlink - no filename 2024-03-23T08:23:14.992 INFO:tasks.workunit.client.1.smithi193.stdout:1/51: dread - da/f12 zero size 2024-03-23T08:23:14.993 INFO:tasks.workunit.client.1.smithi193.stdout:7/35: dwrite d6/fc [12582912,4194304] 0 2024-03-23T08:23:14.993 INFO:tasks.workunit.client.1.smithi193.stdout:2/110: dread - d0/d1b/d19/f1a zero size 2024-03-23T08:23:14.994 INFO:tasks.workunit.client.1.smithi193.stdout:1/52: dread - da/fc zero size 2024-03-23T08:23:14.994 INFO:tasks.workunit.client.1.smithi193.stdout:6/42: fsync d0/f2 0 2024-03-23T08:23:14.996 INFO:tasks.workunit.client.1.smithi193.stdout:8/40: dread f3 [0,4194304] 0 2024-03-23T08:23:14.997 INFO:tasks.workunit.client.1.smithi193.stdout:9/42: creat d2/ff x:0 0 0 2024-03-23T08:23:15.000 INFO:tasks.workunit.client.1.smithi193.stdout:4/51: symlink le 0 2024-03-23T08:23:15.002 INFO:tasks.workunit.client.1.smithi193.stdout:2/111: chown d0/l11 44014693 1 2024-03-23T08:23:15.002 INFO:tasks.workunit.client.1.smithi193.stdout:7/36: dwrite d6/fc [4194304,4194304] 0 2024-03-23T08:23:15.003 INFO:tasks.workunit.client.1.smithi193.stdout:7/37: fdatasync d6/d9/fa 0 2024-03-23T08:23:15.005 INFO:tasks.workunit.client.1.smithi193.stdout:7/38: fsync d6/d9/fa 0 2024-03-23T08:23:15.006 INFO:tasks.workunit.client.1.smithi193.stdout:8/41: creat fc x:0 0 0 2024-03-23T08:23:15.007 INFO:tasks.workunit.client.1.smithi193.stdout:5/34: rmdir d0/db 0 2024-03-23T08:23:15.008 INFO:tasks.workunit.client.1.smithi193.stdout:9/43: mknod d2/d7/db/c10 0 2024-03-23T08:23:15.009 INFO:tasks.workunit.client.1.smithi193.stdout:2/112: dread d0/d1b/f8 [0,4194304] 0 2024-03-23T08:23:15.010 INFO:tasks.workunit.client.1.smithi193.stdout:4/52: rename f1 to ff 0 2024-03-23T08:23:15.012 INFO:tasks.workunit.client.1.smithi193.stdout:4/53: write fd [716616,18513] 0 2024-03-23T08:23:15.012 INFO:tasks.workunit.client.1.smithi193.stdout:1/53: symlink da/d13/l15 0 2024-03-23T08:23:15.013 INFO:tasks.workunit.client.1.smithi193.stdout:4/54: chown l7 870333 1 2024-03-23T08:23:15.013 INFO:tasks.workunit.client.1.smithi193.stdout:4/55: stat f5 0 2024-03-23T08:23:15.013 INFO:tasks.workunit.client.1.smithi193.stdout:1/54: truncate da/f12 487777 0 2024-03-23T08:23:15.013 INFO:tasks.workunit.client.1.smithi193.stdout:6/43: getdents d0/db 0 2024-03-23T08:23:15.013 INFO:tasks.workunit.client.1.smithi193.stdout:7/39: creat d6/d9/fd x:0 0 0 2024-03-23T08:23:15.014 INFO:tasks.workunit.client.1.smithi193.stdout:7/40: dread - d6/d9/fa zero size 2024-03-23T08:23:15.015 INFO:tasks.workunit.client.1.smithi193.stdout:8/42: rename c8 to cd 0 2024-03-23T08:23:15.016 INFO:tasks.workunit.client.1.smithi193.stdout:8/43: write fc [1040002,122598] 0 2024-03-23T08:23:15.017 INFO:tasks.workunit.client.1.smithi193.stdout:9/44: creat d2/d7/f11 x:0 0 0 2024-03-23T08:23:15.021 INFO:tasks.workunit.client.1.smithi193.stdout:0/69: link db/l15 db/d11/d14/l18 0 2024-03-23T08:23:15.023 INFO:tasks.workunit.client.1.smithi193.stdout:1/55: mknod da/dd/df/c16 0 2024-03-23T08:23:15.024 INFO:tasks.workunit.client.1.smithi193.stdout:7/41: creat d6/d9/fe x:0 0 0 2024-03-23T08:23:15.026 INFO:tasks.workunit.client.1.smithi193.stdout:8/44: stat c1 0 2024-03-23T08:23:15.030 INFO:tasks.workunit.client.1.smithi193.stdout:9/45: rename d2/d7/db/cd to d2/c12 0 2024-03-23T08:23:15.030 INFO:tasks.workunit.client.1.smithi193.stdout:9/46: write d2/fc [768707,40465] 0 2024-03-23T08:23:15.031 INFO:tasks.workunit.client.1.smithi193.stdout:4/56: unlink f2 0 2024-03-23T08:23:15.032 INFO:tasks.workunit.client.1.smithi193.stdout:4/57: truncate f9 4610135 0 2024-03-23T08:23:15.032 INFO:tasks.workunit.client.1.smithi193.stdout:6/44: symlink d0/db/lc 0 2024-03-23T08:23:15.033 INFO:tasks.workunit.client.1.smithi193.stdout:4/58: read f9 [2625120,55147] 0 2024-03-23T08:23:15.036 INFO:tasks.workunit.client.1.smithi193.stdout:1/56: creat da/dd/df/f17 x:0 0 0 2024-03-23T08:23:15.036 INFO:tasks.workunit.client.1.smithi193.stdout:7/42: creat d6/ff x:0 0 0 2024-03-23T08:23:15.037 INFO:tasks.workunit.client.1.smithi193.stdout:8/45: symlink le 0 2024-03-23T08:23:15.038 INFO:tasks.workunit.client.1.smithi193.stdout:2/113: link d0/d1b/ca d0/d1b/d14/c1c 0 2024-03-23T08:23:15.039 INFO:tasks.workunit.client.1.smithi193.stdout:1/57: truncate f8 912753 0 2024-03-23T08:23:15.039 INFO:tasks.workunit.client.1.smithi193.stdout:9/47: rename d2/ff to d2/d7/db/f13 0 2024-03-23T08:23:15.040 INFO:tasks.workunit.client.1.smithi193.stdout:2/114: chown d0/d1b/d14 88838527 1 2024-03-23T08:23:15.041 INFO:tasks.workunit.client.1.smithi193.stdout:0/70: creat db/d11/d14/f19 x:0 0 0 2024-03-23T08:23:15.043 INFO:tasks.workunit.client.1.smithi193.stdout:6/45: dwrite d0/f2 [0,4194304] 0 2024-03-23T08:23:15.043 INFO:tasks.workunit.client.1.smithi193.stdout:7/43: creat d6/f10 x:0 0 0 2024-03-23T08:23:15.043 INFO:tasks.workunit.client.1.smithi193.stdout:4/59: dwrite f8 [0,4194304] 0 2024-03-23T08:23:15.044 INFO:tasks.workunit.client.1.smithi193.stdout:8/46: write f3 [4614638,47562] 0 2024-03-23T08:23:15.045 INFO:tasks.workunit.client.1.smithi193.stdout:2/115: write d0/d1b/d19/f1a [344390,60738] 0 2024-03-23T08:23:15.045 INFO:tasks.workunit.client.1.smithi193.stdout:7/44: chown d6/fc 1997 1 2024-03-23T08:23:15.049 INFO:tasks.workunit.client.1.smithi193.stdout:4/60: symlink l10 0 2024-03-23T08:23:15.050 INFO:tasks.workunit.client.1.smithi193.stdout:4/61: read f5 [3094556,29172] 0 2024-03-23T08:23:15.050 INFO:tasks.workunit.client.1.smithi193.stdout:8/47: mkdir df 0 2024-03-23T08:23:15.051 INFO:tasks.workunit.client.1.smithi193.stdout:8/48: truncate fc 1213447 0 2024-03-23T08:23:15.052 INFO:tasks.workunit.client.1.smithi193.stdout:0/71: unlink f5 0 2024-03-23T08:23:15.053 INFO:tasks.workunit.client.1.smithi193.stdout:0/72: write fa [904869,112502] 0 2024-03-23T08:23:15.053 INFO:tasks.workunit.client.1.smithi193.stdout:7/45: mknod d6/d9/c11 0 2024-03-23T08:23:15.054 INFO:tasks.workunit.client.1.smithi193.stdout:7/46: fdatasync d6/f8 0 2024-03-23T08:23:15.055 INFO:tasks.workunit.client.1.smithi193.stdout:6/46: symlink d0/ld 0 2024-03-23T08:23:15.056 INFO:tasks.workunit.client.1.smithi193.stdout:7/47: truncate d6/f8 127655 0 2024-03-23T08:23:15.056 INFO:tasks.workunit.client.1.smithi193.stdout:7/48: read - d6/d9/fe zero size 2024-03-23T08:23:15.057 INFO:tasks.workunit.client.1.smithi193.stdout:4/62: mkdir d11 0 2024-03-23T08:23:15.061 INFO:tasks.workunit.client.1.smithi193.stdout:0/73: readlink db/d11/d14/l18 0 2024-03-23T08:23:15.065 INFO:tasks.workunit.client.1.smithi193.stdout:2/116: creat d0/f1d x:0 0 0 2024-03-23T08:23:15.067 INFO:tasks.workunit.client.1.smithi193.stdout:4/63: creat d11/f12 x:0 0 0 2024-03-23T08:23:15.069 INFO:tasks.workunit.client.1.smithi193.stdout:4/64: creat d11/f13 x:0 0 0 2024-03-23T08:23:15.074 INFO:tasks.workunit.client.1.smithi193.stdout:2/117: dwrite d0/fd [0,4194304] 0 2024-03-23T08:23:15.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:14 smithi193 ceph-mon[48370]: Reconfiguring alertmanager.smithi070 (dependencies changed)... 2024-03-23T08:23:15.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:14 smithi193 ceph-mon[48370]: Reconfiguring daemon alertmanager.smithi070 on smithi070 2024-03-23T08:23:15.077 INFO:tasks.workunit.client.1.smithi193.stdout:2/118: chown d0/l15 146661 1 2024-03-23T08:23:15.078 INFO:tasks.workunit.client.1.smithi193.stdout:4/65: dwrite d11/f12 [0,4194304] 0 2024-03-23T08:23:15.079 INFO:tasks.workunit.client.1.smithi193.stdout:4/66: chown d11/f12 1546941144 1 2024-03-23T08:23:15.080 INFO:tasks.workunit.client.1.smithi193.stdout:4/67: creat d11/f14 x:0 0 0 2024-03-23T08:23:15.081 INFO:tasks.workunit.client.1.smithi193.stdout:2/119: rmdir d0/d1b/d14 39 2024-03-23T08:23:15.083 INFO:tasks.workunit.client.1.smithi193.stdout:4/68: symlink d11/l15 0 2024-03-23T08:23:15.086 INFO:tasks.workunit.client.1.smithi193.stdout:4/69: mkdir d11/d16 0 2024-03-23T08:23:15.086 INFO:tasks.workunit.client.1.smithi193.stdout:4/70: chown fd 0 1 2024-03-23T08:23:15.088 INFO:tasks.workunit.client.1.smithi193.stdout:2/120: dread d0/fc [0,4194304] 0 2024-03-23T08:23:15.090 INFO:tasks.workunit.client.1.smithi193.stdout:2/121: mknod d0/d1b/d19/c1e 0 2024-03-23T08:23:15.093 INFO:tasks.workunit.client.1.smithi193.stdout:4/71: dwrite f8 [0,4194304] 0 2024-03-23T08:23:15.094 INFO:tasks.workunit.client.1.smithi193.stdout:2/122: creat d0/d1b/d14/f1f x:0 0 0 2024-03-23T08:23:15.097 INFO:tasks.workunit.client.1.smithi193.stdout:4/72: symlink d11/d16/l17 0 2024-03-23T08:23:15.100 INFO:tasks.workunit.client.1.smithi193.stdout:2/123: creat d0/f20 x:0 0 0 2024-03-23T08:23:15.116 INFO:tasks.workunit.client.1.smithi193.stdout:4/73: fsync f4 0 2024-03-23T08:23:15.117 INFO:tasks.workunit.client.0.smithi070.stdout:1/29: unlink d8/fa 0 2024-03-23T08:23:15.122 INFO:tasks.workunit.client.0.smithi070.stdout:3/27: write f6 [833527,86707] 0 2024-03-23T08:23:15.122 INFO:tasks.workunit.client.0.smithi070.stdout:3/28: symlink db/lc 0 2024-03-23T08:23:15.124 INFO:tasks.workunit.client.0.smithi070.stdout:3/29: write f6 [860171,92476] 0 2024-03-23T08:23:15.127 INFO:tasks.workunit.client.0.smithi070.stdout:6/37: getdents d0 0 2024-03-23T08:23:15.128 INFO:tasks.workunit.client.0.smithi070.stdout:1/30: dwrite f3 [0,4194304] 0 2024-03-23T08:23:15.131 INFO:tasks.workunit.client.0.smithi070.stdout:6/38: symlink d0/d3/ld 0 2024-03-23T08:23:15.131 INFO:tasks.workunit.client.0.smithi070.stdout:6/39: read d0/d3/fa [3754221,31598] 0 2024-03-23T08:23:15.133 INFO:tasks.workunit.client.0.smithi070.stdout:1/31: unlink f1 0 2024-03-23T08:23:15.133 INFO:tasks.workunit.client.0.smithi070.stdout:1/32: readlink d8/ld 0 2024-03-23T08:23:15.135 INFO:tasks.workunit.client.0.smithi070.stdout:6/40: rename d0/d3/fb to d0/d8/fe 0 2024-03-23T08:23:15.136 INFO:tasks.workunit.client.0.smithi070.stdout:6/41: dread - d0/d8/fe zero size 2024-03-23T08:23:15.142 INFO:tasks.workunit.client.0.smithi070.stdout:1/33: dwrite d8/db/fc [0,4194304] 0 2024-03-23T08:23:15.146 INFO:tasks.workunit.client.0.smithi070.stdout:6/42: link d0/d3/ld d0/lf 0 2024-03-23T08:23:15.148 INFO:tasks.workunit.client.0.smithi070.stdout:6/43: write d0/d8/fe [885641,47452] 0 2024-03-23T08:23:15.151 INFO:tasks.workunit.client.0.smithi070.stdout:6/44: creat d0/d8/f10 x:0 0 0 2024-03-23T08:23:15.168 INFO:tasks.workunit.client.1.smithi193.stdout:3/53: mkdir d1/de/d13 0 2024-03-23T08:23:15.174 INFO:tasks.workunit.client.1.smithi193.stdout:3/54: link d1/l3 d1/l14 0 2024-03-23T08:23:15.178 INFO:tasks.workunit.client.1.smithi193.stdout:3/55: link d1/c5 d1/de/c15 0 2024-03-23T08:23:15.179 INFO:tasks.workunit.client.1.smithi193.stdout:3/56: chown d1/f2 22 1 2024-03-23T08:23:15.180 INFO:tasks.workunit.client.1.smithi193.stdout:5/35: truncate d0/f7 4562701 0 2024-03-23T08:23:15.181 INFO:tasks.workunit.client.1.smithi193.stdout:3/57: read d1/fa [1330817,112919] 0 2024-03-23T08:23:15.182 INFO:tasks.workunit.client.1.smithi193.stdout:5/36: symlink d0/lc 0 2024-03-23T08:23:15.183 INFO:tasks.workunit.client.1.smithi193.stdout:3/58: write d1/fd [6922,98048] 0 2024-03-23T08:23:15.185 INFO:tasks.workunit.client.1.smithi193.stdout:9/48: rename d2/c12 to d2/d7/db/c14 0 2024-03-23T08:23:15.185 INFO:tasks.workunit.client.1.smithi193.stdout:9/49: dread - d2/d7/f11 zero size 2024-03-23T08:23:15.189 INFO:tasks.workunit.client.1.smithi193.stdout:0/74: rename f0 to db/d11/f1a 0 2024-03-23T08:23:15.197 INFO:tasks.workunit.client.1.smithi193.stdout:2/124: rename d0/d1b/d14/f1f to d0/d1b/f21 0 2024-03-23T08:23:15.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/125: chown d0/fd 25 1 2024-03-23T08:23:15.200 INFO:tasks.workunit.client.1.smithi193.stdout:1/58: truncate f5 692370 0 2024-03-23T08:23:15.200 INFO:tasks.workunit.client.1.smithi193.stdout:7/49: rmdir d6 39 2024-03-23T08:23:15.201 INFO:tasks.workunit.client.1.smithi193.stdout:6/47: getdents d0 0 2024-03-23T08:23:15.201 INFO:tasks.workunit.client.1.smithi193.stdout:2/126: dread - d0/d1b/f21 zero size 2024-03-23T08:23:15.201 INFO:tasks.workunit.client.1.smithi193.stdout:1/59: chown da/d13/l15 10 1 2024-03-23T08:23:15.202 INFO:tasks.workunit.client.1.smithi193.stdout:2/127: chown d0/d1b/d14 138 1 2024-03-23T08:23:15.202 INFO:tasks.workunit.client.1.smithi193.stdout:7/50: dread - d6/d9/fe zero size 2024-03-23T08:23:15.202 INFO:tasks.workunit.client.1.smithi193.stdout:1/60: chown da/fc 10531033 1 2024-03-23T08:23:15.203 INFO:tasks.workunit.client.1.smithi193.stdout:6/48: creat d0/fe x:0 0 0 2024-03-23T08:23:15.204 INFO:tasks.workunit.client.1.smithi193.stdout:8/49: write f0 [2401842,51335] 0 2024-03-23T08:23:15.205 INFO:tasks.workunit.client.1.smithi193.stdout:7/51: creat d6/d9/f12 x:0 0 0 2024-03-23T08:23:15.206 INFO:tasks.workunit.client.1.smithi193.stdout:7/52: readlink l4 0 2024-03-23T08:23:15.206 INFO:tasks.workunit.client.1.smithi193.stdout:7/53: chown c2 486879375 1 2024-03-23T08:23:15.208 INFO:tasks.workunit.client.1.smithi193.stdout:6/49: unlink d0/fa 0 2024-03-23T08:23:15.209 INFO:tasks.workunit.client.1.smithi193.stdout:7/54: mknod d6/d9/c13 0 2024-03-23T08:23:15.211 INFO:tasks.workunit.client.1.smithi193.stdout:7/55: write d6/ff [727867,55632] 0 2024-03-23T08:23:15.211 INFO:tasks.workunit.client.1.smithi193.stdout:8/50: mkdir df/d10 0 2024-03-23T08:23:15.212 INFO:tasks.workunit.client.1.smithi193.stdout:8/51: chown ca 565913138 1 2024-03-23T08:23:15.213 INFO:tasks.workunit.client.1.smithi193.stdout:7/56: read d6/f8 [106749,36729] 0 2024-03-23T08:23:15.213 INFO:tasks.workunit.client.1.smithi193.stdout:2/128: rename d0/c1 to d0/d1b/c22 0 2024-03-23T08:23:15.215 INFO:tasks.workunit.client.1.smithi193.stdout:4/74: rmdir d11 39 2024-03-23T08:23:15.216 INFO:tasks.workunit.client.1.smithi193.stdout:2/129: truncate d0/f1d 363303 0 2024-03-23T08:23:15.216 INFO:tasks.workunit.client.1.smithi193.stdout:6/50: dwrite d0/fe [0,4194304] 0 2024-03-23T08:23:15.216 INFO:tasks.workunit.client.1.smithi193.stdout:7/57: creat d6/f14 x:0 0 0 2024-03-23T08:23:15.221 INFO:tasks.workunit.client.0.smithi070.stdout:3/30: truncate f6 184145 0 2024-03-23T08:23:15.227 INFO:tasks.workunit.client.1.smithi193.stdout:6/51: mkdir d0/db/df 0 2024-03-23T08:23:15.232 INFO:tasks.workunit.client.0.smithi070.stdout:6/45: truncate d0/f6 1277782 0 2024-03-23T08:23:15.233 INFO:tasks.workunit.client.1.smithi193.stdout:4/75: rename ff to d11/d16/f18 0 2024-03-23T08:23:15.239 INFO:tasks.workunit.client.0.smithi070.stdout:6/46: creat d0/d8/f11 x:0 0 0 2024-03-23T08:23:15.239 INFO:tasks.workunit.client.1.smithi193.stdout:9/50: rmdir d2 39 2024-03-23T08:23:15.240 INFO:tasks.workunit.client.0.smithi070.stdout:6/47: fdatasync d0/d8/f10 0 2024-03-23T08:23:15.241 INFO:tasks.workunit.client.1.smithi193.stdout:6/52: write d0/f5 [5063927,120001] 0 2024-03-23T08:23:15.242 INFO:tasks.workunit.client.1.smithi193.stdout:3/59: write d1/fa [136440,50442] 0 2024-03-23T08:23:15.244 INFO:tasks.workunit.client.1.smithi193.stdout:2/130: link d0/d1b/ca d0/d1b/d19/c23 0 2024-03-23T08:23:15.244 INFO:tasks.workunit.client.1.smithi193.stdout:0/75: truncate db/d11/f12 313001 0 2024-03-23T08:23:15.249 INFO:tasks.workunit.client.1.smithi193.stdout:3/60: dwrite d1/f2 [0,4194304] 0 2024-03-23T08:23:15.253 INFO:tasks.workunit.client.1.smithi193.stdout:7/58: getdents d6/d9 0 2024-03-23T08:23:15.253 INFO:tasks.workunit.client.1.smithi193.stdout:0/76: readlink db/l15 0 2024-03-23T08:23:15.254 INFO:tasks.workunit.client.1.smithi193.stdout:0/77: readlink db/l15 0 2024-03-23T08:23:15.254 INFO:tasks.workunit.client.1.smithi193.stdout:9/51: rmdir d2/d7 39 2024-03-23T08:23:15.254 INFO:tasks.workunit.client.1.smithi193.stdout:9/52: read d2/f4 [160871,6354] 0 2024-03-23T08:23:15.255 INFO:tasks.workunit.client.1.smithi193.stdout:6/53: symlink d0/d8/l10 0 2024-03-23T08:23:15.257 INFO:tasks.workunit.client.1.smithi193.stdout:3/61: readlink d1/lb 0 2024-03-23T08:23:15.258 INFO:tasks.workunit.client.1.smithi193.stdout:2/131: chown d0/d1b/d19/c23 23716833 1 2024-03-23T08:23:15.260 INFO:tasks.workunit.client.1.smithi193.stdout:0/78: dread f4 [0,4194304] 0 2024-03-23T08:23:15.261 INFO:tasks.workunit.client.1.smithi193.stdout:9/53: dread - d2/d7/db/f13 zero size 2024-03-23T08:23:15.271 INFO:tasks.workunit.client.1.smithi193.stdout:3/62: creat d1/de/f16 x:0 0 0 2024-03-23T08:23:15.273 INFO:tasks.workunit.client.1.smithi193.stdout:3/63: truncate d1/de/f11 162729 0 2024-03-23T08:23:15.273 INFO:tasks.workunit.client.1.smithi193.stdout:4/76: rename d11/d16 to d11/d19 0 2024-03-23T08:23:15.275 INFO:tasks.workunit.client.1.smithi193.stdout:4/77: write fd [1624687,122710] 0 2024-03-23T08:23:15.276 INFO:tasks.workunit.client.1.smithi193.stdout:6/54: dwrite d0/f2 [4194304,4194304] 0 2024-03-23T08:23:15.276 INFO:tasks.workunit.client.0.smithi070.stdout:1/34: rename c7 to d8/cf 0 2024-03-23T08:23:15.277 INFO:tasks.workunit.client.0.smithi070.stdout:1/35: write d8/fe [70624,8852] 0 2024-03-23T08:23:15.282 INFO:tasks.workunit.client.0.smithi070.stdout:1/36: mknod d8/c10 0 2024-03-23T08:23:15.283 INFO:tasks.workunit.client.1.smithi193.stdout:4/78: dwrite f8 [0,4194304] 0 2024-03-23T08:23:15.284 INFO:tasks.workunit.client.1.smithi193.stdout:0/79: symlink db/d11/d14/l1b 0 2024-03-23T08:23:15.292 INFO:tasks.workunit.client.1.smithi193.stdout:4/79: dwrite fd [0,4194304] 0 2024-03-23T08:23:15.292 INFO:tasks.workunit.client.0.smithi070.stdout:6/48: link d0/f6 d0/f12 0 2024-03-23T08:23:15.292 INFO:tasks.workunit.client.1.smithi193.stdout:0/80: dread f7 [0,4194304] 0 2024-03-23T08:23:15.293 INFO:tasks.workunit.client.1.smithi193.stdout:4/80: readlink l10 0 2024-03-23T08:23:15.294 INFO:tasks.workunit.client.1.smithi193.stdout:4/81: chown f5 50324079 1 2024-03-23T08:23:15.295 INFO:tasks.workunit.client.0.smithi070.stdout:1/37: dread f3 [0,4194304] 0 2024-03-23T08:23:15.297 INFO:tasks.workunit.client.0.smithi070.stdout:1/38: stat d8/db 0 2024-03-23T08:23:15.298 INFO:tasks.workunit.client.1.smithi193.stdout:7/59: dwrite d6/f8 [0,4194304] 0 2024-03-23T08:23:15.298 INFO:tasks.workunit.client.0.smithi070.stdout:3/31: dwrite f6 [0,4194304] 0 2024-03-23T08:23:15.299 INFO:tasks.workunit.client.0.smithi070.stdout:1/39: chown d8/fe 322 1 2024-03-23T08:23:15.303 INFO:tasks.workunit.client.0.smithi070.stdout:3/32: dread f1 [0,4194304] 0 2024-03-23T08:23:15.304 INFO:tasks.workunit.client.0.smithi070.stdout:3/33: readlink db/lc 0 2024-03-23T08:23:15.304 INFO:tasks.workunit.client.0.smithi070.stdout:6/49: symlink d0/d3/l13 0 2024-03-23T08:23:15.305 INFO:tasks.workunit.client.0.smithi070.stdout:6/50: dread - d0/d8/f10 zero size 2024-03-23T08:23:15.312 INFO:tasks.workunit.client.0.smithi070.stdout:6/51: rename d0/d8/cc to d0/d8/c14 0 2024-03-23T08:23:15.314 INFO:tasks.workunit.client.0.smithi070.stdout:3/34: dwrite f1 [0,4194304] 0 2024-03-23T08:23:15.314 INFO:tasks.workunit.client.0.smithi070.stdout:3/35: dread - f9 zero size 2024-03-23T08:23:15.318 INFO:tasks.workunit.client.0.smithi070.stdout:6/52: mknod d0/d3/c15 0 2024-03-23T08:23:15.319 INFO:tasks.workunit.client.1.smithi193.stdout:4/82: mkdir d11/d1a 0 2024-03-23T08:23:15.320 INFO:tasks.workunit.client.0.smithi070.stdout:3/36: rename db/lc to db/ld 0 2024-03-23T08:23:15.321 INFO:tasks.workunit.client.1.smithi193.stdout:3/64: symlink d1/l17 0 2024-03-23T08:23:15.322 INFO:tasks.workunit.client.0.smithi070.stdout:1/40: getdents d8 0 2024-03-23T08:23:15.323 INFO:tasks.workunit.client.1.smithi193.stdout:7/60: creat d6/d9/f15 x:0 0 0 2024-03-23T08:23:15.324 INFO:tasks.workunit.client.1.smithi193.stdout:7/61: readlink l1 0 2024-03-23T08:23:15.326 INFO:tasks.workunit.client.0.smithi070.stdout:6/53: readlink d0/d3/ld 0 2024-03-23T08:23:15.326 INFO:tasks.workunit.client.0.smithi070.stdout:1/41: read d8/db/fc [918985,80609] 0 2024-03-23T08:23:15.327 INFO:tasks.workunit.client.1.smithi193.stdout:4/83: dwrite f8 [0,4194304] 0 2024-03-23T08:23:15.328 INFO:tasks.workunit.client.1.smithi193.stdout:5/37: sync 2024-03-23T08:23:15.328 INFO:tasks.workunit.client.1.smithi193.stdout:1/61: sync 2024-03-23T08:23:15.329 INFO:tasks.workunit.client.0.smithi070.stdout:3/37: dwrite fa [0,4194304] 0 2024-03-23T08:23:15.330 INFO:tasks.workunit.client.0.smithi070.stdout:6/54: creat d0/f16 x:0 0 0 2024-03-23T08:23:15.331 INFO:tasks.workunit.client.1.smithi193.stdout:8/52: sync 2024-03-23T08:23:15.331 INFO:tasks.workunit.client.0.smithi070.stdout:1/42: mkdir d8/d11 0 2024-03-23T08:23:15.333 INFO:tasks.workunit.client.1.smithi193.stdout:3/65: dwrite d1/de/f10 [0,4194304] 0 2024-03-23T08:23:15.333 INFO:tasks.workunit.client.1.smithi193.stdout:2/132: link d0/d1b/d19/c1e d0/c24 0 2024-03-23T08:23:15.334 INFO:tasks.workunit.client.1.smithi193.stdout:3/66: fsync d1/f2 0 2024-03-23T08:23:15.336 INFO:tasks.workunit.client.1.smithi193.stdout:3/67: read d1/f2 [1360462,95509] 0 2024-03-23T08:23:15.336 INFO:tasks.workunit.client.0.smithi070.stdout:3/38: dread f6 [0,4194304] 0 2024-03-23T08:23:15.337 INFO:tasks.workunit.client.0.smithi070.stdout:6/55: creat d0/d8/f17 x:0 0 0 2024-03-23T08:23:15.337 INFO:tasks.workunit.client.0.smithi070.stdout:1/43: dread d8/fe [0,4194304] 0 2024-03-23T08:23:15.338 INFO:tasks.workunit.client.0.smithi070.stdout:6/56: dread - d0/d8/f17 zero size 2024-03-23T08:23:15.338 INFO:tasks.workunit.client.0.smithi070.stdout:6/57: readlink d0/lf 0 2024-03-23T08:23:15.341 INFO:tasks.workunit.client.1.smithi193.stdout:7/62: creat d6/d9/f16 x:0 0 0 2024-03-23T08:23:15.341 INFO:tasks.workunit.client.0.smithi070.stdout:6/58: write d0/d8/f10 [321940,40002] 0 2024-03-23T08:23:15.343 INFO:tasks.workunit.client.0.smithi070.stdout:6/59: readlink d0/d3/l13 0 2024-03-23T08:23:15.345 INFO:tasks.workunit.client.0.smithi070.stdout:1/44: chown d8/cf 1705 1 2024-03-23T08:23:15.346 INFO:tasks.workunit.client.0.smithi070.stdout:3/39: dwrite f1 [0,4194304] 0 2024-03-23T08:23:15.350 INFO:tasks.workunit.client.0.smithi070.stdout:6/60: mknod d0/d3/c18 0 2024-03-23T08:23:15.352 INFO:tasks.workunit.client.0.smithi070.stdout:6/61: truncate d0/d8/f17 235667 0 2024-03-23T08:23:15.355 INFO:tasks.workunit.client.0.smithi070.stdout:6/62: dread d0/d8/f17 [0,4194304] 0 2024-03-23T08:23:15.359 INFO:tasks.workunit.client.1.smithi193.stdout:9/54: dwrite d2/f3 [0,4194304] 0 2024-03-23T08:23:15.361 INFO:tasks.workunit.client.1.smithi193.stdout:6/55: truncate d0/f2 4823737 0 2024-03-23T08:23:15.362 INFO:tasks.workunit.client.1.smithi193.stdout:6/56: stat d0/db/lc 0 2024-03-23T08:23:15.365 INFO:tasks.workunit.client.1.smithi193.stdout:4/84: mknod d11/d1a/c1b 0 2024-03-23T08:23:15.365 INFO:tasks.workunit.client.1.smithi193.stdout:4/85: readlink l10 0 2024-03-23T08:23:15.366 INFO:tasks.workunit.client.0.smithi070.stdout:6/63: creat d0/d8/f19 x:0 0 0 2024-03-23T08:23:15.367 INFO:tasks.workunit.client.1.smithi193.stdout:4/86: write d11/f12 [1107758,27216] 0 2024-03-23T08:23:15.367 INFO:tasks.workunit.client.1.smithi193.stdout:1/62: truncate f5 714474 0 2024-03-23T08:23:15.371 INFO:tasks.workunit.client.0.smithi070.stdout:6/64: dread d0/d3/fa [0,4194304] 0 2024-03-23T08:23:15.371 INFO:tasks.workunit.client.1.smithi193.stdout:3/68: mknod d1/c18 0 2024-03-23T08:23:15.372 INFO:tasks.workunit.client.1.smithi193.stdout:4/87: dread f8 [0,4194304] 0 2024-03-23T08:23:15.372 INFO:tasks.workunit.client.1.smithi193.stdout:9/55: creat d2/d7/f15 x:0 0 0 2024-03-23T08:23:15.376 INFO:tasks.workunit.client.0.smithi070.stdout:1/45: write f3 [1269405,50390] 0 2024-03-23T08:23:15.378 INFO:tasks.workunit.client.1.smithi193.stdout:9/56: dread d2/f4 [0,4194304] 0 2024-03-23T08:23:15.378 INFO:tasks.workunit.client.1.smithi193.stdout:6/57: chown d0/c7 122 1 2024-03-23T08:23:15.380 INFO:tasks.workunit.client.0.smithi070.stdout:3/40: truncate f1 4145458 0 2024-03-23T08:23:15.381 INFO:tasks.workunit.client.1.smithi193.stdout:4/88: dread f5 [0,4194304] 0 2024-03-23T08:23:15.381 INFO:tasks.workunit.client.0.smithi070.stdout:1/46: rename d8/db/fc to d8/db/f12 0 2024-03-23T08:23:15.381 INFO:tasks.workunit.client.1.smithi193.stdout:0/81: write db/d11/f12 [397457,13807] 0 2024-03-23T08:23:15.384 INFO:tasks.workunit.client.1.smithi193.stdout:3/69: chown d1/c5 60789909 1 2024-03-23T08:23:15.384 INFO:tasks.workunit.client.1.smithi193.stdout:3/70: stat d1 0 2024-03-23T08:23:15.385 INFO:tasks.workunit.client.1.smithi193.stdout:7/63: link d6/lb d6/l17 0 2024-03-23T08:23:15.387 INFO:tasks.workunit.client.1.smithi193.stdout:2/133: truncate d0/fc 1566059 0 2024-03-23T08:23:15.387 INFO:tasks.workunit.client.1.smithi193.stdout:7/64: dread - d6/d9/fe zero size 2024-03-23T08:23:15.387 INFO:tasks.workunit.client.1.smithi193.stdout:5/38: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:15.387 INFO:tasks.workunit.client.1.smithi193.stdout:8/53: getdents df 0 2024-03-23T08:23:15.388 INFO:tasks.workunit.client.0.smithi070.stdout:3/41: rename db/ld to db/le 0 2024-03-23T08:23:15.390 INFO:tasks.workunit.client.0.smithi070.stdout:1/47: symlink d8/d11/l13 0 2024-03-23T08:23:15.391 INFO:tasks.workunit.client.1.smithi193.stdout:9/57: rmdir d2 39 2024-03-23T08:23:15.393 INFO:tasks.workunit.client.0.smithi070.stdout:3/42: mkdir db/df 0 2024-03-23T08:23:15.395 INFO:tasks.workunit.client.0.smithi070.stdout:1/48: rmdir d8 39 2024-03-23T08:23:15.398 INFO:tasks.workunit.client.1.smithi193.stdout:0/82: unlink c8 0 2024-03-23T08:23:15.400 INFO:tasks.workunit.client.1.smithi193.stdout:1/63: mknod da/c18 0 2024-03-23T08:23:15.400 INFO:tasks.workunit.client.0.smithi070.stdout:6/65: dwrite d0/d3/f4 [0,4194304] 0 2024-03-23T08:23:15.401 INFO:tasks.workunit.client.1.smithi193.stdout:1/64: write f1 [4991943,10491] 0 2024-03-23T08:23:15.403 INFO:tasks.workunit.client.0.smithi070.stdout:3/43: write f6 [4577231,21989] 0 2024-03-23T08:23:15.408 INFO:tasks.workunit.client.0.smithi070.stdout:6/66: dread d0/d8/fe [0,4194304] 0 2024-03-23T08:23:15.409 INFO:tasks.workunit.client.1.smithi193.stdout:7/65: symlink d6/d9/l18 0 2024-03-23T08:23:15.410 INFO:tasks.workunit.client.0.smithi070.stdout:3/44: unlink f1 0 2024-03-23T08:23:15.411 INFO:tasks.workunit.client.1.smithi193.stdout:8/54: rmdir df 39 2024-03-23T08:23:15.417 INFO:tasks.workunit.client.0.smithi070.stdout:3/45: symlink db/l10 0 2024-03-23T08:23:15.419 INFO:tasks.workunit.client.0.smithi070.stdout:3/46: write f6 [2232557,47200] 0 2024-03-23T08:23:15.419 INFO:tasks.workunit.client.1.smithi193.stdout:0/83: dread f7 [4194304,4194304] 0 2024-03-23T08:23:15.419 INFO:tasks.workunit.client.1.smithi193.stdout:0/84: fsync db/d11/f12 0 2024-03-23T08:23:15.420 INFO:tasks.workunit.client.1.smithi193.stdout:3/71: mkdir d1/d19 0 2024-03-23T08:23:15.421 INFO:tasks.workunit.client.0.smithi070.stdout:1/49: write d8/db/f12 [5122074,100436] 0 2024-03-23T08:23:15.424 INFO:tasks.workunit.client.1.smithi193.stdout:7/66: creat d6/d9/f19 x:0 0 0 2024-03-23T08:23:15.424 INFO:tasks.workunit.client.0.smithi070.stdout:6/67: dwrite d0/d8/fe [0,4194304] 0 2024-03-23T08:23:15.424 INFO:tasks.workunit.client.0.smithi070.stdout:1/50: write d8/f9 [804773,48948] 0 2024-03-23T08:23:15.425 INFO:tasks.workunit.client.0.smithi070.stdout:3/47: mknod db/df/c11 0 2024-03-23T08:23:15.426 INFO:tasks.workunit.client.0.smithi070.stdout:3/48: chown f6 574608723 1 2024-03-23T08:23:15.427 INFO:tasks.workunit.client.1.smithi193.stdout:0/85: dwrite fa [0,4194304] 0 2024-03-23T08:23:15.428 INFO:tasks.workunit.client.1.smithi193.stdout:3/72: dread d1/de/f10 [0,4194304] 0 2024-03-23T08:23:15.431 INFO:tasks.workunit.client.1.smithi193.stdout:9/58: symlink d2/d7/l16 0 2024-03-23T08:23:15.432 INFO:tasks.workunit.client.1.smithi193.stdout:9/59: readlink d2/l6 0 2024-03-23T08:23:15.433 INFO:tasks.workunit.client.1.smithi193.stdout:3/73: dread d1/de/f10 [0,4194304] 0 2024-03-23T08:23:15.436 INFO:tasks.workunit.client.0.smithi070.stdout:6/68: fdatasync d0/f12 0 2024-03-23T08:23:15.436 INFO:tasks.workunit.client.0.smithi070.stdout:1/51: dread d8/db/f12 [0,4194304] 0 2024-03-23T08:23:15.436 INFO:tasks.workunit.client.0.smithi070.stdout:3/49: dwrite f6 [0,4194304] 0 2024-03-23T08:23:15.438 INFO:tasks.workunit.client.0.smithi070.stdout:3/50: dread - f9 zero size 2024-03-23T08:23:15.441 INFO:tasks.workunit.client.1.smithi193.stdout:2/134: dwrite d0/f7 [4194304,4194304] 0 2024-03-23T08:23:15.443 INFO:tasks.workunit.client.0.smithi070.stdout:1/52: rename d8/c10 to d8/d11/c14 0 2024-03-23T08:23:15.444 INFO:tasks.workunit.client.1.smithi193.stdout:8/55: symlink df/l11 0 2024-03-23T08:23:15.444 INFO:tasks.workunit.client.0.smithi070.stdout:3/51: symlink db/df/l12 0 2024-03-23T08:23:15.446 INFO:tasks.workunit.client.1.smithi193.stdout:7/67: rename d6/d9/l18 to d6/l1a 0 2024-03-23T08:23:15.447 INFO:tasks.workunit.client.1.smithi193.stdout:8/56: stat cb 0 2024-03-23T08:23:15.448 INFO:tasks.workunit.client.1.smithi193.stdout:8/57: fdatasync f3 0 2024-03-23T08:23:15.449 INFO:tasks.workunit.client.0.smithi070.stdout:3/52: rename f9 to db/df/f13 0 2024-03-23T08:23:15.450 INFO:tasks.workunit.client.1.smithi193.stdout:8/58: write f0 [9041279,13301] 0 2024-03-23T08:23:15.452 INFO:tasks.workunit.client.1.smithi193.stdout:4/89: getdents d11/d1a 0 2024-03-23T08:23:15.455 INFO:tasks.workunit.client.1.smithi193.stdout:7/68: dread d6/f8 [0,4194304] 0 2024-03-23T08:23:15.457 INFO:tasks.workunit.client.1.smithi193.stdout:2/135: mkdir d0/d1b/d14/d25 0 2024-03-23T08:23:15.458 INFO:tasks.workunit.client.1.smithi193.stdout:4/90: dread f4 [0,4194304] 0 2024-03-23T08:23:15.460 INFO:tasks.workunit.client.1.smithi193.stdout:8/59: creat df/f12 x:0 0 0 2024-03-23T08:23:15.461 INFO:tasks.workunit.client.1.smithi193.stdout:3/74: link d1/fa d1/de/d13/f1a 0 2024-03-23T08:23:15.462 INFO:tasks.workunit.client.1.smithi193.stdout:7/69: creat d6/d9/f1b x:0 0 0 2024-03-23T08:23:15.463 INFO:tasks.workunit.client.1.smithi193.stdout:7/70: chown d6/d9 619536 1 2024-03-23T08:23:15.463 INFO:tasks.workunit.client.1.smithi193.stdout:7/71: fdatasync d6/d9/fe 0 2024-03-23T08:23:15.463 INFO:tasks.workunit.client.1.smithi193.stdout:2/136: mknod d0/d1b/d14/c26 0 2024-03-23T08:23:15.465 INFO:tasks.workunit.client.1.smithi193.stdout:8/60: mkdir df/d13 0 2024-03-23T08:23:15.465 INFO:tasks.workunit.client.1.smithi193.stdout:0/86: getdents db/d11/d14 0 2024-03-23T08:23:15.466 INFO:tasks.workunit.client.1.smithi193.stdout:2/137: readlink d0/l11 0 2024-03-23T08:23:15.466 INFO:tasks.workunit.client.1.smithi193.stdout:0/87: write fa [1625352,70108] 0 2024-03-23T08:23:15.467 INFO:tasks.workunit.client.1.smithi193.stdout:7/72: read d6/f8 [3170065,66221] 0 2024-03-23T08:23:15.467 INFO:tasks.workunit.client.1.smithi193.stdout:3/75: getdents d1/d19 0 2024-03-23T08:23:15.469 INFO:tasks.workunit.client.1.smithi193.stdout:7/73: creat d6/d9/f1c x:0 0 0 2024-03-23T08:23:15.471 INFO:tasks.workunit.client.1.smithi193.stdout:2/138: symlink d0/d1b/d14/d25/l27 0 2024-03-23T08:23:15.472 INFO:tasks.workunit.client.1.smithi193.stdout:8/61: fdatasync f0 0 2024-03-23T08:23:15.473 INFO:tasks.workunit.client.1.smithi193.stdout:3/76: dread d1/fc [0,4194304] 0 2024-03-23T08:23:15.474 INFO:tasks.workunit.client.1.smithi193.stdout:0/88: link db/c17 db/c1c 0 2024-03-23T08:23:15.475 INFO:tasks.workunit.client.1.smithi193.stdout:2/139: chown d0/l10 0 1 2024-03-23T08:23:15.479 INFO:tasks.workunit.client.1.smithi193.stdout:7/74: dwrite d6/ff [0,4194304] 0 2024-03-23T08:23:15.479 INFO:tasks.workunit.client.1.smithi193.stdout:8/62: rename cd to df/c14 0 2024-03-23T08:23:15.480 INFO:tasks.workunit.client.1.smithi193.stdout:7/75: fsync d6/d9/f12 0 2024-03-23T08:23:15.480 INFO:tasks.workunit.client.1.smithi193.stdout:2/140: mkdir d0/d28 0 2024-03-23T08:23:15.483 INFO:tasks.workunit.client.1.smithi193.stdout:2/141: dread - d0/d1b/f21 zero size 2024-03-23T08:23:15.484 INFO:tasks.workunit.client.0.smithi070.stdout:6/69: dwrite d0/f12 [0,4194304] 0 2024-03-23T08:23:15.486 INFO:tasks.workunit.client.1.smithi193.stdout:7/76: symlink d6/d9/l1d 0 2024-03-23T08:23:15.487 INFO:tasks.workunit.client.1.smithi193.stdout:7/77: read - d6/f10 zero size 2024-03-23T08:23:15.488 INFO:tasks.workunit.client.1.smithi193.stdout:8/63: dwrite fc [0,4194304] 0 2024-03-23T08:23:15.489 INFO:tasks.workunit.client.0.smithi070.stdout:6/70: unlink d0/d3/l13 0 2024-03-23T08:23:15.489 INFO:tasks.workunit.client.1.smithi193.stdout:2/142: dread d0/d1b/f8 [0,4194304] 0 2024-03-23T08:23:15.492 INFO:tasks.workunit.client.0.smithi070.stdout:6/71: write d0/d8/f19 [180950,75421] 0 2024-03-23T08:23:15.492 INFO:tasks.workunit.client.1.smithi193.stdout:3/77: rename d1/l14 to d1/de/l1b 0 2024-03-23T08:23:15.494 INFO:tasks.workunit.client.0.smithi070.stdout:3/53: truncate fa 2952330 0 2024-03-23T08:23:15.494 INFO:tasks.workunit.client.0.smithi070.stdout:1/53: dwrite d8/fe [0,4194304] 0 2024-03-23T08:23:15.496 INFO:tasks.workunit.client.0.smithi070.stdout:6/72: read d0/f6 [2547609,21818] 0 2024-03-23T08:23:15.497 INFO:tasks.workunit.client.1.smithi193.stdout:7/78: symlink d6/d9/l1e 0 2024-03-23T08:23:15.497 INFO:tasks.workunit.client.0.smithi070.stdout:6/73: stat d0/d3/c15 0 2024-03-23T08:23:15.502 INFO:tasks.workunit.client.0.smithi070.stdout:3/54: mkdir db/d14 0 2024-03-23T08:23:15.502 INFO:tasks.workunit.client.1.smithi193.stdout:8/64: dwrite fc [0,4194304] 0 2024-03-23T08:23:15.502 INFO:tasks.workunit.client.1.smithi193.stdout:2/143: creat d0/d1b/d14/d25/f29 x:0 0 0 2024-03-23T08:23:15.503 INFO:tasks.workunit.client.1.smithi193.stdout:9/60: truncate d2/f3 3724577 0 2024-03-23T08:23:15.505 INFO:tasks.workunit.client.0.smithi070.stdout:1/54: creat d8/db/f15 x:0 0 0 2024-03-23T08:23:15.506 INFO:tasks.workunit.client.0.smithi070.stdout:1/55: write d8/f9 [1561833,79411] 0 2024-03-23T08:23:15.507 INFO:tasks.workunit.client.1.smithi193.stdout:6/58: dwrite d0/f2 [4194304,4194304] 0 2024-03-23T08:23:15.507 INFO:tasks.workunit.client.1.smithi193.stdout:7/79: mkdir d6/d9/d1f 0 2024-03-23T08:23:15.509 INFO:tasks.workunit.client.0.smithi070.stdout:6/74: creat d0/d3/f1a x:0 0 0 2024-03-23T08:23:15.511 INFO:tasks.workunit.client.1.smithi193.stdout:8/65: symlink df/l15 0 2024-03-23T08:23:15.513 INFO:tasks.workunit.client.1.smithi193.stdout:9/61: mkdir d2/d7/db/d17 0 2024-03-23T08:23:15.514 INFO:tasks.workunit.client.0.smithi070.stdout:3/55: mkdir db/df/d15 0 2024-03-23T08:23:15.516 INFO:tasks.workunit.client.1.smithi193.stdout:7/80: chown d6/l1a 1015949 1 2024-03-23T08:23:15.517 INFO:tasks.workunit.client.0.smithi070.stdout:1/56: creat d8/db/f16 x:0 0 0 2024-03-23T08:23:15.521 INFO:tasks.workunit.client.1.smithi193.stdout:4/91: dwrite f9 [0,4194304] 0 2024-03-23T08:23:15.522 INFO:tasks.workunit.client.0.smithi070.stdout:1/57: dread f3 [0,4194304] 0 2024-03-23T08:23:15.526 INFO:tasks.workunit.client.1.smithi193.stdout:8/66: link c4 df/d13/c16 0 2024-03-23T08:23:15.527 INFO:tasks.workunit.client.1.smithi193.stdout:7/81: dwrite d6/d9/f12 [0,4194304] 0 2024-03-23T08:23:15.532 INFO:tasks.workunit.client.1.smithi193.stdout:6/59: fdatasync d0/f2 0 2024-03-23T08:23:15.533 INFO:tasks.workunit.client.1.smithi193.stdout:4/92: creat d11/d1a/f1c x:0 0 0 2024-03-23T08:23:15.537 INFO:tasks.workunit.client.1.smithi193.stdout:0/89: dwrite f7 [4194304,4194304] 0 2024-03-23T08:23:15.538 INFO:tasks.workunit.client.1.smithi193.stdout:0/90: dread - db/d11/f13 zero size 2024-03-23T08:23:15.539 INFO:tasks.workunit.client.0.smithi070.stdout:1/58: dwrite d8/db/f12 [0,4194304] 0 2024-03-23T08:23:15.541 INFO:tasks.workunit.client.0.smithi070.stdout:1/59: chown d8/db/f12 4646 1 2024-03-23T08:23:15.543 INFO:tasks.workunit.client.1.smithi193.stdout:6/60: unlink d0/db/lc 0 2024-03-23T08:23:15.544 INFO:tasks.workunit.client.1.smithi193.stdout:0/91: creat db/f1d x:0 0 0 2024-03-23T08:23:15.545 INFO:tasks.workunit.client.0.smithi070.stdout:1/60: symlink d8/db/l17 0 2024-03-23T08:23:15.546 INFO:tasks.workunit.client.1.smithi193.stdout:3/78: rmdir d1/de 39 2024-03-23T08:23:15.548 INFO:tasks.workunit.client.1.smithi193.stdout:4/93: dwrite f8 [0,4194304] 0 2024-03-23T08:23:15.549 INFO:tasks.workunit.client.1.smithi193.stdout:4/94: fdatasync d11/f13 0 2024-03-23T08:23:15.551 INFO:tasks.workunit.client.1.smithi193.stdout:0/92: dread f7 [0,4194304] 0 2024-03-23T08:23:15.552 INFO:tasks.workunit.client.0.smithi070.stdout:3/56: rename fa to db/df/d15/f16 0 2024-03-23T08:23:15.552 INFO:tasks.workunit.client.1.smithi193.stdout:4/95: rmdir d11 39 2024-03-23T08:23:15.553 INFO:tasks.workunit.client.0.smithi070.stdout:3/57: fsync f6 0 2024-03-23T08:23:15.554 INFO:tasks.workunit.client.1.smithi193.stdout:6/61: mkdir d0/db/df/d11 0 2024-03-23T08:23:15.558 INFO:tasks.workunit.client.1.smithi193.stdout:4/96: readlink d11/l15 0 2024-03-23T08:23:15.559 INFO:tasks.workunit.client.0.smithi070.stdout:3/58: creat db/f17 x:0 0 0 2024-03-23T08:23:15.559 INFO:tasks.workunit.client.0.smithi070.stdout:3/59: rename db/df to db/df/d15/d18 22 2024-03-23T08:23:15.561 INFO:tasks.workunit.client.1.smithi193.stdout:0/93: dwrite db/d11/f12 [0,4194304] 0 2024-03-23T08:23:15.562 INFO:tasks.workunit.client.1.smithi193.stdout:7/82: rmdir d6 39 2024-03-23T08:23:15.563 INFO:tasks.workunit.client.0.smithi070.stdout:3/60: write db/df/f13 [442420,75059] 0 2024-03-23T08:23:15.566 INFO:tasks.workunit.client.1.smithi193.stdout:5/39: sync 2024-03-23T08:23:15.566 INFO:tasks.workunit.client.1.smithi193.stdout:1/65: sync 2024-03-23T08:23:15.569 INFO:tasks.workunit.client.0.smithi070.stdout:6/75: write d0/d8/f17 [1157743,27179] 0 2024-03-23T08:23:15.570 INFO:tasks.workunit.client.0.smithi070.stdout:6/76: read d0/f12 [1259712,2193] 0 2024-03-23T08:23:15.572 INFO:tasks.workunit.client.1.smithi193.stdout:6/62: getdents d0 0 2024-03-23T08:23:15.572 INFO:tasks.workunit.client.1.smithi193.stdout:6/63: fsync d0/fe 0 2024-03-23T08:23:15.573 INFO:tasks.workunit.client.1.smithi193.stdout:5/40: symlink d0/ld 0 2024-03-23T08:23:15.576 INFO:tasks.workunit.client.0.smithi070.stdout:6/77: dread d0/d3/f4 [0,4194304] 0 2024-03-23T08:23:15.576 INFO:tasks.workunit.client.1.smithi193.stdout:5/41: write d0/fa [541911,81563] 0 2024-03-23T08:23:15.576 INFO:tasks.workunit.client.1.smithi193.stdout:0/94: link db/d11/f12 db/f1e 0 2024-03-23T08:23:15.578 INFO:tasks.workunit.client.1.smithi193.stdout:1/66: mknod da/c19 0 2024-03-23T08:23:15.581 INFO:tasks.workunit.client.1.smithi193.stdout:0/95: dread db/f1e [0,4194304] 0 2024-03-23T08:23:15.581 INFO:tasks.workunit.client.0.smithi070.stdout:6/78: symlink d0/d3/l1b 0 2024-03-23T08:23:15.582 INFO:tasks.workunit.client.1.smithi193.stdout:7/83: symlink d6/d9/d1f/l20 0 2024-03-23T08:23:15.582 INFO:tasks.workunit.client.1.smithi193.stdout:7/84: chown l1 16756 1 2024-03-23T08:23:15.587 INFO:tasks.workunit.client.0.smithi070.stdout:6/79: mkdir d0/d8/d1c 0 2024-03-23T08:23:15.587 INFO:tasks.workunit.client.1.smithi193.stdout:5/42: creat d0/fe x:0 0 0 2024-03-23T08:23:15.588 INFO:tasks.workunit.client.1.smithi193.stdout:9/62: write d2/f3 [687738,16628] 0 2024-03-23T08:23:15.589 INFO:tasks.workunit.client.0.smithi070.stdout:3/61: rename db/df/d15/f16 to db/df/f19 0 2024-03-23T08:23:15.589 INFO:tasks.workunit.client.1.smithi193.stdout:4/97: truncate fb 2483630 0 2024-03-23T08:23:15.591 INFO:tasks.workunit.client.1.smithi193.stdout:1/67: mknod da/c1a 0 2024-03-23T08:23:15.592 INFO:tasks.workunit.client.0.smithi070.stdout:6/80: rename d0/c1 to d0/d8/d1c/c1d 0 2024-03-23T08:23:15.594 INFO:tasks.workunit.client.1.smithi193.stdout:1/68: dread - da/dd/f14 zero size 2024-03-23T08:23:15.594 INFO:tasks.workunit.client.1.smithi193.stdout:9/63: dread d2/fc [0,4194304] 0 2024-03-23T08:23:15.597 INFO:tasks.workunit.client.1.smithi193.stdout:7/85: dwrite d6/f10 [0,4194304] 0 2024-03-23T08:23:15.597 INFO:tasks.workunit.client.1.smithi193.stdout:1/69: read f4 [4109315,50600] 0 2024-03-23T08:23:15.597 INFO:tasks.workunit.client.0.smithi070.stdout:6/81: mkdir d0/d8/d1c/d1e 0 2024-03-23T08:23:15.599 INFO:tasks.workunit.client.0.smithi070.stdout:3/62: truncate f6 966740 0 2024-03-23T08:23:15.601 INFO:tasks.workunit.client.1.smithi193.stdout:5/43: dwrite d0/f3 [0,4194304] 0 2024-03-23T08:23:15.603 INFO:tasks.workunit.client.0.smithi070.stdout:3/63: mkdir db/df/d1a 0 2024-03-23T08:23:15.607 INFO:tasks.workunit.client.1.smithi193.stdout:0/96: getdents db 0 2024-03-23T08:23:15.608 INFO:tasks.workunit.client.0.smithi070.stdout:6/82: dread d0/d3/f4 [0,4194304] 0 2024-03-23T08:23:15.609 INFO:tasks.workunit.client.1.smithi193.stdout:7/86: dwrite d6/d9/f16 [0,4194304] 0 2024-03-23T08:23:15.610 INFO:tasks.workunit.client.1.smithi193.stdout:1/70: symlink da/dd/l1b 0 2024-03-23T08:23:15.611 INFO:tasks.workunit.client.0.smithi070.stdout:3/64: creat db/d14/f1b x:0 0 0 2024-03-23T08:23:15.613 INFO:tasks.workunit.client.1.smithi193.stdout:4/98: rename d11/d19/f18 to d11/d1a/f1d 0 2024-03-23T08:23:15.615 INFO:tasks.workunit.client.0.smithi070.stdout:6/83: rename d0/d8/f11 to d0/d8/f1f 0 2024-03-23T08:23:15.619 INFO:tasks.workunit.client.1.smithi193.stdout:5/44: unlink d0/c8 0 2024-03-23T08:23:15.621 INFO:tasks.workunit.client.1.smithi193.stdout:9/64: symlink d2/l18 0 2024-03-23T08:23:15.625 INFO:tasks.workunit.client.0.smithi070.stdout:3/65: creat db/df/d1a/f1c x:0 0 0 2024-03-23T08:23:15.625 INFO:tasks.workunit.client.1.smithi193.stdout:7/87: creat d6/d9/f21 x:0 0 0 2024-03-23T08:23:15.627 INFO:tasks.workunit.client.1.smithi193.stdout:0/97: rename db/f1d to db/d11/d14/f1f 0 2024-03-23T08:23:15.628 INFO:tasks.workunit.client.1.smithi193.stdout:5/45: creat d0/ff x:0 0 0 2024-03-23T08:23:15.629 INFO:tasks.workunit.client.0.smithi070.stdout:6/84: rmdir d0/d8/d1c/d1e 0 2024-03-23T08:23:15.630 INFO:tasks.workunit.client.1.smithi193.stdout:1/71: dread f4 [0,4194304] 0 2024-03-23T08:23:15.630 INFO:tasks.workunit.client.1.smithi193.stdout:1/72: fdatasync da/fc 0 2024-03-23T08:23:15.631 INFO:tasks.workunit.client.1.smithi193.stdout:7/88: mknod d6/c22 0 2024-03-23T08:23:15.632 INFO:tasks.workunit.client.0.smithi070.stdout:6/85: chown d0/d8/d1c/c1d 59730 1 2024-03-23T08:23:15.633 INFO:tasks.workunit.client.0.smithi070.stdout:6/86: chown d0/f12 14 1 2024-03-23T08:23:15.633 INFO:tasks.workunit.client.1.smithi193.stdout:0/98: unlink f4 0 2024-03-23T08:23:15.634 INFO:tasks.workunit.client.1.smithi193.stdout:5/46: creat d0/f10 x:0 0 0 2024-03-23T08:23:15.635 INFO:tasks.workunit.client.1.smithi193.stdout:9/65: symlink d2/d7/db/d17/l19 0 2024-03-23T08:23:15.635 INFO:tasks.workunit.client.0.smithi070.stdout:3/66: rmdir db/d14 39 2024-03-23T08:23:15.637 INFO:tasks.workunit.client.0.smithi070.stdout:6/87: truncate d0/d3/f4 2852536 0 2024-03-23T08:23:15.638 INFO:tasks.workunit.client.1.smithi193.stdout:2/144: sync 2024-03-23T08:23:15.639 INFO:tasks.workunit.client.0.smithi070.stdout:3/67: symlink db/df/l1d 0 2024-03-23T08:23:15.640 INFO:tasks.workunit.client.1.smithi193.stdout:1/73: creat da/dd/f1c x:0 0 0 2024-03-23T08:23:15.643 INFO:tasks.workunit.client.0.smithi070.stdout:3/68: creat db/df/d15/f1e x:0 0 0 2024-03-23T08:23:15.643 INFO:tasks.workunit.client.1.smithi193.stdout:0/99: write db/d11/f1a [2946954,16241] 0 2024-03-23T08:23:15.644 INFO:tasks.workunit.client.1.smithi193.stdout:5/47: dwrite d0/f10 [0,4194304] 0 2024-03-23T08:23:15.645 INFO:tasks.workunit.client.1.smithi193.stdout:0/100: dread - db/d11/d14/f1f zero size 2024-03-23T08:23:15.646 INFO:tasks.workunit.client.1.smithi193.stdout:2/145: rmdir d0/d1b/d14 39 2024-03-23T08:23:15.646 INFO:tasks.workunit.client.1.smithi193.stdout:1/74: dread f4 [0,4194304] 0 2024-03-23T08:23:15.647 INFO:tasks.workunit.client.1.smithi193.stdout:2/146: readlink d0/l15 0 2024-03-23T08:23:15.648 INFO:tasks.workunit.client.0.smithi070.stdout:3/69: write db/d14/f1b [989408,85495] 0 2024-03-23T08:23:15.651 INFO:tasks.workunit.client.1.smithi193.stdout:9/66: unlink d2/d7/db/c14 0 2024-03-23T08:23:15.652 INFO:tasks.workunit.client.1.smithi193.stdout:9/67: chown d2/d7/l16 99 1 2024-03-23T08:23:15.652 INFO:tasks.workunit.client.0.smithi070.stdout:3/70: read db/df/f13 [52426,130247] 0 2024-03-23T08:23:15.654 INFO:tasks.workunit.client.1.smithi193.stdout:5/48: creat d0/f11 x:0 0 0 2024-03-23T08:23:15.655 INFO:tasks.workunit.client.1.smithi193.stdout:4/99: truncate f8 415524 0 2024-03-23T08:23:15.655 INFO:tasks.workunit.client.1.smithi193.stdout:4/100: readlink d11/l15 0 2024-03-23T08:23:15.656 INFO:tasks.workunit.client.1.smithi193.stdout:0/101: symlink db/l20 0 2024-03-23T08:23:15.657 INFO:tasks.workunit.client.0.smithi070.stdout:3/71: symlink db/df/l1f 0 2024-03-23T08:23:15.657 INFO:tasks.workunit.client.0.smithi070.stdout:3/72: fsync db/f17 0 2024-03-23T08:23:15.661 INFO:tasks.workunit.client.1.smithi193.stdout:9/68: dwrite d2/f3 [0,4194304] 0 2024-03-23T08:23:15.662 INFO:tasks.workunit.client.0.smithi070.stdout:6/88: dread d0/d3/f4 [0,4194304] 0 2024-03-23T08:23:15.663 INFO:tasks.workunit.client.1.smithi193.stdout:2/147: write d0/d1b/d14/d25/f29 [283024,89741] 0 2024-03-23T08:23:15.663 INFO:tasks.workunit.client.0.smithi070.stdout:3/73: dread db/d14/f1b [0,4194304] 0 2024-03-23T08:23:15.664 INFO:tasks.workunit.client.1.smithi193.stdout:1/75: rmdir da/dd 39 2024-03-23T08:23:15.668 INFO:tasks.workunit.client.0.smithi070.stdout:6/89: write d0/d3/fa [3810072,517] 0 2024-03-23T08:23:15.670 INFO:tasks.workunit.client.0.smithi070.stdout:6/90: stat d0/d8 0 2024-03-23T08:23:15.672 INFO:tasks.workunit.client.1.smithi193.stdout:5/49: dwrite d0/ff [0,4194304] 0 2024-03-23T08:23:15.673 INFO:tasks.workunit.client.0.smithi070.stdout:6/91: rmdir d0/d8/d1c 39 2024-03-23T08:23:15.673 INFO:tasks.workunit.client.0.smithi070.stdout:6/92: chown d0/d8 6778240 1 2024-03-23T08:23:15.673 INFO:tasks.workunit.client.0.smithi070.stdout:6/93: readlink d0/d3/l1b 0 2024-03-23T08:23:15.674 INFO:tasks.workunit.client.1.smithi193.stdout:4/101: dread d11/f12 [0,4194304] 0 2024-03-23T08:23:15.674 INFO:tasks.workunit.client.0.smithi070.stdout:3/74: mkdir db/d20 0 2024-03-23T08:23:15.676 INFO:tasks.workunit.client.0.smithi070.stdout:6/94: creat d0/f20 x:0 0 0 2024-03-23T08:23:15.677 INFO:tasks.workunit.client.1.smithi193.stdout:2/148: mkdir d0/d1b/d14/d25/d2a 0 2024-03-23T08:23:15.679 INFO:tasks.workunit.client.1.smithi193.stdout:4/102: readlink d11/l15 0 2024-03-23T08:23:15.679 INFO:tasks.workunit.client.1.smithi193.stdout:0/102: dwrite db/d11/f1a [0,4194304] 0 2024-03-23T08:23:15.679 INFO:tasks.workunit.client.1.smithi193.stdout:5/50: creat d0/f12 x:0 0 0 2024-03-23T08:23:15.680 INFO:tasks.workunit.client.0.smithi070.stdout:6/95: dread d0/f6 [0,4194304] 0 2024-03-23T08:23:15.681 INFO:tasks.workunit.client.1.smithi193.stdout:9/69: dwrite d2/f3 [0,4194304] 0 2024-03-23T08:23:15.682 INFO:tasks.workunit.client.0.smithi070.stdout:3/75: creat db/f21 x:0 0 0 2024-03-23T08:23:15.682 INFO:tasks.workunit.client.1.smithi193.stdout:1/76: creat da/dd/df/f1d x:0 0 0 2024-03-23T08:23:15.684 INFO:tasks.workunit.client.1.smithi193.stdout:0/103: mkdir db/d11/d14/d21 0 2024-03-23T08:23:15.685 INFO:tasks.workunit.client.0.smithi070.stdout:3/76: mkdir db/df/d15/d22 0 2024-03-23T08:23:15.687 INFO:tasks.workunit.client.0.smithi070.stdout:6/96: getdents d0/d3 0 2024-03-23T08:23:15.689 INFO:tasks.workunit.client.1.smithi193.stdout:0/104: chown db/d11/d14 3257 1 2024-03-23T08:23:15.689 INFO:tasks.workunit.client.1.smithi193.stdout:9/70: creat d2/d7/f1a x:0 0 0 2024-03-23T08:23:15.689 INFO:tasks.workunit.client.1.smithi193.stdout:1/77: creat da/dd/df/f1e x:0 0 0 2024-03-23T08:23:15.689 INFO:tasks.workunit.client.1.smithi193.stdout:1/78: read f1 [1717394,23932] 0 2024-03-23T08:23:15.690 INFO:tasks.workunit.client.1.smithi193.stdout:1/79: dread - da/fc zero size 2024-03-23T08:23:15.692 INFO:tasks.workunit.client.1.smithi193.stdout:1/80: truncate da/dd/df/f1e 87612 0 2024-03-23T08:23:15.692 INFO:tasks.workunit.client.1.smithi193.stdout:1/81: readlink da/d13/l15 0 2024-03-23T08:23:15.692 INFO:tasks.workunit.client.1.smithi193.stdout:1/82: stat da/d13/l15 0 2024-03-23T08:23:15.693 INFO:tasks.workunit.client.1.smithi193.stdout:1/83: chown f4 0 1 2024-03-23T08:23:15.693 INFO:tasks.workunit.client.1.smithi193.stdout:2/149: mknod d0/d1b/d14/d25/d2a/c2b 0 2024-03-23T08:23:15.694 INFO:tasks.workunit.client.0.smithi070.stdout:6/97: mknod d0/d8/d1c/c21 0 2024-03-23T08:23:15.696 INFO:tasks.workunit.client.1.smithi193.stdout:0/105: rename db/d11/f1a to db/d11/d14/d21/f22 0 2024-03-23T08:23:15.697 INFO:tasks.workunit.client.1.smithi193.stdout:9/71: fsync d2/fc 0 2024-03-23T08:23:15.697 INFO:tasks.workunit.client.1.smithi193.stdout:2/150: dread d0/d1b/d19/f1a [0,4194304] 0 2024-03-23T08:23:15.697 INFO:tasks.workunit.client.1.smithi193.stdout:6/64: sync 2024-03-23T08:23:15.697 INFO:tasks.workunit.client.1.smithi193.stdout:8/67: sync 2024-03-23T08:23:15.698 INFO:tasks.workunit.client.1.smithi193.stdout:3/79: sync 2024-03-23T08:23:15.698 INFO:tasks.workunit.client.1.smithi193.stdout:9/72: stat d2/d7/db 0 2024-03-23T08:23:15.698 INFO:tasks.workunit.client.0.smithi070.stdout:6/98: symlink d0/d3/l22 0 2024-03-23T08:23:15.698 INFO:tasks.workunit.client.0.smithi070.stdout:6/99: readlink d0/lf 0 2024-03-23T08:23:15.699 INFO:tasks.workunit.client.1.smithi193.stdout:3/80: write d1/f2 [1493412,52925] 0 2024-03-23T08:23:15.702 INFO:tasks.workunit.client.1.smithi193.stdout:1/84: creat da/dd/f1f x:0 0 0 2024-03-23T08:23:15.703 INFO:tasks.workunit.client.1.smithi193.stdout:1/85: truncate da/dd/df/f1e 281514 0 2024-03-23T08:23:15.705 INFO:tasks.workunit.client.1.smithi193.stdout:0/106: creat db/d11/d14/d21/f23 x:0 0 0 2024-03-23T08:23:15.707 INFO:tasks.workunit.client.1.smithi193.stdout:5/51: getdents d0 0 2024-03-23T08:23:15.708 INFO:tasks.workunit.client.1.smithi193.stdout:5/52: truncate d0/f12 386054 0 2024-03-23T08:23:15.710 INFO:tasks.workunit.client.1.smithi193.stdout:1/86: symlink da/dd/df/l20 0 2024-03-23T08:23:15.711 INFO:tasks.workunit.client.1.smithi193.stdout:0/107: mkdir db/d11/d14/d24 0 2024-03-23T08:23:15.712 INFO:tasks.workunit.client.1.smithi193.stdout:0/108: read - db/d11/f13 zero size 2024-03-23T08:23:15.713 INFO:tasks.workunit.client.1.smithi193.stdout:8/68: creat df/d10/f17 x:0 0 0 2024-03-23T08:23:15.714 INFO:tasks.workunit.client.0.smithi070.stdout:6/100: rename d0/d8 to d0/d23 0 2024-03-23T08:23:15.715 INFO:tasks.workunit.client.1.smithi193.stdout:8/69: read f3 [2934668,19246] 0 2024-03-23T08:23:15.715 INFO:tasks.workunit.client.0.smithi070.stdout:3/77: dwrite db/df/f19 [0,4194304] 0 2024-03-23T08:23:15.716 INFO:tasks.workunit.client.1.smithi193.stdout:2/151: symlink d0/d28/l2c 0 2024-03-23T08:23:15.718 INFO:tasks.workunit.client.1.smithi193.stdout:4/103: stat f8 0 2024-03-23T08:23:15.719 INFO:tasks.workunit.client.1.smithi193.stdout:8/70: dread f3 [0,4194304] 0 2024-03-23T08:23:15.719 INFO:tasks.workunit.client.1.smithi193.stdout:3/81: creat d1/d19/f1c x:0 0 0 2024-03-23T08:23:15.720 INFO:tasks.workunit.client.0.smithi070.stdout:3/78: rmdir db/df 39 2024-03-23T08:23:15.720 INFO:tasks.workunit.client.1.smithi193.stdout:8/71: write f3 [3757389,77502] 0 2024-03-23T08:23:15.725 INFO:tasks.workunit.client.0.smithi070.stdout:6/101: creat d0/d23/f24 x:0 0 0 2024-03-23T08:23:15.739 INFO:tasks.workunit.client.1.smithi193.stdout:5/53: rename d0/l1 to d0/l13 0 2024-03-23T08:23:15.739 INFO:tasks.workunit.client.1.smithi193.stdout:1/87: rmdir da 39 2024-03-23T08:23:15.739 INFO:tasks.workunit.client.1.smithi193.stdout:4/104: mknod d11/c1e 0 2024-03-23T08:23:15.740 INFO:tasks.workunit.client.1.smithi193.stdout:8/72: creat df/f18 x:0 0 0 2024-03-23T08:23:15.740 INFO:tasks.workunit.client.1.smithi193.stdout:5/54: fsync d0/f10 0 2024-03-23T08:23:15.740 INFO:tasks.workunit.client.1.smithi193.stdout:0/109: creat db/d11/d14/d24/f25 x:0 0 0 2024-03-23T08:23:15.740 INFO:tasks.workunit.client.1.smithi193.stdout:6/65: getdents d0/db/df 0 2024-03-23T08:23:15.748 INFO:tasks.workunit.client.1.smithi193.stdout:6/66: dread d0/f5 [0,4194304] 0 2024-03-23T08:23:15.749 INFO:tasks.workunit.client.0.smithi070.stdout:3/79: rename db/df/d1a/f1c to db/f23 0 2024-03-23T08:23:15.749 INFO:tasks.workunit.client.1.smithi193.stdout:4/105: dwrite d11/d1a/f1c [0,4194304] 0 2024-03-23T08:23:15.749 INFO:tasks.workunit.client.1.smithi193.stdout:5/55: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:15.750 INFO:tasks.workunit.client.0.smithi070.stdout:6/102: dwrite d0/f12 [0,4194304] 0 2024-03-23T08:23:15.750 INFO:tasks.workunit.client.0.smithi070.stdout:3/80: dread - db/df/d15/f1e zero size 2024-03-23T08:23:15.751 INFO:tasks.workunit.client.0.smithi070.stdout:3/81: chown db/df/d15/f1e 12 1 2024-03-23T08:23:15.753 INFO:tasks.workunit.client.0.smithi070.stdout:6/103: dread d0/d3/f4 [0,4194304] 0 2024-03-23T08:23:15.754 INFO:tasks.workunit.client.0.smithi070.stdout:6/104: truncate d0/f16 839726 0 2024-03-23T08:23:15.755 INFO:tasks.workunit.client.1.smithi193.stdout:7/89: sync 2024-03-23T08:23:15.755 INFO:tasks.workunit.client.1.smithi193.stdout:6/67: dread d0/f2 [4194304,4194304] 0 2024-03-23T08:23:15.760 INFO:tasks.workunit.client.1.smithi193.stdout:5/56: dwrite d0/fe [0,4194304] 0 2024-03-23T08:23:15.784 INFO:tasks.workunit.client.1.smithi193.stdout:9/73: sync 2024-03-23T08:23:15.970 INFO:tasks.workunit.client.1.smithi193.stdout:1/88: chown da/dd/df/f1d 102543 1 2024-03-23T08:23:15.973 INFO:tasks.workunit.client.1.smithi193.stdout:8/73: mkdir df/d10/d19 0 2024-03-23T08:23:15.973 INFO:tasks.workunit.client.1.smithi193.stdout:8/74: fsync df/f12 0 2024-03-23T08:23:15.975 INFO:tasks.workunit.client.1.smithi193.stdout:0/110: unlink db/d11/d14/d21/f23 0 2024-03-23T08:23:15.979 INFO:tasks.workunit.client.1.smithi193.stdout:4/106: mkdir d11/d1f 0 2024-03-23T08:23:15.982 INFO:tasks.workunit.client.1.smithi193.stdout:7/90: mkdir d6/d9/d1f/d23 0 2024-03-23T08:23:15.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:15 smithi070 ceph-mon[39699]: pgmap v17: 65 pgs: 65 active+clean; 222 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 413 op/s 2024-03-23T08:23:15.985 INFO:tasks.workunit.client.1.smithi193.stdout:5/57: creat d0/f14 x:0 0 0 2024-03-23T08:23:15.987 INFO:tasks.workunit.client.1.smithi193.stdout:9/74: mkdir d2/d7/db/d1b 0 2024-03-23T08:23:15.989 INFO:tasks.workunit.client.1.smithi193.stdout:2/152: rename d0/d1b/f12 to d0/f2d 0 2024-03-23T08:23:15.989 INFO:tasks.workunit.client.1.smithi193.stdout:3/82: rename d1 to d1/d19/d1d 22 2024-03-23T08:23:15.991 INFO:tasks.workunit.client.1.smithi193.stdout:2/153: write d0/d1b/d14/d25/f29 [1196652,115607] 0 2024-03-23T08:23:15.994 INFO:tasks.workunit.client.1.smithi193.stdout:0/111: creat db/d11/d14/d21/f26 x:0 0 0 2024-03-23T08:23:15.995 INFO:tasks.workunit.client.1.smithi193.stdout:9/75: dwrite d2/fc [0,4194304] 0 2024-03-23T08:23:15.996 INFO:tasks.workunit.client.1.smithi193.stdout:0/112: stat db/d11/d14/d21 0 2024-03-23T08:23:15.996 INFO:tasks.workunit.client.1.smithi193.stdout:9/76: read d2/d7/fe [1624713,53543] 0 2024-03-23T08:23:15.997 INFO:tasks.workunit.client.1.smithi193.stdout:3/83: dread d1/fa [0,4194304] 0 2024-03-23T08:23:15.999 INFO:tasks.workunit.client.1.smithi193.stdout:3/84: dread d1/fc [0,4194304] 0 2024-03-23T08:23:16.000 INFO:tasks.workunit.client.1.smithi193.stdout:9/77: dread d2/d7/fe [0,4194304] 0 2024-03-23T08:23:16.004 INFO:tasks.workunit.client.1.smithi193.stdout:6/68: truncate d0/f2 5867427 0 2024-03-23T08:23:16.007 INFO:tasks.workunit.client.1.smithi193.stdout:5/58: creat d0/f15 x:0 0 0 2024-03-23T08:23:16.013 INFO:tasks.workunit.client.0.smithi070.stdout:3/82: symlink db/df/d1a/l24 0 2024-03-23T08:23:16.013 INFO:tasks.workunit.client.0.smithi070.stdout:3/83: chown db/df/f19 402623 1 2024-03-23T08:23:16.014 INFO:tasks.workunit.client.1.smithi193.stdout:8/75: mknod df/d13/c1a 0 2024-03-23T08:23:16.014 INFO:tasks.workunit.client.0.smithi070.stdout:3/84: write db/df/f19 [2464449,15877] 0 2024-03-23T08:23:16.015 INFO:tasks.workunit.client.1.smithi193.stdout:8/76: write df/f18 [568103,46535] 0 2024-03-23T08:23:16.017 INFO:tasks.workunit.client.0.smithi070.stdout:6/105: creat d0/f25 x:0 0 0 2024-03-23T08:23:16.017 INFO:tasks.workunit.client.1.smithi193.stdout:2/154: dread d0/fc [0,4194304] 0 2024-03-23T08:23:16.019 INFO:tasks.workunit.client.1.smithi193.stdout:8/77: dread f0 [0,4194304] 0 2024-03-23T08:23:16.022 INFO:tasks.workunit.client.1.smithi193.stdout:7/91: mkdir d6/d9/d1f/d23/d24 0 2024-03-23T08:23:16.024 INFO:tasks.workunit.client.0.smithi070.stdout:6/106: dwrite d0/d23/f24 [0,4194304] 0 2024-03-23T08:23:16.026 INFO:tasks.workunit.client.0.smithi070.stdout:3/85: creat db/f25 x:0 0 0 2024-03-23T08:23:16.028 INFO:tasks.workunit.client.0.smithi070.stdout:6/107: symlink d0/l26 0 2024-03-23T08:23:16.030 INFO:tasks.workunit.client.0.smithi070.stdout:3/86: mknod db/d14/c26 0 2024-03-23T08:23:16.031 INFO:tasks.workunit.client.1.smithi193.stdout:6/69: dread d0/f5 [0,4194304] 0 2024-03-23T08:23:16.034 INFO:tasks.workunit.client.1.smithi193.stdout:3/85: truncate d1/de/f10 270973 0 2024-03-23T08:23:16.035 INFO:tasks.workunit.client.1.smithi193.stdout:9/78: symlink d2/d7/db/d1b/l1c 0 2024-03-23T08:23:16.037 INFO:tasks.workunit.client.1.smithi193.stdout:8/78: creat df/d10/d19/f1b x:0 0 0 2024-03-23T08:23:16.038 INFO:tasks.workunit.client.0.smithi070.stdout:3/87: dwrite db/f17 [0,4194304] 0 2024-03-23T08:23:16.039 INFO:tasks.workunit.client.1.smithi193.stdout:9/79: dread d2/d7/fe [0,4194304] 0 2024-03-23T08:23:16.040 INFO:tasks.workunit.client.0.smithi070.stdout:6/108: fdatasync d0/d23/f24 0 2024-03-23T08:23:16.041 INFO:tasks.workunit.client.0.smithi070.stdout:6/109: rename d0 to d0/d23/d1c/d27 22 2024-03-23T08:23:16.042 INFO:tasks.workunit.client.1.smithi193.stdout:3/86: readlink d1/l7 0 2024-03-23T08:23:16.045 INFO:tasks.workunit.client.1.smithi193.stdout:8/79: symlink df/l1c 0 2024-03-23T08:23:16.045 INFO:tasks.workunit.client.1.smithi193.stdout:8/80: readlink le 0 2024-03-23T08:23:16.046 INFO:tasks.workunit.client.1.smithi193.stdout:3/87: dread d1/de/d13/f1a [0,4194304] 0 2024-03-23T08:23:16.048 INFO:tasks.workunit.client.1.smithi193.stdout:3/88: write d1/d19/f1c [757821,12834] 0 2024-03-23T08:23:16.048 INFO:tasks.workunit.client.0.smithi070.stdout:6/110: mkdir d0/d23/d28 0 2024-03-23T08:23:16.049 INFO:tasks.workunit.client.1.smithi193.stdout:9/80: creat d2/d7/f1d x:0 0 0 2024-03-23T08:23:16.050 INFO:tasks.workunit.client.1.smithi193.stdout:2/155: getdents d0/d1b/d14/d25/d2a 0 2024-03-23T08:23:16.051 INFO:tasks.workunit.client.1.smithi193.stdout:2/156: chown d0/d28 67806 1 2024-03-23T08:23:16.051 INFO:tasks.workunit.client.0.smithi070.stdout:6/111: dread d0/f16 [0,4194304] 0 2024-03-23T08:23:16.052 INFO:tasks.workunit.client.1.smithi193.stdout:7/92: link d6/lb d6/d9/l25 0 2024-03-23T08:23:16.053 INFO:tasks.workunit.client.1.smithi193.stdout:8/81: creat df/d10/d19/f1d x:0 0 0 2024-03-23T08:23:16.055 INFO:tasks.workunit.client.0.smithi070.stdout:6/112: rmdir d0/d23 39 2024-03-23T08:23:16.056 INFO:tasks.workunit.client.1.smithi193.stdout:2/157: mkdir d0/d1b/d14/d25/d2e 0 2024-03-23T08:23:16.056 INFO:tasks.workunit.client.1.smithi193.stdout:9/81: dwrite d2/d7/f11 [0,4194304] 0 2024-03-23T08:23:16.058 INFO:tasks.workunit.client.1.smithi193.stdout:7/93: mknod d6/c26 0 2024-03-23T08:23:16.058 INFO:tasks.workunit.client.0.smithi070.stdout:6/113: chown d0/d23/f24 491975816 1 2024-03-23T08:23:16.059 INFO:tasks.workunit.client.1.smithi193.stdout:7/94: write d6/fc [13006891,71894] 0 2024-03-23T08:23:16.061 INFO:tasks.workunit.client.1.smithi193.stdout:2/158: fsync d0/fc 0 2024-03-23T08:23:16.062 INFO:tasks.workunit.client.1.smithi193.stdout:8/82: dwrite f0 [4194304,4194304] 0 2024-03-23T08:23:16.064 INFO:tasks.workunit.client.1.smithi193.stdout:9/82: creat d2/d7/f1e x:0 0 0 2024-03-23T08:23:16.065 INFO:tasks.workunit.client.0.smithi070.stdout:6/114: symlink d0/d23/d28/l29 0 2024-03-23T08:23:16.066 INFO:tasks.workunit.client.0.smithi070.stdout:6/115: write d0/d23/f17 [1239632,113313] 0 2024-03-23T08:23:16.074 INFO:tasks.workunit.client.1.smithi193.stdout:9/83: mkdir d2/d7/db/d17/d1f 0 2024-03-23T08:23:16.074 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:15 smithi193 ceph-mon[48370]: pgmap v17: 65 pgs: 65 active+clean; 222 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 413 op/s 2024-03-23T08:23:16.075 INFO:tasks.workunit.client.1.smithi193.stdout:9/84: truncate d2/d7/f1a 726330 0 2024-03-23T08:23:16.076 INFO:tasks.workunit.client.1.smithi193.stdout:3/89: getdents d1 0 2024-03-23T08:23:16.079 INFO:tasks.workunit.client.0.smithi070.stdout:6/116: rename d0/f6 to d0/d23/f2a 0 2024-03-23T08:23:16.079 INFO:tasks.workunit.client.0.smithi070.stdout:6/117: stat d0/d23/f24 0 2024-03-23T08:23:16.082 INFO:tasks.workunit.client.1.smithi193.stdout:9/85: mknod d2/d7/db/c20 0 2024-03-23T08:23:16.083 INFO:tasks.workunit.client.1.smithi193.stdout:9/86: chown d2/l18 93232041 1 2024-03-23T08:23:16.084 INFO:tasks.workunit.client.1.smithi193.stdout:9/87: truncate d2/d7/f1d 983188 0 2024-03-23T08:23:16.085 INFO:tasks.workunit.client.1.smithi193.stdout:2/159: rmdir d0/d1b/d14/d25/d2e 0 2024-03-23T08:23:16.087 INFO:tasks.workunit.client.1.smithi193.stdout:3/90: link d1/de/f11 d1/f1e 0 2024-03-23T08:23:16.087 INFO:tasks.workunit.client.1.smithi193.stdout:9/88: dread d2/f3 [0,4194304] 0 2024-03-23T08:23:16.088 INFO:tasks.workunit.client.0.smithi070.stdout:6/118: getdents d0/d23/d1c 0 2024-03-23T08:23:16.088 INFO:tasks.workunit.client.1.smithi193.stdout:9/89: write d2/f3 [1355294,83621] 0 2024-03-23T08:23:16.091 INFO:tasks.workunit.client.1.smithi193.stdout:2/160: chown d0/l5 48 1 2024-03-23T08:23:16.098 INFO:tasks.workunit.client.1.smithi193.stdout:2/161: creat d0/d1b/f2f x:0 0 0 2024-03-23T08:23:16.099 INFO:tasks.workunit.client.1.smithi193.stdout:2/162: dread - d0/f20 zero size 2024-03-23T08:23:16.118 INFO:tasks.workunit.client.1.smithi193.stdout:0/113: sync 2024-03-23T08:23:16.120 INFO:tasks.workunit.client.1.smithi193.stdout:0/114: write db/d11/d14/d21/f22 [2433713,42537] 0 2024-03-23T08:23:16.120 INFO:tasks.workunit.client.1.smithi193.stdout:0/115: fsync db/d11/d14/f19 0 2024-03-23T08:23:16.121 INFO:tasks.workunit.client.1.smithi193.stdout:0/116: fdatasync db/d11/d14/d21/f26 0 2024-03-23T08:23:16.125 INFO:tasks.workunit.client.1.smithi193.stdout:0/117: rename f7 to db/d11/d14/d21/f27 0 2024-03-23T08:23:16.126 INFO:tasks.workunit.client.1.smithi193.stdout:0/118: chown db/d11/d14/l1b 6 1 2024-03-23T08:23:16.128 INFO:tasks.workunit.client.1.smithi193.stdout:0/119: creat db/d11/f28 x:0 0 0 2024-03-23T08:23:16.129 INFO:tasks.workunit.client.1.smithi193.stdout:0/120: read db/d11/d14/d21/f27 [2086657,75730] 0 2024-03-23T08:23:16.136 INFO:tasks.workunit.client.1.smithi193.stdout:0/121: dwrite db/d11/d14/d21/f27 [0,4194304] 0 2024-03-23T08:23:16.136 INFO:tasks.workunit.client.1.smithi193.stdout:0/122: dread - db/d11/f28 zero size 2024-03-23T08:23:16.137 INFO:tasks.workunit.client.1.smithi193.stdout:0/123: readlink db/d11/d14/l18 0 2024-03-23T08:23:16.142 INFO:tasks.workunit.client.1.smithi193.stdout:0/124: rename db/l15 to db/d11/d14/d24/l29 0 2024-03-23T08:23:16.163 INFO:tasks.workunit.client.1.smithi193.stdout:5/59: sync 2024-03-23T08:23:16.166 INFO:tasks.workunit.client.1.smithi193.stdout:5/60: dread d0/f10 [0,4194304] 0 2024-03-23T08:23:16.168 INFO:tasks.workunit.client.1.smithi193.stdout:5/61: mknod d0/c16 0 2024-03-23T08:23:16.169 INFO:tasks.workunit.client.1.smithi193.stdout:5/62: creat d0/f17 x:0 0 0 2024-03-23T08:23:16.170 INFO:tasks.workunit.client.1.smithi193.stdout:5/63: readlink d0/l13 0 2024-03-23T08:23:16.172 INFO:tasks.workunit.client.1.smithi193.stdout:5/64: rename d0/lc to d0/l18 0 2024-03-23T08:23:16.172 INFO:tasks.workunit.client.1.smithi193.stdout:5/65: chown d0/f7 1203 1 2024-03-23T08:23:16.173 INFO:tasks.workunit.client.1.smithi193.stdout:5/66: fdatasync d0/f7 0 2024-03-23T08:23:16.183 INFO:tasks.workunit.client.1.smithi193.stdout:6/70: truncate d0/f5 1532558 0 2024-03-23T08:23:16.185 INFO:tasks.workunit.client.1.smithi193.stdout:6/71: unlink d0/f2 0 2024-03-23T08:23:16.190 INFO:tasks.workunit.client.1.smithi193.stdout:1/89: write f5 [118183,69934] 0 2024-03-23T08:23:16.191 INFO:tasks.workunit.client.1.smithi193.stdout:1/90: write da/fb [3861725,65600] 0 2024-03-23T08:23:16.194 INFO:tasks.workunit.client.1.smithi193.stdout:1/91: mknod da/c21 0 2024-03-23T08:23:16.195 INFO:tasks.workunit.client.1.smithi193.stdout:4/107: write fb [889367,66228] 0 2024-03-23T08:23:16.196 INFO:tasks.workunit.client.1.smithi193.stdout:1/92: creat da/dd/df/f22 x:0 0 0 2024-03-23T08:23:16.198 INFO:tasks.workunit.client.1.smithi193.stdout:4/108: creat d11/d19/f20 x:0 0 0 2024-03-23T08:23:16.199 INFO:tasks.workunit.client.1.smithi193.stdout:1/93: mknod da/d13/c23 0 2024-03-23T08:23:16.204 INFO:tasks.workunit.client.1.smithi193.stdout:1/94: mkdir da/d13/d24 0 2024-03-23T08:23:16.206 INFO:tasks.workunit.client.1.smithi193.stdout:4/109: dwrite d11/d1a/f1c [4194304,4194304] 0 2024-03-23T08:23:16.207 INFO:tasks.workunit.client.1.smithi193.stdout:8/83: rmdir df/d10/d19 39 2024-03-23T08:23:16.208 INFO:tasks.workunit.client.1.smithi193.stdout:4/110: write fb [1838813,1555] 0 2024-03-23T08:23:16.211 INFO:tasks.workunit.client.0.smithi070.stdout:3/88: dwrite f6 [0,4194304] 0 2024-03-23T08:23:16.212 INFO:tasks.workunit.client.1.smithi193.stdout:7/95: getdents d6 0 2024-03-23T08:23:16.214 INFO:tasks.workunit.client.1.smithi193.stdout:8/84: dread f0 [4194304,4194304] 0 2024-03-23T08:23:16.214 INFO:tasks.workunit.client.1.smithi193.stdout:4/111: fdatasync f8 0 2024-03-23T08:23:16.215 INFO:tasks.workunit.client.0.smithi070.stdout:3/89: symlink db/d14/l27 0 2024-03-23T08:23:16.217 INFO:tasks.workunit.client.0.smithi070.stdout:6/119: getdents d0/d23/d28 0 2024-03-23T08:23:16.217 INFO:tasks.workunit.client.1.smithi193.stdout:7/96: dread d6/d9/f16 [0,4194304] 0 2024-03-23T08:23:16.219 INFO:tasks.workunit.client.1.smithi193.stdout:8/85: chown df/d10/d19/f1b 2247 1 2024-03-23T08:23:16.219 INFO:tasks.workunit.client.0.smithi070.stdout:3/90: mknod db/df/c28 0 2024-03-23T08:23:16.220 INFO:tasks.workunit.client.1.smithi193.stdout:8/86: dread - df/d10/d19/f1d zero size 2024-03-23T08:23:16.221 INFO:tasks.workunit.client.0.smithi070.stdout:6/120: dread d0/d23/f2a [0,4194304] 0 2024-03-23T08:23:16.222 INFO:tasks.workunit.client.0.smithi070.stdout:6/121: chown d0/d3/fa 189112 1 2024-03-23T08:23:16.222 INFO:tasks.workunit.client.1.smithi193.stdout:4/112: unlink le 0 2024-03-23T08:23:16.223 INFO:tasks.workunit.client.0.smithi070.stdout:6/122: write d0/d23/f17 [2361049,55759] 0 2024-03-23T08:23:16.223 INFO:tasks.workunit.client.0.smithi070.stdout:3/91: symlink db/df/l29 0 2024-03-23T08:23:16.224 INFO:tasks.workunit.client.1.smithi193.stdout:9/90: getdents d2/d7/db 0 2024-03-23T08:23:16.225 INFO:tasks.workunit.client.0.smithi070.stdout:6/123: stat d0/f12 0 2024-03-23T08:23:16.226 INFO:tasks.workunit.client.0.smithi070.stdout:6/124: fdatasync d0/d23/fe 0 2024-03-23T08:23:16.229 INFO:tasks.workunit.client.1.smithi193.stdout:8/87: unlink l7 0 2024-03-23T08:23:16.231 INFO:tasks.workunit.client.1.smithi193.stdout:4/113: rmdir d11 39 2024-03-23T08:23:16.232 INFO:tasks.workunit.client.0.smithi070.stdout:6/125: creat d0/d23/d1c/f2b x:0 0 0 2024-03-23T08:23:16.233 INFO:tasks.workunit.client.0.smithi070.stdout:6/126: read d0/d23/f10 [175075,79847] 0 2024-03-23T08:23:16.235 INFO:tasks.workunit.client.1.smithi193.stdout:0/125: rmdir db/d11 39 2024-03-23T08:23:16.236 INFO:tasks.workunit.client.1.smithi193.stdout:0/126: chown db 13998558 1 2024-03-23T08:23:16.237 INFO:tasks.workunit.client.0.smithi070.stdout:6/127: mkdir d0/d3/d2c 0 2024-03-23T08:23:16.238 INFO:tasks.workunit.client.0.smithi070.stdout:6/128: write d0/d23/f24 [1388140,24529] 0 2024-03-23T08:23:16.239 INFO:tasks.workunit.client.1.smithi193.stdout:8/88: creat df/d13/f1e x:0 0 0 2024-03-23T08:23:16.240 INFO:tasks.workunit.client.1.smithi193.stdout:2/163: dwrite d0/d1b/d19/f1a [0,4194304] 0 2024-03-23T08:23:16.241 INFO:tasks.workunit.client.0.smithi070.stdout:6/129: symlink d0/d3/l2d 0 2024-03-23T08:23:16.241 INFO:tasks.workunit.client.1.smithi193.stdout:3/91: fdatasync d1/f2 0 2024-03-23T08:23:16.242 INFO:tasks.workunit.client.1.smithi193.stdout:9/91: dwrite d2/d7/db/f13 [0,4194304] 0 2024-03-23T08:23:16.242 INFO:tasks.workunit.client.1.smithi193.stdout:4/114: fdatasync f5 0 2024-03-23T08:23:16.244 INFO:tasks.workunit.client.1.smithi193.stdout:7/97: rmdir d6/d9/d1f/d23/d24 0 2024-03-23T08:23:16.246 INFO:tasks.workunit.client.0.smithi070.stdout:6/130: creat d0/d23/d1c/f2e x:0 0 0 2024-03-23T08:23:16.249 INFO:tasks.workunit.client.0.smithi070.stdout:6/131: mkdir d0/d23/d1c/d2f 0 2024-03-23T08:23:16.250 INFO:tasks.workunit.client.1.smithi193.stdout:8/89: creat df/f1f x:0 0 0 2024-03-23T08:23:16.250 INFO:tasks.workunit.client.1.smithi193.stdout:9/92: read d2/d7/f1a [460062,67790] 0 2024-03-23T08:23:16.255 INFO:tasks.workunit.client.1.smithi193.stdout:3/92: truncate d1/fd 1507073 0 2024-03-23T08:23:16.257 INFO:tasks.workunit.client.0.smithi070.stdout:6/132: rename d0/d23/f1f to d0/d23/d1c/d2f/f30 0 2024-03-23T08:23:16.258 INFO:tasks.workunit.client.0.smithi070.stdout:6/133: read d0/d23/f24 [3266873,118246] 0 2024-03-23T08:23:16.259 INFO:tasks.workunit.client.1.smithi193.stdout:2/164: rmdir d0/d1b 39 2024-03-23T08:23:16.263 INFO:tasks.workunit.client.1.smithi193.stdout:7/98: mknod d6/d9/d1f/d23/c27 0 2024-03-23T08:23:16.264 INFO:tasks.workunit.client.1.smithi193.stdout:7/99: write d6/d9/f21 [124068,27824] 0 2024-03-23T08:23:16.265 INFO:tasks.workunit.client.0.smithi070.stdout:6/134: dwrite d0/d23/f17 [0,4194304] 0 2024-03-23T08:23:16.267 INFO:tasks.workunit.client.1.smithi193.stdout:0/127: mkdir db/d11/d14/d24/d2a 0 2024-03-23T08:23:16.268 INFO:tasks.workunit.client.1.smithi193.stdout:8/90: mkdir df/d10/d19/d20 0 2024-03-23T08:23:16.269 INFO:tasks.workunit.client.1.smithi193.stdout:8/91: chown df/d13/f1e 287 1 2024-03-23T08:23:16.271 INFO:tasks.workunit.client.0.smithi070.stdout:6/135: rename d0/d23/fe to d0/d23/d28/f31 0 2024-03-23T08:23:16.272 INFO:tasks.workunit.client.0.smithi070.stdout:6/136: truncate d0/f20 112382 0 2024-03-23T08:23:16.274 INFO:tasks.workunit.client.1.smithi193.stdout:5/67: sync 2024-03-23T08:23:16.275 INFO:tasks.workunit.client.1.smithi193.stdout:5/68: chown d0/l13 5553 1 2024-03-23T08:23:16.276 INFO:tasks.workunit.client.1.smithi193.stdout:5/69: read d0/f10 [997291,34315] 0 2024-03-23T08:23:16.276 INFO:tasks.workunit.client.1.smithi193.stdout:5/70: dread - d0/f17 zero size 2024-03-23T08:23:16.277 INFO:tasks.workunit.client.1.smithi193.stdout:7/100: symlink d6/d9/d1f/l28 0 2024-03-23T08:23:16.278 INFO:tasks.workunit.client.1.smithi193.stdout:5/71: write d0/fa [1668184,111253] 0 2024-03-23T08:23:16.278 INFO:tasks.workunit.client.1.smithi193.stdout:7/101: write d6/d9/f1c [678652,47144] 0 2024-03-23T08:23:16.279 INFO:tasks.workunit.client.1.smithi193.stdout:7/102: read d6/f8 [3950748,20553] 0 2024-03-23T08:23:16.280 INFO:tasks.workunit.client.1.smithi193.stdout:9/93: symlink d2/d7/db/d17/d1f/l21 0 2024-03-23T08:23:16.282 INFO:tasks.workunit.client.1.smithi193.stdout:8/92: creat df/f21 x:0 0 0 2024-03-23T08:23:16.283 INFO:tasks.workunit.client.1.smithi193.stdout:3/93: unlink d1/de/c15 0 2024-03-23T08:23:16.284 INFO:tasks.workunit.client.1.smithi193.stdout:2/165: mkdir d0/d1b/d14/d25/d30 0 2024-03-23T08:23:16.287 INFO:tasks.workunit.client.1.smithi193.stdout:5/72: dwrite d0/f17 [0,4194304] 0 2024-03-23T08:23:16.288 INFO:tasks.workunit.client.1.smithi193.stdout:5/73: dread - d0/f14 zero size 2024-03-23T08:23:16.289 INFO:tasks.workunit.client.1.smithi193.stdout:7/103: mknod d6/d9/c29 0 2024-03-23T08:23:16.291 INFO:tasks.workunit.client.1.smithi193.stdout:8/93: creat df/d13/f22 x:0 0 0 2024-03-23T08:23:16.295 INFO:tasks.workunit.client.1.smithi193.stdout:2/166: write d0/d1b/f21 [569910,63440] 0 2024-03-23T08:23:16.297 INFO:tasks.workunit.client.1.smithi193.stdout:9/94: dwrite d2/f3 [0,4194304] 0 2024-03-23T08:23:16.297 INFO:tasks.workunit.client.1.smithi193.stdout:2/167: write d0/d1b/f21 [962313,86818] 0 2024-03-23T08:23:16.297 INFO:tasks.workunit.client.1.smithi193.stdout:5/74: mknod d0/c19 0 2024-03-23T08:23:16.303 INFO:tasks.workunit.client.1.smithi193.stdout:0/128: getdents db/d11/d14/d21 0 2024-03-23T08:23:16.307 INFO:tasks.workunit.client.1.smithi193.stdout:9/95: creat d2/d7/db/d17/d1f/f22 x:0 0 0 2024-03-23T08:23:16.309 INFO:tasks.workunit.client.1.smithi193.stdout:0/129: rename db/d11/c16 to db/d11/c2b 0 2024-03-23T08:23:16.310 INFO:tasks.workunit.client.1.smithi193.stdout:0/130: write fa [5454597,42944] 0 2024-03-23T08:23:16.312 INFO:tasks.workunit.client.1.smithi193.stdout:9/96: creat d2/d7/f23 x:0 0 0 2024-03-23T08:23:16.314 INFO:tasks.workunit.client.1.smithi193.stdout:0/131: creat db/d11/d14/d21/f2c x:0 0 0 2024-03-23T08:23:16.318 INFO:tasks.workunit.client.0.smithi070.stdout:6/137: dwrite d0/d3/f4 [0,4194304] 0 2024-03-23T08:23:16.320 INFO:tasks.workunit.client.1.smithi193.stdout:0/132: mknod db/d11/d14/d24/d2a/c2d 0 2024-03-23T08:23:16.322 INFO:tasks.workunit.client.1.smithi193.stdout:0/133: creat db/d11/d14/d24/d2a/f2e x:0 0 0 2024-03-23T08:23:16.323 INFO:tasks.workunit.client.1.smithi193.stdout:0/134: readlink db/l20 0 2024-03-23T08:23:16.325 INFO:tasks.workunit.client.1.smithi193.stdout:0/135: write db/d11/d14/d21/f27 [1982590,19402] 0 2024-03-23T08:23:16.325 INFO:tasks.workunit.client.0.smithi070.stdout:6/138: write d0/d23/d28/f31 [307800,117381] 0 2024-03-23T08:23:16.325 INFO:tasks.workunit.client.1.smithi193.stdout:5/75: getdents d0 0 2024-03-23T08:23:16.326 INFO:tasks.workunit.client.1.smithi193.stdout:5/76: fdatasync d0/fa 0 2024-03-23T08:23:16.327 INFO:tasks.workunit.client.1.smithi193.stdout:2/168: write d0/fc [47118,54736] 0 2024-03-23T08:23:16.328 INFO:tasks.workunit.client.0.smithi070.stdout:6/139: mknod d0/d23/d1c/c32 0 2024-03-23T08:23:16.328 INFO:tasks.workunit.client.1.smithi193.stdout:2/169: write d0/f20 [1016443,86289] 0 2024-03-23T08:23:16.329 INFO:tasks.workunit.client.1.smithi193.stdout:7/104: truncate d6/d9/f16 274231 0 2024-03-23T08:23:16.330 INFO:tasks.workunit.client.1.smithi193.stdout:7/105: fsync d6/d9/f12 0 2024-03-23T08:23:16.330 INFO:tasks.workunit.client.0.smithi070.stdout:6/140: chown d0/d23/d28/l29 5 1 2024-03-23T08:23:16.331 INFO:tasks.workunit.client.1.smithi193.stdout:7/106: truncate d6/d9/f21 657629 0 2024-03-23T08:23:16.331 INFO:tasks.workunit.client.1.smithi193.stdout:2/170: rmdir d0/d1b 39 2024-03-23T08:23:16.332 INFO:tasks.workunit.client.1.smithi193.stdout:7/107: write d6/ff [1213515,85136] 0 2024-03-23T08:23:16.332 INFO:tasks.workunit.client.1.smithi193.stdout:9/97: write d2/f4 [1183203,91428] 0 2024-03-23T08:23:16.333 INFO:tasks.workunit.client.1.smithi193.stdout:0/136: symlink db/l2f 0 2024-03-23T08:23:16.336 INFO:tasks.workunit.client.0.smithi070.stdout:6/141: symlink d0/l33 0 2024-03-23T08:23:16.337 INFO:tasks.workunit.client.1.smithi193.stdout:6/72: write d0/f5 [40044,99048] 0 2024-03-23T08:23:16.341 INFO:tasks.workunit.client.1.smithi193.stdout:3/94: dwrite d1/de/f10 [0,4194304] 0 2024-03-23T08:23:16.342 INFO:tasks.workunit.client.0.smithi070.stdout:6/142: creat d0/d3/d2c/f34 x:0 0 0 2024-03-23T08:23:16.342 INFO:tasks.workunit.client.1.smithi193.stdout:5/77: truncate d0/f17 707050 0 2024-03-23T08:23:16.343 INFO:tasks.workunit.client.0.smithi070.stdout:6/143: read - d0/d3/f1a zero size 2024-03-23T08:23:16.346 INFO:tasks.workunit.client.1.smithi193.stdout:9/98: creat d2/d7/db/d17/d1f/f24 x:0 0 0 2024-03-23T08:23:16.346 INFO:tasks.workunit.client.1.smithi193.stdout:5/78: truncate d0/f14 8608 0 2024-03-23T08:23:16.347 INFO:tasks.workunit.client.1.smithi193.stdout:3/95: dread d1/de/d13/f1a [0,4194304] 0 2024-03-23T08:23:16.348 INFO:tasks.workunit.client.1.smithi193.stdout:0/137: mknod db/c30 0 2024-03-23T08:23:16.349 INFO:tasks.workunit.client.1.smithi193.stdout:2/171: write d0/d1b/f8 [307430,76350] 0 2024-03-23T08:23:16.349 INFO:tasks.workunit.client.1.smithi193.stdout:7/108: rename d6/d9/f12 to d6/d9/d1f/f2a 0 2024-03-23T08:23:16.350 INFO:tasks.workunit.client.1.smithi193.stdout:2/172: chown d0/d1b/d14/d25 110556 1 2024-03-23T08:23:16.351 INFO:tasks.workunit.client.1.smithi193.stdout:6/73: mkdir d0/d8/d12 0 2024-03-23T08:23:16.352 INFO:tasks.workunit.client.1.smithi193.stdout:0/138: creat db/d11/d14/d24/d2a/f31 x:0 0 0 2024-03-23T08:23:16.353 INFO:tasks.workunit.client.1.smithi193.stdout:5/79: dread d0/ff [0,4194304] 0 2024-03-23T08:23:16.353 INFO:tasks.workunit.client.1.smithi193.stdout:2/173: read d0/d1b/d14/d25/f29 [950022,74869] 0 2024-03-23T08:23:16.353 INFO:tasks.workunit.client.1.smithi193.stdout:9/99: creat d2/f25 x:0 0 0 2024-03-23T08:23:16.354 INFO:tasks.workunit.client.1.smithi193.stdout:1/95: sync 2024-03-23T08:23:16.355 INFO:tasks.workunit.client.1.smithi193.stdout:3/96: creat d1/d19/f1f x:0 0 0 2024-03-23T08:23:16.358 INFO:tasks.workunit.client.1.smithi193.stdout:3/97: write d1/de/f16 [491177,91453] 0 2024-03-23T08:23:16.359 INFO:tasks.workunit.client.1.smithi193.stdout:9/100: creat d2/d7/db/f26 x:0 0 0 2024-03-23T08:23:16.360 INFO:tasks.workunit.client.1.smithi193.stdout:3/98: write d1/fd [2291193,118500] 0 2024-03-23T08:23:16.361 INFO:tasks.workunit.client.1.smithi193.stdout:1/96: creat da/dd/df/f25 x:0 0 0 2024-03-23T08:23:16.361 INFO:tasks.workunit.client.1.smithi193.stdout:5/80: dread d0/f3 [0,4194304] 0 2024-03-23T08:23:16.362 INFO:tasks.workunit.client.1.smithi193.stdout:1/97: chown da/d13/d24 7627892 1 2024-03-23T08:23:16.363 INFO:tasks.workunit.client.1.smithi193.stdout:5/81: write d0/f11 [5452,6704] 0 2024-03-23T08:23:16.365 INFO:tasks.workunit.client.0.smithi070.stdout:6/144: dwrite d0/d23/f24 [4194304,4194304] 0 2024-03-23T08:23:16.366 INFO:tasks.workunit.client.0.smithi070.stdout:6/145: chown d0/d3/c5 32380 1 2024-03-23T08:23:16.367 INFO:tasks.workunit.client.1.smithi193.stdout:6/74: truncate d0/fe 13108 0 2024-03-23T08:23:16.367 INFO:tasks.workunit.client.0.smithi070.stdout:6/146: dread - d0/d3/d2c/f34 zero size 2024-03-23T08:23:16.368 INFO:tasks.workunit.client.1.smithi193.stdout:2/174: creat d0/f31 x:0 0 0 2024-03-23T08:23:16.368 INFO:tasks.workunit.client.1.smithi193.stdout:6/75: write d0/f5 [1587170,19268] 0 2024-03-23T08:23:16.371 INFO:tasks.workunit.client.0.smithi070.stdout:6/147: mkdir d0/d23/d28/d35 0 2024-03-23T08:23:16.371 INFO:tasks.workunit.client.1.smithi193.stdout:6/76: write d0/f5 [1988695,24244] 0 2024-03-23T08:23:16.371 INFO:tasks.workunit.client.1.smithi193.stdout:2/175: read d0/f20 [714966,105424] 0 2024-03-23T08:23:16.372 INFO:tasks.workunit.client.0.smithi070.stdout:6/148: read d0/d3/f4 [1502164,129116] 0 2024-03-23T08:23:16.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/98: mknod da/d13/c26 0 2024-03-23T08:23:16.372 INFO:tasks.workunit.client.0.smithi070.stdout:6/149: chown d0/d3/l22 3 1 2024-03-23T08:23:16.373 INFO:tasks.workunit.client.1.smithi193.stdout:5/82: dwrite d0/f10 [0,4194304] 0 2024-03-23T08:23:16.374 INFO:tasks.workunit.client.1.smithi193.stdout:2/176: readlink d0/l10 0 2024-03-23T08:23:16.376 INFO:tasks.workunit.client.1.smithi193.stdout:2/177: stat d0/f2d 0 2024-03-23T08:23:16.379 INFO:tasks.workunit.client.1.smithi193.stdout:0/139: getdents db/d11 0 2024-03-23T08:23:16.379 INFO:tasks.workunit.client.1.smithi193.stdout:0/140: stat db 0 2024-03-23T08:23:16.387 INFO:tasks.workunit.client.0.smithi070.stdout:6/150: getdents d0 0 2024-03-23T08:23:16.387 INFO:tasks.workunit.client.0.smithi070.stdout:6/151: read - d0/d23/d1c/d2f/f30 zero size 2024-03-23T08:23:16.389 INFO:tasks.workunit.client.1.smithi193.stdout:3/99: creat d1/f20 x:0 0 0 2024-03-23T08:23:16.391 INFO:tasks.workunit.client.0.smithi070.stdout:6/152: dread d0/d3/fa [0,4194304] 0 2024-03-23T08:23:16.392 INFO:tasks.workunit.client.1.smithi193.stdout:2/178: creat d0/d1b/d14/d25/f32 x:0 0 0 2024-03-23T08:23:16.392 INFO:tasks.workunit.client.0.smithi070.stdout:6/153: write d0/d23/d1c/f2e [243618,126918] 0 2024-03-23T08:23:16.393 INFO:tasks.workunit.client.1.smithi193.stdout:0/141: mknod db/d11/d14/c32 0 2024-03-23T08:23:16.393 INFO:tasks.workunit.client.1.smithi193.stdout:9/101: link d2/d7/db/d17/d1f/l21 d2/d7/db/d17/d1f/l27 0 2024-03-23T08:23:16.394 INFO:tasks.workunit.client.1.smithi193.stdout:5/83: getdents d0 0 2024-03-23T08:23:16.395 INFO:tasks.workunit.client.0.smithi070.stdout:6/154: creat d0/d23/d28/f36 x:0 0 0 2024-03-23T08:23:16.396 INFO:tasks.workunit.client.0.smithi070.stdout:6/155: write d0/d23/f24 [7066151,125747] 0 2024-03-23T08:23:16.396 INFO:tasks.workunit.client.0.smithi070.stdout:6/156: chown d0/d23/d1c/d2f/f30 87169289 1 2024-03-23T08:23:16.398 INFO:tasks.workunit.client.0.smithi070.stdout:6/157: creat d0/d23/d28/f37 x:0 0 0 2024-03-23T08:23:16.399 INFO:tasks.workunit.client.1.smithi193.stdout:2/179: dwrite d0/d1b/f8 [4194304,4194304] 0 2024-03-23T08:23:16.400 INFO:tasks.workunit.client.1.smithi193.stdout:3/100: dread d1/fa [0,4194304] 0 2024-03-23T08:23:16.401 INFO:tasks.workunit.client.1.smithi193.stdout:0/142: mkdir db/d11/d14/d21/d33 0 2024-03-23T08:23:16.401 INFO:tasks.workunit.client.0.smithi070.stdout:6/158: mkdir d0/d23/d1c/d38 0 2024-03-23T08:23:16.403 INFO:tasks.workunit.client.1.smithi193.stdout:5/84: creat d0/f1a x:0 0 0 2024-03-23T08:23:16.404 INFO:tasks.workunit.client.1.smithi193.stdout:2/180: dread d0/fc [0,4194304] 0 2024-03-23T08:23:16.404 INFO:tasks.workunit.client.1.smithi193.stdout:1/99: link da/d13/l15 da/dd/df/l27 0 2024-03-23T08:23:16.404 INFO:tasks.workunit.client.0.smithi070.stdout:6/159: creat d0/d23/d1c/d2f/f39 x:0 0 0 2024-03-23T08:23:16.405 INFO:tasks.workunit.client.1.smithi193.stdout:0/143: write db/d11/f13 [685975,124228] 0 2024-03-23T08:23:16.405 INFO:tasks.workunit.client.1.smithi193.stdout:1/100: fdatasync da/dd/df/f1e 0 2024-03-23T08:23:16.406 INFO:tasks.workunit.client.1.smithi193.stdout:1/101: chown da/dd 24126729 1 2024-03-23T08:23:16.406 INFO:tasks.workunit.client.1.smithi193.stdout:3/101: fsync d1/f1e 0 2024-03-23T08:23:16.407 INFO:tasks.workunit.client.1.smithi193.stdout:2/181: read d0/d1b/d19/f1a [1587034,104588] 0 2024-03-23T08:23:16.408 INFO:tasks.workunit.client.0.smithi070.stdout:6/160: unlink d0/f16 0 2024-03-23T08:23:16.410 INFO:tasks.workunit.client.1.smithi193.stdout:6/77: truncate d0/f5 893223 0 2024-03-23T08:23:16.411 INFO:tasks.workunit.client.1.smithi193.stdout:5/85: rename d0/l13 to d0/l1b 0 2024-03-23T08:23:16.414 INFO:tasks.workunit.client.1.smithi193.stdout:0/144: creat db/d11/d14/f34 x:0 0 0 2024-03-23T08:23:16.415 INFO:tasks.workunit.client.1.smithi193.stdout:1/102: readlink da/dd/df/l27 0 2024-03-23T08:23:16.416 INFO:tasks.workunit.client.1.smithi193.stdout:9/102: creat d2/f28 x:0 0 0 2024-03-23T08:23:16.417 INFO:tasks.workunit.client.1.smithi193.stdout:6/78: readlink d0/l3 0 2024-03-23T08:23:16.418 INFO:tasks.workunit.client.1.smithi193.stdout:5/86: mkdir d0/d1c 0 2024-03-23T08:23:16.419 INFO:tasks.workunit.client.1.smithi193.stdout:0/145: mknod db/d11/c35 0 2024-03-23T08:23:16.421 INFO:tasks.workunit.client.1.smithi193.stdout:5/87: dread d0/ff [0,4194304] 0 2024-03-23T08:23:16.421 INFO:tasks.workunit.client.1.smithi193.stdout:1/103: unlink da/d13/l15 0 2024-03-23T08:23:16.423 INFO:tasks.workunit.client.1.smithi193.stdout:9/103: symlink d2/d7/db/d1b/l29 0 2024-03-23T08:23:16.424 INFO:tasks.workunit.client.1.smithi193.stdout:5/88: dread d0/fe [0,4194304] 0 2024-03-23T08:23:16.425 INFO:tasks.workunit.client.1.smithi193.stdout:6/79: stat d0/db/df/d11 0 2024-03-23T08:23:16.427 INFO:tasks.workunit.client.0.smithi070.stdout:6/161: readlink d0/d3/ld 0 2024-03-23T08:23:16.429 INFO:tasks.workunit.client.1.smithi193.stdout:1/104: rename da/c18 to da/d13/c28 0 2024-03-23T08:23:16.431 INFO:tasks.workunit.client.1.smithi193.stdout:9/104: mknod d2/d7/c2a 0 2024-03-23T08:23:16.432 INFO:tasks.workunit.client.0.smithi070.stdout:6/162: truncate d0/d23/d1c/d2f/f30 235919 0 2024-03-23T08:23:16.433 INFO:tasks.workunit.client.1.smithi193.stdout:2/182: getdents d0/d28 0 2024-03-23T08:23:16.434 INFO:tasks.workunit.client.0.smithi070.stdout:6/163: write d0/f20 [648887,69591] 0 2024-03-23T08:23:16.436 INFO:tasks.workunit.client.1.smithi193.stdout:6/80: mkdir d0/d13 0 2024-03-23T08:23:16.436 INFO:tasks.workunit.client.1.smithi193.stdout:6/81: stat d0/d13 0 2024-03-23T08:23:16.438 INFO:tasks.workunit.client.0.smithi070.stdout:6/164: mkdir d0/d23/d28/d35/d3a 0 2024-03-23T08:23:16.440 INFO:tasks.workunit.client.1.smithi193.stdout:1/105: dwrite da/dd/df/f1e [0,4194304] 0 2024-03-23T08:23:16.441 INFO:tasks.workunit.client.1.smithi193.stdout:2/183: rename d0/d1b/d14/d25/d30 to d0/d28/d33 0 2024-03-23T08:23:16.447 INFO:tasks.workunit.client.1.smithi193.stdout:6/82: unlink d0/c7 0 2024-03-23T08:23:16.449 INFO:tasks.workunit.client.1.smithi193.stdout:9/105: dwrite d2/d7/db/d17/d1f/f22 [0,4194304] 0 2024-03-23T08:23:16.449 INFO:tasks.workunit.client.0.smithi070.stdout:6/165: dwrite d0/f25 [0,4194304] 0 2024-03-23T08:23:16.451 INFO:tasks.workunit.client.1.smithi193.stdout:6/83: readlink d0/l6 0 2024-03-23T08:23:16.455 INFO:tasks.workunit.client.0.smithi070.stdout:6/166: rename d0/f12 to d0/d23/d28/d35/d3a/f3b 0 2024-03-23T08:23:16.457 INFO:tasks.workunit.client.1.smithi193.stdout:2/184: dread d0/fc [0,4194304] 0 2024-03-23T08:23:16.462 INFO:tasks.workunit.client.1.smithi193.stdout:2/185: rmdir d0 39 2024-03-23T08:23:16.464 INFO:tasks.workunit.client.1.smithi193.stdout:9/106: dwrite d2/d7/f1e [0,4194304] 0 2024-03-23T08:23:16.465 INFO:tasks.workunit.client.1.smithi193.stdout:2/186: stat d0/f1d 0 2024-03-23T08:23:16.465 INFO:tasks.workunit.client.1.smithi193.stdout:1/106: readlink da/dd/df/l20 0 2024-03-23T08:23:16.469 INFO:tasks.workunit.client.1.smithi193.stdout:1/107: creat da/dd/df/f29 x:0 0 0 2024-03-23T08:23:16.473 INFO:tasks.workunit.client.1.smithi193.stdout:1/108: mknod da/d13/c2a 0 2024-03-23T08:23:16.476 INFO:tasks.workunit.client.0.smithi070.stdout:6/167: dread - d0/d23/d28/f36 zero size 2024-03-23T08:23:16.478 INFO:tasks.workunit.client.0.smithi070.stdout:6/168: mkdir d0/d23/d28/d35/d3a/d3c 0 2024-03-23T08:23:16.479 INFO:tasks.workunit.client.0.smithi070.stdout:6/169: symlink d0/l3d 0 2024-03-23T08:23:16.480 INFO:tasks.workunit.client.1.smithi193.stdout:1/109: dread da/f12 [0,4194304] 0 2024-03-23T08:23:16.481 INFO:tasks.workunit.client.1.smithi193.stdout:2/187: dwrite d0/f2d [0,4194304] 0 2024-03-23T08:23:16.481 INFO:tasks.workunit.client.0.smithi070.stdout:6/170: chown d0/d23/d28/l29 774623454 1 2024-03-23T08:23:16.481 INFO:tasks.workunit.client.1.smithi193.stdout:9/107: dwrite d2/f28 [0,4194304] 0 2024-03-23T08:23:16.484 INFO:tasks.workunit.client.0.smithi070.stdout:6/171: getdents d0/d3/d2c 0 2024-03-23T08:23:16.485 INFO:tasks.workunit.client.0.smithi070.stdout:6/172: rmdir d0/d23 39 2024-03-23T08:23:16.487 INFO:tasks.workunit.client.1.smithi193.stdout:2/188: truncate d0/f7 8950178 0 2024-03-23T08:23:16.488 INFO:tasks.workunit.client.0.smithi070.stdout:6/173: symlink d0/d23/d28/d35/l3e 0 2024-03-23T08:23:16.488 INFO:tasks.workunit.client.0.smithi070.stdout:6/174: dread - d0/d3/f1a zero size 2024-03-23T08:23:16.490 INFO:tasks.workunit.client.1.smithi193.stdout:4/115: sync 2024-03-23T08:23:16.491 INFO:tasks.workunit.client.1.smithi193.stdout:6/84: dwrite d0/f5 [0,4194304] 0 2024-03-23T08:23:16.495 INFO:tasks.workunit.client.1.smithi193.stdout:9/108: creat d2/d7/f2b x:0 0 0 2024-03-23T08:23:16.495 INFO:tasks.workunit.client.1.smithi193.stdout:2/189: mknod d0/d1b/d19/c34 0 2024-03-23T08:23:16.496 INFO:tasks.workunit.client.1.smithi193.stdout:6/85: readlink d0/ld 0 2024-03-23T08:23:16.496 INFO:tasks.workunit.client.0.smithi070.stdout:6/175: dwrite d0/d23/f2a [0,4194304] 0 2024-03-23T08:23:16.497 INFO:tasks.workunit.client.0.smithi070.stdout:6/176: chown d0/d23/d1c/d38 447 1 2024-03-23T08:23:16.498 INFO:tasks.workunit.client.1.smithi193.stdout:9/109: write d2/d7/db/f26 [570060,69178] 0 2024-03-23T08:23:16.500 INFO:tasks.workunit.client.1.smithi193.stdout:9/110: truncate d2/f25 389278 0 2024-03-23T08:23:16.500 INFO:tasks.workunit.client.0.smithi070.stdout:6/177: mknod d0/d23/d1c/d38/c3f 0 2024-03-23T08:23:16.502 INFO:tasks.workunit.client.1.smithi193.stdout:1/110: mknod da/d13/d24/c2b 0 2024-03-23T08:23:16.505 INFO:tasks.workunit.client.1.smithi193.stdout:4/116: mknod d11/d1f/c21 0 2024-03-23T08:23:16.505 INFO:tasks.workunit.client.0.smithi070.stdout:6/178: rename d0/l26 to d0/d23/d28/l40 0 2024-03-23T08:23:16.507 INFO:tasks.workunit.client.1.smithi193.stdout:6/86: creat d0/db/df/f14 x:0 0 0 2024-03-23T08:23:16.507 INFO:tasks.workunit.client.1.smithi193.stdout:4/117: mkdir d11/d19/d22 0 2024-03-23T08:23:16.509 INFO:tasks.workunit.client.1.smithi193.stdout:4/118: write d11/f14 [448787,8284] 0 2024-03-23T08:23:16.510 INFO:tasks.workunit.client.1.smithi193.stdout:9/111: dread d2/d7/db/d17/d1f/f22 [0,4194304] 0 2024-03-23T08:23:16.512 INFO:tasks.workunit.client.1.smithi193.stdout:4/119: mknod d11/d19/c23 0 2024-03-23T08:23:16.512 INFO:tasks.workunit.client.1.smithi193.stdout:6/87: link d0/d8/l10 d0/d8/l15 0 2024-03-23T08:23:16.515 INFO:tasks.workunit.client.1.smithi193.stdout:1/111: mknod da/c2c 0 2024-03-23T08:23:16.516 INFO:tasks.workunit.client.1.smithi193.stdout:9/112: symlink d2/d7/db/l2c 0 2024-03-23T08:23:16.517 INFO:tasks.workunit.client.1.smithi193.stdout:4/120: mknod d11/d1a/c24 0 2024-03-23T08:23:16.518 INFO:tasks.workunit.client.1.smithi193.stdout:1/112: dread - da/dd/df/f25 zero size 2024-03-23T08:23:16.521 INFO:tasks.workunit.client.1.smithi193.stdout:9/113: write d2/d7/f1e [4410388,95511] 0 2024-03-23T08:23:16.526 INFO:tasks.workunit.client.1.smithi193.stdout:6/88: dwrite d0/db/df/f14 [0,4194304] 0 2024-03-23T08:23:16.527 INFO:tasks.workunit.client.1.smithi193.stdout:9/114: write d2/d7/f1e [5482099,18390] 0 2024-03-23T08:23:16.528 INFO:tasks.workunit.client.0.smithi070.stdout:6/179: chown d0/d23/d28/l40 30 1 2024-03-23T08:23:16.529 INFO:tasks.workunit.client.0.smithi070.stdout:6/180: read d0/d23/d28/f31 [3121234,31630] 0 2024-03-23T08:23:16.531 INFO:tasks.workunit.client.1.smithi193.stdout:4/121: rename d11/d1a/f1d to d11/d19/d22/f25 0 2024-03-23T08:23:16.532 INFO:tasks.workunit.client.1.smithi193.stdout:1/113: getdents da/d13/d24 0 2024-03-23T08:23:16.535 INFO:tasks.workunit.client.1.smithi193.stdout:2/190: dwrite d0/f20 [0,4194304] 0 2024-03-23T08:23:16.539 INFO:tasks.workunit.client.0.smithi070.stdout:6/181: dread d0/f20 [0,4194304] 0 2024-03-23T08:23:16.545 INFO:tasks.workunit.client.1.smithi193.stdout:9/115: dwrite d2/d7/fe [0,4194304] 0 2024-03-23T08:23:16.546 INFO:tasks.workunit.client.1.smithi193.stdout:9/116: dread - d2/d7/db/d17/d1f/f24 zero size 2024-03-23T08:23:16.555 INFO:tasks.workunit.client.1.smithi193.stdout:9/117: dwrite d2/d7/f1d [0,4194304] 0 2024-03-23T08:23:16.563 INFO:tasks.workunit.client.1.smithi193.stdout:8/94: sync 2024-03-23T08:23:16.563 INFO:tasks.workunit.client.1.smithi193.stdout:3/102: sync 2024-03-23T08:23:16.563 INFO:tasks.workunit.client.1.smithi193.stdout:7/109: sync 2024-03-23T08:23:16.564 INFO:tasks.workunit.client.1.smithi193.stdout:8/95: write df/d10/d19/f1b [834248,20076] 0 2024-03-23T08:23:16.569 INFO:tasks.workunit.client.1.smithi193.stdout:7/110: dwrite d6/d9/fd [0,4194304] 0 2024-03-23T08:23:16.570 INFO:tasks.workunit.client.1.smithi193.stdout:7/111: write d6/d9/f19 [607882,68448] 0 2024-03-23T08:23:16.571 INFO:tasks.workunit.client.1.smithi193.stdout:7/112: read d6/d9/d1f/f2a [281383,7711] 0 2024-03-23T08:23:16.665 INFO:tasks.workunit.client.1.smithi193.stdout:2/191: fdatasync d0/d1b/d14/d25/f29 0 2024-03-23T08:23:16.667 INFO:tasks.workunit.client.1.smithi193.stdout:6/89: link d0/f5 d0/db/df/d11/f16 0 2024-03-23T08:23:16.669 INFO:tasks.workunit.client.0.smithi070.stdout:6/182: symlink d0/d23/d28/d35/d3a/d3c/l41 0 2024-03-23T08:23:16.679 INFO:tasks.workunit.client.1.smithi193.stdout:0/146: sync 2024-03-23T08:23:16.679 INFO:tasks.workunit.client.1.smithi193.stdout:5/89: sync 2024-03-23T08:23:16.680 INFO:tasks.workunit.client.1.smithi193.stdout:0/147: fdatasync db/d11/d14/d21/f27 0 2024-03-23T08:23:16.680 INFO:tasks.workunit.client.1.smithi193.stdout:0/148: chown db/d11 437318 1 2024-03-23T08:23:16.696 INFO:tasks.workunit.client.0.smithi070.stdout:6/183: symlink d0/d3/d2c/l42 0 2024-03-23T08:23:16.697 INFO:tasks.workunit.client.1.smithi193.stdout:9/118: chown d2/d7/db/d17/d1f/l27 0 1 2024-03-23T08:23:16.699 INFO:tasks.workunit.client.0.smithi070.stdout:6/184: symlink d0/d23/d1c/l43 0 2024-03-23T08:23:16.700 INFO:tasks.workunit.client.1.smithi193.stdout:9/119: truncate d2/d7/f23 756926 0 2024-03-23T08:23:16.701 INFO:tasks.workunit.client.1.smithi193.stdout:4/122: dwrite f5 [0,4194304] 0 2024-03-23T08:23:16.701 INFO:tasks.workunit.client.0.smithi070.stdout:6/185: mknod d0/d23/d28/c44 0 2024-03-23T08:23:16.702 INFO:tasks.workunit.client.1.smithi193.stdout:4/123: dread - d11/d19/f20 zero size 2024-03-23T08:23:16.703 INFO:tasks.workunit.client.1.smithi193.stdout:2/192: mknod d0/d1b/d19/c35 0 2024-03-23T08:23:16.704 INFO:tasks.workunit.client.1.smithi193.stdout:6/90: fsync d0/f5 0 2024-03-23T08:23:16.707 INFO:tasks.workunit.client.1.smithi193.stdout:1/114: rename da/dd/f1f to da/dd/df/f2d 0 2024-03-23T08:23:16.708 INFO:tasks.workunit.client.1.smithi193.stdout:0/149: write db/d11/f12 [741480,33761] 0 2024-03-23T08:23:16.712 INFO:tasks.workunit.client.0.smithi070.stdout:6/186: write d0/d23/d28/f31 [1365127,122123] 0 2024-03-23T08:23:16.713 INFO:tasks.workunit.client.1.smithi193.stdout:2/193: chown d0/d1b/d19/c1e 3477 1 2024-03-23T08:23:16.716 INFO:tasks.workunit.client.0.smithi070.stdout:6/187: creat d0/d3/f45 x:0 0 0 2024-03-23T08:23:16.716 INFO:tasks.workunit.client.0.smithi070.stdout:6/188: fsync d0/d3/d2c/f34 0 2024-03-23T08:23:16.718 INFO:tasks.workunit.client.0.smithi070.stdout:6/189: chown d0/d23/d1c/d2f/f39 38 1 2024-03-23T08:23:16.718 INFO:tasks.workunit.client.1.smithi193.stdout:2/194: dread d0/fd [0,4194304] 0 2024-03-23T08:23:16.720 INFO:tasks.workunit.client.1.smithi193.stdout:7/113: truncate d6/ff 1128874 0 2024-03-23T08:23:16.721 INFO:tasks.workunit.client.1.smithi193.stdout:1/115: fdatasync da/dd/df/f1d 0 2024-03-23T08:23:16.722 INFO:tasks.workunit.client.0.smithi070.stdout:6/190: link d0/d23/f17 d0/d23/d28/d35/f46 0 2024-03-23T08:23:16.722 INFO:tasks.workunit.client.1.smithi193.stdout:2/195: write d0/d1b/d14/d25/f29 [409911,124538] 0 2024-03-23T08:23:16.722 INFO:tasks.workunit.client.1.smithi193.stdout:1/116: fsync da/dd/df/f29 0 2024-03-23T08:23:16.723 INFO:tasks.workunit.client.1.smithi193.stdout:2/196: stat d0/d1b/d14/d25/d2a 0 2024-03-23T08:23:16.724 INFO:tasks.workunit.client.0.smithi070.stdout:6/191: write d0/f20 [45991,33229] 0 2024-03-23T08:23:16.729 INFO:tasks.workunit.client.1.smithi193.stdout:8/96: rename c6 to df/d10/c23 0 2024-03-23T08:23:16.729 INFO:tasks.workunit.client.1.smithi193.stdout:5/90: truncate d0/ff 2279152 0 2024-03-23T08:23:16.730 INFO:tasks.workunit.client.0.smithi070.stdout:6/192: rmdir d0/d23/d1c/d2f 39 2024-03-23T08:23:16.730 INFO:tasks.workunit.client.1.smithi193.stdout:5/91: write d0/f14 [913080,1504] 0 2024-03-23T08:23:16.732 INFO:tasks.workunit.client.1.smithi193.stdout:9/120: truncate d2/d7/fe 1635861 0 2024-03-23T08:23:16.732 INFO:tasks.workunit.client.1.smithi193.stdout:1/117: dread - da/dd/df/f22 zero size 2024-03-23T08:23:16.735 INFO:tasks.workunit.client.1.smithi193.stdout:4/124: rename d11/f14 to d11/d19/d22/f26 0 2024-03-23T08:23:16.737 INFO:tasks.workunit.client.1.smithi193.stdout:0/150: truncate db/f1e 2411190 0 2024-03-23T08:23:16.738 INFO:tasks.workunit.client.1.smithi193.stdout:8/97: dwrite df/f1f [0,4194304] 0 2024-03-23T08:23:16.746 INFO:tasks.workunit.client.0.smithi070.stdout:6/193: truncate d0/d23/d28/d35/d3a/f3b 182410 0 2024-03-23T08:23:16.762 INFO:tasks.workunit.client.0.smithi070.stdout:6/194: write d0/d3/f4 [2838897,20000] 0 2024-03-23T08:23:16.767 INFO:tasks.workunit.client.1.smithi193.stdout:3/103: sync 2024-03-23T08:23:16.771 INFO:tasks.workunit.client.1.smithi193.stdout:5/92: read d0/f17 [49539,29854] 0 2024-03-23T08:23:16.772 INFO:tasks.workunit.client.1.smithi193.stdout:1/118: chown da/dd/df/f17 597299 1 2024-03-23T08:23:16.774 INFO:tasks.workunit.client.1.smithi193.stdout:5/93: write d0/f14 [511802,40181] 0 2024-03-23T08:23:16.774 INFO:tasks.workunit.client.1.smithi193.stdout:9/121: dread d2/d7/f1d [0,4194304] 0 2024-03-23T08:23:16.777 INFO:tasks.workunit.client.1.smithi193.stdout:5/94: chown d0/f7 6683 1 2024-03-23T08:23:16.777 INFO:tasks.workunit.client.1.smithi193.stdout:7/114: dwrite d6/f8 [0,4194304] 0 2024-03-23T08:23:16.780 INFO:tasks.workunit.client.1.smithi193.stdout:7/115: chown d6/d9/d1f/d23/c27 61 1 2024-03-23T08:23:16.780 INFO:tasks.workunit.client.1.smithi193.stdout:9/122: dread d2/d7/f23 [0,4194304] 0 2024-03-23T08:23:16.781 INFO:tasks.workunit.client.1.smithi193.stdout:2/197: dwrite d0/fc [0,4194304] 0 2024-03-23T08:23:16.782 INFO:tasks.workunit.client.1.smithi193.stdout:9/123: write d2/f4 [1323416,105495] 0 2024-03-23T08:23:16.785 INFO:tasks.workunit.client.1.smithi193.stdout:7/116: dread - d6/d9/fa zero size 2024-03-23T08:23:16.789 INFO:tasks.workunit.client.1.smithi193.stdout:9/124: dwrite d2/d7/f11 [0,4194304] 0 2024-03-23T08:23:16.822 INFO:tasks.workunit.client.1.smithi193.stdout:6/91: sync 2024-03-23T08:23:16.829 INFO:tasks.workunit.client.1.smithi193.stdout:0/151: symlink db/l36 0 2024-03-23T08:23:16.835 INFO:tasks.workunit.client.1.smithi193.stdout:5/95: creat d0/f1d x:0 0 0 2024-03-23T08:23:16.840 INFO:tasks.workunit.client.1.smithi193.stdout:2/198: symlink d0/d28/l36 0 2024-03-23T08:23:16.842 INFO:tasks.workunit.client.1.smithi193.stdout:7/117: rename d6/c7 to d6/d9/d1f/d23/c2b 0 2024-03-23T08:23:16.843 INFO:tasks.workunit.client.1.smithi193.stdout:9/125: mknod d2/d7/db/d17/d1f/c2d 0 2024-03-23T08:23:16.845 INFO:tasks.workunit.client.1.smithi193.stdout:9/126: dread - d2/d7/f15 zero size 2024-03-23T08:23:16.846 INFO:tasks.workunit.client.1.smithi193.stdout:6/92: dread d0/fe [0,4194304] 0 2024-03-23T08:23:16.848 INFO:tasks.workunit.client.1.smithi193.stdout:7/118: dwrite d6/d9/f15 [0,4194304] 0 2024-03-23T08:23:16.860 INFO:tasks.workunit.client.1.smithi193.stdout:3/104: unlink d1/de/l1b 0 2024-03-23T08:23:16.863 INFO:tasks.workunit.client.1.smithi193.stdout:8/98: rmdir df/d10/d19/d20 0 2024-03-23T08:23:16.864 INFO:tasks.workunit.client.1.smithi193.stdout:8/99: fdatasync df/f18 0 2024-03-23T08:23:16.864 INFO:tasks.workunit.client.1.smithi193.stdout:4/125: sync 2024-03-23T08:23:16.866 INFO:tasks.workunit.client.1.smithi193.stdout:8/100: read df/d10/d19/f1b [41695,61615] 0 2024-03-23T08:23:16.867 INFO:tasks.workunit.client.1.smithi193.stdout:2/199: mkdir d0/d37 0 2024-03-23T08:23:16.874 INFO:tasks.workunit.client.1.smithi193.stdout:4/126: mkdir d11/d27 0 2024-03-23T08:23:16.878 INFO:tasks.workunit.client.1.smithi193.stdout:8/101: mknod df/d13/c24 0 2024-03-23T08:23:16.885 INFO:tasks.workunit.client.1.smithi193.stdout:4/127: creat d11/d1f/f28 x:0 0 0 2024-03-23T08:23:16.886 INFO:tasks.workunit.client.1.smithi193.stdout:3/105: dwrite d1/fc [0,4194304] 0 2024-03-23T08:23:16.890 INFO:tasks.workunit.client.1.smithi193.stdout:2/200: symlink d0/d1b/l38 0 2024-03-23T08:23:16.890 INFO:tasks.workunit.client.1.smithi193.stdout:4/128: dread d11/f12 [0,4194304] 0 2024-03-23T08:23:16.892 INFO:tasks.workunit.client.1.smithi193.stdout:9/127: getdents d2/d7/db/d17 0 2024-03-23T08:23:16.893 INFO:tasks.workunit.client.1.smithi193.stdout:8/102: dread df/d10/d19/f1b [0,4194304] 0 2024-03-23T08:23:16.894 INFO:tasks.workunit.client.1.smithi193.stdout:6/93: link d0/l4 d0/d8/l17 0 2024-03-23T08:23:16.899 INFO:tasks.workunit.client.1.smithi193.stdout:4/129: chown d11/d19/d22/f25 2 1 2024-03-23T08:23:16.899 INFO:tasks.workunit.client.1.smithi193.stdout:4/130: dread - d11/f13 zero size 2024-03-23T08:23:16.901 INFO:tasks.workunit.client.1.smithi193.stdout:9/128: creat d2/d7/db/f2e x:0 0 0 2024-03-23T08:23:16.902 INFO:tasks.workunit.client.1.smithi193.stdout:9/129: stat d2/d7/db/d17/d1f/l21 0 2024-03-23T08:23:16.903 INFO:tasks.workunit.client.1.smithi193.stdout:8/103: creat df/d13/f25 x:0 0 0 2024-03-23T08:23:16.906 INFO:tasks.workunit.client.1.smithi193.stdout:4/131: unlink f8 0 2024-03-23T08:23:16.916 INFO:tasks.workunit.client.1.smithi193.stdout:6/94: rename d0/d8/l15 to d0/l18 0 2024-03-23T08:23:16.917 INFO:tasks.workunit.client.1.smithi193.stdout:6/95: read d0/f5 [2868426,15870] 0 2024-03-23T08:23:16.937 INFO:tasks.workunit.client.1.smithi193.stdout:1/119: dwrite f4 [0,4194304] 0 2024-03-23T08:23:16.944 INFO:tasks.workunit.client.1.smithi193.stdout:5/96: getdents d0 0 2024-03-23T08:23:16.945 INFO:tasks.workunit.client.1.smithi193.stdout:1/120: mkdir da/dd/df/d2e 0 2024-03-23T08:23:16.949 INFO:tasks.workunit.client.1.smithi193.stdout:2/201: dwrite d0/fc [4194304,4194304] 0 2024-03-23T08:23:16.951 INFO:tasks.workunit.client.1.smithi193.stdout:2/202: write d0/f20 [3111301,107109] 0 2024-03-23T08:23:16.952 INFO:tasks.workunit.client.1.smithi193.stdout:5/97: creat d0/d1c/f1e x:0 0 0 2024-03-23T08:23:16.954 INFO:tasks.workunit.client.1.smithi193.stdout:1/121: mknod da/dd/df/c2f 0 2024-03-23T08:23:16.955 INFO:tasks.workunit.client.1.smithi193.stdout:1/122: stat da/dd/l1b 0 2024-03-23T08:23:16.966 INFO:tasks.workunit.client.1.smithi193.stdout:1/123: unlink da/dd/df/f1e 0 2024-03-23T08:23:16.967 INFO:tasks.workunit.client.1.smithi193.stdout:1/124: chown da/d13/c23 1 1 2024-03-23T08:23:16.968 INFO:tasks.workunit.client.0.smithi070.stdout:6/195: rename d0/d23/d28/c44 to d0/d23/d1c/c47 0 2024-03-23T08:23:16.970 INFO:tasks.workunit.client.0.smithi070.stdout:6/196: mknod d0/d23/d1c/d38/c48 0 2024-03-23T08:23:16.972 INFO:tasks.workunit.client.1.smithi193.stdout:1/125: link f5 da/dd/f30 0 2024-03-23T08:23:16.973 INFO:tasks.workunit.client.1.smithi193.stdout:1/126: write da/dd/df/f22 [56263,113513] 0 2024-03-23T08:23:16.975 INFO:tasks.workunit.client.1.smithi193.stdout:1/127: symlink da/d13/d24/l31 0 2024-03-23T08:23:16.975 INFO:tasks.workunit.client.1.smithi193.stdout:1/128: dread - da/dd/f1c zero size 2024-03-23T08:23:16.977 INFO:tasks.workunit.client.0.smithi070.stdout:6/197: dwrite d0/d23/d28/f36 [0,4194304] 0 2024-03-23T08:23:16.978 INFO:tasks.workunit.client.1.smithi193.stdout:1/129: dread f5 [0,4194304] 0 2024-03-23T08:23:16.978 INFO:tasks.workunit.client.1.smithi193.stdout:1/130: chown f1 3 1 2024-03-23T08:23:16.981 INFO:tasks.workunit.client.0.smithi070.stdout:6/198: creat d0/d3/d2c/f49 x:0 0 0 2024-03-23T08:23:16.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:16 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:23:16.986 INFO:tasks.workunit.client.1.smithi193.stdout:1/131: dwrite da/dd/df/f2d [0,4194304] 0 2024-03-23T08:23:16.988 INFO:tasks.workunit.client.1.smithi193.stdout:9/130: rmdir d2 39 2024-03-23T08:23:16.990 INFO:tasks.workunit.client.1.smithi193.stdout:8/104: sync 2024-03-23T08:23:16.990 INFO:tasks.workunit.client.1.smithi193.stdout:3/106: sync 2024-03-23T08:23:16.993 INFO:tasks.workunit.client.1.smithi193.stdout:9/131: mkdir d2/d2f 0 2024-03-23T08:23:16.994 INFO:tasks.workunit.client.1.smithi193.stdout:3/107: dread d1/f1e [0,4194304] 0 2024-03-23T08:23:16.996 INFO:tasks.workunit.client.1.smithi193.stdout:8/105: creat df/d10/d19/f26 x:0 0 0 2024-03-23T08:23:16.996 INFO:tasks.workunit.client.1.smithi193.stdout:9/132: truncate d2/f4 2447187 0 2024-03-23T08:23:16.997 INFO:tasks.workunit.client.1.smithi193.stdout:1/132: dwrite da/fc [0,4194304] 0 2024-03-23T08:23:16.997 INFO:tasks.workunit.client.1.smithi193.stdout:4/132: link f4 d11/d1f/f29 0 2024-03-23T08:23:16.999 INFO:tasks.workunit.client.1.smithi193.stdout:9/133: stat d2/d7/f2b 0 2024-03-23T08:23:17.000 INFO:tasks.workunit.client.1.smithi193.stdout:1/133: chown da/dd/f1c 31323420 1 2024-03-23T08:23:17.002 INFO:tasks.workunit.client.1.smithi193.stdout:1/134: chown da/dd/f30 678306 1 2024-03-23T08:23:17.003 INFO:tasks.workunit.client.1.smithi193.stdout:7/119: write d6/ff [239682,3944] 0 2024-03-23T08:23:17.004 INFO:tasks.workunit.client.1.smithi193.stdout:9/134: getdents d2/d2f 0 2024-03-23T08:23:17.005 INFO:tasks.workunit.client.1.smithi193.stdout:8/106: creat df/d10/f27 x:0 0 0 2024-03-23T08:23:17.007 INFO:tasks.workunit.client.1.smithi193.stdout:6/96: dwrite d0/f5 [0,4194304] 0 2024-03-23T08:23:17.009 INFO:tasks.workunit.client.1.smithi193.stdout:4/133: dwrite fb [0,4194304] 0 2024-03-23T08:23:17.010 INFO:tasks.workunit.client.1.smithi193.stdout:8/107: chown df/d13/c24 1478337661 1 2024-03-23T08:23:17.012 INFO:tasks.workunit.client.1.smithi193.stdout:1/135: unlink da/dd/df/f11 0 2024-03-23T08:23:17.012 INFO:tasks.workunit.client.1.smithi193.stdout:6/97: write d0/db/df/f14 [3676227,76696] 0 2024-03-23T08:23:17.015 INFO:tasks.workunit.client.1.smithi193.stdout:8/108: write f0 [4946442,127762] 0 2024-03-23T08:23:17.021 INFO:tasks.workunit.client.1.smithi193.stdout:7/120: getdents d6/d9 0 2024-03-23T08:23:17.024 INFO:tasks.workunit.client.1.smithi193.stdout:2/203: write d0/d1b/f13 [1829119,85575] 0 2024-03-23T08:23:17.025 INFO:tasks.workunit.client.1.smithi193.stdout:5/98: write d0/ff [157286,33430] 0 2024-03-23T08:23:17.026 INFO:tasks.workunit.client.1.smithi193.stdout:0/152: dwrite db/f1e [0,4194304] 0 2024-03-23T08:23:17.027 INFO:tasks.workunit.client.0.smithi070.stdout:6/199: read d0/d23/f2a [100852,108570] 0 2024-03-23T08:23:17.030 INFO:tasks.workunit.client.1.smithi193.stdout:8/109: dwrite df/d10/f27 [0,4194304] 0 2024-03-23T08:23:17.032 INFO:tasks.workunit.client.1.smithi193.stdout:7/121: rename d6/f10 to d6/d9/d1f/f2c 0 2024-03-23T08:23:17.032 INFO:tasks.workunit.client.1.smithi193.stdout:5/99: mkdir d0/d1f 0 2024-03-23T08:23:17.035 INFO:tasks.workunit.client.1.smithi193.stdout:1/136: rmdir da/d13 39 2024-03-23T08:23:17.036 INFO:tasks.workunit.client.1.smithi193.stdout:2/204: getdents d0/d37 0 2024-03-23T08:23:17.037 INFO:tasks.workunit.client.1.smithi193.stdout:8/110: symlink df/d13/l28 0 2024-03-23T08:23:17.038 INFO:tasks.workunit.client.0.smithi070.stdout:6/200: dwrite d0/d23/d1c/d2f/f39 [0,4194304] 0 2024-03-23T08:23:17.039 INFO:tasks.workunit.client.0.smithi070.stdout:6/201: chown d0/d3/c7 14891 1 2024-03-23T08:23:17.042 INFO:tasks.workunit.client.1.smithi193.stdout:5/100: dwrite d0/d1c/f1e [0,4194304] 0 2024-03-23T08:23:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:16 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:23:17.141 INFO:tasks.workunit.client.1.smithi193.stdout:6/98: sync 2024-03-23T08:23:17.146 INFO:tasks.workunit.client.1.smithi193.stdout:6/99: dwrite d0/db/df/f14 [4194304,4194304] 0 2024-03-23T08:23:17.147 INFO:tasks.workunit.client.1.smithi193.stdout:6/100: chown d0/db/df/d11 734598 1 2024-03-23T08:23:17.175 INFO:tasks.workunit.client.0.smithi070.stdout:6/202: creat d0/d23/d28/f4a x:0 0 0 2024-03-23T08:23:17.175 INFO:tasks.workunit.client.1.smithi193.stdout:3/108: write d1/de/d13/f1a [358313,75701] 0 2024-03-23T08:23:17.178 INFO:tasks.workunit.client.1.smithi193.stdout:9/135: write d2/d7/f23 [1344193,35832] 0 2024-03-23T08:23:17.180 INFO:tasks.workunit.client.1.smithi193.stdout:9/136: dread d2/f25 [0,4194304] 0 2024-03-23T08:23:17.180 INFO:tasks.workunit.client.1.smithi193.stdout:5/101: creat d0/d1c/f20 x:0 0 0 2024-03-23T08:23:17.182 INFO:tasks.workunit.client.1.smithi193.stdout:4/134: truncate fd 1060801 0 2024-03-23T08:23:17.184 INFO:tasks.workunit.client.1.smithi193.stdout:0/153: link db/d11/d14/d24/d2a/c2d db/d11/d14/d21/c37 0 2024-03-23T08:23:17.184 INFO:tasks.workunit.client.1.smithi193.stdout:0/154: readlink db/l2f 0 2024-03-23T08:23:17.185 INFO:tasks.workunit.client.1.smithi193.stdout:0/155: dread - db/d11/d14/d21/f2c zero size 2024-03-23T08:23:17.187 INFO:tasks.workunit.client.1.smithi193.stdout:5/102: creat d0/d1c/f21 x:0 0 0 2024-03-23T08:23:17.189 INFO:tasks.workunit.client.1.smithi193.stdout:9/137: creat d2/d7/db/d1b/f30 x:0 0 0 2024-03-23T08:23:17.189 INFO:tasks.workunit.client.0.smithi070.stdout:6/203: write d0/d23/f10 [66309,48716] 0 2024-03-23T08:23:17.190 INFO:tasks.workunit.client.1.smithi193.stdout:4/135: mknod d11/d19/c2a 0 2024-03-23T08:23:17.191 INFO:tasks.workunit.client.1.smithi193.stdout:2/205: sync 2024-03-23T08:23:17.192 INFO:tasks.workunit.client.1.smithi193.stdout:4/136: chown d11/l15 192638 1 2024-03-23T08:23:17.192 INFO:tasks.workunit.client.1.smithi193.stdout:9/138: chown d2/d7/db/d1b/l29 8555 1 2024-03-23T08:23:17.193 INFO:tasks.workunit.client.1.smithi193.stdout:8/111: rename df/d10/d19/f1d to df/d10/f29 0 2024-03-23T08:23:17.193 INFO:tasks.workunit.client.1.smithi193.stdout:2/206: stat d0/d1b/d19/c1e 0 2024-03-23T08:23:17.194 INFO:tasks.workunit.client.1.smithi193.stdout:7/122: getdents d6/d9/d1f 0 2024-03-23T08:23:17.196 INFO:tasks.workunit.client.1.smithi193.stdout:1/137: dwrite da/f12 [0,4194304] 0 2024-03-23T08:23:17.196 INFO:tasks.workunit.client.1.smithi193.stdout:0/156: dwrite db/d11/d14/d21/f26 [0,4194304] 0 2024-03-23T08:23:17.197 INFO:tasks.workunit.client.0.smithi070.stdout:6/204: rename d0/d3/l1b to d0/d23/d1c/l4b 0 2024-03-23T08:23:17.197 INFO:tasks.workunit.client.1.smithi193.stdout:8/112: write fc [3554909,1498] 0 2024-03-23T08:23:17.197 INFO:tasks.workunit.client.1.smithi193.stdout:9/139: readlink d2/d7/db/d17/l19 0 2024-03-23T08:23:17.198 INFO:tasks.workunit.client.0.smithi070.stdout:6/205: chown d0/d3/f4 1 1 2024-03-23T08:23:17.198 INFO:tasks.workunit.client.0.smithi070.stdout:6/206: fsync d0/f20 0 2024-03-23T08:23:17.200 INFO:tasks.workunit.client.1.smithi193.stdout:0/157: write db/d11/f13 [231052,28050] 0 2024-03-23T08:23:17.201 INFO:tasks.workunit.client.1.smithi193.stdout:7/123: write d6/d9/fe [183311,80032] 0 2024-03-23T08:23:17.204 INFO:tasks.workunit.client.1.smithi193.stdout:6/101: rmdir d0/d13 0 2024-03-23T08:23:17.205 INFO:tasks.workunit.client.1.smithi193.stdout:7/124: write d6/d9/f15 [3765294,120230] 0 2024-03-23T08:23:17.209 INFO:tasks.workunit.client.0.smithi070.stdout:6/207: creat d0/d23/d1c/d2f/f4c x:0 0 0 2024-03-23T08:23:17.212 INFO:tasks.workunit.client.1.smithi193.stdout:7/125: dread d6/d9/fe [0,4194304] 0 2024-03-23T08:23:17.212 INFO:tasks.workunit.client.1.smithi193.stdout:4/137: creat d11/d19/d22/f2b x:0 0 0 2024-03-23T08:23:17.212 INFO:tasks.workunit.client.1.smithi193.stdout:9/140: dread d2/d7/db/f13 [0,4194304] 0 2024-03-23T08:23:17.214 INFO:tasks.workunit.client.1.smithi193.stdout:9/141: read - d2/d7/f2b zero size 2024-03-23T08:23:17.215 INFO:tasks.workunit.client.0.smithi070.stdout:6/208: rmdir d0/d23/d28 39 2024-03-23T08:23:17.215 INFO:tasks.workunit.client.0.smithi070.stdout:6/209: dread - d0/d23/d1c/d2f/f4c zero size 2024-03-23T08:23:17.218 INFO:tasks.workunit.client.1.smithi193.stdout:9/142: write d2/d7/db/f13 [4433561,76816] 0 2024-03-23T08:23:17.218 INFO:tasks.workunit.client.1.smithi193.stdout:5/103: creat d0/d1f/f22 x:0 0 0 2024-03-23T08:23:17.218 INFO:tasks.workunit.client.1.smithi193.stdout:3/109: truncate d1/de/f10 3473570 0 2024-03-23T08:23:17.221 INFO:tasks.workunit.client.1.smithi193.stdout:0/158: symlink db/d11/l38 0 2024-03-23T08:23:17.221 INFO:tasks.workunit.client.1.smithi193.stdout:0/159: stat db/d11/d14/d21/d33 0 2024-03-23T08:23:17.223 INFO:tasks.workunit.client.0.smithi070.stdout:6/210: dwrite d0/d23/f19 [0,4194304] 0 2024-03-23T08:23:17.223 INFO:tasks.workunit.client.1.smithi193.stdout:7/126: creat d6/f2d x:0 0 0 2024-03-23T08:23:17.225 INFO:tasks.workunit.client.1.smithi193.stdout:5/104: mknod d0/d1c/c23 0 2024-03-23T08:23:17.228 INFO:tasks.workunit.client.1.smithi193.stdout:2/207: creat d0/d37/f39 x:0 0 0 2024-03-23T08:23:17.230 INFO:tasks.workunit.client.1.smithi193.stdout:9/143: rename d2/d2f to d2/d7/d31 0 2024-03-23T08:23:17.231 INFO:tasks.workunit.client.0.smithi070.stdout:6/211: dread d0/d23/d28/d35/f46 [0,4194304] 0 2024-03-23T08:23:17.232 INFO:tasks.workunit.client.0.smithi070.stdout:6/212: write d0/d23/f24 [1356906,83680] 0 2024-03-23T08:23:17.233 INFO:tasks.workunit.client.1.smithi193.stdout:1/138: mknod da/c32 0 2024-03-23T08:23:17.237 INFO:tasks.workunit.client.1.smithi193.stdout:6/102: unlink d0/d8/l10 0 2024-03-23T08:23:17.238 INFO:tasks.workunit.client.1.smithi193.stdout:6/103: dread d0/fe [0,4194304] 0 2024-03-23T08:23:17.241 INFO:tasks.workunit.client.1.smithi193.stdout:7/127: creat d6/d9/d1f/d23/f2e x:0 0 0 2024-03-23T08:23:17.243 INFO:tasks.workunit.client.1.smithi193.stdout:0/160: rename db/d11/d14/d24/d2a/f31 to db/d11/d14/d24/f39 0 2024-03-23T08:23:17.244 INFO:tasks.workunit.client.1.smithi193.stdout:9/144: mkdir d2/d7/db/d32 0 2024-03-23T08:23:17.245 INFO:tasks.workunit.client.1.smithi193.stdout:3/110: mkdir d1/de/d13/d21 0 2024-03-23T08:23:17.246 INFO:tasks.workunit.client.1.smithi193.stdout:6/104: creat d0/d8/f19 x:0 0 0 2024-03-23T08:23:17.247 INFO:tasks.workunit.client.1.smithi193.stdout:4/138: link l7 d11/d1a/l2c 0 2024-03-23T08:23:17.248 INFO:tasks.workunit.client.1.smithi193.stdout:7/128: mknod d6/d9/d1f/c2f 0 2024-03-23T08:23:17.248 INFO:tasks.workunit.client.1.smithi193.stdout:7/129: fdatasync d6/f2d 0 2024-03-23T08:23:17.249 INFO:tasks.workunit.client.1.smithi193.stdout:5/105: rename d0/f10 to d0/d1c/f24 0 2024-03-23T08:23:17.250 INFO:tasks.workunit.client.1.smithi193.stdout:9/145: dwrite d2/f28 [0,4194304] 0 2024-03-23T08:23:17.252 INFO:tasks.workunit.client.1.smithi193.stdout:1/139: symlink da/l33 0 2024-03-23T08:23:17.253 INFO:tasks.workunit.client.1.smithi193.stdout:4/139: mkdir d11/d2d 0 2024-03-23T08:23:17.254 INFO:tasks.workunit.client.1.smithi193.stdout:6/105: symlink d0/db/l1a 0 2024-03-23T08:23:17.255 INFO:tasks.workunit.client.1.smithi193.stdout:5/106: dread d0/f12 [0,4194304] 0 2024-03-23T08:23:17.256 INFO:tasks.workunit.client.1.smithi193.stdout:5/107: write d0/d1c/f20 [133884,120832] 0 2024-03-23T08:23:17.257 INFO:tasks.workunit.client.1.smithi193.stdout:6/106: dread d0/fe [0,4194304] 0 2024-03-23T08:23:17.257 INFO:tasks.workunit.client.1.smithi193.stdout:2/208: rename d0/f20 to d0/d1b/d14/d25/f3a 0 2024-03-23T08:23:17.258 INFO:tasks.workunit.client.1.smithi193.stdout:7/130: rename d6/d9/d1f/f2c to d6/d9/d1f/f30 0 2024-03-23T08:23:17.260 INFO:tasks.workunit.client.0.smithi070.stdout:6/213: dwrite d0/d23/f19 [4194304,4194304] 0 2024-03-23T08:23:17.261 INFO:tasks.workunit.client.1.smithi193.stdout:9/146: mkdir d2/d7/db/d17/d33 0 2024-03-23T08:23:17.261 INFO:tasks.workunit.client.1.smithi193.stdout:7/131: chown d6/d9/f19 97312 1 2024-03-23T08:23:17.262 INFO:tasks.workunit.client.1.smithi193.stdout:3/111: mkdir d1/d22 0 2024-03-23T08:23:17.262 INFO:tasks.workunit.client.0.smithi070.stdout:6/214: write d0/d23/d1c/d2f/f4c [563860,112095] 0 2024-03-23T08:23:17.263 INFO:tasks.workunit.client.0.smithi070.stdout:6/215: chown d0/d23/f19 25732 1 2024-03-23T08:23:17.264 INFO:tasks.workunit.client.0.smithi070.stdout:6/216: stat d0/d23/d28/d35/l3e 0 2024-03-23T08:23:17.265 INFO:tasks.workunit.client.0.smithi070.stdout:6/217: chown d0 0 1 2024-03-23T08:23:17.266 INFO:tasks.workunit.client.1.smithi193.stdout:6/107: dwrite d0/d8/f19 [0,4194304] 0 2024-03-23T08:23:17.266 INFO:tasks.workunit.client.0.smithi070.stdout:6/218: write d0/f20 [1430374,83599] 0 2024-03-23T08:23:17.268 INFO:tasks.workunit.client.1.smithi193.stdout:5/108: write d0/d1c/f24 [4572077,89389] 0 2024-03-23T08:23:17.271 INFO:tasks.workunit.client.1.smithi193.stdout:2/209: unlink d0/d37/f39 0 2024-03-23T08:23:17.272 INFO:tasks.workunit.client.1.smithi193.stdout:2/210: read - d0/d1b/d14/d25/f32 zero size 2024-03-23T08:23:17.274 INFO:tasks.workunit.client.1.smithi193.stdout:2/211: write d0/d1b/f21 [843115,75365] 0 2024-03-23T08:23:17.274 INFO:tasks.workunit.client.1.smithi193.stdout:2/212: stat d0/d28/d33 0 2024-03-23T08:23:17.274 INFO:tasks.workunit.client.1.smithi193.stdout:8/113: sync 2024-03-23T08:23:17.278 INFO:tasks.workunit.client.0.smithi070.stdout:6/219: mkdir d0/d23/d4d 0 2024-03-23T08:23:17.281 INFO:tasks.workunit.client.0.smithi070.stdout:6/220: symlink d0/d23/d1c/d38/l4e 0 2024-03-23T08:23:17.282 INFO:tasks.workunit.client.0.smithi070.stdout:6/221: chown d0/d23/d1c/c21 13096502 1 2024-03-23T08:23:17.283 INFO:tasks.workunit.client.0.smithi070.stdout:6/222: fsync d0/d3/f45 0 2024-03-23T08:23:17.283 INFO:tasks.workunit.client.0.smithi070.stdout:6/223: read - d0/d23/d28/f37 zero size 2024-03-23T08:23:17.284 INFO:tasks.workunit.client.1.smithi193.stdout:7/132: creat d6/d9/f31 x:0 0 0 2024-03-23T08:23:17.285 INFO:tasks.workunit.client.1.smithi193.stdout:3/112: stat d1/l17 0 2024-03-23T08:23:17.285 INFO:tasks.workunit.client.0.smithi070.stdout:6/224: truncate d0/d3/d2c/f34 850589 0 2024-03-23T08:23:17.286 INFO:tasks.workunit.client.1.smithi193.stdout:9/147: fsync d2/d7/db/d17/d1f/f22 0 2024-03-23T08:23:17.286 INFO:tasks.workunit.client.1.smithi193.stdout:9/148: readlink d2/d7/db/d17/d1f/l21 0 2024-03-23T08:23:17.290 INFO:tasks.workunit.client.1.smithi193.stdout:0/161: rename db/c1c to db/d11/d14/c3a 0 2024-03-23T08:23:17.293 INFO:tasks.workunit.client.1.smithi193.stdout:8/114: mkdir df/d13/d2a 0 2024-03-23T08:23:17.294 INFO:tasks.workunit.client.1.smithi193.stdout:1/140: truncate da/fb 3150399 0 2024-03-23T08:23:17.294 INFO:tasks.workunit.client.0.smithi070.stdout:6/225: getdents d0/d23/d28 0 2024-03-23T08:23:17.295 INFO:tasks.workunit.client.1.smithi193.stdout:1/141: chown da/dd 0 1 2024-03-23T08:23:17.297 INFO:tasks.workunit.client.0.smithi070.stdout:6/226: mknod d0/d23/d28/d35/c4f 0 2024-03-23T08:23:17.297 INFO:tasks.workunit.client.0.smithi070.stdout:6/227: read d0/f20 [213309,32533] 0 2024-03-23T08:23:17.298 INFO:tasks.workunit.client.1.smithi193.stdout:3/113: rmdir d1/de 39 2024-03-23T08:23:17.298 INFO:tasks.workunit.client.1.smithi193.stdout:1/142: chown da/dd/f14 10868347 1 2024-03-23T08:23:17.299 INFO:tasks.workunit.client.1.smithi193.stdout:0/162: dwrite db/d11/d14/d21/f22 [4194304,4194304] 0 2024-03-23T08:23:17.300 INFO:tasks.workunit.client.1.smithi193.stdout:7/133: dread d6/fc [8388608,4194304] 0 2024-03-23T08:23:17.303 INFO:tasks.workunit.client.1.smithi193.stdout:4/140: rename fa to d11/f2e 0 2024-03-23T08:23:17.306 INFO:tasks.workunit.client.1.smithi193.stdout:6/108: stat d0/db/df/f14 0 2024-03-23T08:23:17.309 INFO:tasks.workunit.client.0.smithi070.stdout:6/228: creat d0/f50 x:0 0 0 2024-03-23T08:23:17.312 INFO:tasks.workunit.client.1.smithi193.stdout:8/115: rmdir df/d10/d19 39 2024-03-23T08:23:17.313 INFO:tasks.workunit.client.1.smithi193.stdout:9/149: getdents d2/d7/d31 0 2024-03-23T08:23:17.315 INFO:tasks.workunit.client.1.smithi193.stdout:7/134: dwrite d6/d9/f1b [0,4194304] 0 2024-03-23T08:23:17.317 INFO:tasks.workunit.client.0.smithi070.stdout:6/229: dwrite d0/d23/f10 [0,4194304] 0 2024-03-23T08:23:17.318 INFO:tasks.workunit.client.1.smithi193.stdout:2/213: dwrite d0/d1b/d19/f1a [0,4194304] 0 2024-03-23T08:23:17.318 INFO:tasks.workunit.client.1.smithi193.stdout:0/163: dwrite db/d11/d14/d24/d2a/f2e [0,4194304] 0 2024-03-23T08:23:17.319 INFO:tasks.workunit.client.1.smithi193.stdout:4/141: readlink d11/d1a/l2c 0 2024-03-23T08:23:17.323 INFO:tasks.workunit.client.1.smithi193.stdout:5/109: getdents d0/d1c 0 2024-03-23T08:23:17.324 INFO:tasks.workunit.client.0.smithi070.stdout:6/230: creat d0/d23/d28/d35/d3a/d3c/f51 x:0 0 0 2024-03-23T08:23:17.328 INFO:tasks.workunit.client.1.smithi193.stdout:3/114: rename d1/de/d13/f1a to d1/de/d13/d21/f23 0 2024-03-23T08:23:17.329 INFO:tasks.workunit.client.0.smithi070.stdout:6/231: chown d0/d23/c14 2041 1 2024-03-23T08:23:17.332 INFO:tasks.workunit.client.1.smithi193.stdout:5/110: mkdir d0/d1f/d25 0 2024-03-23T08:23:17.334 INFO:tasks.workunit.client.1.smithi193.stdout:2/214: dread d0/d1b/f21 [0,4194304] 0 2024-03-23T08:23:17.334 INFO:tasks.workunit.client.0.smithi070.stdout:6/232: mkdir d0/d23/d1c/d38/d52 0 2024-03-23T08:23:17.335 INFO:tasks.workunit.client.1.smithi193.stdout:6/109: unlink d0/db/df/d11/f16 0 2024-03-23T08:23:17.335 INFO:tasks.workunit.client.1.smithi193.stdout:2/215: read d0/f1d [361196,124023] 0 2024-03-23T08:23:17.336 INFO:tasks.workunit.client.1.smithi193.stdout:2/216: read d0/d1b/d19/f1a [3600585,3784] 0 2024-03-23T08:23:17.337 INFO:tasks.workunit.client.1.smithi193.stdout:2/217: fsync d0/d1b/f2f 0 2024-03-23T08:23:17.338 INFO:tasks.workunit.client.0.smithi070.stdout:6/233: dread d0/d23/f10 [0,4194304] 0 2024-03-23T08:23:17.338 INFO:tasks.workunit.client.1.smithi193.stdout:2/218: write d0/fc [5367642,80642] 0 2024-03-23T08:23:17.340 INFO:tasks.workunit.client.1.smithi193.stdout:8/116: creat df/d13/d2a/f2b x:0 0 0 2024-03-23T08:23:17.341 INFO:tasks.workunit.client.0.smithi070.stdout:6/234: creat d0/d23/d28/d35/d3a/d3c/f53 x:0 0 0 2024-03-23T08:23:17.342 INFO:tasks.workunit.client.0.smithi070.stdout:6/235: write d0/d3/d2c/f49 [676907,86272] 0 2024-03-23T08:23:17.343 INFO:tasks.workunit.client.1.smithi193.stdout:1/143: unlink da/d13/c28 0 2024-03-23T08:23:17.346 INFO:tasks.workunit.client.0.smithi070.stdout:6/236: creat d0/d23/d4d/f54 x:0 0 0 2024-03-23T08:23:17.346 INFO:tasks.workunit.client.1.smithi193.stdout:0/164: dwrite db/d11/d14/f1f [0,4194304] 0 2024-03-23T08:23:17.348 INFO:tasks.workunit.client.0.smithi070.stdout:6/237: truncate d0/d23/d28/f37 161580 0 2024-03-23T08:23:17.349 INFO:tasks.workunit.client.0.smithi070.stdout:6/238: write d0/d23/d28/f4a [946158,7720] 0 2024-03-23T08:23:17.350 INFO:tasks.workunit.client.1.smithi193.stdout:4/142: truncate f9 3055930 0 2024-03-23T08:23:17.351 INFO:tasks.workunit.client.1.smithi193.stdout:7/135: dwrite d6/d9/d1f/f2a [0,4194304] 0 2024-03-23T08:23:17.353 INFO:tasks.workunit.client.0.smithi070.stdout:6/239: unlink d0/d3/f1a 0 2024-03-23T08:23:17.355 INFO:tasks.workunit.client.1.smithi193.stdout:8/117: dwrite df/d10/f27 [0,4194304] 0 2024-03-23T08:23:17.358 INFO:tasks.workunit.client.1.smithi193.stdout:1/144: creat da/d13/f34 x:0 0 0 2024-03-23T08:23:17.358 INFO:tasks.workunit.client.1.smithi193.stdout:8/118: fsync f0 0 2024-03-23T08:23:17.365 INFO:tasks.workunit.client.1.smithi193.stdout:9/150: rename d2/d7/fe to d2/d7/db/f34 0 2024-03-23T08:23:17.366 INFO:tasks.workunit.client.1.smithi193.stdout:3/115: dwrite d1/de/d13/d21/f23 [0,4194304] 0 2024-03-23T08:23:17.366 INFO:tasks.workunit.client.1.smithi193.stdout:7/136: creat d6/f32 x:0 0 0 2024-03-23T08:23:17.366 INFO:tasks.workunit.client.1.smithi193.stdout:9/151: fsync d2/f4 0 2024-03-23T08:23:17.366 INFO:tasks.workunit.client.1.smithi193.stdout:4/143: creat d11/d19/f2f x:0 0 0 2024-03-23T08:23:17.367 INFO:tasks.workunit.client.1.smithi193.stdout:2/219: mkdir d0/d1b/d3b 0 2024-03-23T08:23:17.367 INFO:tasks.workunit.client.1.smithi193.stdout:5/111: rmdir d0/d1f/d25 0 2024-03-23T08:23:17.370 INFO:tasks.workunit.client.1.smithi193.stdout:7/137: read d6/d9/f19 [409278,100206] 0 2024-03-23T08:23:17.370 INFO:tasks.workunit.client.1.smithi193.stdout:0/165: dread db/d11/d14/d21/f22 [4194304,4194304] 0 2024-03-23T08:23:17.373 INFO:tasks.workunit.client.1.smithi193.stdout:6/110: creat d0/d8/d12/f1b x:0 0 0 2024-03-23T08:23:17.375 INFO:tasks.workunit.client.1.smithi193.stdout:9/152: fsync d2/d7/f1d 0 2024-03-23T08:23:17.376 INFO:tasks.workunit.client.1.smithi193.stdout:8/119: creat df/f2c x:0 0 0 2024-03-23T08:23:17.376 INFO:tasks.workunit.client.1.smithi193.stdout:2/220: rmdir d0/d1b/d14 39 2024-03-23T08:23:17.377 INFO:tasks.workunit.client.1.smithi193.stdout:5/112: creat d0/d1c/f26 x:0 0 0 2024-03-23T08:23:17.379 INFO:tasks.workunit.client.1.smithi193.stdout:6/111: write d0/d8/f19 [4014949,126312] 0 2024-03-23T08:23:17.379 INFO:tasks.workunit.client.1.smithi193.stdout:1/145: dread da/dd/df/f2d [0,4194304] 0 2024-03-23T08:23:17.380 INFO:tasks.workunit.client.1.smithi193.stdout:8/120: fdatasync df/f2c 0 2024-03-23T08:23:17.381 INFO:tasks.workunit.client.1.smithi193.stdout:5/113: fsync d0/f14 0 2024-03-23T08:23:17.381 INFO:tasks.workunit.client.1.smithi193.stdout:4/144: write d11/d19/d22/f26 [1369724,29857] 0 2024-03-23T08:23:17.383 INFO:tasks.workunit.client.1.smithi193.stdout:6/112: readlink d0/ld 0 2024-03-23T08:23:17.385 INFO:tasks.workunit.client.1.smithi193.stdout:8/121: chown c9 63873054 1 2024-03-23T08:23:17.386 INFO:tasks.workunit.client.1.smithi193.stdout:1/146: unlink da/fb 0 2024-03-23T08:23:17.387 INFO:tasks.workunit.client.1.smithi193.stdout:9/153: mknod d2/d7/db/d17/d33/c35 0 2024-03-23T08:23:17.390 INFO:tasks.workunit.client.1.smithi193.stdout:3/116: dwrite d1/fd [0,4194304] 0 2024-03-23T08:23:17.390 INFO:tasks.workunit.client.1.smithi193.stdout:7/138: dwrite d6/f2d [0,4194304] 0 2024-03-23T08:23:17.394 INFO:tasks.workunit.client.1.smithi193.stdout:8/122: mknod df/d13/d2a/c2d 0 2024-03-23T08:23:17.395 INFO:tasks.workunit.client.1.smithi193.stdout:1/147: mknod da/d13/d24/c35 0 2024-03-23T08:23:17.397 INFO:tasks.workunit.client.1.smithi193.stdout:1/148: chown f5 0 1 2024-03-23T08:23:17.398 INFO:tasks.workunit.client.1.smithi193.stdout:7/139: unlink l1 0 2024-03-23T08:23:17.399 INFO:tasks.workunit.client.1.smithi193.stdout:0/166: truncate db/d11/d14/d21/f22 6215410 0 2024-03-23T08:23:17.399 INFO:tasks.workunit.client.1.smithi193.stdout:7/140: readlink l0 0 2024-03-23T08:23:17.400 INFO:tasks.workunit.client.1.smithi193.stdout:1/149: fdatasync da/dd/df/f22 0 2024-03-23T08:23:17.402 INFO:tasks.workunit.client.1.smithi193.stdout:6/113: dread d0/f5 [0,4194304] 0 2024-03-23T08:23:17.402 INFO:tasks.workunit.client.1.smithi193.stdout:8/123: mknod df/d13/d2a/c2e 0 2024-03-23T08:23:17.402 INFO:tasks.workunit.client.1.smithi193.stdout:7/141: fsync d6/d9/fa 0 2024-03-23T08:23:17.404 INFO:tasks.workunit.client.1.smithi193.stdout:4/145: write fd [283403,95854] 0 2024-03-23T08:23:17.404 INFO:tasks.workunit.client.1.smithi193.stdout:8/124: dread - df/f2c zero size 2024-03-23T08:23:17.407 INFO:tasks.workunit.client.1.smithi193.stdout:3/117: link d1/de/d13/d21/f23 d1/de/d13/f24 0 2024-03-23T08:23:17.407 INFO:tasks.workunit.client.1.smithi193.stdout:9/154: link d2/f3 d2/d7/f36 0 2024-03-23T08:23:17.408 INFO:tasks.workunit.client.1.smithi193.stdout:2/221: dwrite d0/d1b/d14/d25/f3a [0,4194304] 0 2024-03-23T08:23:17.420 INFO:tasks.workunit.client.1.smithi193.stdout:1/150: dwrite da/f12 [0,4194304] 0 2024-03-23T08:23:17.423 INFO:tasks.workunit.client.1.smithi193.stdout:1/151: chown da/f12 49243948 1 2024-03-23T08:23:17.426 INFO:tasks.workunit.client.1.smithi193.stdout:1/152: chown c3 119472343 1 2024-03-23T08:23:17.426 INFO:tasks.workunit.client.1.smithi193.stdout:4/146: dwrite d11/d19/f20 [0,4194304] 0 2024-03-23T08:23:17.429 INFO:tasks.workunit.client.1.smithi193.stdout:9/155: dwrite d2/d7/db/d17/d1f/f22 [0,4194304] 0 2024-03-23T08:23:17.429 INFO:tasks.workunit.client.1.smithi193.stdout:4/147: write d11/d1f/f28 [1044785,116527] 0 2024-03-23T08:23:17.446 INFO:tasks.workunit.client.1.smithi193.stdout:4/148: dread fd [0,4194304] 0 2024-03-23T08:23:17.447 INFO:tasks.workunit.client.1.smithi193.stdout:5/114: getdents d0/d1c 0 2024-03-23T08:23:17.447 INFO:tasks.workunit.client.1.smithi193.stdout:0/167: mkdir db/d11/d14/d3b 0 2024-03-23T08:23:17.450 INFO:tasks.workunit.client.1.smithi193.stdout:7/142: rmdir d6 39 2024-03-23T08:23:17.450 INFO:tasks.workunit.client.1.smithi193.stdout:7/143: readlink l4 0 2024-03-23T08:23:17.451 INFO:tasks.workunit.client.1.smithi193.stdout:4/149: read d11/d1a/f1c [4523099,7100] 0 2024-03-23T08:23:17.452 INFO:tasks.workunit.client.1.smithi193.stdout:6/114: mknod d0/db/df/d11/c1c 0 2024-03-23T08:23:17.457 INFO:tasks.workunit.client.1.smithi193.stdout:3/118: chown d1/l7 273282082 1 2024-03-23T08:23:17.457 INFO:tasks.workunit.client.1.smithi193.stdout:8/125: dread df/d10/d19/f1b [0,4194304] 0 2024-03-23T08:23:17.457 INFO:tasks.workunit.client.1.smithi193.stdout:2/222: symlink d0/d1b/d19/l3c 0 2024-03-23T08:23:17.459 INFO:tasks.workunit.client.1.smithi193.stdout:1/153: mknod da/d13/d24/c36 0 2024-03-23T08:23:17.460 INFO:tasks.workunit.client.1.smithi193.stdout:9/156: unlink d2/d7/f1a 0 2024-03-23T08:23:17.461 INFO:tasks.workunit.client.1.smithi193.stdout:5/115: creat d0/f27 x:0 0 0 2024-03-23T08:23:17.465 INFO:tasks.workunit.client.1.smithi193.stdout:5/116: dread - d0/d1c/f26 zero size 2024-03-23T08:23:17.466 INFO:tasks.workunit.client.1.smithi193.stdout:4/150: write d11/f12 [4693897,95976] 0 2024-03-23T08:23:17.466 INFO:tasks.workunit.client.1.smithi193.stdout:3/119: mkdir d1/de/d13/d21/d25 0 2024-03-23T08:23:17.469 INFO:tasks.workunit.client.1.smithi193.stdout:5/117: chown d0/c19 36 1 2024-03-23T08:23:17.471 INFO:tasks.workunit.client.1.smithi193.stdout:2/223: dread d0/d1b/d14/d25/f29 [0,4194304] 0 2024-03-23T08:23:17.473 INFO:tasks.workunit.client.1.smithi193.stdout:6/115: dwrite d0/db/df/f14 [0,4194304] 0 2024-03-23T08:23:17.476 INFO:tasks.workunit.client.1.smithi193.stdout:0/168: symlink db/d11/d14/d21/d33/l3c 0 2024-03-23T08:23:17.478 INFO:tasks.workunit.client.1.smithi193.stdout:7/144: read d6/d9/fe [243014,113495] 0 2024-03-23T08:23:17.479 INFO:tasks.workunit.client.1.smithi193.stdout:9/157: dwrite d2/d7/db/d1b/f30 [0,4194304] 0 2024-03-23T08:23:17.480 INFO:tasks.workunit.client.1.smithi193.stdout:8/126: unlink c1 0 2024-03-23T08:23:17.480 INFO:tasks.workunit.client.1.smithi193.stdout:6/116: rmdir d0/d8/d12 39 2024-03-23T08:23:17.489 INFO:tasks.workunit.client.1.smithi193.stdout:7/145: unlink d6/c26 0 2024-03-23T08:23:17.491 INFO:tasks.workunit.client.1.smithi193.stdout:2/224: dwrite d0/d1b/d14/d25/f29 [0,4194304] 0 2024-03-23T08:23:17.492 INFO:tasks.workunit.client.1.smithi193.stdout:8/127: dwrite f3 [0,4194304] 0 2024-03-23T08:23:17.507 INFO:tasks.workunit.client.1.smithi193.stdout:5/118: rmdir d0 39 2024-03-23T08:23:17.509 INFO:tasks.workunit.client.1.smithi193.stdout:7/146: getdents d6/d9/d1f 0 2024-03-23T08:23:17.513 INFO:tasks.workunit.client.1.smithi193.stdout:7/147: unlink d6/d9/c29 0 2024-03-23T08:23:17.517 INFO:tasks.workunit.client.1.smithi193.stdout:9/158: dread d2/fc [0,4194304] 0 2024-03-23T08:23:17.518 INFO:tasks.workunit.client.1.smithi193.stdout:3/120: write d1/de/d13/f24 [4028049,3373] 0 2024-03-23T08:23:17.518 INFO:tasks.workunit.client.1.smithi193.stdout:8/128: dread fc [0,4194304] 0 2024-03-23T08:23:17.520 INFO:tasks.workunit.client.1.smithi193.stdout:8/129: readlink df/l15 0 2024-03-23T08:23:17.522 INFO:tasks.workunit.client.1.smithi193.stdout:7/148: dread d6/f8 [0,4194304] 0 2024-03-23T08:23:17.523 INFO:tasks.workunit.client.1.smithi193.stdout:9/159: write d2/d7/db/d17/d1f/f22 [2481680,94821] 0 2024-03-23T08:23:17.525 INFO:tasks.workunit.client.1.smithi193.stdout:7/149: write d6/f14 [433426,32105] 0 2024-03-23T08:23:17.526 INFO:tasks.workunit.client.1.smithi193.stdout:6/117: truncate d0/d8/f19 327507 0 2024-03-23T08:23:17.527 INFO:tasks.workunit.client.1.smithi193.stdout:7/150: chown d6 128 1 2024-03-23T08:23:17.527 INFO:tasks.workunit.client.1.smithi193.stdout:2/225: write d0/fd [1382887,130984] 0 2024-03-23T08:23:17.528 INFO:tasks.workunit.client.1.smithi193.stdout:3/121: dread d1/de/d13/f24 [0,4194304] 0 2024-03-23T08:23:17.529 INFO:tasks.workunit.client.1.smithi193.stdout:7/151: chown d6/d9/f16 166764 1 2024-03-23T08:23:17.531 INFO:tasks.workunit.client.1.smithi193.stdout:9/160: dread d2/d7/db/f34 [0,4194304] 0 2024-03-23T08:23:17.531 INFO:tasks.workunit.client.1.smithi193.stdout:2/226: symlink d0/d1b/d14/d25/l3d 0 2024-03-23T08:23:17.532 INFO:tasks.workunit.client.1.smithi193.stdout:2/227: write d0/d1b/f2f [874433,40272] 0 2024-03-23T08:23:17.532 INFO:tasks.workunit.client.1.smithi193.stdout:7/152: mkdir d6/d9/d1f/d23/d33 0 2024-03-23T08:23:17.534 INFO:tasks.workunit.client.1.smithi193.stdout:5/119: write d0/fe [3226305,50750] 0 2024-03-23T08:23:17.535 INFO:tasks.workunit.client.1.smithi193.stdout:3/122: symlink d1/l26 0 2024-03-23T08:23:17.535 INFO:tasks.workunit.client.1.smithi193.stdout:7/153: write d6/d9/d1f/f2a [1630976,60028] 0 2024-03-23T08:23:17.536 INFO:tasks.workunit.client.1.smithi193.stdout:3/123: readlink d1/l26 0 2024-03-23T08:23:17.538 INFO:tasks.workunit.client.1.smithi193.stdout:2/228: truncate d0/d1b/d19/f1a 2994711 0 2024-03-23T08:23:17.539 INFO:tasks.workunit.client.1.smithi193.stdout:6/118: dread d0/d8/f19 [0,4194304] 0 2024-03-23T08:23:17.545 INFO:tasks.workunit.client.1.smithi193.stdout:5/120: creat d0/d1f/f28 x:0 0 0 2024-03-23T08:23:17.548 INFO:tasks.workunit.client.1.smithi193.stdout:0/169: dwrite db/d11/d14/d21/f22 [0,4194304] 0 2024-03-23T08:23:17.548 INFO:tasks.workunit.client.1.smithi193.stdout:3/124: rename d1/d19 to d1/de/d13/d21/d27 0 2024-03-23T08:23:17.551 INFO:tasks.workunit.client.1.smithi193.stdout:8/130: creat df/d10/f2f x:0 0 0 2024-03-23T08:23:17.551 INFO:tasks.workunit.client.1.smithi193.stdout:9/161: dread d2/f3 [0,4194304] 0 2024-03-23T08:23:17.553 INFO:tasks.workunit.client.1.smithi193.stdout:5/121: creat d0/d1f/f29 x:0 0 0 2024-03-23T08:23:17.553 INFO:tasks.workunit.client.1.smithi193.stdout:7/154: dwrite d6/fc [12582912,4194304] 0 2024-03-23T08:23:17.554 INFO:tasks.workunit.client.1.smithi193.stdout:6/119: creat d0/d8/f1d x:0 0 0 2024-03-23T08:23:17.555 INFO:tasks.workunit.client.1.smithi193.stdout:0/170: rmdir db/d11/d14 39 2024-03-23T08:23:17.555 INFO:tasks.workunit.client.1.smithi193.stdout:5/122: stat d0/d1c 0 2024-03-23T08:23:17.558 INFO:tasks.workunit.client.1.smithi193.stdout:9/162: dread - d2/d7/f2b zero size 2024-03-23T08:23:17.559 INFO:tasks.workunit.client.1.smithi193.stdout:6/120: chown d0/d8/d12 1506 1 2024-03-23T08:23:17.564 INFO:tasks.workunit.client.1.smithi193.stdout:6/121: creat d0/d8/d12/f1e x:0 0 0 2024-03-23T08:23:17.565 INFO:tasks.workunit.client.1.smithi193.stdout:3/125: dread d1/fa [0,4194304] 0 2024-03-23T08:23:17.565 INFO:tasks.workunit.client.1.smithi193.stdout:0/171: symlink db/d11/d14/d21/l3d 0 2024-03-23T08:23:17.574 INFO:tasks.workunit.client.1.smithi193.stdout:3/126: mkdir d1/de/d28 0 2024-03-23T08:23:17.575 INFO:tasks.workunit.client.1.smithi193.stdout:7/155: dwrite d6/d9/d1f/d23/f2e [0,4194304] 0 2024-03-23T08:23:17.576 INFO:tasks.workunit.client.1.smithi193.stdout:6/122: write d0/d8/f1d [434262,27550] 0 2024-03-23T08:23:17.578 INFO:tasks.workunit.client.1.smithi193.stdout:5/123: dwrite d0/fa [0,4194304] 0 2024-03-23T08:23:17.578 INFO:tasks.workunit.client.1.smithi193.stdout:8/131: dwrite fc [0,4194304] 0 2024-03-23T08:23:17.581 INFO:tasks.workunit.client.1.smithi193.stdout:2/229: write d0/d1b/d19/f1a [2008262,19833] 0 2024-03-23T08:23:17.581 INFO:tasks.workunit.client.1.smithi193.stdout:3/127: mknod d1/c29 0 2024-03-23T08:23:17.582 INFO:tasks.workunit.client.1.smithi193.stdout:5/124: creat d0/f2a x:0 0 0 2024-03-23T08:23:17.584 INFO:tasks.workunit.client.1.smithi193.stdout:3/128: dread - d1/f20 zero size 2024-03-23T08:23:17.584 INFO:tasks.workunit.client.1.smithi193.stdout:6/123: mknod d0/db/c1f 0 2024-03-23T08:23:17.584 INFO:tasks.workunit.client.1.smithi193.stdout:9/163: dread d2/d7/db/d17/d1f/f22 [0,4194304] 0 2024-03-23T08:23:17.589 INFO:tasks.workunit.client.1.smithi193.stdout:0/172: dwrite db/d11/d14/d24/f39 [0,4194304] 0 2024-03-23T08:23:17.589 INFO:tasks.workunit.client.1.smithi193.stdout:5/125: creat d0/d1f/f2b x:0 0 0 2024-03-23T08:23:17.590 INFO:tasks.workunit.client.1.smithi193.stdout:9/164: creat d2/d7/f37 x:0 0 0 2024-03-23T08:23:17.592 INFO:tasks.workunit.client.1.smithi193.stdout:8/132: dread df/f18 [0,4194304] 0 2024-03-23T08:23:17.592 INFO:tasks.workunit.client.1.smithi193.stdout:5/126: chown d0/c19 136 1 2024-03-23T08:23:17.598 INFO:tasks.workunit.client.1.smithi193.stdout:6/124: creat d0/db/f20 x:0 0 0 2024-03-23T08:23:17.598 INFO:tasks.workunit.client.1.smithi193.stdout:5/127: unlink d0/c19 0 2024-03-23T08:23:17.598 INFO:tasks.workunit.client.1.smithi193.stdout:2/230: link d0/c24 d0/d1b/c3e 0 2024-03-23T08:23:17.599 INFO:tasks.workunit.client.1.smithi193.stdout:5/128: chown d0/d1c 35757263 1 2024-03-23T08:23:17.601 INFO:tasks.workunit.client.1.smithi193.stdout:8/133: read - df/d10/f29 zero size 2024-03-23T08:23:17.601 INFO:tasks.workunit.client.1.smithi193.stdout:6/125: mkdir d0/d8/d12/d21 0 2024-03-23T08:23:17.604 INFO:tasks.workunit.client.1.smithi193.stdout:5/129: chown d0/f1d 1275 1 2024-03-23T08:23:17.604 INFO:tasks.workunit.client.1.smithi193.stdout:0/173: creat db/f3e x:0 0 0 2024-03-23T08:23:17.605 INFO:tasks.workunit.client.1.smithi193.stdout:5/130: chown d0/d1c 250 1 2024-03-23T08:23:17.605 INFO:tasks.workunit.client.1.smithi193.stdout:2/231: dread d0/f1d [0,4194304] 0 2024-03-23T08:23:17.606 INFO:tasks.workunit.client.1.smithi193.stdout:6/126: dread d0/d8/f19 [0,4194304] 0 2024-03-23T08:23:17.608 INFO:tasks.workunit.client.1.smithi193.stdout:8/134: rename df to df/d10/d19/d30 22 2024-03-23T08:23:17.610 INFO:tasks.workunit.client.1.smithi193.stdout:0/174: symlink db/d11/d14/d21/d33/l3f 0 2024-03-23T08:23:17.611 INFO:tasks.workunit.client.1.smithi193.stdout:9/165: truncate d2/f3 4135831 0 2024-03-23T08:23:17.612 INFO:tasks.workunit.client.1.smithi193.stdout:9/166: readlink d2/l6 0 2024-03-23T08:23:17.612 INFO:tasks.workunit.client.1.smithi193.stdout:8/135: mknod df/d13/c31 0 2024-03-23T08:23:17.614 INFO:tasks.workunit.client.1.smithi193.stdout:2/232: link d0/d1b/l38 d0/d28/l3f 0 2024-03-23T08:23:17.614 INFO:tasks.workunit.client.1.smithi193.stdout:9/167: dread - d2/d7/db/f2e zero size 2024-03-23T08:23:17.616 INFO:tasks.workunit.client.1.smithi193.stdout:6/127: rename d0/d8/l17 to d0/d8/l22 0 2024-03-23T08:23:17.617 INFO:tasks.workunit.client.1.smithi193.stdout:0/175: creat db/d11/f40 x:0 0 0 2024-03-23T08:23:17.619 INFO:tasks.workunit.client.1.smithi193.stdout:8/136: truncate df/f1f 5219640 0 2024-03-23T08:23:17.620 INFO:tasks.workunit.client.1.smithi193.stdout:5/131: dwrite d0/f11 [0,4194304] 0 2024-03-23T08:23:17.626 INFO:tasks.workunit.client.1.smithi193.stdout:3/129: dread d1/de/f10 [0,4194304] 0 2024-03-23T08:23:17.633 INFO:tasks.workunit.client.1.smithi193.stdout:6/128: write d0/f5 [4317688,107016] 0 2024-03-23T08:23:17.636 INFO:tasks.workunit.client.1.smithi193.stdout:3/130: symlink d1/d22/l2a 0 2024-03-23T08:23:17.636 INFO:tasks.workunit.client.1.smithi193.stdout:2/233: dwrite d0/d1b/f8 [0,4194304] 0 2024-03-23T08:23:17.639 INFO:tasks.workunit.client.1.smithi193.stdout:9/168: dwrite d2/f28 [0,4194304] 0 2024-03-23T08:23:17.640 INFO:tasks.workunit.client.1.smithi193.stdout:0/176: truncate db/d11/f13 432750 0 2024-03-23T08:23:17.641 INFO:tasks.workunit.client.1.smithi193.stdout:8/137: getdents df/d13/d2a 0 2024-03-23T08:23:17.642 INFO:tasks.workunit.client.1.smithi193.stdout:6/129: symlink d0/l23 0 2024-03-23T08:23:17.642 INFO:tasks.workunit.client.1.smithi193.stdout:0/177: chown db/d11/c2b 1693 1 2024-03-23T08:23:17.642 INFO:tasks.workunit.client.1.smithi193.stdout:8/138: chown df/d13/f25 1390 1 2024-03-23T08:23:17.644 INFO:tasks.workunit.client.1.smithi193.stdout:9/169: symlink d2/d7/d31/l38 0 2024-03-23T08:23:17.651 INFO:tasks.workunit.client.1.smithi193.stdout:2/234: link d0/fc d0/d1b/d14/f40 0 2024-03-23T08:23:17.652 INFO:tasks.workunit.client.1.smithi193.stdout:6/130: mkdir d0/d24 0 2024-03-23T08:23:17.653 INFO:tasks.workunit.client.1.smithi193.stdout:0/178: chown db/d11/d14/d24/d2a/c2d 69 1 2024-03-23T08:23:17.654 INFO:tasks.workunit.client.1.smithi193.stdout:8/139: dwrite df/d13/d2a/f2b [0,4194304] 0 2024-03-23T08:23:17.655 INFO:tasks.workunit.client.1.smithi193.stdout:2/235: stat d0/d1b/l38 0 2024-03-23T08:23:17.656 INFO:tasks.workunit.client.1.smithi193.stdout:5/132: dread d0/d1c/f24 [0,4194304] 0 2024-03-23T08:23:17.656 INFO:tasks.workunit.client.1.smithi193.stdout:6/131: creat d0/db/df/f25 x:0 0 0 2024-03-23T08:23:17.657 INFO:tasks.workunit.client.1.smithi193.stdout:8/140: dread - df/d13/f25 zero size 2024-03-23T08:23:17.658 INFO:tasks.workunit.client.1.smithi193.stdout:8/141: write df/d10/d19/f26 [250379,91333] 0 2024-03-23T08:23:17.658 INFO:tasks.workunit.client.1.smithi193.stdout:0/179: creat db/d11/d14/d21/f41 x:0 0 0 2024-03-23T08:23:17.660 INFO:tasks.workunit.client.1.smithi193.stdout:2/236: creat d0/d1b/d14/f41 x:0 0 0 2024-03-23T08:23:17.661 INFO:tasks.workunit.client.1.smithi193.stdout:5/133: readlink d0/l1b 0 2024-03-23T08:23:17.664 INFO:tasks.workunit.client.1.smithi193.stdout:5/134: dread - d0/d1c/f26 zero size 2024-03-23T08:23:17.668 INFO:tasks.workunit.client.1.smithi193.stdout:8/142: mknod df/d13/c32 0 2024-03-23T08:23:17.668 INFO:tasks.workunit.client.1.smithi193.stdout:6/132: chown d0/fe 6 1 2024-03-23T08:23:17.669 INFO:tasks.workunit.client.1.smithi193.stdout:0/180: dwrite db/f3e [0,4194304] 0 2024-03-23T08:23:17.670 INFO:tasks.workunit.client.1.smithi193.stdout:2/237: creat d0/d1b/d3b/f42 x:0 0 0 2024-03-23T08:23:17.671 INFO:tasks.workunit.client.1.smithi193.stdout:5/135: rename d0/l1b to d0/d1c/l2c 0 2024-03-23T08:23:17.672 INFO:tasks.workunit.client.1.smithi193.stdout:2/238: write d0/f7 [9167969,95412] 0 2024-03-23T08:23:17.672 INFO:tasks.workunit.client.1.smithi193.stdout:5/136: dread - d0/f15 zero size 2024-03-23T08:23:17.674 INFO:tasks.workunit.client.1.smithi193.stdout:9/170: truncate d2/f3 2400411 0 2024-03-23T08:23:17.674 INFO:tasks.workunit.client.1.smithi193.stdout:8/143: rmdir df/d10/d19 39 2024-03-23T08:23:17.675 INFO:tasks.workunit.client.1.smithi193.stdout:6/133: mknod d0/db/c26 0 2024-03-23T08:23:17.677 INFO:tasks.workunit.client.1.smithi193.stdout:2/239: chown d0/l4 63 1 2024-03-23T08:23:17.677 INFO:tasks.workunit.client.1.smithi193.stdout:8/144: write df/d13/d2a/f2b [1912704,77020] 0 2024-03-23T08:23:17.679 INFO:tasks.workunit.client.1.smithi193.stdout:5/137: symlink d0/d1f/l2d 0 2024-03-23T08:23:17.679 INFO:tasks.workunit.client.1.smithi193.stdout:2/240: read d0/fd [975091,80913] 0 2024-03-23T08:23:17.679 INFO:tasks.workunit.client.1.smithi193.stdout:5/138: dread - d0/f2a zero size 2024-03-23T08:23:17.680 INFO:tasks.workunit.client.1.smithi193.stdout:5/139: chown d0/f27 65071 1 2024-03-23T08:23:17.682 INFO:tasks.workunit.client.1.smithi193.stdout:6/134: mkdir d0/d27 0 2024-03-23T08:23:17.683 INFO:tasks.workunit.client.1.smithi193.stdout:8/145: creat df/f33 x:0 0 0 2024-03-23T08:23:17.684 INFO:tasks.workunit.client.1.smithi193.stdout:2/241: creat d0/d28/f43 x:0 0 0 2024-03-23T08:23:17.685 INFO:tasks.workunit.client.1.smithi193.stdout:6/135: symlink d0/db/df/l28 0 2024-03-23T08:23:17.687 INFO:tasks.workunit.client.1.smithi193.stdout:6/136: stat d0/f5 0 2024-03-23T08:23:17.688 INFO:tasks.workunit.client.1.smithi193.stdout:9/171: fsync d2/d7/f36 0 2024-03-23T08:23:17.690 INFO:tasks.workunit.client.1.smithi193.stdout:5/140: dwrite d0/d1c/f20 [0,4194304] 0 2024-03-23T08:23:17.692 INFO:tasks.workunit.client.1.smithi193.stdout:0/181: write db/d11/f13 [602251,113388] 0 2024-03-23T08:23:17.693 INFO:tasks.workunit.client.1.smithi193.stdout:5/141: unlink d0/c16 0 2024-03-23T08:23:17.694 INFO:tasks.workunit.client.1.smithi193.stdout:0/182: write db/d11/d14/d21/f2c [834341,100965] 0 2024-03-23T08:23:17.694 INFO:tasks.workunit.client.1.smithi193.stdout:9/172: rename d2/d7/db/d17/d1f/c2d to d2/d7/db/d17/d33/c39 0 2024-03-23T08:23:17.695 INFO:tasks.workunit.client.1.smithi193.stdout:5/142: write d0/d1c/f26 [930788,87379] 0 2024-03-23T08:23:17.696 INFO:tasks.workunit.client.1.smithi193.stdout:5/143: chown d0/d1f/l2d 24722505 1 2024-03-23T08:23:17.697 INFO:tasks.workunit.client.1.smithi193.stdout:9/173: mknod d2/d7/d31/c3a 0 2024-03-23T08:23:17.697 INFO:tasks.workunit.client.1.smithi193.stdout:5/144: write d0/ff [2427891,113325] 0 2024-03-23T08:23:17.698 INFO:tasks.workunit.client.1.smithi193.stdout:9/174: fdatasync d2/d7/f37 0 2024-03-23T08:23:17.698 INFO:tasks.workunit.client.1.smithi193.stdout:0/183: rename c9 to db/d11/d14/d21/d33/c42 0 2024-03-23T08:23:17.701 INFO:tasks.workunit.client.1.smithi193.stdout:9/175: creat d2/d7/db/f3b x:0 0 0 2024-03-23T08:23:17.702 INFO:tasks.workunit.client.1.smithi193.stdout:0/184: creat db/d11/d14/d21/d33/f43 x:0 0 0 2024-03-23T08:23:17.704 INFO:tasks.workunit.client.1.smithi193.stdout:0/185: write db/d11/d14/d21/f2c [239691,55783] 0 2024-03-23T08:23:17.707 INFO:tasks.workunit.client.1.smithi193.stdout:5/145: dwrite d0/d1c/f1e [0,4194304] 0 2024-03-23T08:23:17.707 INFO:tasks.workunit.client.1.smithi193.stdout:0/186: write db/d11/d14/d24/f39 [2765235,72567] 0 2024-03-23T08:23:17.709 INFO:tasks.workunit.client.1.smithi193.stdout:5/146: creat d0/d1f/f2e x:0 0 0 2024-03-23T08:23:17.711 INFO:tasks.workunit.client.1.smithi193.stdout:0/187: mknod db/d11/d14/d24/d2a/c44 0 2024-03-23T08:23:17.711 INFO:tasks.workunit.client.1.smithi193.stdout:9/176: rmdir d2/d7 39 2024-03-23T08:23:17.712 INFO:tasks.workunit.client.1.smithi193.stdout:5/147: mkdir d0/d2f 0 2024-03-23T08:23:17.714 INFO:tasks.workunit.client.1.smithi193.stdout:9/177: dread - d2/d7/f15 zero size 2024-03-23T08:23:17.717 INFO:tasks.workunit.client.1.smithi193.stdout:0/188: link db/d11/d14/d21/d33/f43 db/d11/d14/d21/f45 0 2024-03-23T08:23:17.718 INFO:tasks.workunit.client.1.smithi193.stdout:9/178: mkdir d2/d7/d31/d3c 0 2024-03-23T08:23:17.718 INFO:tasks.workunit.client.1.smithi193.stdout:5/148: dwrite d0/d1c/f21 [0,4194304] 0 2024-03-23T08:23:17.719 INFO:tasks.workunit.client.1.smithi193.stdout:0/189: mknod db/d11/d14/d24/d2a/c46 0 2024-03-23T08:23:17.720 INFO:tasks.workunit.client.1.smithi193.stdout:5/149: dread - d0/f2a zero size 2024-03-23T08:23:17.759 INFO:tasks.workunit.client.1.smithi193.stdout:1/154: sync 2024-03-23T08:23:17.759 INFO:tasks.workunit.client.1.smithi193.stdout:5/150: rename d0 to d0/d30 22 2024-03-23T08:23:17.763 INFO:tasks.workunit.client.1.smithi193.stdout:0/190: mkdir db/d11/d14/d24/d47 0 2024-03-23T08:23:17.764 INFO:tasks.workunit.client.1.smithi193.stdout:9/179: getdents d2 0 2024-03-23T08:23:17.766 INFO:tasks.workunit.client.1.smithi193.stdout:5/151: symlink d0/d1c/l31 0 2024-03-23T08:23:17.766 INFO:tasks.workunit.client.1.smithi193.stdout:1/155: dwrite da/f12 [4194304,4194304] 0 2024-03-23T08:23:17.768 INFO:tasks.workunit.client.1.smithi193.stdout:0/191: mkdir db/d11/d14/d21/d33/d48 0 2024-03-23T08:23:17.770 INFO:tasks.workunit.client.1.smithi193.stdout:9/180: truncate d2/f3 558039 0 2024-03-23T08:23:17.775 INFO:tasks.workunit.client.1.smithi193.stdout:5/152: mkdir d0/d32 0 2024-03-23T08:23:17.775 INFO:tasks.workunit.client.1.smithi193.stdout:0/192: creat db/d11/d14/d21/f49 x:0 0 0 2024-03-23T08:23:17.779 INFO:tasks.workunit.client.1.smithi193.stdout:9/181: rename d2/d7/c9 to d2/c3d 0 2024-03-23T08:23:17.781 INFO:tasks.workunit.client.1.smithi193.stdout:0/193: unlink db/d11/d14/d24/d2a/c2d 0 2024-03-23T08:23:17.784 INFO:tasks.workunit.client.1.smithi193.stdout:5/153: rename d0/d1f/l2d to d0/d32/l33 0 2024-03-23T08:23:17.784 INFO:tasks.workunit.client.1.smithi193.stdout:1/156: getdents da/dd 0 2024-03-23T08:23:17.785 INFO:tasks.workunit.client.1.smithi193.stdout:5/154: read d0/f14 [93774,108812] 0 2024-03-23T08:23:17.785 INFO:tasks.workunit.client.1.smithi193.stdout:1/157: read - da/dd/df/f25 zero size 2024-03-23T08:23:17.786 INFO:tasks.workunit.client.1.smithi193.stdout:5/155: dread - d0/f15 zero size 2024-03-23T08:23:17.786 INFO:tasks.workunit.client.1.smithi193.stdout:5/156: truncate d0/fa 5056313 0 2024-03-23T08:23:17.788 INFO:tasks.workunit.client.1.smithi193.stdout:0/194: rename db/d11/d14/f34 to db/d11/d14/d24/f4a 0 2024-03-23T08:23:17.792 INFO:tasks.workunit.client.1.smithi193.stdout:5/157: mknod d0/d1f/c34 0 2024-03-23T08:23:17.794 INFO:tasks.workunit.client.1.smithi193.stdout:0/195: mkdir db/d11/d14/d24/d2a/d4b 0 2024-03-23T08:23:17.796 INFO:tasks.workunit.client.1.smithi193.stdout:0/196: truncate db/d11/d14/f19 284300 0 2024-03-23T08:23:17.798 INFO:tasks.workunit.client.1.smithi193.stdout:5/158: dwrite d0/d1f/f28 [0,4194304] 0 2024-03-23T08:23:17.799 INFO:tasks.workunit.client.1.smithi193.stdout:5/159: dread - d0/f15 zero size 2024-03-23T08:23:17.806 INFO:tasks.workunit.client.1.smithi193.stdout:5/160: mknod d0/d1f/c35 0 2024-03-23T08:23:17.812 INFO:tasks.workunit.client.1.smithi193.stdout:5/161: dwrite d0/f14 [0,4194304] 0 2024-03-23T08:23:17.813 INFO:tasks.workunit.client.1.smithi193.stdout:1/158: truncate f4 578116 0 2024-03-23T08:23:17.814 INFO:tasks.workunit.client.1.smithi193.stdout:1/159: stat da/dd/df/f29 0 2024-03-23T08:23:17.814 INFO:tasks.workunit.client.1.smithi193.stdout:1/160: dread - da/dd/f14 zero size 2024-03-23T08:23:17.817 INFO:tasks.workunit.client.1.smithi193.stdout:0/197: link db/d11/d14/d24/d2a/c46 db/d11/d14/d24/d47/c4c 0 2024-03-23T08:23:17.818 INFO:tasks.workunit.client.1.smithi193.stdout:0/198: fsync db/d11/d14/f1f 0 2024-03-23T08:23:17.819 INFO:tasks.workunit.client.1.smithi193.stdout:0/199: read - db/d11/f28 zero size 2024-03-23T08:23:17.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/161: dread da/fc [0,4194304] 0 2024-03-23T08:23:17.820 INFO:tasks.workunit.client.1.smithi193.stdout:0/200: chown db/d11/f40 533953599 1 2024-03-23T08:23:17.821 INFO:tasks.workunit.client.1.smithi193.stdout:1/162: write da/dd/df/f17 [933074,106789] 0 2024-03-23T08:23:17.821 INFO:tasks.workunit.client.1.smithi193.stdout:9/182: link d2/d7/db/c10 d2/d7/c3e 0 2024-03-23T08:23:17.829 INFO:tasks.workunit.client.1.smithi193.stdout:5/162: link d0/fa d0/f36 0 2024-03-23T08:23:17.831 INFO:tasks.workunit.client.1.smithi193.stdout:0/201: mkdir db/d11/d14/d24/d2a/d4d 0 2024-03-23T08:23:17.831 INFO:tasks.workunit.client.1.smithi193.stdout:5/163: dread d0/fa [4194304,4194304] 0 2024-03-23T08:23:17.833 INFO:tasks.workunit.client.1.smithi193.stdout:5/164: write d0/f15 [595474,112399] 0 2024-03-23T08:23:17.839 INFO:tasks.workunit.client.1.smithi193.stdout:5/165: dwrite d0/f27 [0,4194304] 0 2024-03-23T08:23:17.839 INFO:tasks.workunit.client.1.smithi193.stdout:0/202: mknod db/d11/d14/d24/d2a/c4e 0 2024-03-23T08:23:17.840 INFO:tasks.workunit.client.1.smithi193.stdout:5/166: chown d0/f27 33105 1 2024-03-23T08:23:17.841 INFO:tasks.workunit.client.1.smithi193.stdout:1/163: symlink da/l37 0 2024-03-23T08:23:17.841 INFO:tasks.workunit.client.1.smithi193.stdout:5/167: truncate d0/d1c/f21 5067271 0 2024-03-23T08:23:17.844 INFO:tasks.workunit.client.1.smithi193.stdout:9/183: link d2/d7/f2b d2/d7/db/d32/f3f 0 2024-03-23T08:23:17.845 INFO:tasks.workunit.client.1.smithi193.stdout:9/184: stat d2/d7/c2a 0 2024-03-23T08:23:17.847 INFO:tasks.workunit.client.1.smithi193.stdout:0/203: dread db/d11/f12 [0,4194304] 0 2024-03-23T08:23:17.850 INFO:tasks.workunit.client.1.smithi193.stdout:1/164: dwrite da/d13/f34 [0,4194304] 0 2024-03-23T08:23:17.860 INFO:tasks.workunit.client.1.smithi193.stdout:1/165: symlink da/l38 0 2024-03-23T08:23:17.861 INFO:tasks.workunit.client.1.smithi193.stdout:1/166: chown da/dd/df/l20 11253993 1 2024-03-23T08:23:17.864 INFO:tasks.workunit.client.1.smithi193.stdout:1/167: creat da/dd/f39 x:0 0 0 2024-03-23T08:23:17.865 INFO:tasks.workunit.client.1.smithi193.stdout:1/168: write da/dd/df/f25 [239057,109870] 0 2024-03-23T08:23:17.868 INFO:tasks.workunit.client.1.smithi193.stdout:1/169: symlink da/dd/df/d2e/l3a 0 2024-03-23T08:23:17.871 INFO:tasks.workunit.client.1.smithi193.stdout:9/185: dwrite d2/d7/db/f34 [0,4194304] 0 2024-03-23T08:23:17.876 INFO:tasks.workunit.client.1.smithi193.stdout:1/170: creat da/dd/df/f3b x:0 0 0 2024-03-23T08:23:17.880 INFO:tasks.workunit.client.1.smithi193.stdout:9/186: dwrite d2/d7/db/f34 [0,4194304] 0 2024-03-23T08:23:17.886 INFO:tasks.workunit.client.1.smithi193.stdout:9/187: rmdir d2/d7/db/d17/d1f 39 2024-03-23T08:23:17.886 INFO:tasks.workunit.client.1.smithi193.stdout:1/171: dwrite da/dd/df/f22 [0,4194304] 0 2024-03-23T08:23:17.895 INFO:tasks.workunit.client.1.smithi193.stdout:0/204: dwrite db/d11/d14/d21/d33/f43 [0,4194304] 0 2024-03-23T08:23:17.899 INFO:tasks.workunit.client.1.smithi193.stdout:1/172: dwrite da/dd/df/f29 [0,4194304] 0 2024-03-23T08:23:17.901 INFO:tasks.workunit.client.1.smithi193.stdout:9/188: dread d2/d7/db/f26 [0,4194304] 0 2024-03-23T08:23:17.901 INFO:tasks.workunit.client.1.smithi193.stdout:4/151: sync 2024-03-23T08:23:17.903 INFO:tasks.workunit.client.1.smithi193.stdout:9/189: write d2/d7/f15 [779772,121567] 0 2024-03-23T08:23:17.905 INFO:tasks.workunit.client.1.smithi193.stdout:9/190: chown d2/d7/db 10176 1 2024-03-23T08:23:17.908 INFO:tasks.workunit.client.1.smithi193.stdout:1/173: dwrite da/dd/df/f25 [0,4194304] 0 2024-03-23T08:23:17.909 INFO:tasks.workunit.client.1.smithi193.stdout:1/174: chown da/dd/f30 153 1 2024-03-23T08:23:17.911 INFO:tasks.workunit.client.1.smithi193.stdout:4/152: mknod d11/d27/c30 0 2024-03-23T08:23:17.912 INFO:tasks.workunit.client.1.smithi193.stdout:9/191: dread d2/d7/db/f13 [0,4194304] 0 2024-03-23T08:23:17.915 INFO:tasks.workunit.client.1.smithi193.stdout:9/192: truncate d2/d7/db/f26 1166748 0 2024-03-23T08:23:17.916 INFO:tasks.workunit.client.1.smithi193.stdout:1/175: rmdir da/dd/df 39 2024-03-23T08:23:17.918 INFO:tasks.workunit.client.1.smithi193.stdout:4/153: creat d11/d19/d22/f31 x:0 0 0 2024-03-23T08:23:17.919 INFO:tasks.workunit.client.1.smithi193.stdout:0/205: fdatasync db/d11/d14/d21/d33/f43 0 2024-03-23T08:23:17.921 INFO:tasks.workunit.client.1.smithi193.stdout:9/193: creat d2/d7/d31/f40 x:0 0 0 2024-03-23T08:23:17.926 INFO:tasks.workunit.client.1.smithi193.stdout:9/194: mknod d2/d7/db/c41 0 2024-03-23T08:23:17.927 INFO:tasks.workunit.client.1.smithi193.stdout:9/195: write d2/d7/db/f3b [170444,87829] 0 2024-03-23T08:23:17.934 INFO:tasks.workunit.client.1.smithi193.stdout:9/196: dread d2/f28 [0,4194304] 0 2024-03-23T08:23:17.938 INFO:tasks.workunit.client.1.smithi193.stdout:0/206: getdents db/d11/d14/d24 0 2024-03-23T08:23:17.948 INFO:tasks.workunit.client.1.smithi193.stdout:0/207: mkdir db/d11/d14/d21/d33/d48/d4f 0 2024-03-23T08:23:17.948 INFO:tasks.workunit.client.1.smithi193.stdout:0/208: chown db/d11/d14/d24/d47 91 1 2024-03-23T08:23:17.950 INFO:tasks.workunit.client.1.smithi193.stdout:9/197: getdents d2/d7/db/d1b 0 2024-03-23T08:23:17.954 INFO:tasks.workunit.client.1.smithi193.stdout:0/209: dread db/d11/f12 [0,4194304] 0 2024-03-23T08:23:17.955 INFO:tasks.workunit.client.1.smithi193.stdout:9/198: creat d2/d7/db/d17/d1f/f42 x:0 0 0 2024-03-23T08:23:17.956 INFO:tasks.workunit.client.1.smithi193.stdout:9/199: fdatasync d2/d7/f23 0 2024-03-23T08:23:17.957 INFO:tasks.workunit.client.1.smithi193.stdout:0/210: write db/d11/d14/d21/f26 [1560528,104327] 0 2024-03-23T08:23:17.957 INFO:tasks.workunit.client.1.smithi193.stdout:0/211: stat db/d11/d14 0 2024-03-23T08:23:17.958 INFO:tasks.workunit.client.1.smithi193.stdout:0/212: write db/d11/f40 [396710,50297] 0 2024-03-23T08:23:17.959 INFO:tasks.workunit.client.1.smithi193.stdout:0/213: read - db/d11/d14/d24/f4a zero size 2024-03-23T08:23:17.959 INFO:tasks.workunit.client.1.smithi193.stdout:9/200: mkdir d2/d7/db/d1b/d43 0 2024-03-23T08:23:17.960 INFO:tasks.workunit.client.1.smithi193.stdout:9/201: stat d2/d7/f11 0 2024-03-23T08:23:17.960 INFO:tasks.workunit.client.1.smithi193.stdout:1/176: write f5 [696211,34544] 0 2024-03-23T08:23:17.966 INFO:tasks.workunit.client.1.smithi193.stdout:0/214: dread db/d11/d14/d21/f2c [0,4194304] 0 2024-03-23T08:23:17.969 INFO:tasks.workunit.client.1.smithi193.stdout:9/202: rename d2/d7/db/d17/d1f/f24 to d2/d7/db/d1b/f44 0 2024-03-23T08:23:17.973 INFO:tasks.workunit.client.1.smithi193.stdout:1/177: rmdir da/dd/df 39 2024-03-23T08:23:17.974 INFO:tasks.workunit.client.1.smithi193.stdout:1/178: write da/dd/f30 [1141672,34393] 0 2024-03-23T08:23:17.974 INFO:tasks.workunit.client.1.smithi193.stdout:1/179: stat f8 0 2024-03-23T08:23:17.975 INFO:tasks.workunit.client.1.smithi193.stdout:4/154: dwrite f5 [0,4194304] 0 2024-03-23T08:23:17.976 INFO:tasks.workunit.client.1.smithi193.stdout:0/215: chown db/c17 54003 1 2024-03-23T08:23:17.977 INFO:tasks.workunit.client.1.smithi193.stdout:0/216: write db/d11/d14/f19 [140961,30837] 0 2024-03-23T08:23:17.979 INFO:tasks.workunit.client.1.smithi193.stdout:0/217: fdatasync db/d11/f40 0 2024-03-23T08:23:17.981 INFO:tasks.workunit.client.1.smithi193.stdout:4/155: dread f5 [0,4194304] 0 2024-03-23T08:23:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:17 smithi070 ceph-mon[39699]: pgmap v18: 65 pgs: 65 active+clean; 222 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 315 op/s 2024-03-23T08:23:17.983 INFO:tasks.workunit.client.1.smithi193.stdout:4/156: read d11/f12 [3726183,107364] 0 2024-03-23T08:23:17.985 INFO:tasks.workunit.client.1.smithi193.stdout:1/180: rmdir da 39 2024-03-23T08:23:17.992 INFO:tasks.workunit.client.1.smithi193.stdout:0/218: write db/d11/d14/d21/f27 [4843800,48154] 0 2024-03-23T08:23:18.000 INFO:tasks.workunit.client.1.smithi193.stdout:9/203: link d2/f4 d2/d7/db/d17/d33/f45 0 2024-03-23T08:23:18.006 INFO:tasks.workunit.client.1.smithi193.stdout:0/219: dwrite db/d11/d14/d21/f41 [0,4194304] 0 2024-03-23T08:23:18.007 INFO:tasks.workunit.client.1.smithi193.stdout:9/204: write d2/d7/db/f26 [706282,65342] 0 2024-03-23T08:23:18.008 INFO:tasks.workunit.client.1.smithi193.stdout:0/220: write db/d11/d14/d21/f27 [4189842,122322] 0 2024-03-23T08:23:18.012 INFO:tasks.workunit.client.1.smithi193.stdout:7/156: sync 2024-03-23T08:23:18.012 INFO:tasks.workunit.client.1.smithi193.stdout:4/157: getdents d11/d27 0 2024-03-23T08:23:18.013 INFO:tasks.workunit.client.1.smithi193.stdout:9/205: fsync d2/d7/db/f13 0 2024-03-23T08:23:18.015 INFO:tasks.workunit.client.1.smithi193.stdout:1/181: rename da/dd/f1c to da/f3c 0 2024-03-23T08:23:18.016 INFO:tasks.workunit.client.1.smithi193.stdout:7/157: dread d6/f2d [0,4194304] 0 2024-03-23T08:23:18.017 INFO:tasks.workunit.client.1.smithi193.stdout:0/221: readlink db/d11/d14/d24/l29 0 2024-03-23T08:23:18.018 INFO:tasks.workunit.client.1.smithi193.stdout:4/158: rmdir d11/d27 39 2024-03-23T08:23:18.023 INFO:tasks.workunit.client.1.smithi193.stdout:9/206: chown d2/d7/f36 1516225 1 2024-03-23T08:23:18.023 INFO:tasks.workunit.client.1.smithi193.stdout:9/207: write d2/d7/db/f34 [47697,129033] 0 2024-03-23T08:23:18.024 INFO:tasks.workunit.client.1.smithi193.stdout:4/159: dread d11/d1f/f28 [0,4194304] 0 2024-03-23T08:23:18.025 INFO:tasks.workunit.client.1.smithi193.stdout:4/160: truncate d11/d1f/f28 1767105 0 2024-03-23T08:23:18.033 INFO:tasks.workunit.client.1.smithi193.stdout:4/161: dwrite d11/d19/d22/f31 [0,4194304] 0 2024-03-23T08:23:18.033 INFO:tasks.workunit.client.1.smithi193.stdout:1/182: dread da/dd/df/f22 [0,4194304] 0 2024-03-23T08:23:18.038 INFO:tasks.workunit.client.1.smithi193.stdout:0/222: creat db/d11/d14/d24/d2a/d4d/f50 x:0 0 0 2024-03-23T08:23:18.042 INFO:tasks.workunit.client.1.smithi193.stdout:0/223: read db/d11/d14/d21/f26 [1299752,111691] 0 2024-03-23T08:23:18.043 INFO:tasks.workunit.client.1.smithi193.stdout:4/162: mknod d11/d1a/c32 0 2024-03-23T08:23:18.048 INFO:tasks.workunit.client.1.smithi193.stdout:4/163: dread f5 [0,4194304] 0 2024-03-23T08:23:18.049 INFO:tasks.workunit.client.1.smithi193.stdout:7/158: dwrite d6/d9/f19 [0,4194304] 0 2024-03-23T08:23:18.050 INFO:tasks.workunit.client.1.smithi193.stdout:7/159: stat d6/d9/f31 0 2024-03-23T08:23:18.051 INFO:tasks.workunit.client.1.smithi193.stdout:7/160: chown d6/d9/d1f/l28 106888 1 2024-03-23T08:23:18.052 INFO:tasks.workunit.client.1.smithi193.stdout:9/208: rename d2/l6 to d2/d7/db/d17/l46 0 2024-03-23T08:23:18.057 INFO:tasks.workunit.client.1.smithi193.stdout:1/183: rename da/d13/c26 to da/d13/d24/c3d 0 2024-03-23T08:23:18.058 INFO:tasks.workunit.client.1.smithi193.stdout:1/184: chown da/dd/f14 0 1 2024-03-23T08:23:18.062 INFO:tasks.workunit.client.1.smithi193.stdout:7/161: creat d6/d9/d1f/d23/d33/f34 x:0 0 0 2024-03-23T08:23:18.063 INFO:tasks.workunit.client.1.smithi193.stdout:1/185: write da/f3c [50551,21789] 0 2024-03-23T08:23:18.065 INFO:tasks.workunit.client.1.smithi193.stdout:1/186: write da/dd/df/f1d [894700,27941] 0 2024-03-23T08:23:18.072 INFO:tasks.workunit.client.1.smithi193.stdout:1/187: dwrite da/dd/df/f1d [0,4194304] 0 2024-03-23T08:23:18.073 INFO:tasks.workunit.client.1.smithi193.stdout:1/188: write da/dd/df/f25 [1908310,106666] 0 2024-03-23T08:23:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:17 smithi193 ceph-mon[48370]: pgmap v18: 65 pgs: 65 active+clean; 222 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 315 op/s 2024-03-23T08:23:18.079 INFO:tasks.workunit.client.1.smithi193.stdout:9/209: symlink d2/d7/d31/d3c/l47 0 2024-03-23T08:23:18.082 INFO:tasks.workunit.client.1.smithi193.stdout:9/210: unlink d2/d7/f23 0 2024-03-23T08:23:18.082 INFO:tasks.workunit.client.1.smithi193.stdout:4/164: dwrite f9 [0,4194304] 0 2024-03-23T08:23:18.083 INFO:tasks.workunit.client.1.smithi193.stdout:7/162: dread d6/fc [4194304,4194304] 0 2024-03-23T08:23:18.083 INFO:tasks.workunit.client.1.smithi193.stdout:1/189: dread da/dd/df/f29 [0,4194304] 0 2024-03-23T08:23:18.086 INFO:tasks.workunit.client.1.smithi193.stdout:4/165: mkdir d11/d1a/d33 0 2024-03-23T08:23:18.086 INFO:tasks.workunit.client.1.smithi193.stdout:7/163: chown d6/d9/d1f/f30 1001964 1 2024-03-23T08:23:18.086 INFO:tasks.workunit.client.1.smithi193.stdout:9/211: unlink d2/d7/db/d1b/f44 0 2024-03-23T08:23:18.087 INFO:tasks.workunit.client.1.smithi193.stdout:4/166: unlink d11/d1a/l2c 0 2024-03-23T08:23:18.090 INFO:tasks.workunit.client.1.smithi193.stdout:9/212: dread - d2/d7/db/f2e zero size 2024-03-23T08:23:18.091 INFO:tasks.workunit.client.1.smithi193.stdout:9/213: stat d2/d7/db/l2c 0 2024-03-23T08:23:18.091 INFO:tasks.workunit.client.1.smithi193.stdout:9/214: chown d2/d7/f1e 3 1 2024-03-23T08:23:18.091 INFO:tasks.workunit.client.1.smithi193.stdout:7/164: creat d6/d9/f35 x:0 0 0 2024-03-23T08:23:18.092 INFO:tasks.workunit.client.1.smithi193.stdout:4/167: write f5 [2138639,105659] 0 2024-03-23T08:23:18.093 INFO:tasks.workunit.client.1.smithi193.stdout:1/190: dwrite da/dd/df/f25 [0,4194304] 0 2024-03-23T08:23:18.093 INFO:tasks.workunit.client.1.smithi193.stdout:9/215: write d2/d7/db/f26 [1092600,112409] 0 2024-03-23T08:23:18.097 INFO:tasks.workunit.client.1.smithi193.stdout:7/165: creat d6/d9/f36 x:0 0 0 2024-03-23T08:23:18.099 INFO:tasks.workunit.client.1.smithi193.stdout:4/168: stat l7 0 2024-03-23T08:23:18.100 INFO:tasks.workunit.client.1.smithi193.stdout:9/216: truncate d2/d7/f36 501273 0 2024-03-23T08:23:18.102 INFO:tasks.workunit.client.1.smithi193.stdout:4/169: symlink d11/d2d/l34 0 2024-03-23T08:23:18.103 INFO:tasks.workunit.client.1.smithi193.stdout:9/217: symlink d2/d7/d31/d3c/l48 0 2024-03-23T08:23:18.103 INFO:tasks.workunit.client.1.smithi193.stdout:7/166: getdents d6/d9/d1f/d23/d33 0 2024-03-23T08:23:18.104 INFO:tasks.workunit.client.1.smithi193.stdout:4/170: creat d11/d19/d22/f35 x:0 0 0 2024-03-23T08:23:18.105 INFO:tasks.workunit.client.1.smithi193.stdout:7/167: mknod d6/d9/c37 0 2024-03-23T08:23:18.106 INFO:tasks.workunit.client.1.smithi193.stdout:9/218: symlink d2/d7/db/d17/l49 0 2024-03-23T08:23:18.109 INFO:tasks.workunit.client.1.smithi193.stdout:9/219: mknod d2/d7/d31/c4a 0 2024-03-23T08:23:18.111 INFO:tasks.workunit.client.1.smithi193.stdout:1/191: truncate da/d13/f34 713283 0 2024-03-23T08:23:18.113 INFO:tasks.workunit.client.1.smithi193.stdout:4/171: dwrite d11/d19/f20 [0,4194304] 0 2024-03-23T08:23:18.114 INFO:tasks.workunit.client.1.smithi193.stdout:9/220: fsync d2/d7/db/d17/d1f/f22 0 2024-03-23T08:23:18.115 INFO:tasks.workunit.client.1.smithi193.stdout:1/192: rmdir da/d13 39 2024-03-23T08:23:18.116 INFO:tasks.workunit.client.1.smithi193.stdout:7/168: dread d6/ff [0,4194304] 0 2024-03-23T08:23:18.116 INFO:tasks.workunit.client.1.smithi193.stdout:1/193: write da/f12 [6989853,71798] 0 2024-03-23T08:23:18.119 INFO:tasks.workunit.client.1.smithi193.stdout:4/172: creat d11/d27/f36 x:0 0 0 2024-03-23T08:23:18.121 INFO:tasks.workunit.client.1.smithi193.stdout:4/173: rmdir d11 39 2024-03-23T08:23:18.125 INFO:tasks.workunit.client.1.smithi193.stdout:1/194: link da/c1a da/dd/df/d2e/c3e 0 2024-03-23T08:23:18.126 INFO:tasks.workunit.client.1.smithi193.stdout:7/169: dwrite d6/d9/f35 [0,4194304] 0 2024-03-23T08:23:18.130 INFO:tasks.workunit.client.1.smithi193.stdout:7/170: mkdir d6/d9/d1f/d23/d33/d38 0 2024-03-23T08:23:18.132 INFO:tasks.workunit.client.1.smithi193.stdout:1/195: mkdir da/d13/d3f 0 2024-03-23T08:23:18.135 INFO:tasks.workunit.client.1.smithi193.stdout:7/171: chown d6/d9/d1f/d23/f2e 7 1 2024-03-23T08:23:18.135 INFO:tasks.workunit.client.1.smithi193.stdout:1/196: chown da/dd/f14 8 1 2024-03-23T08:23:18.137 INFO:tasks.workunit.client.1.smithi193.stdout:8/146: sync 2024-03-23T08:23:18.138 INFO:tasks.workunit.client.1.smithi193.stdout:3/131: sync 2024-03-23T08:23:18.138 INFO:tasks.workunit.client.1.smithi193.stdout:9/221: dwrite d2/d7/db/d17/d33/f45 [0,4194304] 0 2024-03-23T08:23:18.138 INFO:tasks.workunit.client.1.smithi193.stdout:7/172: unlink l0 0 2024-03-23T08:23:18.138 INFO:tasks.workunit.client.1.smithi193.stdout:4/174: dwrite d11/d19/f2f [0,4194304] 0 2024-03-23T08:23:18.139 INFO:tasks.workunit.client.1.smithi193.stdout:8/147: read - df/d13/f22 zero size 2024-03-23T08:23:18.144 INFO:tasks.workunit.client.1.smithi193.stdout:7/173: unlink d6/d9/d1f/c2f 0 2024-03-23T08:23:18.144 INFO:tasks.workunit.client.1.smithi193.stdout:1/197: mkdir da/dd/d40 0 2024-03-23T08:23:18.145 INFO:tasks.workunit.client.1.smithi193.stdout:8/148: creat df/f34 x:0 0 0 2024-03-23T08:23:18.147 INFO:tasks.workunit.client.1.smithi193.stdout:4/175: mkdir d11/d1a/d37 0 2024-03-23T08:23:18.147 INFO:tasks.workunit.client.1.smithi193.stdout:9/222: stat d2/d7/db/d17/l46 0 2024-03-23T08:23:18.149 INFO:tasks.workunit.client.1.smithi193.stdout:3/132: dread d1/de/f16 [0,4194304] 0 2024-03-23T08:23:18.149 INFO:tasks.workunit.client.1.smithi193.stdout:7/174: creat d6/f39 x:0 0 0 2024-03-23T08:23:18.150 INFO:tasks.workunit.client.1.smithi193.stdout:8/149: fsync df/d10/f17 0 2024-03-23T08:23:18.152 INFO:tasks.workunit.client.1.smithi193.stdout:4/176: unlink d11/d1a/c24 0 2024-03-23T08:23:18.153 INFO:tasks.workunit.client.1.smithi193.stdout:1/198: creat da/d13/d3f/f41 x:0 0 0 2024-03-23T08:23:18.155 INFO:tasks.workunit.client.1.smithi193.stdout:7/175: write d6/fc [3301782,114644] 0 2024-03-23T08:23:18.157 INFO:tasks.workunit.client.1.smithi193.stdout:4/177: creat d11/d2d/f38 x:0 0 0 2024-03-23T08:23:18.158 INFO:tasks.workunit.client.1.smithi193.stdout:9/223: mkdir d2/d7/db/d17/d4b 0 2024-03-23T08:23:18.160 INFO:tasks.workunit.client.1.smithi193.stdout:1/199: rename c6 to da/d13/d3f/c42 0 2024-03-23T08:23:18.161 INFO:tasks.workunit.client.1.smithi193.stdout:7/176: symlink d6/d9/d1f/d23/d33/l3a 0 2024-03-23T08:23:18.163 INFO:tasks.workunit.client.1.smithi193.stdout:1/200: truncate da/dd/f14 358429 0 2024-03-23T08:23:18.163 INFO:tasks.workunit.client.1.smithi193.stdout:4/178: dread d11/d1f/f29 [0,4194304] 0 2024-03-23T08:23:18.164 INFO:tasks.workunit.client.1.smithi193.stdout:9/224: link d2/d7/db/d1b/f30 d2/d7/db/d1b/f4c 0 2024-03-23T08:23:18.166 INFO:tasks.workunit.client.1.smithi193.stdout:7/177: dread d6/ff [0,4194304] 0 2024-03-23T08:23:18.166 INFO:tasks.workunit.client.1.smithi193.stdout:2/242: sync 2024-03-23T08:23:18.166 INFO:tasks.workunit.client.1.smithi193.stdout:6/137: sync 2024-03-23T08:23:18.166 INFO:tasks.workunit.client.1.smithi193.stdout:5/168: sync 2024-03-23T08:23:18.167 INFO:tasks.workunit.client.1.smithi193.stdout:4/179: unlink d11/f13 0 2024-03-23T08:23:18.170 INFO:tasks.workunit.client.1.smithi193.stdout:8/150: dwrite df/d10/f29 [0,4194304] 0 2024-03-23T08:23:18.171 INFO:tasks.workunit.client.1.smithi193.stdout:1/201: rename f4 to da/d13/d24/f43 0 2024-03-23T08:23:18.172 INFO:tasks.workunit.client.1.smithi193.stdout:5/169: truncate d0/d1f/f22 852415 0 2024-03-23T08:23:18.173 INFO:tasks.workunit.client.1.smithi193.stdout:1/202: stat da/fc 0 2024-03-23T08:23:18.175 INFO:tasks.workunit.client.1.smithi193.stdout:9/225: creat d2/d7/db/d1b/f4d x:0 0 0 2024-03-23T08:23:18.175 INFO:tasks.workunit.client.1.smithi193.stdout:2/243: creat d0/d28/f44 x:0 0 0 2024-03-23T08:23:18.176 INFO:tasks.workunit.client.1.smithi193.stdout:9/226: stat d2/d7/db/f34 0 2024-03-23T08:23:18.177 INFO:tasks.workunit.client.1.smithi193.stdout:8/151: creat df/f35 x:0 0 0 2024-03-23T08:23:18.177 INFO:tasks.workunit.client.1.smithi193.stdout:5/170: mknod d0/c37 0 2024-03-23T08:23:18.178 INFO:tasks.workunit.client.1.smithi193.stdout:1/203: creat da/d13/d3f/f44 x:0 0 0 2024-03-23T08:23:18.180 INFO:tasks.workunit.client.1.smithi193.stdout:7/178: mkdir d6/d9/d1f/d23/d33/d38/d3b 0 2024-03-23T08:23:18.183 INFO:tasks.workunit.client.1.smithi193.stdout:9/227: chown d2/d7/db/d1b/l1c 37 1 2024-03-23T08:23:18.183 INFO:tasks.workunit.client.1.smithi193.stdout:3/133: dread d1/f2 [0,4194304] 0 2024-03-23T08:23:18.183 INFO:tasks.workunit.client.1.smithi193.stdout:8/152: symlink df/d13/d2a/l36 0 2024-03-23T08:23:18.185 INFO:tasks.workunit.client.1.smithi193.stdout:7/179: truncate d6/d9/f31 370240 0 2024-03-23T08:23:18.187 INFO:tasks.workunit.client.1.smithi193.stdout:6/138: dwrite d0/db/df/f14 [0,4194304] 0 2024-03-23T08:23:18.190 INFO:tasks.workunit.client.1.smithi193.stdout:6/139: chown d0/db/df/f25 22665 1 2024-03-23T08:23:18.191 INFO:tasks.workunit.client.1.smithi193.stdout:8/153: symlink df/d10/l37 0 2024-03-23T08:23:18.191 INFO:tasks.workunit.client.1.smithi193.stdout:5/171: dread d0/f14 [0,4194304] 0 2024-03-23T08:23:18.191 INFO:tasks.workunit.client.1.smithi193.stdout:8/154: chown df/d13 6 1 2024-03-23T08:23:18.191 INFO:tasks.workunit.client.1.smithi193.stdout:7/180: mkdir d6/d9/d1f/d23/d33/d3c 0 2024-03-23T08:23:18.192 INFO:tasks.workunit.client.1.smithi193.stdout:6/140: write d0/d8/d12/f1b [409530,128025] 0 2024-03-23T08:23:18.192 INFO:tasks.workunit.client.1.smithi193.stdout:8/155: read - df/f35 zero size 2024-03-23T08:23:18.194 INFO:tasks.workunit.client.1.smithi193.stdout:3/134: dread d1/de/d13/f24 [0,4194304] 0 2024-03-23T08:23:18.194 INFO:tasks.workunit.client.1.smithi193.stdout:4/180: dread d11/f2e [0,4194304] 0 2024-03-23T08:23:18.194 INFO:tasks.workunit.client.1.smithi193.stdout:1/204: rmdir da/dd/d40 0 2024-03-23T08:23:18.196 INFO:tasks.workunit.client.1.smithi193.stdout:9/228: creat d2/d7/db/d32/f4e x:0 0 0 2024-03-23T08:23:18.198 INFO:tasks.workunit.client.1.smithi193.stdout:2/244: getdents d0/d28 0 2024-03-23T08:23:18.198 INFO:tasks.workunit.client.1.smithi193.stdout:5/172: dread d0/d1c/f1e [0,4194304] 0 2024-03-23T08:23:18.199 INFO:tasks.workunit.client.1.smithi193.stdout:7/181: creat d6/d9/f3d x:0 0 0 2024-03-23T08:23:18.200 INFO:tasks.workunit.client.1.smithi193.stdout:7/182: write d6/d9/f1b [797149,89653] 0 2024-03-23T08:23:18.200 INFO:tasks.workunit.client.1.smithi193.stdout:8/156: rename fc to df/f38 0 2024-03-23T08:23:18.201 INFO:tasks.workunit.client.1.smithi193.stdout:7/183: chown d6/d9/c37 11057 1 2024-03-23T08:23:18.202 INFO:tasks.workunit.client.1.smithi193.stdout:1/205: creat da/d13/d24/f45 x:0 0 0 2024-03-23T08:23:18.203 INFO:tasks.workunit.client.1.smithi193.stdout:1/206: write da/d13/d3f/f44 [188742,23316] 0 2024-03-23T08:23:18.203 INFO:tasks.workunit.client.1.smithi193.stdout:9/229: creat d2/d7/d31/f4f x:0 0 0 2024-03-23T08:23:18.206 INFO:tasks.workunit.client.1.smithi193.stdout:2/245: unlink d0/f31 0 2024-03-23T08:23:18.207 INFO:tasks.workunit.client.1.smithi193.stdout:0/224: sync 2024-03-23T08:23:18.210 INFO:tasks.workunit.client.1.smithi193.stdout:0/225: write db/d11/d14/d24/d2a/f2e [138133,91271] 0 2024-03-23T08:23:18.211 INFO:tasks.workunit.client.1.smithi193.stdout:6/141: rename d0/f5 to d0/d8/d12/d21/f29 0 2024-03-23T08:23:18.212 INFO:tasks.workunit.client.1.smithi193.stdout:7/184: mkdir d6/d9/d1f/d3e 0 2024-03-23T08:23:18.212 INFO:tasks.workunit.client.1.smithi193.stdout:3/135: mkdir d1/de/d13/d21/d25/d2b 0 2024-03-23T08:23:18.215 INFO:tasks.workunit.client.1.smithi193.stdout:9/230: symlink d2/d7/db/d17/d1f/l50 0 2024-03-23T08:23:18.217 INFO:tasks.workunit.client.1.smithi193.stdout:2/246: stat d0/d1b/c9 0 2024-03-23T08:23:18.219 INFO:tasks.workunit.client.1.smithi193.stdout:6/142: dwrite d0/d8/f1d [0,4194304] 0 2024-03-23T08:23:18.221 INFO:tasks.workunit.client.1.smithi193.stdout:6/143: chown d0/db/df/d11/c1c 7752 1 2024-03-23T08:23:18.222 INFO:tasks.workunit.client.1.smithi193.stdout:8/157: mknod df/d10/c39 0 2024-03-23T08:23:18.224 INFO:tasks.workunit.client.1.smithi193.stdout:4/181: rename d11/d19/d22/f35 to d11/d1a/d33/f39 0 2024-03-23T08:23:18.225 INFO:tasks.workunit.client.1.smithi193.stdout:7/185: symlink d6/l3f 0 2024-03-23T08:23:18.226 INFO:tasks.workunit.client.1.smithi193.stdout:7/186: chown d6/d9/f15 1 1 2024-03-23T08:23:18.226 INFO:tasks.workunit.client.1.smithi193.stdout:0/226: readlink db/d11/d14/d21/l3d 0 2024-03-23T08:23:18.230 INFO:tasks.workunit.client.1.smithi193.stdout:7/187: dread d6/d9/fe [0,4194304] 0 2024-03-23T08:23:18.230 INFO:tasks.workunit.client.1.smithi193.stdout:3/136: symlink d1/de/d13/d21/l2c 0 2024-03-23T08:23:18.232 INFO:tasks.workunit.client.1.smithi193.stdout:5/173: link d0/fa d0/d2f/f38 0 2024-03-23T08:23:18.232 INFO:tasks.workunit.client.1.smithi193.stdout:3/137: readlink d1/l26 0 2024-03-23T08:23:18.232 INFO:tasks.workunit.client.1.smithi193.stdout:6/144: mkdir d0/d8/d12/d21/d2a 0 2024-03-23T08:23:18.233 INFO:tasks.workunit.client.1.smithi193.stdout:9/231: chown d2/d7/db/d17/d33/c39 48586 1 2024-03-23T08:23:18.233 INFO:tasks.workunit.client.1.smithi193.stdout:2/247: chown d0/d1b/d19/c1e 1302860088 1 2024-03-23T08:23:18.234 INFO:tasks.workunit.client.1.smithi193.stdout:8/158: mknod df/c3a 0 2024-03-23T08:23:18.235 INFO:tasks.workunit.client.1.smithi193.stdout:4/182: unlink d11/f12 0 2024-03-23T08:23:18.236 INFO:tasks.workunit.client.1.smithi193.stdout:3/138: dread - d1/f20 zero size 2024-03-23T08:23:18.238 INFO:tasks.workunit.client.1.smithi193.stdout:5/174: write d0/d1f/f28 [4053096,82521] 0 2024-03-23T08:23:18.240 INFO:tasks.workunit.client.1.smithi193.stdout:6/145: fdatasync d0/fe 0 2024-03-23T08:23:18.240 INFO:tasks.workunit.client.1.smithi193.stdout:8/159: write df/d13/f1e [79857,18186] 0 2024-03-23T08:23:18.241 INFO:tasks.workunit.client.1.smithi193.stdout:0/227: dwrite db/d11/f40 [0,4194304] 0 2024-03-23T08:23:18.244 INFO:tasks.workunit.client.1.smithi193.stdout:6/146: chown d0/d8/d12 32837 1 2024-03-23T08:23:18.246 INFO:tasks.workunit.client.1.smithi193.stdout:4/183: mkdir d11/d2d/d3a 0 2024-03-23T08:23:18.247 INFO:tasks.workunit.client.1.smithi193.stdout:1/207: write da/d13/d24/f43 [1507841,24131] 0 2024-03-23T08:23:18.248 INFO:tasks.workunit.client.1.smithi193.stdout:7/188: symlink d6/d9/d1f/d3e/l40 0 2024-03-23T08:23:18.249 INFO:tasks.workunit.client.1.smithi193.stdout:4/184: readlink d11/d19/l17 0 2024-03-23T08:23:18.250 INFO:tasks.workunit.client.1.smithi193.stdout:2/248: dwrite d0/d28/f43 [0,4194304] 0 2024-03-23T08:23:18.250 INFO:tasks.workunit.client.1.smithi193.stdout:5/175: rename d0/c2 to d0/d32/c39 0 2024-03-23T08:23:18.251 INFO:tasks.workunit.client.1.smithi193.stdout:0/228: mkdir db/d11/d14/d51 0 2024-03-23T08:23:18.251 INFO:tasks.workunit.client.1.smithi193.stdout:4/185: chown d11/d1a/c32 5216 1 2024-03-23T08:23:18.252 INFO:tasks.workunit.client.1.smithi193.stdout:6/147: symlink d0/d8/d12/l2b 0 2024-03-23T08:23:18.253 INFO:tasks.workunit.client.1.smithi193.stdout:9/232: write d2/d7/db/d1b/f30 [523022,87828] 0 2024-03-23T08:23:18.254 INFO:tasks.workunit.client.1.smithi193.stdout:2/249: truncate d0/f2d 4542753 0 2024-03-23T08:23:18.256 INFO:tasks.workunit.client.1.smithi193.stdout:3/139: write d1/de/f10 [858509,116957] 0 2024-03-23T08:23:18.256 INFO:tasks.workunit.client.1.smithi193.stdout:5/176: creat d0/d1c/f3a x:0 0 0 2024-03-23T08:23:18.259 INFO:tasks.workunit.client.1.smithi193.stdout:8/160: creat df/d10/f3b x:0 0 0 2024-03-23T08:23:18.260 INFO:tasks.workunit.client.1.smithi193.stdout:0/229: chown db/d11/d14/d21/d33/c42 5791766 1 2024-03-23T08:23:18.260 INFO:tasks.workunit.client.1.smithi193.stdout:7/189: mkdir d6/d9/d1f/d23/d33/d3c/d41 0 2024-03-23T08:23:18.260 INFO:tasks.workunit.client.1.smithi193.stdout:4/186: mkdir d11/d1a/d33/d3b 0 2024-03-23T08:23:18.260 INFO:tasks.workunit.client.1.smithi193.stdout:9/233: truncate d2/d7/f37 355375 0 2024-03-23T08:23:18.261 INFO:tasks.workunit.client.1.smithi193.stdout:6/148: unlink d0/db/df/f14 0 2024-03-23T08:23:18.261 INFO:tasks.workunit.client.1.smithi193.stdout:1/208: creat da/f46 x:0 0 0 2024-03-23T08:23:18.262 INFO:tasks.workunit.client.1.smithi193.stdout:8/161: truncate df/d10/f2f 892623 0 2024-03-23T08:23:18.262 INFO:tasks.workunit.client.1.smithi193.stdout:2/250: creat d0/d1b/d14/f45 x:0 0 0 2024-03-23T08:23:18.263 INFO:tasks.workunit.client.1.smithi193.stdout:9/234: stat d2/d7/db/d17/d1f/l27 0 2024-03-23T08:23:18.264 INFO:tasks.workunit.client.1.smithi193.stdout:4/187: mkdir d11/d1a/d33/d3c 0 2024-03-23T08:23:18.264 INFO:tasks.workunit.client.1.smithi193.stdout:0/230: mkdir db/d52 0 2024-03-23T08:23:18.264 INFO:tasks.workunit.client.1.smithi193.stdout:9/235: dread - d2/d7/db/d32/f4e zero size 2024-03-23T08:23:18.265 INFO:tasks.workunit.client.1.smithi193.stdout:3/140: link d1/de/f10 d1/d22/f2d 0 2024-03-23T08:23:18.267 INFO:tasks.workunit.client.1.smithi193.stdout:9/236: unlink d2/d7/d31/d3c/l48 0 2024-03-23T08:23:18.270 INFO:tasks.workunit.client.1.smithi193.stdout:8/162: fsync df/f33 0 2024-03-23T08:23:18.271 INFO:tasks.workunit.client.1.smithi193.stdout:9/237: dread d2/f25 [0,4194304] 0 2024-03-23T08:23:18.271 INFO:tasks.workunit.client.1.smithi193.stdout:7/190: stat d6/d9/d1f/d23/d33/d38/d3b 0 2024-03-23T08:23:18.271 INFO:tasks.workunit.client.1.smithi193.stdout:1/209: fsync da/dd/df/f25 0 2024-03-23T08:23:18.274 INFO:tasks.workunit.client.1.smithi193.stdout:6/149: creat d0/d8/d12/d21/f2c x:0 0 0 2024-03-23T08:23:18.274 INFO:tasks.workunit.client.1.smithi193.stdout:3/141: write d1/de/d13/d21/d27/f1f [249364,28123] 0 2024-03-23T08:23:18.274 INFO:tasks.workunit.client.1.smithi193.stdout:0/231: chown db/d11/d14/d24/d2a/c4e 41239039 1 2024-03-23T08:23:18.278 INFO:tasks.workunit.client.1.smithi193.stdout:4/188: getdents d11/d1a/d33/d3b 0 2024-03-23T08:23:18.279 INFO:tasks.workunit.client.1.smithi193.stdout:6/150: rmdir d0/db/df/d11 39 2024-03-23T08:23:18.280 INFO:tasks.workunit.client.1.smithi193.stdout:0/232: chown db/d11/d14/d21/l3d 297482 1 2024-03-23T08:23:18.280 INFO:tasks.workunit.client.1.smithi193.stdout:3/142: symlink d1/de/d13/d21/l2e 0 2024-03-23T08:23:18.281 INFO:tasks.workunit.client.1.smithi193.stdout:1/210: symlink da/l47 0 2024-03-23T08:23:18.283 INFO:tasks.workunit.client.1.smithi193.stdout:6/151: readlink d0/l18 0 2024-03-23T08:23:18.286 INFO:tasks.workunit.client.1.smithi193.stdout:3/143: unlink d1/d22/l2a 0 2024-03-23T08:23:18.286 INFO:tasks.workunit.client.1.smithi193.stdout:1/211: read - da/dd/df/f3b zero size 2024-03-23T08:23:18.290 INFO:tasks.workunit.client.1.smithi193.stdout:4/189: symlink d11/d1a/d33/d3c/l3d 0 2024-03-23T08:23:18.292 INFO:tasks.workunit.client.1.smithi193.stdout:2/251: dread d0/d1b/d19/f1a [0,4194304] 0 2024-03-23T08:23:18.294 INFO:tasks.workunit.client.1.smithi193.stdout:0/233: dread db/d11/d14/f19 [0,4194304] 0 2024-03-23T08:23:18.294 INFO:tasks.workunit.client.1.smithi193.stdout:8/163: dwrite df/d13/f1e [0,4194304] 0 2024-03-23T08:23:18.295 INFO:tasks.workunit.client.1.smithi193.stdout:5/177: dwrite d0/d2f/f38 [4194304,4194304] 0 2024-03-23T08:23:18.296 INFO:tasks.workunit.client.1.smithi193.stdout:6/152: write d0/fe [632688,73383] 0 2024-03-23T08:23:18.297 INFO:tasks.workunit.client.1.smithi193.stdout:7/191: dread d6/d9/d1f/f2a [0,4194304] 0 2024-03-23T08:23:18.298 INFO:tasks.workunit.client.1.smithi193.stdout:4/190: chown d11/d19/d22/f26 6860666 1 2024-03-23T08:23:18.299 INFO:tasks.workunit.client.1.smithi193.stdout:5/178: fsync d0/ff 0 2024-03-23T08:23:18.300 INFO:tasks.workunit.client.1.smithi193.stdout:9/238: dwrite d2/d7/db/f34 [0,4194304] 0 2024-03-23T08:23:18.300 INFO:tasks.workunit.client.1.smithi193.stdout:3/144: truncate d1/de/f11 677317 0 2024-03-23T08:23:18.303 INFO:tasks.workunit.client.1.smithi193.stdout:6/153: write d0/db/f20 [319129,87775] 0 2024-03-23T08:23:18.304 INFO:tasks.workunit.client.1.smithi193.stdout:1/212: unlink da/d13/d3f/c42 0 2024-03-23T08:23:18.306 INFO:tasks.workunit.client.1.smithi193.stdout:7/192: mkdir d6/d9/d1f/d3e/d42 0 2024-03-23T08:23:18.306 INFO:tasks.workunit.client.1.smithi193.stdout:3/145: chown d1/f20 104258 1 2024-03-23T08:23:18.308 INFO:tasks.workunit.client.1.smithi193.stdout:8/164: creat df/d13/d2a/f3c x:0 0 0 2024-03-23T08:23:18.309 INFO:tasks.workunit.client.1.smithi193.stdout:7/193: fsync d6/f14 0 2024-03-23T08:23:18.311 INFO:tasks.workunit.client.1.smithi193.stdout:6/154: write d0/d8/d12/f1b [283183,89127] 0 2024-03-23T08:23:18.312 INFO:tasks.workunit.client.1.smithi193.stdout:4/191: dread f5 [0,4194304] 0 2024-03-23T08:23:18.312 INFO:tasks.workunit.client.1.smithi193.stdout:6/155: dread - d0/db/df/f25 zero size 2024-03-23T08:23:18.313 INFO:tasks.workunit.client.1.smithi193.stdout:1/213: truncate f1 3362674 0 2024-03-23T08:23:18.313 INFO:tasks.workunit.client.1.smithi193.stdout:4/192: readlink l10 0 2024-03-23T08:23:18.314 INFO:tasks.workunit.client.1.smithi193.stdout:8/165: unlink df/f35 0 2024-03-23T08:23:18.314 INFO:tasks.workunit.client.1.smithi193.stdout:9/239: mknod d2/d7/db/d17/d4b/c51 0 2024-03-23T08:23:18.316 INFO:tasks.workunit.client.1.smithi193.stdout:5/179: getdents d0/d32 0 2024-03-23T08:23:18.318 INFO:tasks.workunit.client.1.smithi193.stdout:7/194: rename d6/d9/l1e to d6/d9/d1f/d3e/d42/l43 0 2024-03-23T08:23:18.318 INFO:tasks.workunit.client.1.smithi193.stdout:0/234: dread db/d11/f12 [0,4194304] 0 2024-03-23T08:23:18.318 INFO:tasks.workunit.client.1.smithi193.stdout:2/252: dwrite d0/d1b/d14/d25/f3a [0,4194304] 0 2024-03-23T08:23:18.318 INFO:tasks.workunit.client.1.smithi193.stdout:3/146: creat d1/f2f x:0 0 0 2024-03-23T08:23:18.320 INFO:tasks.workunit.client.1.smithi193.stdout:1/214: creat da/dd/df/d2e/f48 x:0 0 0 2024-03-23T08:23:18.320 INFO:tasks.workunit.client.1.smithi193.stdout:8/166: truncate df/d10/f3b 746397 0 2024-03-23T08:23:18.321 INFO:tasks.workunit.client.1.smithi193.stdout:5/180: mkdir d0/d1c/d3b 0 2024-03-23T08:23:18.322 INFO:tasks.workunit.client.1.smithi193.stdout:9/240: write d2/d7/db/d32/f4e [369319,107014] 0 2024-03-23T08:23:18.324 INFO:tasks.workunit.client.1.smithi193.stdout:6/156: mknod d0/db/df/d11/c2d 0 2024-03-23T08:23:18.324 INFO:tasks.workunit.client.1.smithi193.stdout:5/181: write d0/f1a [416498,24318] 0 2024-03-23T08:23:18.325 INFO:tasks.workunit.client.1.smithi193.stdout:1/215: write da/dd/df/d2e/f48 [882800,57492] 0 2024-03-23T08:23:18.325 INFO:tasks.workunit.client.1.smithi193.stdout:7/195: creat d6/d9/d1f/d23/d33/d38/f44 x:0 0 0 2024-03-23T08:23:18.330 INFO:tasks.workunit.client.1.smithi193.stdout:2/253: creat d0/d1b/d3b/f46 x:0 0 0 2024-03-23T08:23:18.330 INFO:tasks.workunit.client.1.smithi193.stdout:9/241: write d2/d7/db/d17/d33/f45 [3611996,70977] 0 2024-03-23T08:23:18.331 INFO:tasks.workunit.client.1.smithi193.stdout:6/157: write d0/db/f20 [1105466,106605] 0 2024-03-23T08:23:18.332 INFO:tasks.workunit.client.1.smithi193.stdout:8/167: rmdir df/d10 39 2024-03-23T08:23:18.336 INFO:tasks.workunit.client.1.smithi193.stdout:0/235: dread db/d11/f40 [0,4194304] 0 2024-03-23T08:23:18.337 INFO:tasks.workunit.client.1.smithi193.stdout:7/196: creat d6/d9/d1f/d23/f45 x:0 0 0 2024-03-23T08:23:18.338 INFO:tasks.workunit.client.1.smithi193.stdout:1/216: creat da/d13/f49 x:0 0 0 2024-03-23T08:23:18.338 INFO:tasks.workunit.client.1.smithi193.stdout:3/147: dwrite d1/de/d13/d21/d27/f1f [0,4194304] 0 2024-03-23T08:23:18.340 INFO:tasks.workunit.client.1.smithi193.stdout:6/158: readlink d0/l3 0 2024-03-23T08:23:18.340 INFO:tasks.workunit.client.1.smithi193.stdout:5/182: rename d0/d1f/f29 to d0/d1c/d3b/f3c 0 2024-03-23T08:23:18.341 INFO:tasks.workunit.client.1.smithi193.stdout:1/217: dread - da/d13/f49 zero size 2024-03-23T08:23:18.343 INFO:tasks.workunit.client.1.smithi193.stdout:3/148: chown d1/l17 24101701 1 2024-03-23T08:23:18.345 INFO:tasks.workunit.client.1.smithi193.stdout:0/236: mknod db/d11/d14/d24/d2a/d4d/c53 0 2024-03-23T08:23:18.345 INFO:tasks.workunit.client.1.smithi193.stdout:8/168: fdatasync df/d10/f3b 0 2024-03-23T08:23:18.346 INFO:tasks.workunit.client.1.smithi193.stdout:4/193: dwrite fd [0,4194304] 0 2024-03-23T08:23:18.346 INFO:tasks.workunit.client.1.smithi193.stdout:7/197: mkdir d6/d9/d1f/d23/d33/d38/d46 0 2024-03-23T08:23:18.347 INFO:tasks.workunit.client.1.smithi193.stdout:0/237: write fa [2135705,87935] 0 2024-03-23T08:23:18.351 INFO:tasks.workunit.client.1.smithi193.stdout:2/254: write d0/d1b/d14/f40 [3932268,19681] 0 2024-03-23T08:23:18.353 INFO:tasks.workunit.client.1.smithi193.stdout:2/255: fsync d0/d1b/d14/d25/f3a 0 2024-03-23T08:23:18.353 INFO:tasks.workunit.client.1.smithi193.stdout:6/159: creat d0/db/df/d11/f2e x:0 0 0 2024-03-23T08:23:18.354 INFO:tasks.workunit.client.1.smithi193.stdout:3/149: creat d1/d22/f30 x:0 0 0 2024-03-23T08:23:18.355 INFO:tasks.workunit.client.1.smithi193.stdout:6/160: read - d0/d8/d12/f1e zero size 2024-03-23T08:23:18.355 INFO:tasks.workunit.client.1.smithi193.stdout:2/256: write d0/d1b/d14/d25/f29 [535822,50404] 0 2024-03-23T08:23:18.356 INFO:tasks.workunit.client.1.smithi193.stdout:2/257: stat d0/d1b/d14/f40 0 2024-03-23T08:23:18.360 INFO:tasks.workunit.client.1.smithi193.stdout:5/183: dread d0/ff [0,4194304] 0 2024-03-23T08:23:18.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/258: write d0/d1b/d14/d25/f29 [2029336,111164] 0 2024-03-23T08:23:18.361 INFO:tasks.workunit.client.1.smithi193.stdout:3/150: unlink d1/f2 0 2024-03-23T08:23:18.362 INFO:tasks.workunit.client.1.smithi193.stdout:2/259: readlink d0/l4 0 2024-03-23T08:23:18.363 INFO:tasks.workunit.client.1.smithi193.stdout:8/169: rename df/c14 to df/d13/c3d 0 2024-03-23T08:23:18.364 INFO:tasks.workunit.client.1.smithi193.stdout:7/198: getdents d6/d9/d1f/d23/d33/d3c/d41 0 2024-03-23T08:23:18.365 INFO:tasks.workunit.client.1.smithi193.stdout:6/161: rmdir d0/db/df 39 2024-03-23T08:23:18.365 INFO:tasks.workunit.client.1.smithi193.stdout:6/162: readlink d0/db/l1a 0 2024-03-23T08:23:18.365 INFO:tasks.workunit.client.1.smithi193.stdout:9/242: link d2/d7/db/d1b/l29 d2/l52 0 2024-03-23T08:23:18.367 INFO:tasks.workunit.client.1.smithi193.stdout:5/184: mknod d0/d1c/c3d 0 2024-03-23T08:23:18.368 INFO:tasks.workunit.client.1.smithi193.stdout:5/185: chown d0/fa 1275 1 2024-03-23T08:23:18.369 INFO:tasks.workunit.client.1.smithi193.stdout:2/260: unlink d0/d28/l3f 0 2024-03-23T08:23:18.370 INFO:tasks.workunit.client.1.smithi193.stdout:1/218: link da/c32 da/c4a 0 2024-03-23T08:23:18.370 INFO:tasks.workunit.client.1.smithi193.stdout:5/186: write d0/f11 [3064111,109872] 0 2024-03-23T08:23:18.372 INFO:tasks.workunit.client.1.smithi193.stdout:2/261: chown d0/d1b/d14/d25/l27 104136141 1 2024-03-23T08:23:18.375 INFO:tasks.workunit.client.1.smithi193.stdout:5/187: write d0/ff [1490309,117328] 0 2024-03-23T08:23:18.375 INFO:tasks.workunit.client.1.smithi193.stdout:0/238: getdents db/d11/d14 0 2024-03-23T08:23:18.375 INFO:tasks.workunit.client.1.smithi193.stdout:7/199: mkdir d6/d9/d1f/d23/d33/d3c/d41/d47 0 2024-03-23T08:23:18.375 INFO:tasks.workunit.client.1.smithi193.stdout:2/262: fdatasync d0/d1b/f8 0 2024-03-23T08:23:18.375 INFO:tasks.workunit.client.1.smithi193.stdout:8/170: mkdir df/d10/d3e 0 2024-03-23T08:23:18.377 INFO:tasks.workunit.client.1.smithi193.stdout:4/194: dwrite d11/d1a/f1c [8388608,4194304] 0 2024-03-23T08:23:18.377 INFO:tasks.workunit.client.1.smithi193.stdout:6/163: mknod d0/d8/c2f 0 2024-03-23T08:23:18.381 INFO:tasks.workunit.client.1.smithi193.stdout:6/164: stat d0/d8/d12/d21/f2c 0 2024-03-23T08:23:18.381 INFO:tasks.workunit.client.1.smithi193.stdout:9/243: write d2/d7/db/d17/d1f/f42 [803042,100131] 0 2024-03-23T08:23:18.381 INFO:tasks.workunit.client.1.smithi193.stdout:5/188: creat d0/f3e x:0 0 0 2024-03-23T08:23:18.382 INFO:tasks.workunit.client.1.smithi193.stdout:4/195: unlink cc 0 2024-03-23T08:23:18.383 INFO:tasks.workunit.client.1.smithi193.stdout:2/263: creat d0/d28/f47 x:0 0 0 2024-03-23T08:23:18.386 INFO:tasks.workunit.client.1.smithi193.stdout:3/151: getdents d1 0 2024-03-23T08:23:18.386 INFO:tasks.workunit.client.1.smithi193.stdout:9/244: chown d2/d7/f11 675738 1 2024-03-23T08:23:18.386 INFO:tasks.workunit.client.1.smithi193.stdout:2/264: write d0/d28/f43 [1482282,71914] 0 2024-03-23T08:23:18.387 INFO:tasks.workunit.client.1.smithi193.stdout:5/189: symlink d0/l3f 0 2024-03-23T08:23:18.390 INFO:tasks.workunit.client.1.smithi193.stdout:1/219: link da/dd/df/f1d da/dd/df/f4b 0 2024-03-23T08:23:18.392 INFO:tasks.workunit.client.1.smithi193.stdout:9/245: write d2/d7/d31/f4f [490529,107563] 0 2024-03-23T08:23:18.393 INFO:tasks.workunit.client.1.smithi193.stdout:2/265: mkdir d0/d37/d48 0 2024-03-23T08:23:18.394 INFO:tasks.workunit.client.1.smithi193.stdout:5/190: stat d0/d2f/f38 0 2024-03-23T08:23:18.394 INFO:tasks.workunit.client.1.smithi193.stdout:6/165: dread d0/db/f20 [0,4194304] 0 2024-03-23T08:23:18.394 INFO:tasks.workunit.client.1.smithi193.stdout:1/220: readlink da/dd/df/d2e/l3a 0 2024-03-23T08:23:18.394 INFO:tasks.workunit.client.1.smithi193.stdout:0/239: getdents db 0 2024-03-23T08:23:18.397 INFO:tasks.workunit.client.1.smithi193.stdout:3/152: link d1/fa d1/de/d13/d21/f31 0 2024-03-23T08:23:18.398 INFO:tasks.workunit.client.1.smithi193.stdout:5/191: symlink d0/l40 0 2024-03-23T08:23:18.399 INFO:tasks.workunit.client.1.smithi193.stdout:1/221: read f1 [197387,28431] 0 2024-03-23T08:23:18.401 INFO:tasks.workunit.client.1.smithi193.stdout:0/240: stat db/d11/d14/d21/f22 0 2024-03-23T08:23:18.401 INFO:tasks.workunit.client.1.smithi193.stdout:2/266: mkdir d0/d1b/d49 0 2024-03-23T08:23:18.402 INFO:tasks.workunit.client.1.smithi193.stdout:4/196: dwrite d11/f2e [0,4194304] 0 2024-03-23T08:23:18.402 INFO:tasks.workunit.client.1.smithi193.stdout:9/246: truncate d2/f25 89047 0 2024-03-23T08:23:18.404 INFO:tasks.workunit.client.1.smithi193.stdout:8/171: dread df/f1f [0,4194304] 0 2024-03-23T08:23:18.406 INFO:tasks.workunit.client.1.smithi193.stdout:5/192: rmdir d0/d1f 39 2024-03-23T08:23:18.406 INFO:tasks.workunit.client.1.smithi193.stdout:2/267: read - d0/d1b/d14/f45 zero size 2024-03-23T08:23:18.406 INFO:tasks.workunit.client.1.smithi193.stdout:8/172: dread - df/f2c zero size 2024-03-23T08:23:18.406 INFO:tasks.workunit.client.1.smithi193.stdout:9/247: chown d2/d7/db/d1b/f30 685187718 1 2024-03-23T08:23:18.407 INFO:tasks.workunit.client.1.smithi193.stdout:6/166: creat d0/db/df/d11/f30 x:0 0 0 2024-03-23T08:23:18.407 INFO:tasks.workunit.client.1.smithi193.stdout:7/200: dread d6/f8 [0,4194304] 0 2024-03-23T08:23:18.410 INFO:tasks.workunit.client.1.smithi193.stdout:7/201: fdatasync d6/f8 0 2024-03-23T08:23:18.412 INFO:tasks.workunit.client.1.smithi193.stdout:8/173: truncate df/d13/f22 668808 0 2024-03-23T08:23:18.415 INFO:tasks.workunit.client.1.smithi193.stdout:8/174: chown cb 54483 1 2024-03-23T08:23:18.415 INFO:tasks.workunit.client.1.smithi193.stdout:0/241: dread db/d11/d14/d21/d33/f43 [0,4194304] 0 2024-03-23T08:23:18.417 INFO:tasks.workunit.client.1.smithi193.stdout:4/197: rename d11/d19/l17 to d11/l3e 0 2024-03-23T08:23:18.418 INFO:tasks.workunit.client.1.smithi193.stdout:7/202: write d6/d9/d1f/d23/f2e [4698464,54694] 0 2024-03-23T08:23:18.418 INFO:tasks.workunit.client.1.smithi193.stdout:5/193: rename d0/d1f/f2b to d0/d1f/f41 0 2024-03-23T08:23:18.420 INFO:tasks.workunit.client.1.smithi193.stdout:2/268: creat d0/d37/d48/f4a x:0 0 0 2024-03-23T08:23:18.423 INFO:tasks.workunit.client.1.smithi193.stdout:4/198: fsync f4 0 2024-03-23T08:23:18.424 INFO:tasks.workunit.client.1.smithi193.stdout:9/248: creat d2/d7/db/d17/f53 x:0 0 0 2024-03-23T08:23:18.424 INFO:tasks.workunit.client.1.smithi193.stdout:5/194: write d0/d1c/f3a [230551,71888] 0 2024-03-23T08:23:18.426 INFO:tasks.workunit.client.1.smithi193.stdout:1/222: dread da/dd/df/f22 [0,4194304] 0 2024-03-23T08:23:18.426 INFO:tasks.workunit.client.1.smithi193.stdout:2/269: read - d0/d1b/d14/f45 zero size 2024-03-23T08:23:18.426 INFO:tasks.workunit.client.1.smithi193.stdout:6/167: dwrite d0/db/df/d11/f2e [0,4194304] 0 2024-03-23T08:23:18.428 INFO:tasks.workunit.client.1.smithi193.stdout:0/242: truncate db/d11/f28 183413 0 2024-03-23T08:23:18.430 INFO:tasks.workunit.client.1.smithi193.stdout:9/249: readlink d2/d7/d31/l38 0 2024-03-23T08:23:18.430 INFO:tasks.workunit.client.1.smithi193.stdout:7/203: creat d6/d9/d1f/d23/d33/d38/d3b/f48 x:0 0 0 2024-03-23T08:23:18.432 INFO:tasks.workunit.client.1.smithi193.stdout:6/168: chown d0/d8/d12 0 1 2024-03-23T08:23:18.433 INFO:tasks.workunit.client.1.smithi193.stdout:2/270: fdatasync d0/d28/f47 0 2024-03-23T08:23:18.434 INFO:tasks.workunit.client.1.smithi193.stdout:3/153: dwrite d1/de/f11 [0,4194304] 0 2024-03-23T08:23:18.436 INFO:tasks.workunit.client.1.smithi193.stdout:1/223: symlink da/d13/l4c 0 2024-03-23T08:23:18.438 INFO:tasks.workunit.client.1.smithi193.stdout:5/195: symlink d0/d2f/l42 0 2024-03-23T08:23:18.440 INFO:tasks.workunit.client.1.smithi193.stdout:3/154: mknod d1/de/d13/d21/d27/c32 0 2024-03-23T08:23:18.440 INFO:tasks.workunit.client.1.smithi193.stdout:2/271: symlink d0/d28/d33/l4b 0 2024-03-23T08:23:18.441 INFO:tasks.workunit.client.1.smithi193.stdout:8/175: dread df/d10/d19/f26 [0,4194304] 0 2024-03-23T08:23:18.442 INFO:tasks.workunit.client.1.smithi193.stdout:6/169: write d0/db/df/d11/f2e [4550282,122242] 0 2024-03-23T08:23:18.443 INFO:tasks.workunit.client.1.smithi193.stdout:1/224: creat da/dd/df/d2e/f4d x:0 0 0 2024-03-23T08:23:18.443 INFO:tasks.workunit.client.1.smithi193.stdout:3/155: stat d1/de/d13/d21/d25/d2b 0 2024-03-23T08:23:18.445 INFO:tasks.workunit.client.1.smithi193.stdout:8/176: chown df/d10/c39 1662 1 2024-03-23T08:23:18.445 INFO:tasks.workunit.client.1.smithi193.stdout:7/204: write d6/f2d [2888485,66681] 0 2024-03-23T08:23:18.445 INFO:tasks.workunit.client.1.smithi193.stdout:2/272: write d0/d1b/f13 [577964,4302] 0 2024-03-23T08:23:18.448 INFO:tasks.workunit.client.1.smithi193.stdout:0/243: link db/d11/d14/d21/f2c db/d11/d14/d21/d33/d48/f54 0 2024-03-23T08:23:18.449 INFO:tasks.workunit.client.1.smithi193.stdout:2/273: chown d0/fc 960286473 1 2024-03-23T08:23:18.450 INFO:tasks.workunit.client.1.smithi193.stdout:4/199: dwrite d11/d1a/f1c [0,4194304] 0 2024-03-23T08:23:18.450 INFO:tasks.workunit.client.1.smithi193.stdout:3/156: creat d1/de/d13/f33 x:0 0 0 2024-03-23T08:23:18.452 INFO:tasks.workunit.client.1.smithi193.stdout:5/196: creat d0/f43 x:0 0 0 2024-03-23T08:23:18.455 INFO:tasks.workunit.client.1.smithi193.stdout:4/200: write f9 [1960945,17465] 0 2024-03-23T08:23:18.455 INFO:tasks.workunit.client.1.smithi193.stdout:3/157: chown d1/lb 57734 1 2024-03-23T08:23:18.457 INFO:tasks.workunit.client.1.smithi193.stdout:8/177: creat df/d10/f3f x:0 0 0 2024-03-23T08:23:18.459 INFO:tasks.workunit.client.1.smithi193.stdout:5/197: creat d0/d32/f44 x:0 0 0 2024-03-23T08:23:18.463 INFO:tasks.workunit.client.1.smithi193.stdout:4/201: chown d11/d1a/c32 26166 1 2024-03-23T08:23:18.463 INFO:tasks.workunit.client.1.smithi193.stdout:2/274: truncate d0/d1b/d3b/f46 699828 0 2024-03-23T08:23:18.463 INFO:tasks.workunit.client.1.smithi193.stdout:5/198: read d0/ff [1783411,2602] 0 2024-03-23T08:23:18.463 INFO:tasks.workunit.client.1.smithi193.stdout:3/158: readlink d1/de/d13/d21/l2c 0 2024-03-23T08:23:18.463 INFO:tasks.workunit.client.1.smithi193.stdout:8/178: dread - df/d10/f3f zero size 2024-03-23T08:23:18.464 INFO:tasks.workunit.client.1.smithi193.stdout:9/250: dwrite d2/f3 [0,4194304] 0 2024-03-23T08:23:18.467 INFO:tasks.workunit.client.1.smithi193.stdout:6/170: truncate d0/db/f20 444879 0 2024-03-23T08:23:18.470 INFO:tasks.workunit.client.1.smithi193.stdout:1/225: dwrite da/d13/d24/f43 [0,4194304] 0 2024-03-23T08:23:18.470 INFO:tasks.workunit.client.1.smithi193.stdout:7/205: dread d6/f2d [0,4194304] 0 2024-03-23T08:23:18.473 INFO:tasks.workunit.client.1.smithi193.stdout:6/171: read - d0/d8/d12/f1e zero size 2024-03-23T08:23:18.477 INFO:tasks.workunit.client.1.smithi193.stdout:3/159: read d1/de/f10 [253511,87800] 0 2024-03-23T08:23:18.478 INFO:tasks.workunit.client.1.smithi193.stdout:7/206: unlink d6/d9/c11 0 2024-03-23T08:23:18.478 INFO:tasks.workunit.client.1.smithi193.stdout:8/179: write df/d13/f1e [2482141,106304] 0 2024-03-23T08:23:18.479 INFO:tasks.workunit.client.1.smithi193.stdout:4/202: creat d11/d1a/d37/f3f x:0 0 0 2024-03-23T08:23:18.479 INFO:tasks.workunit.client.1.smithi193.stdout:0/244: truncate db/d11/d14/d21/f2c 1581748 0 2024-03-23T08:23:18.483 INFO:tasks.workunit.client.1.smithi193.stdout:8/180: rename df/d13 to df/d13/d40 22 2024-03-23T08:23:18.483 INFO:tasks.workunit.client.1.smithi193.stdout:9/251: creat d2/d7/db/d17/f54 x:0 0 0 2024-03-23T08:23:18.484 INFO:tasks.workunit.client.1.smithi193.stdout:1/226: creat da/f4e x:0 0 0 2024-03-23T08:23:18.485 INFO:tasks.workunit.client.1.smithi193.stdout:0/245: chown db/d11/d14/d21/f26 8377482 1 2024-03-23T08:23:18.485 INFO:tasks.workunit.client.1.smithi193.stdout:3/160: mkdir d1/de/d13/d21/d25/d34 0 2024-03-23T08:23:18.487 INFO:tasks.workunit.client.1.smithi193.stdout:4/203: mknod d11/d1a/d33/c40 0 2024-03-23T08:23:18.487 INFO:tasks.workunit.client.1.smithi193.stdout:0/246: readlink db/l36 0 2024-03-23T08:23:18.487 INFO:tasks.workunit.client.1.smithi193.stdout:2/275: dwrite d0/d1b/d3b/f42 [0,4194304] 0 2024-03-23T08:23:18.489 INFO:tasks.workunit.client.1.smithi193.stdout:6/172: rename d0/db/c1f to d0/db/c31 0 2024-03-23T08:23:18.494 INFO:tasks.workunit.client.1.smithi193.stdout:0/247: write db/f3e [1563496,3755] 0 2024-03-23T08:23:18.496 INFO:tasks.workunit.client.1.smithi193.stdout:6/173: write d0/db/df/f25 [677419,121450] 0 2024-03-23T08:23:18.498 INFO:tasks.workunit.client.1.smithi193.stdout:2/276: truncate d0/d1b/d14/f45 1851 0 2024-03-23T08:23:18.498 INFO:tasks.workunit.client.1.smithi193.stdout:8/181: rename c4 to df/d13/c41 0 2024-03-23T08:23:18.500 INFO:tasks.workunit.client.1.smithi193.stdout:7/207: link d6/f8 d6/d9/d1f/d23/d33/d3c/d41/d47/f49 0 2024-03-23T08:23:18.500 INFO:tasks.workunit.client.1.smithi193.stdout:5/199: dwrite d0/f12 [0,4194304] 0 2024-03-23T08:23:18.502 INFO:tasks.workunit.client.1.smithi193.stdout:3/161: creat d1/de/d13/d21/d25/d2b/f35 x:0 0 0 2024-03-23T08:23:18.504 INFO:tasks.workunit.client.1.smithi193.stdout:6/174: write d0/db/df/d11/f30 [525979,84646] 0 2024-03-23T08:23:18.504 INFO:tasks.workunit.client.1.smithi193.stdout:9/252: write d2/f28 [4258051,24763] 0 2024-03-23T08:23:18.506 INFO:tasks.workunit.client.1.smithi193.stdout:1/227: dread da/dd/df/f29 [0,4194304] 0 2024-03-23T08:23:18.507 INFO:tasks.workunit.client.1.smithi193.stdout:0/248: symlink db/d11/d14/d21/l55 0 2024-03-23T08:23:18.508 INFO:tasks.workunit.client.1.smithi193.stdout:2/277: creat d0/d37/f4c x:0 0 0 2024-03-23T08:23:18.508 INFO:tasks.workunit.client.1.smithi193.stdout:3/162: truncate d1/f2f 725899 0 2024-03-23T08:23:18.509 INFO:tasks.workunit.client.1.smithi193.stdout:5/200: truncate d0/f17 63574 0 2024-03-23T08:23:18.510 INFO:tasks.workunit.client.1.smithi193.stdout:8/182: mknod df/c42 0 2024-03-23T08:23:18.511 INFO:tasks.workunit.client.1.smithi193.stdout:6/175: mknod d0/db/c32 0 2024-03-23T08:23:18.512 INFO:tasks.workunit.client.1.smithi193.stdout:7/208: write d6/d9/d1f/d23/d33/d38/d3b/f48 [400999,113894] 0 2024-03-23T08:23:18.514 INFO:tasks.workunit.client.1.smithi193.stdout:5/201: read d0/fa [1805028,457] 0 2024-03-23T08:23:18.514 INFO:tasks.workunit.client.1.smithi193.stdout:4/204: dwrite d11/d1a/d33/f39 [0,4194304] 0 2024-03-23T08:23:18.518 INFO:tasks.workunit.client.1.smithi193.stdout:8/183: dread df/d13/f1e [0,4194304] 0 2024-03-23T08:23:18.526 INFO:tasks.workunit.client.1.smithi193.stdout:2/278: dread d0/d1b/f8 [4194304,4194304] 0 2024-03-23T08:23:18.604 INFO:tasks.workunit.client.1.smithi193.stdout:9/253: mknod d2/d7/db/d17/c55 0 2024-03-23T08:23:18.605 INFO:tasks.workunit.client.1.smithi193.stdout:9/254: chown d2/d7/db/d32 38768424 1 2024-03-23T08:23:18.605 INFO:tasks.workunit.client.1.smithi193.stdout:0/249: write db/d11/d14/f19 [1228944,122658] 0 2024-03-23T08:23:18.607 INFO:tasks.workunit.client.1.smithi193.stdout:0/250: write db/d11/f28 [921099,119642] 0 2024-03-23T08:23:18.609 INFO:tasks.workunit.client.1.smithi193.stdout:7/209: chown d6/l17 9 1 2024-03-23T08:23:18.642 INFO:tasks.workunit.client.1.smithi193.stdout:3/163: mknod d1/de/d13/d21/d27/c36 0 2024-03-23T08:23:18.643 INFO:tasks.workunit.client.1.smithi193.stdout:3/164: read d1/fa [1128128,88511] 0 2024-03-23T08:23:18.644 INFO:tasks.workunit.client.1.smithi193.stdout:5/202: mkdir d0/d32/d45 0 2024-03-23T08:23:18.645 INFO:tasks.workunit.client.1.smithi193.stdout:3/165: fdatasync d1/de/d13/d21/d25/d2b/f35 0 2024-03-23T08:23:18.645 INFO:tasks.workunit.client.1.smithi193.stdout:5/203: chown d0/l3f 808839 1 2024-03-23T08:23:18.653 INFO:tasks.workunit.client.1.smithi193.stdout:4/205: write d11/d19/d22/f25 [2839847,114844] 0 2024-03-23T08:23:18.654 INFO:tasks.workunit.client.1.smithi193.stdout:3/166: mkdir d1/de/d13/d21/d37 0 2024-03-23T08:23:18.655 INFO:tasks.workunit.client.1.smithi193.stdout:8/184: mknod df/d10/d3e/c43 0 2024-03-23T08:23:18.658 INFO:tasks.workunit.client.1.smithi193.stdout:6/176: creat d0/db/f33 x:0 0 0 2024-03-23T08:23:18.660 INFO:tasks.workunit.client.1.smithi193.stdout:6/177: read - d0/d8/d12/f1e zero size 2024-03-23T08:23:18.661 INFO:tasks.workunit.client.1.smithi193.stdout:5/204: rename d0/d2f/l42 to d0/l46 0 2024-03-23T08:23:18.662 INFO:tasks.workunit.client.1.smithi193.stdout:2/279: truncate d0/d1b/d19/f1a 2671954 0 2024-03-23T08:23:18.662 INFO:tasks.workunit.client.1.smithi193.stdout:8/185: dwrite f0 [8388608,4194304] 0 2024-03-23T08:23:18.662 INFO:tasks.workunit.client.1.smithi193.stdout:7/210: getdents d6/d9/d1f/d23/d33/d3c 0 2024-03-23T08:23:18.664 INFO:tasks.workunit.client.1.smithi193.stdout:7/211: readlink d6/l17 0 2024-03-23T08:23:18.668 INFO:tasks.workunit.client.1.smithi193.stdout:4/206: rename d11/d1a/c1b to d11/d2d/c41 0 2024-03-23T08:23:18.668 INFO:tasks.workunit.client.1.smithi193.stdout:6/178: mkdir d0/db/d34 0 2024-03-23T08:23:18.668 INFO:tasks.workunit.client.1.smithi193.stdout:8/186: chown df/d13/d2a/c2d 253912 1 2024-03-23T08:23:18.668 INFO:tasks.workunit.client.1.smithi193.stdout:7/212: write d6/d9/f21 [204796,102975] 0 2024-03-23T08:23:18.668 INFO:tasks.workunit.client.1.smithi193.stdout:8/187: readlink le 0 2024-03-23T08:23:18.668 INFO:tasks.workunit.client.1.smithi193.stdout:5/205: mknod d0/c47 0 2024-03-23T08:23:18.673 INFO:tasks.workunit.client.1.smithi193.stdout:5/206: creat d0/d32/f48 x:0 0 0 2024-03-23T08:23:18.674 INFO:tasks.workunit.client.1.smithi193.stdout:0/251: dwrite db/d11/d14/d21/f2c [0,4194304] 0 2024-03-23T08:23:18.674 INFO:tasks.workunit.client.1.smithi193.stdout:3/167: rename d1/l3 to d1/de/d13/d21/d27/l38 0 2024-03-23T08:23:18.675 INFO:tasks.workunit.client.1.smithi193.stdout:2/280: link d0/d1b/d19/c35 d0/d37/c4d 0 2024-03-23T08:23:18.683 INFO:tasks.workunit.client.1.smithi193.stdout:4/207: rename d11/d1f to d11/d1a/d33/d42 0 2024-03-23T08:23:18.684 INFO:tasks.workunit.client.1.smithi193.stdout:6/179: dwrite d0/d8/d12/f1e [0,4194304] 0 2024-03-23T08:23:18.684 INFO:tasks.workunit.client.1.smithi193.stdout:8/188: creat df/f44 x:0 0 0 2024-03-23T08:23:18.687 INFO:tasks.workunit.client.1.smithi193.stdout:2/281: mkdir d0/d1b/d14/d25/d2a/d4e 0 2024-03-23T08:23:18.687 INFO:tasks.workunit.client.1.smithi193.stdout:6/180: chown d0/db/df/d11/f30 108859 1 2024-03-23T08:23:18.690 INFO:tasks.workunit.client.1.smithi193.stdout:7/213: rename d6/d9/fe to d6/d9/d1f/d23/d33/d3c/d41/f4a 0 2024-03-23T08:23:18.692 INFO:tasks.workunit.client.1.smithi193.stdout:8/189: creat df/d13/f45 x:0 0 0 2024-03-23T08:23:18.694 INFO:tasks.workunit.client.1.smithi193.stdout:0/252: write db/d11/f12 [1669348,128428] 0 2024-03-23T08:23:18.695 INFO:tasks.workunit.client.1.smithi193.stdout:8/190: dread - df/f33 zero size 2024-03-23T08:23:18.695 INFO:tasks.workunit.client.1.smithi193.stdout:5/207: rename d0/f3 to d0/d1f/f49 0 2024-03-23T08:23:18.696 INFO:tasks.workunit.client.1.smithi193.stdout:3/168: dwrite d1/d22/f2d [0,4194304] 0 2024-03-23T08:23:18.707 INFO:tasks.workunit.client.1.smithi193.stdout:2/282: creat d0/f4f x:0 0 0 2024-03-23T08:23:18.707 INFO:tasks.workunit.client.1.smithi193.stdout:3/169: creat d1/de/d13/d21/d25/d2b/f39 x:0 0 0 2024-03-23T08:23:18.707 INFO:tasks.workunit.client.1.smithi193.stdout:6/181: rename d0/d8/d12/d21/f2c to d0/d8/d12/f35 0 2024-03-23T08:23:18.707 INFO:tasks.workunit.client.1.smithi193.stdout:4/208: link d11/l3e d11/d1a/d33/d3c/l43 0 2024-03-23T08:23:18.707 INFO:tasks.workunit.client.1.smithi193.stdout:8/191: mkdir df/d46 0 2024-03-23T08:23:18.707 INFO:tasks.workunit.client.1.smithi193.stdout:2/283: fsync d0/d1b/d14/d25/f32 0 2024-03-23T08:23:18.707 INFO:tasks.workunit.client.1.smithi193.stdout:0/253: creat db/d11/d14/d51/f56 x:0 0 0 2024-03-23T08:23:18.708 INFO:tasks.workunit.client.1.smithi193.stdout:3/170: symlink d1/de/d13/d21/l3a 0 2024-03-23T08:23:18.709 INFO:tasks.workunit.client.1.smithi193.stdout:4/209: mkdir d11/d1a/d33/d44 0 2024-03-23T08:23:18.709 INFO:tasks.workunit.client.1.smithi193.stdout:5/208: dread d0/d1f/f22 [0,4194304] 0 2024-03-23T08:23:18.711 INFO:tasks.workunit.client.1.smithi193.stdout:6/182: rename d0/db/d34 to d0/db/df/d11/d36 0 2024-03-23T08:23:18.711 INFO:tasks.workunit.client.1.smithi193.stdout:8/192: creat df/d46/f47 x:0 0 0 2024-03-23T08:23:18.712 INFO:tasks.workunit.client.1.smithi193.stdout:4/210: write d11/d1a/d33/f39 [4302152,6780] 0 2024-03-23T08:23:18.714 INFO:tasks.workunit.client.1.smithi193.stdout:3/171: truncate d1/de/f10 4252451 0 2024-03-23T08:23:18.715 INFO:tasks.workunit.client.1.smithi193.stdout:6/183: creat d0/db/df/d11/f37 x:0 0 0 2024-03-23T08:23:18.715 INFO:tasks.workunit.client.1.smithi193.stdout:5/209: getdents d0/d32/d45 0 2024-03-23T08:23:18.716 INFO:tasks.workunit.client.1.smithi193.stdout:5/210: dread - d0/d32/f44 zero size 2024-03-23T08:23:18.718 INFO:tasks.workunit.client.1.smithi193.stdout:3/172: symlink d1/de/l3b 0 2024-03-23T08:23:18.720 INFO:tasks.workunit.client.1.smithi193.stdout:3/173: chown d1/de/f16 235733054 1 2024-03-23T08:23:18.720 INFO:tasks.workunit.client.1.smithi193.stdout:0/254: link db/d11/c35 db/d11/d14/d21/c57 0 2024-03-23T08:23:18.721 INFO:tasks.workunit.client.1.smithi193.stdout:5/211: mkdir d0/d32/d4a 0 2024-03-23T08:23:18.723 INFO:tasks.workunit.client.1.smithi193.stdout:4/211: link fd d11/d19/d22/f45 0 2024-03-23T08:23:18.726 INFO:tasks.workunit.client.1.smithi193.stdout:0/255: symlink db/d11/d14/d51/l58 0 2024-03-23T08:23:18.726 INFO:tasks.workunit.client.1.smithi193.stdout:7/214: truncate d6/d9/f35 351165 0 2024-03-23T08:23:18.726 INFO:tasks.workunit.client.1.smithi193.stdout:4/212: chown f4 7 1 2024-03-23T08:23:18.726 INFO:tasks.workunit.client.1.smithi193.stdout:6/184: dwrite d0/db/df/f25 [0,4194304] 0 2024-03-23T08:23:18.727 INFO:tasks.workunit.client.1.smithi193.stdout:5/212: mkdir d0/d1c/d3b/d4b 0 2024-03-23T08:23:18.729 INFO:tasks.workunit.client.1.smithi193.stdout:4/213: stat l7 0 2024-03-23T08:23:18.730 INFO:tasks.workunit.client.1.smithi193.stdout:6/185: fsync d0/d8/d12/f1b 0 2024-03-23T08:23:18.730 INFO:tasks.workunit.client.1.smithi193.stdout:8/193: rmdir df/d46 39 2024-03-23T08:23:18.731 INFO:tasks.workunit.client.1.smithi193.stdout:5/213: chown d0/f12 25657 1 2024-03-23T08:23:18.732 INFO:tasks.workunit.client.1.smithi193.stdout:8/194: stat df/d10/f29 0 2024-03-23T08:23:18.733 INFO:tasks.workunit.client.1.smithi193.stdout:3/174: dwrite d1/de/f11 [0,4194304] 0 2024-03-23T08:23:18.735 INFO:tasks.workunit.client.1.smithi193.stdout:7/215: creat d6/d9/d1f/d23/d33/d3c/f4b x:0 0 0 2024-03-23T08:23:18.736 INFO:tasks.workunit.client.1.smithi193.stdout:4/214: rmdir d11/d1a/d33/d3c 39 2024-03-23T08:23:18.738 INFO:tasks.workunit.client.1.smithi193.stdout:7/216: fsync d6/d9/d1f/d23/f45 0 2024-03-23T08:23:18.739 INFO:tasks.workunit.client.1.smithi193.stdout:3/175: mkdir d1/de/d13/d21/d25/d2b/d3c 0 2024-03-23T08:23:18.739 INFO:tasks.workunit.client.1.smithi193.stdout:5/214: mkdir d0/d1c/d3b/d4b/d4c 0 2024-03-23T08:23:18.740 INFO:tasks.workunit.client.1.smithi193.stdout:8/195: write df/d10/d19/f26 [1244609,61002] 0 2024-03-23T08:23:18.741 INFO:tasks.workunit.client.1.smithi193.stdout:4/215: mknod d11/d1a/d33/d44/c46 0 2024-03-23T08:23:18.746 INFO:tasks.workunit.client.1.smithi193.stdout:5/215: write d0/d1c/f3a [1220138,37144] 0 2024-03-23T08:23:18.747 INFO:tasks.workunit.client.1.smithi193.stdout:6/186: dread d0/d8/d12/f1e [0,4194304] 0 2024-03-23T08:23:18.749 INFO:tasks.workunit.client.1.smithi193.stdout:5/216: chown d0/d32 533839 1 2024-03-23T08:23:18.749 INFO:tasks.workunit.client.1.smithi193.stdout:4/216: symlink d11/d1a/l47 0 2024-03-23T08:23:18.750 INFO:tasks.workunit.client.1.smithi193.stdout:8/196: creat df/f48 x:0 0 0 2024-03-23T08:23:18.753 INFO:tasks.workunit.client.1.smithi193.stdout:7/217: write d6/d9/d1f/d23/d33/d38/d3b/f48 [1432391,127260] 0 2024-03-23T08:23:18.753 INFO:tasks.workunit.client.1.smithi193.stdout:5/217: chown d0/d1f/f22 47102424 1 2024-03-23T08:23:18.753 INFO:tasks.workunit.client.1.smithi193.stdout:8/197: write df/f34 [937585,19679] 0 2024-03-23T08:23:18.755 INFO:tasks.workunit.client.1.smithi193.stdout:6/187: write d0/d8/d12/f1b [1078289,68795] 0 2024-03-23T08:23:18.755 INFO:tasks.workunit.client.1.smithi193.stdout:4/217: mkdir d11/d48 0 2024-03-23T08:23:18.758 INFO:tasks.workunit.client.1.smithi193.stdout:8/198: chown df/d13 6033 1 2024-03-23T08:23:18.759 INFO:tasks.workunit.client.1.smithi193.stdout:3/176: dwrite d1/de/d13/f33 [0,4194304] 0 2024-03-23T08:23:18.761 INFO:tasks.workunit.client.1.smithi193.stdout:6/188: mknod d0/db/df/d11/c38 0 2024-03-23T08:23:18.765 INFO:tasks.workunit.client.1.smithi193.stdout:0/256: dread db/d11/d14/d21/f26 [0,4194304] 0 2024-03-23T08:23:18.767 INFO:tasks.workunit.client.1.smithi193.stdout:8/199: write df/f38 [1834537,65021] 0 2024-03-23T08:23:18.770 INFO:tasks.workunit.client.1.smithi193.stdout:4/218: rename d11/d1a/d33/d3b to d11/d1a/d33/d3c/d49 0 2024-03-23T08:23:18.771 INFO:tasks.workunit.client.1.smithi193.stdout:6/189: write d0/fe [1168539,119623] 0 2024-03-23T08:23:18.771 INFO:tasks.workunit.client.1.smithi193.stdout:8/200: rmdir df/d46 39 2024-03-23T08:23:18.772 INFO:tasks.workunit.client.1.smithi193.stdout:0/257: mkdir db/d11/d14/d24/d47/d59 0 2024-03-23T08:23:18.774 INFO:tasks.workunit.client.1.smithi193.stdout:4/219: truncate d11/d19/d22/f2b 658753 0 2024-03-23T08:23:18.774 INFO:tasks.workunit.client.1.smithi193.stdout:7/218: dwrite d6/f2d [0,4194304] 0 2024-03-23T08:23:18.774 INFO:tasks.workunit.client.1.smithi193.stdout:0/258: fdatasync db/f1e 0 2024-03-23T08:23:18.780 INFO:tasks.workunit.client.1.smithi193.stdout:4/220: chown d11/d19/d22 115078 1 2024-03-23T08:23:18.782 INFO:tasks.workunit.client.1.smithi193.stdout:5/218: dread d0/fe [0,4194304] 0 2024-03-23T08:23:18.783 INFO:tasks.workunit.client.1.smithi193.stdout:5/219: chown d0/d1f/f28 98903513 1 2024-03-23T08:23:18.784 INFO:tasks.workunit.client.1.smithi193.stdout:6/190: rename d0/db/df/d11/c38 to d0/d8/d12/d21/c39 0 2024-03-23T08:23:18.784 INFO:tasks.workunit.client.1.smithi193.stdout:7/219: dread d6/f2d [0,4194304] 0 2024-03-23T08:23:18.786 INFO:tasks.workunit.client.1.smithi193.stdout:0/259: chown db/d11/d14/d21/c37 14093924 1 2024-03-23T08:23:18.789 INFO:tasks.workunit.client.1.smithi193.stdout:4/221: rename d11/f2e to d11/d1a/d33/d44/f4a 0 2024-03-23T08:23:18.790 INFO:tasks.workunit.client.1.smithi193.stdout:4/222: fsync d11/d1a/d37/f3f 0 2024-03-23T08:23:18.793 INFO:tasks.workunit.client.1.smithi193.stdout:0/260: dread db/d11/f13 [0,4194304] 0 2024-03-23T08:23:18.797 INFO:tasks.workunit.client.1.smithi193.stdout:4/223: creat d11/d2d/f4b x:0 0 0 2024-03-23T08:23:18.802 INFO:tasks.workunit.client.1.smithi193.stdout:5/220: rename d0/d1c/f26 to d0/f4d 0 2024-03-23T08:23:18.806 INFO:tasks.workunit.client.1.smithi193.stdout:0/261: rename db/d11/c2b to db/d11/d14/d24/d2a/c5a 0 2024-03-23T08:23:18.809 INFO:tasks.workunit.client.1.smithi193.stdout:0/262: dread db/f1e [0,4194304] 0 2024-03-23T08:23:18.813 INFO:tasks.workunit.client.1.smithi193.stdout:5/221: dwrite d0/d1f/f41 [0,4194304] 0 2024-03-23T08:23:18.814 INFO:tasks.workunit.client.1.smithi193.stdout:0/263: symlink db/d11/d14/d24/d47/l5b 0 2024-03-23T08:23:18.823 INFO:tasks.workunit.client.1.smithi193.stdout:6/191: truncate d0/d8/d12/d21/f29 2414861 0 2024-03-23T08:23:18.824 INFO:tasks.workunit.client.1.smithi193.stdout:9/255: sync 2024-03-23T08:23:18.824 INFO:tasks.workunit.client.1.smithi193.stdout:1/228: sync 2024-03-23T08:23:18.825 INFO:tasks.workunit.client.1.smithi193.stdout:4/224: rmdir d11 39 2024-03-23T08:23:18.826 INFO:tasks.workunit.client.1.smithi193.stdout:5/222: fsync d0/d1f/f41 0 2024-03-23T08:23:18.826 INFO:tasks.workunit.client.1.smithi193.stdout:0/264: dwrite db/d11/f13 [0,4194304] 0 2024-03-23T08:23:18.830 INFO:tasks.workunit.client.1.smithi193.stdout:9/256: creat d2/d7/f56 x:0 0 0 2024-03-23T08:23:18.831 INFO:tasks.workunit.client.1.smithi193.stdout:9/257: fdatasync d2/d7/f1e 0 2024-03-23T08:23:18.832 INFO:tasks.workunit.client.1.smithi193.stdout:1/229: creat da/dd/df/d2e/f4f x:0 0 0 2024-03-23T08:23:18.852 INFO:tasks.workunit.client.1.smithi193.stdout:6/192: getdents d0/d27 0 2024-03-23T08:23:18.853 INFO:tasks.workunit.client.1.smithi193.stdout:1/230: dread - da/d13/f49 zero size 2024-03-23T08:23:18.854 INFO:tasks.workunit.client.1.smithi193.stdout:9/258: dread d2/d7/db/d32/f4e [0,4194304] 0 2024-03-23T08:23:18.855 INFO:tasks.workunit.client.1.smithi193.stdout:0/265: creat db/d52/f5c x:0 0 0 2024-03-23T08:23:18.855 INFO:tasks.workunit.client.1.smithi193.stdout:6/193: dread - d0/db/df/d11/f37 zero size 2024-03-23T08:23:18.856 INFO:tasks.workunit.client.1.smithi193.stdout:4/225: mkdir d11/d48/d4c 0 2024-03-23T08:23:18.856 INFO:tasks.workunit.client.1.smithi193.stdout:0/266: stat db/d11/d14/c3a 0 2024-03-23T08:23:18.857 INFO:tasks.workunit.client.1.smithi193.stdout:4/226: chown f4 60780 1 2024-03-23T08:23:18.858 INFO:tasks.workunit.client.1.smithi193.stdout:4/227: fsync f9 0 2024-03-23T08:23:18.860 INFO:tasks.workunit.client.1.smithi193.stdout:1/231: rename da/dd/df/d2e/f4d to da/dd/df/d2e/f50 0 2024-03-23T08:23:18.865 INFO:tasks.workunit.client.1.smithi193.stdout:6/194: dwrite d0/d8/d12/f35 [0,4194304] 0 2024-03-23T08:23:18.865 INFO:tasks.workunit.client.1.smithi193.stdout:0/267: dread db/d11/d14/f19 [0,4194304] 0 2024-03-23T08:23:18.866 INFO:tasks.workunit.client.1.smithi193.stdout:4/228: creat d11/d1a/d33/d42/f4d x:0 0 0 2024-03-23T08:23:18.866 INFO:tasks.workunit.client.1.smithi193.stdout:5/223: dread d0/d2f/f38 [0,4194304] 0 2024-03-23T08:23:18.870 INFO:tasks.workunit.client.1.smithi193.stdout:1/232: mknod da/dd/df/d2e/c51 0 2024-03-23T08:23:18.871 INFO:tasks.workunit.client.1.smithi193.stdout:1/233: fdatasync da/d13/d3f/f41 0 2024-03-23T08:23:18.878 INFO:tasks.workunit.client.1.smithi193.stdout:4/229: mknod d11/d1a/d33/d42/c4e 0 2024-03-23T08:23:18.880 INFO:tasks.workunit.client.1.smithi193.stdout:5/224: rename d0/l18 to d0/d1c/l4e 0 2024-03-23T08:23:18.882 INFO:tasks.workunit.client.1.smithi193.stdout:9/259: link d2/d7/db/d17/c55 d2/d7/db/d17/d1f/c57 0 2024-03-23T08:23:18.883 INFO:tasks.workunit.client.1.smithi193.stdout:0/268: dwrite db/d11/d14/f19 [0,4194304] 0 2024-03-23T08:23:18.883 INFO:tasks.workunit.client.1.smithi193.stdout:6/195: mkdir d0/d8/d3a 0 2024-03-23T08:23:18.884 INFO:tasks.workunit.client.1.smithi193.stdout:6/196: chown d0/d8/f1d 45 1 2024-03-23T08:23:18.886 INFO:tasks.workunit.client.1.smithi193.stdout:4/230: creat d11/d1a/d33/d3c/d49/f4f x:0 0 0 2024-03-23T08:23:18.889 INFO:tasks.workunit.client.1.smithi193.stdout:1/234: mkdir da/dd/df/d52 0 2024-03-23T08:23:18.891 INFO:tasks.workunit.client.1.smithi193.stdout:1/235: dread - da/dd/df/d2e/f50 zero size 2024-03-23T08:23:18.893 INFO:tasks.workunit.client.1.smithi193.stdout:9/260: chown d2/c3d 5897 1 2024-03-23T08:23:18.893 INFO:tasks.workunit.client.1.smithi193.stdout:1/236: chown da/dd/f30 5556587 1 2024-03-23T08:23:18.894 INFO:tasks.workunit.client.1.smithi193.stdout:4/231: dwrite d11/d19/d22/f25 [4194304,4194304] 0 2024-03-23T08:23:18.894 INFO:tasks.workunit.client.1.smithi193.stdout:0/269: fdatasync db/d11/d14/d21/f26 0 2024-03-23T08:23:18.896 INFO:tasks.workunit.client.1.smithi193.stdout:6/197: chown d0/l4 1 1 2024-03-23T08:23:18.903 INFO:tasks.workunit.client.1.smithi193.stdout:4/232: dread d11/d19/d22/f26 [0,4194304] 0 2024-03-23T08:23:18.909 INFO:tasks.workunit.client.1.smithi193.stdout:6/198: mkdir d0/db/df/d11/d3b 0 2024-03-23T08:23:18.913 INFO:tasks.workunit.client.1.smithi193.stdout:9/261: write d2/fc [2980612,88050] 0 2024-03-23T08:23:18.913 INFO:tasks.workunit.client.1.smithi193.stdout:0/270: dread db/d11/d14/d21/f27 [0,4194304] 0 2024-03-23T08:23:18.914 INFO:tasks.workunit.client.1.smithi193.stdout:0/271: readlink db/d11/d14/d51/l58 0 2024-03-23T08:23:18.915 INFO:tasks.workunit.client.1.smithi193.stdout:0/272: chown db/d11/d14/d24/d47/l5b 15630871 1 2024-03-23T08:23:18.918 INFO:tasks.workunit.client.1.smithi193.stdout:4/233: rename d11/d1a/d33/d3c to d11/d48/d50 0 2024-03-23T08:23:18.919 INFO:tasks.workunit.client.1.smithi193.stdout:9/262: dwrite d2/f3 [0,4194304] 0 2024-03-23T08:23:18.920 INFO:tasks.workunit.client.1.smithi193.stdout:4/234: truncate d11/d19/d22/f2b 982328 0 2024-03-23T08:23:18.923 INFO:tasks.workunit.client.1.smithi193.stdout:5/225: getdents d0/d32 0 2024-03-23T08:23:18.929 INFO:tasks.workunit.client.1.smithi193.stdout:9/263: dwrite d2/d7/d31/f4f [0,4194304] 0 2024-03-23T08:23:18.929 INFO:tasks.workunit.client.1.smithi193.stdout:4/235: dread f4 [0,4194304] 0 2024-03-23T08:23:18.930 INFO:tasks.workunit.client.1.smithi193.stdout:4/236: chown fb 1708 1 2024-03-23T08:23:18.930 INFO:tasks.workunit.client.1.smithi193.stdout:4/237: stat d11/d1a/l47 0 2024-03-23T08:23:18.930 INFO:tasks.workunit.client.1.smithi193.stdout:5/226: mkdir d0/d2f/d4f 0 2024-03-23T08:23:18.932 INFO:tasks.workunit.client.1.smithi193.stdout:5/227: read d0/f7 [2526956,20381] 0 2024-03-23T08:23:18.934 INFO:tasks.workunit.client.1.smithi193.stdout:5/228: write d0/f43 [612521,107634] 0 2024-03-23T08:23:18.936 INFO:tasks.workunit.client.1.smithi193.stdout:5/229: read d0/d1c/f1e [3983655,123781] 0 2024-03-23T08:23:18.936 INFO:tasks.workunit.client.1.smithi193.stdout:9/264: chown d2/l18 37 1 2024-03-23T08:23:18.939 INFO:tasks.workunit.client.1.smithi193.stdout:9/265: write d2/d7/f15 [1451571,64536] 0 2024-03-23T08:23:18.942 INFO:tasks.workunit.client.1.smithi193.stdout:4/238: dwrite d11/d19/d22/f2b [0,4194304] 0 2024-03-23T08:23:18.944 INFO:tasks.workunit.client.1.smithi193.stdout:5/230: mknod d0/d32/d45/c50 0 2024-03-23T08:23:18.946 INFO:tasks.workunit.client.1.smithi193.stdout:9/266: symlink d2/d7/db/d32/l58 0 2024-03-23T08:23:18.949 INFO:tasks.workunit.client.1.smithi193.stdout:5/231: chown d0/c47 3 1 2024-03-23T08:23:18.950 INFO:tasks.workunit.client.1.smithi193.stdout:6/199: dwrite d0/d8/f19 [0,4194304] 0 2024-03-23T08:23:18.952 INFO:tasks.workunit.client.1.smithi193.stdout:9/267: mkdir d2/d7/db/d17/d4b/d59 0 2024-03-23T08:23:18.953 INFO:tasks.workunit.client.1.smithi193.stdout:2/284: sync 2024-03-23T08:23:18.956 INFO:tasks.workunit.client.1.smithi193.stdout:9/268: chown d2/d7/db/f2e 10894463 1 2024-03-23T08:23:18.957 INFO:tasks.workunit.client.1.smithi193.stdout:4/239: dwrite d11/d19/d22/f31 [4194304,4194304] 0 2024-03-23T08:23:18.957 INFO:tasks.workunit.client.1.smithi193.stdout:6/200: symlink d0/d8/d12/d21/d2a/l3c 0 2024-03-23T08:23:18.960 INFO:tasks.workunit.client.1.smithi193.stdout:2/285: mkdir d0/d1b/d14/d25/d2a/d50 0 2024-03-23T08:23:18.963 INFO:tasks.workunit.client.1.smithi193.stdout:2/286: fsync d0/d1b/d3b/f46 0 2024-03-23T08:23:18.964 INFO:tasks.workunit.client.1.smithi193.stdout:5/232: dwrite d0/f14 [0,4194304] 0 2024-03-23T08:23:18.967 INFO:tasks.workunit.client.1.smithi193.stdout:5/233: write d0/f27 [184129,64828] 0 2024-03-23T08:23:18.969 INFO:tasks.workunit.client.1.smithi193.stdout:6/201: dwrite d0/db/f33 [0,4194304] 0 2024-03-23T08:23:18.970 INFO:tasks.workunit.client.1.smithi193.stdout:4/240: truncate fd 3492948 0 2024-03-23T08:23:19.041 INFO:tasks.workunit.client.1.smithi193.stdout:9/269: getdents d2/d7/db/d32 0 2024-03-23T08:23:19.042 INFO:tasks.workunit.client.1.smithi193.stdout:6/202: mknod d0/d8/d3a/c3d 0 2024-03-23T08:23:19.044 INFO:tasks.workunit.client.1.smithi193.stdout:2/287: dwrite d0/d1b/d3b/f46 [0,4194304] 0 2024-03-23T08:23:19.044 INFO:tasks.workunit.client.1.smithi193.stdout:4/241: link d11/d48/d50/l3d d11/d1a/d37/l51 0 2024-03-23T08:23:19.078 INFO:tasks.workunit.client.1.smithi193.stdout:3/177: sync 2024-03-23T08:23:19.079 INFO:tasks.workunit.client.1.smithi193.stdout:6/203: fsync d0/d8/d12/f35 0 2024-03-23T08:23:19.080 INFO:tasks.workunit.client.1.smithi193.stdout:5/234: dwrite d0/f11 [0,4194304] 0 2024-03-23T08:23:19.083 INFO:tasks.workunit.client.1.smithi193.stdout:2/288: write d0/fc [8541076,127675] 0 2024-03-23T08:23:19.083 INFO:tasks.workunit.client.1.smithi193.stdout:4/242: dread - d11/d48/d50/d49/f4f zero size 2024-03-23T08:23:19.083 INFO:tasks.workunit.client.1.smithi193.stdout:3/178: mkdir d1/de/d3d 0 2024-03-23T08:23:19.087 INFO:tasks.workunit.client.1.smithi193.stdout:6/204: write d0/db/df/d11/f2e [108297,61060] 0 2024-03-23T08:23:19.090 INFO:tasks.workunit.client.1.smithi193.stdout:6/205: write d0/d8/f1d [3838822,98638] 0 2024-03-23T08:23:19.097 INFO:tasks.workunit.client.1.smithi193.stdout:2/289: dwrite d0/f2d [4194304,4194304] 0 2024-03-23T08:23:19.097 INFO:tasks.workunit.client.1.smithi193.stdout:9/270: dread d2/fc [0,4194304] 0 2024-03-23T08:23:19.098 INFO:tasks.workunit.client.1.smithi193.stdout:2/290: truncate d0/d1b/d14/f41 982225 0 2024-03-23T08:23:19.098 INFO:tasks.workunit.client.1.smithi193.stdout:4/243: rename d11/d1a/d37 to d11/d1a/d33/d42/d52 0 2024-03-23T08:23:19.099 INFO:tasks.workunit.client.1.smithi193.stdout:6/206: mknod d0/db/df/d11/d36/c3e 0 2024-03-23T08:23:19.099 INFO:tasks.workunit.client.1.smithi193.stdout:3/179: symlink d1/de/d13/d21/d25/d2b/d3c/l3e 0 2024-03-23T08:23:19.102 INFO:tasks.workunit.client.1.smithi193.stdout:9/271: rename d2/f3 to d2/d7/db/d32/f5a 0 2024-03-23T08:23:19.104 INFO:tasks.workunit.client.1.smithi193.stdout:2/291: symlink d0/d1b/l51 0 2024-03-23T08:23:19.106 INFO:tasks.workunit.client.1.smithi193.stdout:5/235: dread d0/d1c/f20 [0,4194304] 0 2024-03-23T08:23:19.106 INFO:tasks.workunit.client.1.smithi193.stdout:9/272: mkdir d2/d7/db/d17/d1f/d5b 0 2024-03-23T08:23:19.108 INFO:tasks.workunit.client.1.smithi193.stdout:6/207: dread d0/db/f20 [0,4194304] 0 2024-03-23T08:23:19.109 INFO:tasks.workunit.client.1.smithi193.stdout:2/292: fsync d0/d1b/d14/d25/f3a 0 2024-03-23T08:23:19.110 INFO:tasks.workunit.client.1.smithi193.stdout:5/236: fsync d0/fa 0 2024-03-23T08:23:19.111 INFO:tasks.workunit.client.1.smithi193.stdout:9/273: chown d2/la 10026583 1 2024-03-23T08:23:19.114 INFO:tasks.workunit.client.1.smithi193.stdout:9/274: fsync d2/d7/f56 0 2024-03-23T08:23:19.114 INFO:tasks.workunit.client.1.smithi193.stdout:5/237: creat d0/d1f/f51 x:0 0 0 2024-03-23T08:23:19.115 INFO:tasks.workunit.client.1.smithi193.stdout:4/244: dwrite d11/d1a/d33/d42/f29 [4194304,4194304] 0 2024-03-23T08:23:19.115 INFO:tasks.workunit.client.1.smithi193.stdout:3/180: dwrite d1/de/d13/f33 [0,4194304] 0 2024-03-23T08:23:19.118 INFO:tasks.workunit.client.1.smithi193.stdout:4/245: chown d11/d48/d4c 394929 1 2024-03-23T08:23:19.118 INFO:tasks.workunit.client.1.smithi193.stdout:1/237: sync 2024-03-23T08:23:19.118 INFO:tasks.workunit.client.1.smithi193.stdout:8/201: sync 2024-03-23T08:23:19.118 INFO:tasks.workunit.client.1.smithi193.stdout:7/220: sync 2024-03-23T08:23:19.119 INFO:tasks.workunit.client.1.smithi193.stdout:5/238: write d0/f15 [1716998,74820] 0 2024-03-23T08:23:19.121 INFO:tasks.workunit.client.1.smithi193.stdout:2/293: creat d0/d1b/d14/d25/d2a/d50/f52 x:0 0 0 2024-03-23T08:23:19.122 INFO:tasks.workunit.client.1.smithi193.stdout:9/275: mkdir d2/d7/db/d1b/d5c 0 2024-03-23T08:23:19.124 INFO:tasks.workunit.client.1.smithi193.stdout:8/202: chown df/d46 15302 1 2024-03-23T08:23:19.126 INFO:tasks.workunit.client.1.smithi193.stdout:5/239: chown d0/d1f/f2e 0 1 2024-03-23T08:23:19.126 INFO:tasks.workunit.client.1.smithi193.stdout:4/246: creat d11/d2d/f53 x:0 0 0 2024-03-23T08:23:19.127 INFO:tasks.workunit.client.1.smithi193.stdout:7/221: unlink d6/d9/d1f/l20 0 2024-03-23T08:23:19.128 INFO:tasks.workunit.client.1.smithi193.stdout:6/208: dwrite d0/fe [0,4194304] 0 2024-03-23T08:23:19.129 INFO:tasks.workunit.client.1.smithi193.stdout:2/294: mknod d0/d1b/d14/d25/c53 0 2024-03-23T08:23:19.129 INFO:tasks.workunit.client.1.smithi193.stdout:4/247: fdatasync d11/d19/f20 0 2024-03-23T08:23:19.131 INFO:tasks.workunit.client.1.smithi193.stdout:6/209: chown d0/d8/d12/d21 104688 1 2024-03-23T08:23:19.131 INFO:tasks.workunit.client.1.smithi193.stdout:8/203: rmdir df/d13/d2a 39 2024-03-23T08:23:19.134 INFO:tasks.workunit.client.1.smithi193.stdout:7/222: creat d6/d9/d1f/d23/d33/d3c/f4c x:0 0 0 2024-03-23T08:23:19.134 INFO:tasks.workunit.client.1.smithi193.stdout:4/248: readlink d11/d1a/l47 0 2024-03-23T08:23:19.134 INFO:tasks.workunit.client.1.smithi193.stdout:1/238: creat da/dd/df/d52/f53 x:0 0 0 2024-03-23T08:23:19.135 INFO:tasks.workunit.client.1.smithi193.stdout:8/204: fdatasync df/d10/f17 0 2024-03-23T08:23:19.136 INFO:tasks.workunit.client.1.smithi193.stdout:5/240: write d0/d1f/f49 [472971,39522] 0 2024-03-23T08:23:19.136 INFO:tasks.workunit.client.1.smithi193.stdout:2/295: creat d0/d1b/d3b/f54 x:0 0 0 2024-03-23T08:23:19.138 INFO:tasks.workunit.client.1.smithi193.stdout:6/210: write d0/d8/d12/f35 [3050288,35404] 0 2024-03-23T08:23:19.139 INFO:tasks.workunit.client.1.smithi193.stdout:2/296: fdatasync d0/d1b/f13 0 2024-03-23T08:23:19.139 INFO:tasks.workunit.client.1.smithi193.stdout:3/181: dread d1/de/d13/f33 [0,4194304] 0 2024-03-23T08:23:19.141 INFO:tasks.workunit.client.1.smithi193.stdout:1/239: creat da/d13/d3f/f54 x:0 0 0 2024-03-23T08:23:19.142 INFO:tasks.workunit.client.1.smithi193.stdout:7/223: write d6/d9/d1f/f30 [1059914,6205] 0 2024-03-23T08:23:19.143 INFO:tasks.workunit.client.1.smithi193.stdout:8/205: creat df/d13/d2a/f49 x:0 0 0 2024-03-23T08:23:19.143 INFO:tasks.workunit.client.1.smithi193.stdout:9/276: dread d2/d7/db/d32/f5a [0,4194304] 0 2024-03-23T08:23:19.144 INFO:tasks.workunit.client.1.smithi193.stdout:2/297: readlink d0/d1b/d14/d25/l3d 0 2024-03-23T08:23:19.145 INFO:tasks.workunit.client.1.smithi193.stdout:5/241: mknod d0/d1c/d3b/d4b/d4c/c52 0 2024-03-23T08:23:19.145 INFO:tasks.workunit.client.1.smithi193.stdout:3/182: mknod d1/de/d13/d21/d25/c3f 0 2024-03-23T08:23:19.148 INFO:tasks.workunit.client.1.smithi193.stdout:1/240: chown da/dd/f39 28 1 2024-03-23T08:23:19.149 INFO:tasks.workunit.client.1.smithi193.stdout:5/242: chown d0/d1c/f3a 5357493 1 2024-03-23T08:23:19.149 INFO:tasks.workunit.client.1.smithi193.stdout:2/298: chown d0/d1b/f21 12144 1 2024-03-23T08:23:19.149 INFO:tasks.workunit.client.1.smithi193.stdout:8/206: write df/d10/f27 [1046280,48961] 0 2024-03-23T08:23:19.151 INFO:tasks.workunit.client.1.smithi193.stdout:6/211: rename d0/db/df/d11/c1c to d0/d8/d12/d21/c3f 0 2024-03-23T08:23:19.152 INFO:tasks.workunit.client.1.smithi193.stdout:2/299: chown d0/d1b/d19 8617097 1 2024-03-23T08:23:19.153 INFO:tasks.workunit.client.1.smithi193.stdout:9/277: creat d2/d7/db/d17/d33/f5d x:0 0 0 2024-03-23T08:23:19.153 INFO:tasks.workunit.client.1.smithi193.stdout:4/249: dwrite d11/d2d/f38 [0,4194304] 0 2024-03-23T08:23:19.154 INFO:tasks.workunit.client.1.smithi193.stdout:5/243: creat d0/d1c/d3b/f53 x:0 0 0 2024-03-23T08:23:19.154 INFO:tasks.workunit.client.1.smithi193.stdout:9/278: stat d2/d7/f2b 0 2024-03-23T08:23:19.154 INFO:tasks.workunit.client.1.smithi193.stdout:2/300: readlink d0/d28/d33/l4b 0 2024-03-23T08:23:19.155 INFO:tasks.workunit.client.1.smithi193.stdout:3/183: mkdir d1/de/d13/d21/d25/d2b/d3c/d40 0 2024-03-23T08:23:19.156 INFO:tasks.workunit.client.1.smithi193.stdout:1/241: symlink da/dd/l55 0 2024-03-23T08:23:19.157 INFO:tasks.workunit.client.1.smithi193.stdout:4/250: rmdir d11/d48/d50 39 2024-03-23T08:23:19.158 INFO:tasks.workunit.client.1.smithi193.stdout:0/273: sync 2024-03-23T08:23:19.158 INFO:tasks.workunit.client.1.smithi193.stdout:7/224: rmdir d6/d9/d1f/d23/d33/d3c 39 2024-03-23T08:23:19.159 INFO:tasks.workunit.client.1.smithi193.stdout:3/184: chown d1/de/d13/d21/d27/c32 13 1 2024-03-23T08:23:19.159 INFO:tasks.workunit.client.1.smithi193.stdout:4/251: chown d11/d48 30 1 2024-03-23T08:23:19.162 INFO:tasks.workunit.client.1.smithi193.stdout:2/301: creat d0/d37/d48/f55 x:0 0 0 2024-03-23T08:23:19.162 INFO:tasks.workunit.client.1.smithi193.stdout:5/244: mknod d0/c54 0 2024-03-23T08:23:19.163 INFO:tasks.workunit.client.1.smithi193.stdout:1/242: creat da/d13/d3f/f56 x:0 0 0 2024-03-23T08:23:19.164 INFO:tasks.workunit.client.1.smithi193.stdout:6/212: creat d0/f40 x:0 0 0 2024-03-23T08:23:19.164 INFO:tasks.workunit.client.1.smithi193.stdout:1/243: readlink da/l33 0 2024-03-23T08:23:19.166 INFO:tasks.workunit.client.1.smithi193.stdout:5/245: write d0/d1c/f3a [908198,122819] 0 2024-03-23T08:23:19.169 INFO:tasks.workunit.client.1.smithi193.stdout:4/252: creat d11/d1a/d33/d44/f54 x:0 0 0 2024-03-23T08:23:19.169 INFO:tasks.workunit.client.1.smithi193.stdout:2/302: write d0/d1b/d14/d25/f29 [3411340,43832] 0 2024-03-23T08:23:19.171 INFO:tasks.workunit.client.1.smithi193.stdout:9/279: write d2/d7/f2b [494492,98498] 0 2024-03-23T08:23:19.171 INFO:tasks.workunit.client.1.smithi193.stdout:0/274: creat db/d11/d14/d21/d33/d48/f5d x:0 0 0 2024-03-23T08:23:19.171 INFO:tasks.workunit.client.1.smithi193.stdout:3/185: dread d1/de/f10 [0,4194304] 0 2024-03-23T08:23:19.172 INFO:tasks.workunit.client.1.smithi193.stdout:1/244: truncate da/f46 713505 0 2024-03-23T08:23:19.172 INFO:tasks.workunit.client.1.smithi193.stdout:8/207: link c2 df/d10/c4a 0 2024-03-23T08:23:19.173 INFO:tasks.workunit.client.1.smithi193.stdout:6/213: creat d0/db/df/d11/d36/f41 x:0 0 0 2024-03-23T08:23:19.175 INFO:tasks.workunit.client.1.smithi193.stdout:7/225: dread d6/f8 [0,4194304] 0 2024-03-23T08:23:19.175 INFO:tasks.workunit.client.1.smithi193.stdout:2/303: unlink d0/f4f 0 2024-03-23T08:23:19.175 INFO:tasks.workunit.client.1.smithi193.stdout:1/245: chown da/d13/d3f/f41 162503409 1 2024-03-23T08:23:19.176 INFO:tasks.workunit.client.1.smithi193.stdout:3/186: read - d1/de/d13/d21/d25/d2b/f39 zero size 2024-03-23T08:23:19.176 INFO:tasks.workunit.client.1.smithi193.stdout:6/214: read d0/db/df/d11/f2e [1253088,66723] 0 2024-03-23T08:23:19.178 INFO:tasks.workunit.client.1.smithi193.stdout:0/275: unlink db/d11/l38 0 2024-03-23T08:23:19.179 INFO:tasks.workunit.client.1.smithi193.stdout:1/246: read - da/dd/df/d2e/f50 zero size 2024-03-23T08:23:19.179 INFO:tasks.workunit.client.1.smithi193.stdout:4/253: symlink d11/d48/l55 0 2024-03-23T08:23:19.180 INFO:tasks.workunit.client.1.smithi193.stdout:6/215: read - d0/db/df/d11/f37 zero size 2024-03-23T08:23:19.181 INFO:tasks.workunit.client.1.smithi193.stdout:5/246: rename d0/d32/d4a to d0/d1c/d3b/d4b/d55 0 2024-03-23T08:23:19.181 INFO:tasks.workunit.client.1.smithi193.stdout:6/216: stat d0/d8/f19 0 2024-03-23T08:23:19.182 INFO:tasks.workunit.client.1.smithi193.stdout:1/247: write da/d13/d24/f43 [2247972,84281] 0 2024-03-23T08:23:19.182 INFO:tasks.workunit.client.1.smithi193.stdout:7/226: mkdir d6/d9/d1f/d4d 0 2024-03-23T08:23:19.183 INFO:tasks.workunit.client.1.smithi193.stdout:8/208: mkdir df/d4b 0 2024-03-23T08:23:19.184 INFO:tasks.workunit.client.1.smithi193.stdout:9/280: truncate d2/f25 1098308 0 2024-03-23T08:23:19.185 INFO:tasks.workunit.client.1.smithi193.stdout:0/276: symlink db/d11/d14/l5e 0 2024-03-23T08:23:19.186 INFO:tasks.workunit.client.1.smithi193.stdout:9/281: chown d2/d7/db/f13 11 1 2024-03-23T08:23:19.187 INFO:tasks.workunit.client.1.smithi193.stdout:3/187: truncate d1/de/d13/d21/f31 4044151 0 2024-03-23T08:23:19.188 INFO:tasks.workunit.client.1.smithi193.stdout:0/277: readlink db/d11/d14/l18 0 2024-03-23T08:23:19.189 INFO:tasks.workunit.client.1.smithi193.stdout:5/247: creat d0/d32/d45/f56 x:0 0 0 2024-03-23T08:23:19.190 INFO:tasks.workunit.client.1.smithi193.stdout:3/188: chown d1/lb 2072469 1 2024-03-23T08:23:19.191 INFO:tasks.workunit.client.1.smithi193.stdout:9/282: write d2/f28 [4237263,13137] 0 2024-03-23T08:23:19.191 INFO:tasks.workunit.client.1.smithi193.stdout:6/217: rename d0/d8/d3a/c3d to d0/db/df/d11/d3b/c42 0 2024-03-23T08:23:19.192 INFO:tasks.workunit.client.1.smithi193.stdout:4/254: symlink d11/d1a/d33/l56 0 2024-03-23T08:23:19.196 INFO:tasks.workunit.client.1.smithi193.stdout:2/304: dread d0/d1b/d14/f40 [0,4194304] 0 2024-03-23T08:23:19.196 INFO:tasks.workunit.client.1.smithi193.stdout:9/283: read d2/d7/db/f34 [613959,53509] 0 2024-03-23T08:23:19.198 INFO:tasks.workunit.client.1.smithi193.stdout:8/209: dwrite df/d13/d2a/f2b [0,4194304] 0 2024-03-23T08:23:19.198 INFO:tasks.workunit.client.1.smithi193.stdout:5/248: fsync d0/fa 0 2024-03-23T08:23:19.199 INFO:tasks.workunit.client.1.smithi193.stdout:7/227: dwrite d6/d9/f21 [0,4194304] 0 2024-03-23T08:23:19.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/305: dread - d0/d37/f4c zero size 2024-03-23T08:23:19.200 INFO:tasks.workunit.client.1.smithi193.stdout:5/249: stat d0/d1c/f3a 0 2024-03-23T08:23:19.202 INFO:tasks.workunit.client.1.smithi193.stdout:3/189: symlink d1/l41 0 2024-03-23T08:23:19.206 INFO:tasks.workunit.client.1.smithi193.stdout:9/284: read d2/d7/db/f3b [190705,86977] 0 2024-03-23T08:23:19.206 INFO:tasks.workunit.client.1.smithi193.stdout:5/250: chown d0/fe 736 1 2024-03-23T08:23:19.206 INFO:tasks.workunit.client.1.smithi193.stdout:2/306: write d0/d1b/d14/d25/f32 [593804,5697] 0 2024-03-23T08:23:19.206 INFO:tasks.workunit.client.1.smithi193.stdout:8/210: symlink df/d13/d2a/l4c 0 2024-03-23T08:23:19.211 INFO:tasks.workunit.client.1.smithi193.stdout:9/285: read d2/d7/db/d32/f3f [179607,12179] 0 2024-03-23T08:23:19.212 INFO:tasks.workunit.client.1.smithi193.stdout:0/278: dwrite db/d11/d14/d21/f49 [0,4194304] 0 2024-03-23T08:23:19.219 INFO:tasks.workunit.client.1.smithi193.stdout:6/218: fdatasync d0/d8/d12/d21/f29 0 2024-03-23T08:23:19.219 INFO:tasks.workunit.client.1.smithi193.stdout:3/190: symlink d1/de/d13/d21/d25/l42 0 2024-03-23T08:23:19.220 INFO:tasks.workunit.client.1.smithi193.stdout:9/286: write d2/d7/d31/f4f [2958484,122437] 0 2024-03-23T08:23:19.220 INFO:tasks.workunit.client.1.smithi193.stdout:5/251: dwrite d0/d1f/f51 [0,4194304] 0 2024-03-23T08:23:19.223 INFO:tasks.workunit.client.1.smithi193.stdout:1/248: write da/d13/f34 [1269464,27950] 0 2024-03-23T08:23:19.228 INFO:tasks.workunit.client.1.smithi193.stdout:7/228: unlink d6/d9/d1f/d23/d33/d3c/d41/d47/f49 0 2024-03-23T08:23:19.230 INFO:tasks.workunit.client.1.smithi193.stdout:1/249: write da/dd/df/d52/f53 [349230,35249] 0 2024-03-23T08:23:19.231 INFO:tasks.workunit.client.1.smithi193.stdout:2/307: getdents d0/d1b/d14/d25/d2a/d4e 0 2024-03-23T08:23:19.231 INFO:tasks.workunit.client.1.smithi193.stdout:9/287: mknod d2/d7/db/d17/d33/c5e 0 2024-03-23T08:23:19.232 INFO:tasks.workunit.client.1.smithi193.stdout:3/191: symlink d1/de/d13/d21/d25/d2b/d3c/l43 0 2024-03-23T08:23:19.232 INFO:tasks.workunit.client.1.smithi193.stdout:1/250: fsync da/d13/d24/f43 0 2024-03-23T08:23:19.233 INFO:tasks.workunit.client.1.smithi193.stdout:7/229: mkdir d6/d9/d4e 0 2024-03-23T08:23:19.236 INFO:tasks.workunit.client.1.smithi193.stdout:6/219: dread d0/fe [0,4194304] 0 2024-03-23T08:23:19.239 INFO:tasks.workunit.client.1.smithi193.stdout:8/211: rename df/d13 to df/d4b/d4d 0 2024-03-23T08:23:19.240 INFO:tasks.workunit.client.1.smithi193.stdout:2/308: dwrite d0/d37/f4c [0,4194304] 0 2024-03-23T08:23:19.240 INFO:tasks.workunit.client.1.smithi193.stdout:1/251: symlink da/l57 0 2024-03-23T08:23:19.240 INFO:tasks.workunit.client.1.smithi193.stdout:8/212: stat df/d4b 0 2024-03-23T08:23:19.242 INFO:tasks.workunit.client.1.smithi193.stdout:2/309: chown d0/l5 2352 1 2024-03-23T08:23:19.243 INFO:tasks.workunit.client.1.smithi193.stdout:9/288: getdents d2/d7/db/d1b/d5c 0 2024-03-23T08:23:19.243 INFO:tasks.workunit.client.1.smithi193.stdout:3/192: mkdir d1/de/d13/d21/d37/d44 0 2024-03-23T08:23:19.243 INFO:tasks.workunit.client.1.smithi193.stdout:7/230: mknod d6/d9/d4e/c4f 0 2024-03-23T08:23:19.244 INFO:tasks.workunit.client.1.smithi193.stdout:2/310: stat d0/d37/d48 0 2024-03-23T08:23:19.245 INFO:tasks.workunit.client.1.smithi193.stdout:5/252: rename d0/d1c/d3b/d4b/d4c to d0/d32/d57 0 2024-03-23T08:23:19.249 INFO:tasks.workunit.client.1.smithi193.stdout:1/252: dread da/dd/f30 [0,4194304] 0 2024-03-23T08:23:19.251 INFO:tasks.workunit.client.1.smithi193.stdout:1/253: write da/d13/f49 [23007,97798] 0 2024-03-23T08:23:19.251 INFO:tasks.workunit.client.1.smithi193.stdout:7/231: dread d6/d9/f1c [0,4194304] 0 2024-03-23T08:23:19.251 INFO:tasks.workunit.client.1.smithi193.stdout:9/289: mkdir d2/d7/d31/d5f 0 2024-03-23T08:23:19.254 INFO:tasks.workunit.client.1.smithi193.stdout:2/311: dread d0/d1b/f21 [0,4194304] 0 2024-03-23T08:23:19.254 INFO:tasks.workunit.client.1.smithi193.stdout:7/232: read d6/d9/d1f/f2a [2433685,87919] 0 2024-03-23T08:23:19.254 INFO:tasks.workunit.client.1.smithi193.stdout:2/312: read d0/d1b/f21 [569600,50670] 0 2024-03-23T08:23:19.255 INFO:tasks.workunit.client.1.smithi193.stdout:1/254: creat da/d13/d24/f58 x:0 0 0 2024-03-23T08:23:19.260 INFO:tasks.workunit.client.1.smithi193.stdout:7/233: dread d6/d9/f1c [0,4194304] 0 2024-03-23T08:23:19.260 INFO:tasks.workunit.client.1.smithi193.stdout:2/313: mkdir d0/d1b/d14/d56 0 2024-03-23T08:23:19.261 INFO:tasks.workunit.client.1.smithi193.stdout:7/234: chown d6/l1a 0 1 2024-03-23T08:23:19.262 INFO:tasks.workunit.client.1.smithi193.stdout:5/253: rmdir d0/d2f/d4f 0 2024-03-23T08:23:19.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/220: truncate d0/db/f33 1728926 0 2024-03-23T08:23:19.266 INFO:tasks.workunit.client.1.smithi193.stdout:2/314: dwrite d0/d28/f47 [0,4194304] 0 2024-03-23T08:23:19.266 INFO:tasks.workunit.client.1.smithi193.stdout:7/235: creat d6/d9/d1f/f50 x:0 0 0 2024-03-23T08:23:19.267 INFO:tasks.workunit.client.1.smithi193.stdout:8/213: write df/d10/d19/f1b [1541119,75367] 0 2024-03-23T08:23:19.267 INFO:tasks.workunit.client.1.smithi193.stdout:5/254: mknod d0/d32/d57/c58 0 2024-03-23T08:23:19.270 INFO:tasks.workunit.client.1.smithi193.stdout:1/255: rename da/fc to da/f59 0 2024-03-23T08:23:19.270 INFO:tasks.workunit.client.1.smithi193.stdout:5/255: truncate d0/f43 1474829 0 2024-03-23T08:23:19.270 INFO:tasks.workunit.client.1.smithi193.stdout:6/221: read d0/db/df/d11/f2e [1401326,72574] 0 2024-03-23T08:23:19.271 INFO:tasks.workunit.client.1.smithi193.stdout:1/256: write da/f4e [793356,41739] 0 2024-03-23T08:23:19.273 INFO:tasks.workunit.client.1.smithi193.stdout:8/214: mkdir df/d10/d3e/d4e 0 2024-03-23T08:23:19.274 INFO:tasks.workunit.client.1.smithi193.stdout:7/236: creat d6/d9/d1f/d23/d33/d3c/d41/f51 x:0 0 0 2024-03-23T08:23:19.277 INFO:tasks.workunit.client.1.smithi193.stdout:1/257: mkdir da/d13/d3f/d5a 0 2024-03-23T08:23:19.278 INFO:tasks.workunit.client.1.smithi193.stdout:2/315: rmdir d0/d1b/d14 39 2024-03-23T08:23:19.279 INFO:tasks.workunit.client.1.smithi193.stdout:8/215: truncate df/d10/f3f 426897 0 2024-03-23T08:23:19.280 INFO:tasks.workunit.client.1.smithi193.stdout:8/216: fdatasync f3 0 2024-03-23T08:23:19.282 INFO:tasks.workunit.client.1.smithi193.stdout:1/258: unlink da/dd/l1b 0 2024-03-23T08:23:19.282 INFO:tasks.workunit.client.1.smithi193.stdout:9/290: dwrite d2/d7/db/f3b [0,4194304] 0 2024-03-23T08:23:19.283 INFO:tasks.workunit.client.1.smithi193.stdout:7/237: truncate d6/d9/d1f/f50 1017864 0 2024-03-23T08:23:19.284 INFO:tasks.workunit.client.1.smithi193.stdout:6/222: dwrite d0/d8/d12/f1b [0,4194304] 0 2024-03-23T08:23:19.307 INFO:tasks.workunit.client.1.smithi193.stdout:4/255: sync 2024-03-23T08:23:19.319 INFO:tasks.workunit.client.1.smithi193.stdout:1/259: symlink da/dd/df/d2e/l5b 0 2024-03-23T08:23:19.321 INFO:tasks.workunit.client.1.smithi193.stdout:4/256: chown d11/d1a/d33/d42/f4d 123275096 1 2024-03-23T08:23:19.321 INFO:tasks.workunit.client.1.smithi193.stdout:6/223: symlink d0/d8/d12/d21/l43 0 2024-03-23T08:23:19.323 INFO:tasks.workunit.client.1.smithi193.stdout:7/238: creat d6/d9/d1f/d23/d33/d38/d3b/f52 x:0 0 0 2024-03-23T08:23:19.323 INFO:tasks.workunit.client.1.smithi193.stdout:1/260: rename da/dd to da/dd/df/d5c 22 2024-03-23T08:23:19.325 INFO:tasks.workunit.client.1.smithi193.stdout:9/291: getdents d2/d7/d31 0 2024-03-23T08:23:19.325 INFO:tasks.workunit.client.1.smithi193.stdout:5/256: dwrite d0/fa [0,4194304] 0 2024-03-23T08:23:19.326 INFO:tasks.workunit.client.1.smithi193.stdout:6/224: creat d0/db/df/d11/d36/f44 x:0 0 0 2024-03-23T08:23:19.328 INFO:tasks.workunit.client.1.smithi193.stdout:1/261: write da/d13/d24/f58 [657777,9382] 0 2024-03-23T08:23:19.328 INFO:tasks.workunit.client.1.smithi193.stdout:7/239: creat d6/d9/d1f/d23/d33/d38/f53 x:0 0 0 2024-03-23T08:23:19.329 INFO:tasks.workunit.client.1.smithi193.stdout:6/225: rmdir d0/db/df 39 2024-03-23T08:23:19.330 INFO:tasks.workunit.client.1.smithi193.stdout:5/257: creat d0/d1c/d3b/d4b/d55/f59 x:0 0 0 2024-03-23T08:23:19.335 INFO:tasks.workunit.client.1.smithi193.stdout:6/226: dread d0/db/df/f25 [0,4194304] 0 2024-03-23T08:23:19.338 INFO:tasks.workunit.client.1.smithi193.stdout:7/240: symlink d6/d9/d1f/d23/d33/l54 0 2024-03-23T08:23:19.339 INFO:tasks.workunit.client.1.smithi193.stdout:1/262: mknod da/d13/d3f/d5a/c5d 0 2024-03-23T08:23:19.340 INFO:tasks.workunit.client.1.smithi193.stdout:1/263: readlink da/d13/d24/l31 0 2024-03-23T08:23:19.341 INFO:tasks.workunit.client.1.smithi193.stdout:5/258: mkdir d0/d5a 0 2024-03-23T08:23:19.343 INFO:tasks.workunit.client.1.smithi193.stdout:8/217: dread df/f38 [0,4194304] 0 2024-03-23T08:23:19.343 INFO:tasks.workunit.client.1.smithi193.stdout:5/259: chown d0/d32 633638 1 2024-03-23T08:23:19.345 INFO:tasks.workunit.client.1.smithi193.stdout:9/292: write d2/f25 [479156,3134] 0 2024-03-23T08:23:19.345 INFO:tasks.workunit.client.1.smithi193.stdout:8/218: readlink df/l11 0 2024-03-23T08:23:19.346 INFO:tasks.workunit.client.1.smithi193.stdout:1/264: chown da/dd/df/d2e/c51 267434761 1 2024-03-23T08:23:19.347 INFO:tasks.workunit.client.1.smithi193.stdout:5/260: read d0/f36 [5392191,130014] 0 2024-03-23T08:23:19.348 INFO:tasks.workunit.client.1.smithi193.stdout:4/257: dread d11/d1a/d33/f39 [0,4194304] 0 2024-03-23T08:23:19.350 INFO:tasks.workunit.client.1.smithi193.stdout:8/219: fdatasync df/d4b/d4d/f45 0 2024-03-23T08:23:19.350 INFO:tasks.workunit.client.1.smithi193.stdout:9/293: rmdir d2/d7/db 39 2024-03-23T08:23:19.351 INFO:tasks.workunit.client.1.smithi193.stdout:7/241: dwrite d6/f32 [0,4194304] 0 2024-03-23T08:23:19.352 INFO:tasks.workunit.client.1.smithi193.stdout:4/258: symlink d11/d27/l57 0 2024-03-23T08:23:19.354 INFO:tasks.workunit.client.1.smithi193.stdout:1/265: rename da/dd/df/f22 to da/d13/d3f/f5e 0 2024-03-23T08:23:19.355 INFO:tasks.workunit.client.1.smithi193.stdout:8/220: creat df/d10/d19/f4f x:0 0 0 2024-03-23T08:23:19.356 INFO:tasks.workunit.client.1.smithi193.stdout:9/294: creat d2/d7/db/f60 x:0 0 0 2024-03-23T08:23:19.358 INFO:tasks.workunit.client.1.smithi193.stdout:1/266: read - da/dd/df/d2e/f4f zero size 2024-03-23T08:23:19.358 INFO:tasks.workunit.client.1.smithi193.stdout:6/227: dwrite d0/d8/d12/d21/f29 [0,4194304] 0 2024-03-23T08:23:19.358 INFO:tasks.workunit.client.1.smithi193.stdout:4/259: rename d11/d48/d4c to d11/d2d/d58 0 2024-03-23T08:23:19.361 INFO:tasks.workunit.client.1.smithi193.stdout:7/242: creat d6/d9/d1f/d23/d33/d38/d3b/f55 x:0 0 0 2024-03-23T08:23:19.363 INFO:tasks.workunit.client.1.smithi193.stdout:8/221: fdatasync df/f1f 0 2024-03-23T08:23:19.364 INFO:tasks.workunit.client.1.smithi193.stdout:5/261: dwrite d0/d32/f44 [0,4194304] 0 2024-03-23T08:23:19.364 INFO:tasks.workunit.client.1.smithi193.stdout:6/228: creat d0/d8/d3a/f45 x:0 0 0 2024-03-23T08:23:19.367 INFO:tasks.workunit.client.1.smithi193.stdout:4/260: creat d11/d1a/d33/d42/f59 x:0 0 0 2024-03-23T08:23:19.368 INFO:tasks.workunit.client.1.smithi193.stdout:5/262: fsync d0/d1f/f41 0 2024-03-23T08:23:19.369 INFO:tasks.workunit.client.1.smithi193.stdout:7/243: write d6/d9/f21 [1791565,51963] 0 2024-03-23T08:23:19.370 INFO:tasks.workunit.client.1.smithi193.stdout:1/267: symlink da/dd/l5f 0 2024-03-23T08:23:19.373 INFO:tasks.workunit.client.1.smithi193.stdout:8/222: mknod df/c50 0 2024-03-23T08:23:19.376 INFO:tasks.workunit.client.1.smithi193.stdout:8/223: stat df/d10/f17 0 2024-03-23T08:23:19.376 INFO:tasks.workunit.client.1.smithi193.stdout:1/268: write da/dd/df/d2e/f4f [766900,99231] 0 2024-03-23T08:23:19.378 INFO:tasks.workunit.client.1.smithi193.stdout:7/244: rename d6/ff to d6/d9/d1f/d4d/f56 0 2024-03-23T08:23:19.381 INFO:tasks.workunit.client.1.smithi193.stdout:6/229: dwrite d0/d8/d12/d21/f29 [0,4194304] 0 2024-03-23T08:23:19.382 INFO:tasks.workunit.client.1.smithi193.stdout:8/224: symlink df/d4b/d4d/l51 0 2024-03-23T08:23:19.383 INFO:tasks.workunit.client.1.smithi193.stdout:5/263: dread d0/f15 [0,4194304] 0 2024-03-23T08:23:19.385 INFO:tasks.workunit.client.1.smithi193.stdout:0/279: sync 2024-03-23T08:23:19.389 INFO:tasks.workunit.client.1.smithi193.stdout:8/225: truncate df/d10/d19/f4f 517523 0 2024-03-23T08:23:19.390 INFO:tasks.workunit.client.1.smithi193.stdout:9/295: dread d2/d7/f1e [0,4194304] 0 2024-03-23T08:23:19.390 INFO:tasks.workunit.client.1.smithi193.stdout:5/264: creat d0/d1f/f5b x:0 0 0 2024-03-23T08:23:19.393 INFO:tasks.workunit.client.1.smithi193.stdout:7/245: unlink d6/d9/d1f/d4d/f56 0 2024-03-23T08:23:19.394 INFO:tasks.workunit.client.1.smithi193.stdout:4/261: dread d11/d19/f20 [0,4194304] 0 2024-03-23T08:23:19.395 INFO:tasks.workunit.client.1.smithi193.stdout:7/246: stat d6/d9/l25 0 2024-03-23T08:23:19.395 INFO:tasks.workunit.client.1.smithi193.stdout:5/265: creat d0/d1c/f5c x:0 0 0 2024-03-23T08:23:19.395 INFO:tasks.workunit.client.1.smithi193.stdout:0/280: dread db/d11/f28 [0,4194304] 0 2024-03-23T08:23:19.397 INFO:tasks.workunit.client.1.smithi193.stdout:8/226: dread df/d4b/d4d/f22 [0,4194304] 0 2024-03-23T08:23:19.398 INFO:tasks.workunit.client.1.smithi193.stdout:6/230: dwrite d0/d8/d3a/f45 [0,4194304] 0 2024-03-23T08:23:19.398 INFO:tasks.workunit.client.1.smithi193.stdout:9/296: read d2/d7/db/d1b/f30 [1895013,79094] 0 2024-03-23T08:23:19.399 INFO:tasks.workunit.client.1.smithi193.stdout:0/281: readlink db/d11/d14/l1b 0 2024-03-23T08:23:19.400 INFO:tasks.workunit.client.1.smithi193.stdout:8/227: chown ca 0 1 2024-03-23T08:23:19.401 INFO:tasks.workunit.client.1.smithi193.stdout:5/266: mknod d0/d5a/c5d 0 2024-03-23T08:23:19.401 INFO:tasks.workunit.client.1.smithi193.stdout:8/228: stat f3 0 2024-03-23T08:23:19.403 INFO:tasks.workunit.client.1.smithi193.stdout:5/267: chown d0/c37 2485 1 2024-03-23T08:23:19.404 INFO:tasks.workunit.client.1.smithi193.stdout:7/247: creat d6/d9/d1f/d23/d33/d3c/d41/d47/f57 x:0 0 0 2024-03-23T08:23:19.405 INFO:tasks.workunit.client.1.smithi193.stdout:1/269: dwrite da/dd/df/f1d [0,4194304] 0 2024-03-23T08:23:19.410 INFO:tasks.workunit.client.1.smithi193.stdout:0/282: rename db/f3e to db/d11/d14/d24/f5f 0 2024-03-23T08:23:19.411 INFO:tasks.workunit.client.1.smithi193.stdout:6/231: write d0/db/f20 [214440,21906] 0 2024-03-23T08:23:19.411 INFO:tasks.workunit.client.1.smithi193.stdout:5/268: unlink d0/d1c/l31 0 2024-03-23T08:23:19.412 INFO:tasks.workunit.client.1.smithi193.stdout:7/248: mknod d6/d9/d1f/c58 0 2024-03-23T08:23:19.414 INFO:tasks.workunit.client.1.smithi193.stdout:7/249: fdatasync d6/d9/f19 0 2024-03-23T08:23:19.417 INFO:tasks.workunit.client.1.smithi193.stdout:8/229: dwrite df/d10/f17 [0,4194304] 0 2024-03-23T08:23:19.417 INFO:tasks.workunit.client.1.smithi193.stdout:6/232: chown d0/d8/d12/d21/f29 37501 1 2024-03-23T08:23:19.418 INFO:tasks.workunit.client.1.smithi193.stdout:5/269: unlink d0/d32/d57/c58 0 2024-03-23T08:23:19.419 INFO:tasks.workunit.client.1.smithi193.stdout:8/230: fdatasync f3 0 2024-03-23T08:23:19.423 INFO:tasks.workunit.client.1.smithi193.stdout:5/270: write d0/d32/d45/f56 [661436,75748] 0 2024-03-23T08:23:19.423 INFO:tasks.workunit.client.1.smithi193.stdout:6/233: rename d0/db/df/d11/c2d to d0/d27/c46 0 2024-03-23T08:23:19.424 INFO:tasks.workunit.client.1.smithi193.stdout:1/270: dwrite da/d13/d24/f43 [0,4194304] 0 2024-03-23T08:23:19.426 INFO:tasks.workunit.client.1.smithi193.stdout:5/271: fsync d0/d1f/f2e 0 2024-03-23T08:23:19.428 INFO:tasks.workunit.client.1.smithi193.stdout:6/234: chown d0/db/df/l28 1 1 2024-03-23T08:23:19.430 INFO:tasks.workunit.client.1.smithi193.stdout:9/297: dread d2/f4 [0,4194304] 0 2024-03-23T08:23:19.432 INFO:tasks.workunit.client.1.smithi193.stdout:7/250: dwrite d6/d9/d1f/f30 [4194304,4194304] 0 2024-03-23T08:23:19.432 INFO:tasks.workunit.client.1.smithi193.stdout:5/272: chown d0/d1c/f20 6550 1 2024-03-23T08:23:19.434 INFO:tasks.workunit.client.1.smithi193.stdout:8/231: rename df/l11 to df/d4b/d4d/l52 0 2024-03-23T08:23:19.435 INFO:tasks.workunit.client.1.smithi193.stdout:1/271: symlink da/dd/l60 0 2024-03-23T08:23:19.469 INFO:tasks.workunit.client.1.smithi193.stdout:3/193: sync 2024-03-23T08:23:19.469 INFO:tasks.workunit.client.1.smithi193.stdout:2/316: sync 2024-03-23T08:23:19.483 INFO:tasks.workunit.client.1.smithi193.stdout:6/235: dwrite d0/f40 [0,4194304] 0 2024-03-23T08:23:19.483 INFO:tasks.workunit.client.1.smithi193.stdout:1/272: rename da/dd/f30 to da/d13/d3f/d5a/f61 0 2024-03-23T08:23:19.485 INFO:tasks.workunit.client.1.smithi193.stdout:3/194: chown d1/de/d13/d21/d25 3460831 1 2024-03-23T08:23:19.485 INFO:tasks.workunit.client.1.smithi193.stdout:1/273: fdatasync da/f12 0 2024-03-23T08:23:19.486 INFO:tasks.workunit.client.1.smithi193.stdout:8/232: symlink df/l53 0 2024-03-23T08:23:19.487 INFO:tasks.workunit.client.1.smithi193.stdout:9/298: rmdir d2/d7/db/d17/d1f/d5b 0 2024-03-23T08:23:19.488 INFO:tasks.workunit.client.1.smithi193.stdout:5/273: dread d0/f36 [0,4194304] 0 2024-03-23T08:23:19.492 INFO:tasks.workunit.client.1.smithi193.stdout:6/236: rename d0/d27/c46 to d0/db/df/d11/c47 0 2024-03-23T08:23:19.492 INFO:tasks.workunit.client.1.smithi193.stdout:8/233: mkdir df/d10/d3e/d54 0 2024-03-23T08:23:19.492 INFO:tasks.workunit.client.1.smithi193.stdout:2/317: getdents d0/d37 0 2024-03-23T08:23:19.493 INFO:tasks.workunit.client.1.smithi193.stdout:5/274: mknod d0/d1c/c5e 0 2024-03-23T08:23:19.494 INFO:tasks.workunit.client.1.smithi193.stdout:3/195: mknod d1/de/d13/d21/d25/d2b/d3c/c45 0 2024-03-23T08:23:19.494 INFO:tasks.workunit.client.1.smithi193.stdout:6/237: chown d0/d8/d12/d21/l43 1417941 1 2024-03-23T08:23:19.495 INFO:tasks.workunit.client.1.smithi193.stdout:1/274: link da/dd/f14 da/dd/f62 0 2024-03-23T08:23:19.496 INFO:tasks.workunit.client.1.smithi193.stdout:3/196: mknod d1/de/c46 0 2024-03-23T08:23:19.498 INFO:tasks.workunit.client.1.smithi193.stdout:1/275: symlink da/dd/df/d52/l63 0 2024-03-23T08:23:19.499 INFO:tasks.workunit.client.1.smithi193.stdout:8/234: rename df/l53 to df/l55 0 2024-03-23T08:23:19.501 INFO:tasks.workunit.client.1.smithi193.stdout:8/235: chown df/d10/f2f 0 1 2024-03-23T08:23:19.502 INFO:tasks.workunit.client.1.smithi193.stdout:3/197: unlink d1/fd 0 2024-03-23T08:23:19.502 INFO:tasks.workunit.client.1.smithi193.stdout:0/283: dread db/f1e [0,4194304] 0 2024-03-23T08:23:19.503 INFO:tasks.workunit.client.1.smithi193.stdout:6/238: creat d0/d8/f48 x:0 0 0 2024-03-23T08:23:19.503 INFO:tasks.workunit.client.1.smithi193.stdout:2/318: symlink d0/d1b/d14/d25/d2a/d4e/l57 0 2024-03-23T08:23:19.504 INFO:tasks.workunit.client.1.smithi193.stdout:8/236: chown df/d4b/d4d/f22 29 1 2024-03-23T08:23:19.504 INFO:tasks.workunit.client.1.smithi193.stdout:5/275: dwrite d0/d32/f48 [0,4194304] 0 2024-03-23T08:23:19.506 INFO:tasks.workunit.client.1.smithi193.stdout:2/319: chown d0/d28/f47 0 1 2024-03-23T08:23:19.506 INFO:tasks.workunit.client.1.smithi193.stdout:1/276: rmdir da/d13 39 2024-03-23T08:23:19.508 INFO:tasks.workunit.client.1.smithi193.stdout:6/239: truncate d0/d8/f48 895293 0 2024-03-23T08:23:19.509 INFO:tasks.workunit.client.1.smithi193.stdout:5/276: chown d0/l46 470904 1 2024-03-23T08:23:19.512 INFO:tasks.workunit.client.1.smithi193.stdout:1/277: truncate da/dd/df/f2d 5113939 0 2024-03-23T08:23:19.513 INFO:tasks.workunit.client.1.smithi193.stdout:0/284: write db/d11/f40 [3031075,48358] 0 2024-03-23T08:23:19.514 INFO:tasks.workunit.client.1.smithi193.stdout:9/299: dread d2/d7/db/d17/d1f/f22 [0,4194304] 0 2024-03-23T08:23:19.515 INFO:tasks.workunit.client.1.smithi193.stdout:9/300: chown d2/d7/db/d1b/d43 213339 1 2024-03-23T08:23:19.525 INFO:tasks.workunit.client.1.smithi193.stdout:6/240: unlink d0/l6 0 2024-03-23T08:23:19.525 INFO:tasks.workunit.client.1.smithi193.stdout:4/262: sync 2024-03-23T08:23:19.525 INFO:tasks.workunit.client.1.smithi193.stdout:7/251: sync 2024-03-23T08:23:19.526 INFO:tasks.workunit.client.1.smithi193.stdout:2/320: rename d0/d1b/d14/d25/f32 to d0/d1b/d3b/f58 0 2024-03-23T08:23:19.527 INFO:tasks.workunit.client.1.smithi193.stdout:2/321: stat d0/d1b/c9 0 2024-03-23T08:23:19.528 INFO:tasks.workunit.client.1.smithi193.stdout:2/322: write d0/d1b/f8 [846200,73361] 0 2024-03-23T08:23:19.528 INFO:tasks.workunit.client.1.smithi193.stdout:5/277: creat d0/d32/d45/f5f x:0 0 0 2024-03-23T08:23:19.530 INFO:tasks.workunit.client.1.smithi193.stdout:8/237: link df/f1f df/f56 0 2024-03-23T08:23:19.532 INFO:tasks.workunit.client.1.smithi193.stdout:0/285: mknod db/d11/d14/d51/c60 0 2024-03-23T08:23:19.534 INFO:tasks.workunit.client.1.smithi193.stdout:7/252: creat d6/d9/d1f/d3e/d42/f59 x:0 0 0 2024-03-23T08:23:19.536 INFO:tasks.workunit.client.1.smithi193.stdout:2/323: unlink d0/d1b/f2f 0 2024-03-23T08:23:19.537 INFO:tasks.workunit.client.1.smithi193.stdout:5/278: dwrite d0/f14 [4194304,4194304] 0 2024-03-23T08:23:19.537 INFO:tasks.workunit.client.1.smithi193.stdout:9/301: mknod d2/d7/db/d1b/d5c/c61 0 2024-03-23T08:23:19.540 INFO:tasks.workunit.client.1.smithi193.stdout:8/238: dread df/d10/d19/f1b [0,4194304] 0 2024-03-23T08:23:19.540 INFO:tasks.workunit.client.1.smithi193.stdout:1/278: creat da/dd/f64 x:0 0 0 2024-03-23T08:23:19.540 INFO:tasks.workunit.client.1.smithi193.stdout:4/263: link d11/d27/l57 d11/d2d/d3a/l5a 0 2024-03-23T08:23:19.544 INFO:tasks.workunit.client.1.smithi193.stdout:3/198: dwrite d1/fa [0,4194304] 0 2024-03-23T08:23:19.547 INFO:tasks.workunit.client.1.smithi193.stdout:5/279: rename d0/c47 to d0/d1c/c60 0 2024-03-23T08:23:19.555 INFO:tasks.workunit.client.1.smithi193.stdout:9/302: creat d2/d7/db/d17/d33/f62 x:0 0 0 2024-03-23T08:23:19.555 INFO:tasks.workunit.client.1.smithi193.stdout:4/264: rename d11/d2d/f53 to d11/d48/d50/f5b 0 2024-03-23T08:23:19.556 INFO:tasks.workunit.client.1.smithi193.stdout:0/286: truncate db/d11/f28 165487 0 2024-03-23T08:23:19.557 INFO:tasks.workunit.client.1.smithi193.stdout:7/253: rename d6/d9/d1f/d23/d33/d3c/d41/d47/f57 to d6/d9/d1f/d23/d33/d38/d3b/f5a 0 2024-03-23T08:23:19.558 INFO:tasks.workunit.client.1.smithi193.stdout:5/280: creat d0/d1c/d3b/d4b/f61 x:0 0 0 2024-03-23T08:23:19.559 INFO:tasks.workunit.client.1.smithi193.stdout:4/265: creat d11/d48/d50/d49/f5c x:0 0 0 2024-03-23T08:23:19.560 INFO:tasks.workunit.client.1.smithi193.stdout:8/239: dwrite df/d10/f29 [0,4194304] 0 2024-03-23T08:23:19.560 INFO:tasks.workunit.client.1.smithi193.stdout:9/303: creat d2/d7/db/d17/d33/f63 x:0 0 0 2024-03-23T08:23:19.561 INFO:tasks.workunit.client.1.smithi193.stdout:3/199: dwrite d1/de/d13/d21/d25/d2b/f39 [0,4194304] 0 2024-03-23T08:23:19.562 INFO:tasks.workunit.client.1.smithi193.stdout:5/281: rmdir d0/d32 39 2024-03-23T08:23:19.563 INFO:tasks.workunit.client.1.smithi193.stdout:4/266: dread - d11/d1a/d33/d42/f59 zero size 2024-03-23T08:23:19.567 INFO:tasks.workunit.client.1.smithi193.stdout:5/282: fdatasync d0/d1c/d3b/d4b/d55/f59 0 2024-03-23T08:23:19.568 INFO:tasks.workunit.client.1.smithi193.stdout:7/254: truncate d6/d9/d1f/d23/f45 1030781 0 2024-03-23T08:23:19.569 INFO:tasks.workunit.client.1.smithi193.stdout:9/304: creat d2/d7/db/d17/d33/f64 x:0 0 0 2024-03-23T08:23:19.569 INFO:tasks.workunit.client.1.smithi193.stdout:8/240: symlink df/d10/l57 0 2024-03-23T08:23:19.572 INFO:tasks.workunit.client.1.smithi193.stdout:7/255: fsync d6/d9/fd 0 2024-03-23T08:23:19.573 INFO:tasks.workunit.client.1.smithi193.stdout:0/287: dwrite db/d11/d14/f1f [0,4194304] 0 2024-03-23T08:23:19.574 INFO:tasks.workunit.client.1.smithi193.stdout:3/200: creat d1/de/d13/d21/d25/d34/f47 x:0 0 0 2024-03-23T08:23:19.575 INFO:tasks.workunit.client.1.smithi193.stdout:4/267: rename d11/d27/f36 to d11/d48/f5d 0 2024-03-23T08:23:19.576 INFO:tasks.workunit.client.1.smithi193.stdout:1/279: fsync da/dd/df/f2d 0 2024-03-23T08:23:19.577 INFO:tasks.workunit.client.1.smithi193.stdout:4/268: chown d11/d19/c23 0 1 2024-03-23T08:23:19.577 INFO:tasks.workunit.client.1.smithi193.stdout:7/256: truncate d6/d9/f15 4535911 0 2024-03-23T08:23:19.578 INFO:tasks.workunit.client.1.smithi193.stdout:8/241: symlink df/d4b/d4d/d2a/l58 0 2024-03-23T08:23:19.579 INFO:tasks.workunit.client.1.smithi193.stdout:3/201: symlink d1/d22/l48 0 2024-03-23T08:23:19.579 INFO:tasks.workunit.client.1.smithi193.stdout:9/305: rename d2/d7/db/c20 to d2/d7/db/d17/d1f/c65 0 2024-03-23T08:23:19.579 INFO:tasks.workunit.client.1.smithi193.stdout:4/269: readlink d11/l15 0 2024-03-23T08:23:19.579 INFO:tasks.workunit.client.1.smithi193.stdout:1/280: rmdir da/dd/df/d2e 39 2024-03-23T08:23:19.581 INFO:tasks.workunit.client.1.smithi193.stdout:3/202: write d1/de/d13/d21/d27/f1c [1635207,91160] 0 2024-03-23T08:23:19.583 INFO:tasks.workunit.client.1.smithi193.stdout:7/257: rmdir d6/d9/d1f/d3e/d42 39 2024-03-23T08:23:19.585 INFO:tasks.workunit.client.1.smithi193.stdout:7/258: stat d6/d9/d1f/d23/d33/l54 0 2024-03-23T08:23:19.588 INFO:tasks.workunit.client.1.smithi193.stdout:5/283: rmdir d0/d1c 39 2024-03-23T08:23:19.589 INFO:tasks.workunit.client.1.smithi193.stdout:7/259: dread - d6/d9/d1f/d23/d33/d3c/f4c zero size 2024-03-23T08:23:19.590 INFO:tasks.workunit.client.1.smithi193.stdout:9/306: dwrite d2/fc [0,4194304] 0 2024-03-23T08:23:19.591 INFO:tasks.workunit.client.1.smithi193.stdout:3/203: symlink d1/de/d13/d21/d25/d34/l49 0 2024-03-23T08:23:19.592 INFO:tasks.workunit.client.1.smithi193.stdout:3/204: chown d1/de/d28 50 1 2024-03-23T08:23:19.592 INFO:tasks.workunit.client.1.smithi193.stdout:4/270: creat d11/d2d/d3a/f5e x:0 0 0 2024-03-23T08:23:19.594 INFO:tasks.workunit.client.1.smithi193.stdout:1/281: creat da/d13/d3f/d5a/f65 x:0 0 0 2024-03-23T08:23:19.595 INFO:tasks.workunit.client.1.smithi193.stdout:5/284: mkdir d0/d1f/d62 0 2024-03-23T08:23:19.595 INFO:tasks.workunit.client.1.smithi193.stdout:9/307: chown d2/d7/db/d17/d33/c39 28531 1 2024-03-23T08:23:19.597 INFO:tasks.workunit.client.1.smithi193.stdout:1/282: stat da/d13/d24/c35 0 2024-03-23T08:23:19.597 INFO:tasks.workunit.client.1.smithi193.stdout:8/242: getdents df/d46 0 2024-03-23T08:23:19.597 INFO:tasks.workunit.client.1.smithi193.stdout:9/308: readlink d2/d7/l16 0 2024-03-23T08:23:19.598 INFO:tasks.workunit.client.1.smithi193.stdout:8/243: chown le 40 1 2024-03-23T08:23:19.598 INFO:tasks.workunit.client.1.smithi193.stdout:4/271: fdatasync d11/d1a/d33/d42/f28 0 2024-03-23T08:23:19.599 INFO:tasks.workunit.client.1.smithi193.stdout:3/205: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a 0 2024-03-23T08:23:19.600 INFO:tasks.workunit.client.1.smithi193.stdout:0/288: truncate db/d11/d14/f1f 483316 0 2024-03-23T08:23:19.602 INFO:tasks.workunit.client.1.smithi193.stdout:4/272: chown fb 252 1 2024-03-23T08:23:19.603 INFO:tasks.workunit.client.1.smithi193.stdout:8/244: readlink df/d10/l57 0 2024-03-23T08:23:19.627 INFO:tasks.workunit.client.1.smithi193.stdout:6/241: sync 2024-03-23T08:23:19.641 INFO:tasks.workunit.client.1.smithi193.stdout:7/260: dwrite d6/d9/f1b [4194304,4194304] 0 2024-03-23T08:23:19.643 INFO:tasks.workunit.client.1.smithi193.stdout:3/206: write d1/de/d13/f24 [3952262,10221] 0 2024-03-23T08:23:19.646 INFO:tasks.workunit.client.1.smithi193.stdout:0/289: creat db/d11/d14/d24/d47/f61 x:0 0 0 2024-03-23T08:23:19.646 INFO:tasks.workunit.client.1.smithi193.stdout:8/245: write f3 [3763635,73804] 0 2024-03-23T08:23:19.648 INFO:tasks.workunit.client.1.smithi193.stdout:9/309: dread d2/d7/f15 [0,4194304] 0 2024-03-23T08:23:19.648 INFO:tasks.workunit.client.1.smithi193.stdout:6/242: fsync d0/db/df/d11/f30 0 2024-03-23T08:23:19.649 INFO:tasks.workunit.client.1.smithi193.stdout:3/207: chown d1/c5 64 1 2024-03-23T08:23:19.652 INFO:tasks.workunit.client.1.smithi193.stdout:5/285: dwrite d0/f2a [0,4194304] 0 2024-03-23T08:23:19.654 INFO:tasks.workunit.client.1.smithi193.stdout:0/290: creat db/d11/d14/d24/d2a/f62 x:0 0 0 2024-03-23T08:23:19.655 INFO:tasks.workunit.client.1.smithi193.stdout:8/246: symlink df/d10/d3e/l59 0 2024-03-23T08:23:19.656 INFO:tasks.workunit.client.1.smithi193.stdout:5/286: chown d0/f11 5 1 2024-03-23T08:23:19.657 INFO:tasks.workunit.client.1.smithi193.stdout:4/273: getdents d11/d1a/d33/d44 0 2024-03-23T08:23:19.658 INFO:tasks.workunit.client.1.smithi193.stdout:9/310: mknod d2/d7/db/d17/d1f/c66 0 2024-03-23T08:23:19.659 INFO:tasks.workunit.client.1.smithi193.stdout:5/287: chown d0/d1f/d62 49 1 2024-03-23T08:23:19.659 INFO:tasks.workunit.client.1.smithi193.stdout:7/261: dwrite d6/d9/f19 [0,4194304] 0 2024-03-23T08:23:19.661 INFO:tasks.workunit.client.1.smithi193.stdout:3/208: rename d1/l26 to d1/de/d13/d21/d25/l4b 0 2024-03-23T08:23:19.665 INFO:tasks.workunit.client.1.smithi193.stdout:9/311: creat d2/d7/db/d32/f67 x:0 0 0 2024-03-23T08:23:19.668 INFO:tasks.workunit.client.1.smithi193.stdout:7/262: creat d6/d9/d1f/d23/d33/d38/d46/f5b x:0 0 0 2024-03-23T08:23:19.668 INFO:tasks.workunit.client.1.smithi193.stdout:5/288: rmdir d0/d1f/d62 0 2024-03-23T08:23:19.669 INFO:tasks.workunit.client.1.smithi193.stdout:3/209: symlink d1/de/d13/d21/d37/l4c 0 2024-03-23T08:23:19.674 INFO:tasks.workunit.client.1.smithi193.stdout:8/247: dwrite df/d4b/d4d/d2a/f49 [0,4194304] 0 2024-03-23T08:23:19.675 INFO:tasks.workunit.client.1.smithi193.stdout:0/291: write db/d11/d14/d21/f26 [934718,1220] 0 2024-03-23T08:23:19.675 INFO:tasks.workunit.client.1.smithi193.stdout:9/312: truncate d2/d7/db/d17/f53 130918 0 2024-03-23T08:23:19.678 INFO:tasks.workunit.client.1.smithi193.stdout:5/289: creat d0/d1c/d3b/d4b/d55/f63 x:0 0 0 2024-03-23T08:23:19.679 INFO:tasks.workunit.client.1.smithi193.stdout:4/274: dwrite d11/d19/f2f [0,4194304] 0 2024-03-23T08:23:19.679 INFO:tasks.workunit.client.1.smithi193.stdout:8/248: dread - df/d46/f47 zero size 2024-03-23T08:23:19.680 INFO:tasks.workunit.client.1.smithi193.stdout:3/210: creat d1/de/d13/d21/d25/d2b/f4d x:0 0 0 2024-03-23T08:23:19.683 INFO:tasks.workunit.client.1.smithi193.stdout:5/290: fdatasync d0/f15 0 2024-03-23T08:23:19.683 INFO:tasks.workunit.client.1.smithi193.stdout:9/313: chown d2/d7/db/d17/d33/c39 69 1 2024-03-23T08:23:19.685 INFO:tasks.workunit.client.1.smithi193.stdout:4/275: mknod d11/d19/c5f 0 2024-03-23T08:23:19.691 INFO:tasks.workunit.client.1.smithi193.stdout:6/243: dwrite d0/db/f33 [0,4194304] 0 2024-03-23T08:23:19.693 INFO:tasks.workunit.client.1.smithi193.stdout:8/249: creat df/f5a x:0 0 0 2024-03-23T08:23:19.693 INFO:tasks.workunit.client.1.smithi193.stdout:7/263: dwrite d6/d9/f31 [0,4194304] 0 2024-03-23T08:23:19.694 INFO:tasks.workunit.client.1.smithi193.stdout:8/250: stat df/f5a 0 2024-03-23T08:23:19.696 INFO:tasks.workunit.client.1.smithi193.stdout:3/211: truncate d1/de/d13/d21/d25/d2b/f35 725603 0 2024-03-23T08:23:19.696 INFO:tasks.workunit.client.1.smithi193.stdout:4/276: read d11/d19/d22/f25 [4350550,31195] 0 2024-03-23T08:23:19.697 INFO:tasks.workunit.client.1.smithi193.stdout:0/292: dwrite db/d11/d14/d21/f49 [0,4194304] 0 2024-03-23T08:23:19.704 INFO:tasks.workunit.client.1.smithi193.stdout:2/324: sync 2024-03-23T08:23:19.716 INFO:tasks.workunit.client.1.smithi193.stdout:5/291: link d0/d32/c39 d0/d1c/c64 0 2024-03-23T08:23:19.717 INFO:tasks.workunit.client.1.smithi193.stdout:0/293: fdatasync db/d11/d14/f19 0 2024-03-23T08:23:19.717 INFO:tasks.workunit.client.1.smithi193.stdout:8/251: truncate df/f56 3791392 0 2024-03-23T08:23:19.720 INFO:tasks.workunit.client.1.smithi193.stdout:9/314: truncate d2/fc 1544395 0 2024-03-23T08:23:19.721 INFO:tasks.workunit.client.1.smithi193.stdout:3/212: rename d1/de/d13/d21/d27/c32 to d1/c4e 0 2024-03-23T08:23:19.722 INFO:tasks.workunit.client.1.smithi193.stdout:7/264: dwrite d6/fc [4194304,4194304] 0 2024-03-23T08:23:19.722 INFO:tasks.workunit.client.1.smithi193.stdout:9/315: fsync d2/d7/f11 0 2024-03-23T08:23:19.726 INFO:tasks.workunit.client.1.smithi193.stdout:8/252: rename df/f44 to df/d10/d3e/d4e/f5b 0 2024-03-23T08:23:19.727 INFO:tasks.workunit.client.1.smithi193.stdout:0/294: truncate db/d11/d14/d24/d47/f61 329991 0 2024-03-23T08:23:19.728 INFO:tasks.workunit.client.1.smithi193.stdout:5/292: write d0/d1c/f24 [5077962,13337] 0 2024-03-23T08:23:19.730 INFO:tasks.workunit.client.1.smithi193.stdout:8/253: fsync df/f33 0 2024-03-23T08:23:19.733 INFO:tasks.workunit.client.1.smithi193.stdout:9/316: dread d2/d7/db/d17/f53 [0,4194304] 0 2024-03-23T08:23:19.735 INFO:tasks.workunit.client.1.smithi193.stdout:8/254: chown df/c42 21632662 1 2024-03-23T08:23:19.735 INFO:tasks.workunit.client.1.smithi193.stdout:3/213: creat d1/de/d13/d21/d37/d44/f4f x:0 0 0 2024-03-23T08:23:19.739 INFO:tasks.workunit.client.1.smithi193.stdout:2/325: dwrite d0/d1b/d3b/f46 [0,4194304] 0 2024-03-23T08:23:19.739 INFO:tasks.workunit.client.1.smithi193.stdout:8/255: fsync df/f12 0 2024-03-23T08:23:19.741 INFO:tasks.workunit.client.1.smithi193.stdout:4/277: dwrite d11/d48/d50/f5b [0,4194304] 0 2024-03-23T08:23:19.745 INFO:tasks.workunit.client.1.smithi193.stdout:4/278: mkdir d11/d1a/d60 0 2024-03-23T08:23:19.746 INFO:tasks.workunit.client.1.smithi193.stdout:9/317: creat d2/d7/d31/d5f/f68 x:0 0 0 2024-03-23T08:23:19.747 INFO:tasks.workunit.client.1.smithi193.stdout:0/295: dread db/d11/f12 [0,4194304] 0 2024-03-23T08:23:19.747 INFO:tasks.workunit.client.1.smithi193.stdout:5/293: truncate d0/fe 2881899 0 2024-03-23T08:23:19.747 INFO:tasks.workunit.client.1.smithi193.stdout:2/326: readlink d0/d1b/l38 0 2024-03-23T08:23:19.748 INFO:tasks.workunit.client.1.smithi193.stdout:3/214: truncate d1/de/d13/d21/f31 4846001 0 2024-03-23T08:23:19.748 INFO:tasks.workunit.client.1.smithi193.stdout:7/265: dwrite d6/d9/d1f/d23/d33/d3c/f4b [0,4194304] 0 2024-03-23T08:23:19.753 INFO:tasks.workunit.client.1.smithi193.stdout:9/318: fsync d2/d7/db/f60 0 2024-03-23T08:23:19.753 INFO:tasks.workunit.client.1.smithi193.stdout:4/279: creat d11/d48/d50/f61 x:0 0 0 2024-03-23T08:23:19.755 INFO:tasks.workunit.client.1.smithi193.stdout:8/256: link df/d4b/d4d/f22 df/d4b/d4d/f5c 0 2024-03-23T08:23:19.755 INFO:tasks.workunit.client.1.smithi193.stdout:2/327: dread - d0/d37/d48/f4a zero size 2024-03-23T08:23:19.756 INFO:tasks.workunit.client.1.smithi193.stdout:5/294: creat d0/d1c/f65 x:0 0 0 2024-03-23T08:23:19.756 INFO:tasks.workunit.client.1.smithi193.stdout:3/215: write d1/de/d13/d21/d25/d2b/f35 [1027016,113775] 0 2024-03-23T08:23:19.759 INFO:tasks.workunit.client.1.smithi193.stdout:8/257: stat df/d10/d3e 0 2024-03-23T08:23:19.762 INFO:tasks.workunit.client.1.smithi193.stdout:4/280: write d11/d1a/d33/d42/f29 [499409,72545] 0 2024-03-23T08:23:19.763 INFO:tasks.workunit.client.1.smithi193.stdout:7/266: symlink d6/d9/d1f/d23/d33/d3c/d41/d47/l5c 0 2024-03-23T08:23:19.766 INFO:tasks.workunit.client.1.smithi193.stdout:9/319: getdents d2/d7/db/d17/d4b/d59 0 2024-03-23T08:23:19.767 INFO:tasks.workunit.client.1.smithi193.stdout:4/281: fsync d11/d19/d22/f2b 0 2024-03-23T08:23:19.771 INFO:tasks.workunit.client.1.smithi193.stdout:3/216: rename d1/de/d13/d21/d37 to d1/de/d13/d21/d50 0 2024-03-23T08:23:19.772 INFO:tasks.workunit.client.1.smithi193.stdout:2/328: dread d0/d37/f4c [0,4194304] 0 2024-03-23T08:23:19.772 INFO:tasks.workunit.client.1.smithi193.stdout:4/282: fdatasync d11/d19/d22/f31 0 2024-03-23T08:23:19.773 INFO:tasks.workunit.client.1.smithi193.stdout:7/267: symlink d6/d9/d1f/d3e/d42/l5d 0 2024-03-23T08:23:19.773 INFO:tasks.workunit.client.1.smithi193.stdout:9/320: symlink d2/d7/db/d32/l69 0 2024-03-23T08:23:19.774 INFO:tasks.workunit.client.1.smithi193.stdout:0/296: dwrite db/d11/f40 [4194304,4194304] 0 2024-03-23T08:23:19.780 INFO:tasks.workunit.client.1.smithi193.stdout:4/283: mknod d11/d27/c62 0 2024-03-23T08:23:19.785 INFO:tasks.workunit.client.1.smithi193.stdout:1/283: sync 2024-03-23T08:23:19.786 INFO:tasks.workunit.client.1.smithi193.stdout:9/321: creat d2/d7/db/d1b/f6a x:0 0 0 2024-03-23T08:23:19.790 INFO:tasks.workunit.client.1.smithi193.stdout:5/295: dwrite d0/ff [0,4194304] 0 2024-03-23T08:23:19.791 INFO:tasks.workunit.client.1.smithi193.stdout:8/258: dread df/f1f [0,4194304] 0 2024-03-23T08:23:19.794 INFO:tasks.workunit.client.1.smithi193.stdout:2/329: truncate d0/f1d 1143780 0 2024-03-23T08:23:19.794 INFO:tasks.workunit.client.1.smithi193.stdout:7/268: creat d6/d9/d1f/d3e/f5e x:0 0 0 2024-03-23T08:23:19.795 INFO:tasks.workunit.client.1.smithi193.stdout:4/284: mknod d11/d19/d22/c63 0 2024-03-23T08:23:19.795 INFO:tasks.workunit.client.1.smithi193.stdout:1/284: truncate da/d13/d24/f45 460382 0 2024-03-23T08:23:19.796 INFO:tasks.workunit.client.1.smithi193.stdout:8/259: unlink df/c50 0 2024-03-23T08:23:19.797 INFO:tasks.workunit.client.1.smithi193.stdout:0/297: creat db/d11/d14/d3b/f63 x:0 0 0 2024-03-23T08:23:19.797 INFO:tasks.workunit.client.1.smithi193.stdout:3/217: dwrite d1/fc [0,4194304] 0 2024-03-23T08:23:19.799 INFO:tasks.workunit.client.1.smithi193.stdout:5/296: rename d0/d1f/f28 to d0/d32/d45/f66 0 2024-03-23T08:23:19.799 INFO:tasks.workunit.client.1.smithi193.stdout:1/285: readlink da/dd/l60 0 2024-03-23T08:23:19.801 INFO:tasks.workunit.client.1.smithi193.stdout:5/297: chown d0/d1c/f24 273505 1 2024-03-23T08:23:19.802 INFO:tasks.workunit.client.1.smithi193.stdout:9/322: dread d2/d7/f1e [0,4194304] 0 2024-03-23T08:23:19.802 INFO:tasks.workunit.client.1.smithi193.stdout:1/286: unlink da/f46 0 2024-03-23T08:23:19.804 INFO:tasks.workunit.client.1.smithi193.stdout:8/260: symlink df/d4b/d4d/l5d 0 2024-03-23T08:23:19.805 INFO:tasks.workunit.client.1.smithi193.stdout:0/298: mknod db/d11/d14/d21/d33/c64 0 2024-03-23T08:23:19.807 INFO:tasks.workunit.client.1.smithi193.stdout:9/323: fsync d2/d7/f11 0 2024-03-23T08:23:19.807 INFO:tasks.workunit.client.1.smithi193.stdout:1/287: write da/dd/f39 [522965,50994] 0 2024-03-23T08:23:19.809 INFO:tasks.workunit.client.1.smithi193.stdout:9/324: write d2/d7/f1e [1484874,45007] 0 2024-03-23T08:23:19.809 INFO:tasks.workunit.client.1.smithi193.stdout:5/298: rename d0/f36 to d0/d2f/f67 0 2024-03-23T08:23:19.815 INFO:tasks.workunit.client.1.smithi193.stdout:2/330: link d0/d1b/d14/d25/c53 d0/d1b/d14/d25/c59 0 2024-03-23T08:23:19.816 INFO:tasks.workunit.client.1.smithi193.stdout:2/331: chown d0/d37/f4c 1 1 2024-03-23T08:23:19.816 INFO:tasks.workunit.client.1.smithi193.stdout:4/285: link d11/d1a/d33/d44/c46 d11/d48/d50/d49/c64 0 2024-03-23T08:23:19.819 INFO:tasks.workunit.client.1.smithi193.stdout:1/288: dwrite da/f3c [0,4194304] 0 2024-03-23T08:23:19.819 INFO:tasks.workunit.client.1.smithi193.stdout:5/299: symlink d0/d32/d57/l68 0 2024-03-23T08:23:19.820 INFO:tasks.workunit.client.1.smithi193.stdout:2/332: creat d0/d1b/d14/d25/f5a x:0 0 0 2024-03-23T08:23:19.821 INFO:tasks.workunit.client.1.smithi193.stdout:3/218: dread d1/de/d13/d21/d27/f1c [0,4194304] 0 2024-03-23T08:23:19.824 INFO:tasks.workunit.client.1.smithi193.stdout:8/261: getdents df/d10 0 2024-03-23T08:23:19.825 INFO:tasks.workunit.client.1.smithi193.stdout:5/300: write d0/f14 [1400394,69070] 0 2024-03-23T08:23:19.825 INFO:tasks.workunit.client.1.smithi193.stdout:4/286: rename d11/d1a/d60 to d11/d1a/d65 0 2024-03-23T08:23:19.827 INFO:tasks.workunit.client.1.smithi193.stdout:1/289: readlink da/dd/l5f 0 2024-03-23T08:23:19.829 INFO:tasks.workunit.client.1.smithi193.stdout:8/262: dread - df/d4b/d4d/f45 zero size 2024-03-23T08:23:19.832 INFO:tasks.workunit.client.1.smithi193.stdout:2/333: getdents d0/d1b/d3b 0 2024-03-23T08:23:19.833 INFO:tasks.workunit.client.1.smithi193.stdout:1/290: mknod da/dd/df/d52/c66 0 2024-03-23T08:23:19.834 INFO:tasks.workunit.client.1.smithi193.stdout:8/263: chown df/d10/d19/f1b 452955 1 2024-03-23T08:23:19.835 INFO:tasks.workunit.client.1.smithi193.stdout:0/299: dwrite db/d11/d14/d24/f4a [0,4194304] 0 2024-03-23T08:23:19.836 INFO:tasks.workunit.client.1.smithi193.stdout:5/301: rename d0/c54 to d0/d1c/d3b/d4b/d55/c69 0 2024-03-23T08:23:19.839 INFO:tasks.workunit.client.1.smithi193.stdout:1/291: creat da/d13/d3f/d5a/f67 x:0 0 0 2024-03-23T08:23:19.843 INFO:tasks.workunit.client.1.smithi193.stdout:5/302: symlink d0/d5a/l6a 0 2024-03-23T08:23:19.845 INFO:tasks.workunit.client.1.smithi193.stdout:8/264: chown df/d4b/d4d/d2a/f49 9 1 2024-03-23T08:23:19.845 INFO:tasks.workunit.client.1.smithi193.stdout:0/300: rename db/d11/d14/d21/d33/d48/d4f to db/d11/d14/d21/d33/d48/d65 0 2024-03-23T08:23:19.847 INFO:tasks.workunit.client.1.smithi193.stdout:2/334: creat d0/d1b/d14/d25/d2a/d4e/f5b x:0 0 0 2024-03-23T08:23:19.847 INFO:tasks.workunit.client.1.smithi193.stdout:9/325: dwrite d2/d7/db/d17/f53 [0,4194304] 0 2024-03-23T08:23:19.849 INFO:tasks.workunit.client.1.smithi193.stdout:5/303: mknod d0/c6b 0 2024-03-23T08:23:19.850 INFO:tasks.workunit.client.1.smithi193.stdout:1/292: dread da/dd/f39 [0,4194304] 0 2024-03-23T08:23:19.851 INFO:tasks.workunit.client.1.smithi193.stdout:8/265: rename df/l1c to df/d46/l5e 0 2024-03-23T08:23:19.853 INFO:tasks.workunit.client.1.smithi193.stdout:9/326: write d2/d7/f1e [502288,113497] 0 2024-03-23T08:23:19.855 INFO:tasks.workunit.client.1.smithi193.stdout:0/301: creat db/d11/d14/d24/d2a/d4d/f66 x:0 0 0 2024-03-23T08:23:19.858 INFO:tasks.workunit.client.1.smithi193.stdout:4/287: dwrite d11/d48/f5d [0,4194304] 0 2024-03-23T08:23:19.860 INFO:tasks.workunit.client.1.smithi193.stdout:1/293: creat da/dd/df/d52/f68 x:0 0 0 2024-03-23T08:23:19.860 INFO:tasks.workunit.client.1.smithi193.stdout:8/266: creat df/d10/d3e/f5f x:0 0 0 2024-03-23T08:23:19.861 INFO:tasks.workunit.client.1.smithi193.stdout:9/327: mkdir d2/d7/db/d32/d6b 0 2024-03-23T08:23:19.862 INFO:tasks.workunit.client.1.smithi193.stdout:5/304: link d0/d32/d45/f66 d0/d32/f6c 0 2024-03-23T08:23:19.863 INFO:tasks.workunit.client.1.smithi193.stdout:4/288: symlink d11/d2d/d3a/l66 0 2024-03-23T08:23:19.864 INFO:tasks.workunit.client.1.smithi193.stdout:9/328: chown d2/d7/f1d 394 1 2024-03-23T08:23:19.865 INFO:tasks.workunit.client.1.smithi193.stdout:0/302: symlink db/d11/d14/d21/d33/d48/l67 0 2024-03-23T08:23:19.866 INFO:tasks.workunit.client.1.smithi193.stdout:5/305: mkdir d0/d32/d57/d6d 0 2024-03-23T08:23:19.868 INFO:tasks.workunit.client.1.smithi193.stdout:4/289: chown d11/d1a/d33/d42/f29 272785173 1 2024-03-23T08:23:19.870 INFO:tasks.workunit.client.1.smithi193.stdout:8/267: getdents df/d10/d19 0 2024-03-23T08:23:19.871 INFO:tasks.workunit.client.1.smithi193.stdout:9/329: creat d2/d7/db/d32/d6b/f6c x:0 0 0 2024-03-23T08:23:19.873 INFO:tasks.workunit.client.1.smithi193.stdout:1/294: dwrite da/f12 [0,4194304] 0 2024-03-23T08:23:19.874 INFO:tasks.workunit.client.1.smithi193.stdout:5/306: truncate d0/ff 4556281 0 2024-03-23T08:23:19.875 INFO:tasks.workunit.client.1.smithi193.stdout:1/295: stat f5 0 2024-03-23T08:23:19.875 INFO:tasks.workunit.client.1.smithi193.stdout:0/303: rename db/d11/f13 to db/d11/d14/d21/d33/f68 0 2024-03-23T08:23:19.878 INFO:tasks.workunit.client.1.smithi193.stdout:1/296: readlink da/l38 0 2024-03-23T08:23:19.879 INFO:tasks.workunit.client.1.smithi193.stdout:8/268: write df/d10/f17 [2604774,26494] 0 2024-03-23T08:23:19.879 INFO:tasks.workunit.client.1.smithi193.stdout:9/330: symlink d2/d7/db/d17/d4b/l6d 0 2024-03-23T08:23:19.882 INFO:tasks.workunit.client.1.smithi193.stdout:5/307: write d0/d1f/f49 [5225060,129757] 0 2024-03-23T08:23:19.883 INFO:tasks.workunit.client.1.smithi193.stdout:2/335: dwrite d0/d1b/d19/f1a [0,4194304] 0 2024-03-23T08:23:19.884 INFO:tasks.workunit.client.1.smithi193.stdout:1/297: fsync da/dd/df/f2d 0 2024-03-23T08:23:19.884 INFO:tasks.workunit.client.1.smithi193.stdout:0/304: mkdir db/d11/d14/d21/d33/d69 0 2024-03-23T08:23:19.886 INFO:tasks.workunit.client.1.smithi193.stdout:6/244: sync 2024-03-23T08:23:19.887 INFO:tasks.workunit.client.1.smithi193.stdout:9/331: truncate d2/d7/db/d32/f67 92316 0 2024-03-23T08:23:19.887 INFO:tasks.workunit.client.1.smithi193.stdout:8/269: unlink df/d10/l37 0 2024-03-23T08:23:19.888 INFO:tasks.workunit.client.1.smithi193.stdout:1/298: dread - da/d13/d3f/f56 zero size 2024-03-23T08:23:19.889 INFO:tasks.workunit.client.1.smithi193.stdout:4/290: dwrite d11/d48/d50/d49/f5c [0,4194304] 0 2024-03-23T08:23:19.889 INFO:tasks.workunit.client.1.smithi193.stdout:5/308: mkdir d0/d32/d45/d6e 0 2024-03-23T08:23:19.891 INFO:tasks.workunit.client.1.smithi193.stdout:2/336: creat d0/d1b/d19/f5c x:0 0 0 2024-03-23T08:23:19.892 INFO:tasks.workunit.client.1.smithi193.stdout:5/309: chown d0/d1c/c5e 53616085 1 2024-03-23T08:23:19.893 INFO:tasks.workunit.client.1.smithi193.stdout:1/299: dread - da/d13/d3f/d5a/f65 zero size 2024-03-23T08:23:19.894 INFO:tasks.workunit.client.1.smithi193.stdout:6/245: write d0/d8/d3a/f45 [598805,19842] 0 2024-03-23T08:23:19.894 INFO:tasks.workunit.client.1.smithi193.stdout:0/305: symlink db/d11/d14/d24/d2a/d4d/l6a 0 2024-03-23T08:23:19.896 INFO:tasks.workunit.client.1.smithi193.stdout:0/306: write db/d11/d14/d21/f49 [253642,83545] 0 2024-03-23T08:23:19.898 INFO:tasks.workunit.client.1.smithi193.stdout:0/307: chown db/d11/d14/d24/f4a 70432 1 2024-03-23T08:23:19.901 INFO:tasks.workunit.client.1.smithi193.stdout:4/291: creat d11/d48/d50/d49/f67 x:0 0 0 2024-03-23T08:23:19.903 INFO:tasks.workunit.client.1.smithi193.stdout:0/308: symlink db/d11/d14/d3b/l6b 0 2024-03-23T08:23:19.903 INFO:tasks.workunit.client.1.smithi193.stdout:1/300: dwrite da/dd/df/f25 [0,4194304] 0 2024-03-23T08:23:19.903 INFO:tasks.workunit.client.1.smithi193.stdout:6/246: write d0/d8/d12/f1e [4442845,48086] 0 2024-03-23T08:23:19.904 INFO:tasks.workunit.client.1.smithi193.stdout:5/310: mknod d0/d32/d57/d6d/c6f 0 2024-03-23T08:23:19.906 INFO:tasks.workunit.client.1.smithi193.stdout:9/332: getdents d2 0 2024-03-23T08:23:19.906 INFO:tasks.workunit.client.1.smithi193.stdout:8/270: write df/d4b/d4d/f22 [223420,60075] 0 2024-03-23T08:23:19.908 INFO:tasks.workunit.client.1.smithi193.stdout:0/309: readlink db/l2f 0 2024-03-23T08:23:19.908 INFO:tasks.workunit.client.1.smithi193.stdout:6/247: readlink d0/l18 0 2024-03-23T08:23:19.909 INFO:tasks.workunit.client.1.smithi193.stdout:8/271: stat df/f18 0 2024-03-23T08:23:19.911 INFO:tasks.workunit.client.1.smithi193.stdout:1/301: chown da/d13/d3f/d5a/f65 283 1 2024-03-23T08:23:19.912 INFO:tasks.workunit.client.1.smithi193.stdout:2/337: link d0/d1b/d14/d25/c53 d0/d1b/d14/c5d 0 2024-03-23T08:23:19.913 INFO:tasks.workunit.client.1.smithi193.stdout:8/272: write df/f21 [72676,123444] 0 2024-03-23T08:23:19.913 INFO:tasks.workunit.client.1.smithi193.stdout:9/333: write d2/d7/db/d17/d33/f63 [203558,110320] 0 2024-03-23T08:23:19.917 INFO:tasks.workunit.client.1.smithi193.stdout:6/248: creat d0/db/df/d11/d3b/f49 x:0 0 0 2024-03-23T08:23:19.918 INFO:tasks.workunit.client.1.smithi193.stdout:2/338: creat d0/d37/d48/f5e x:0 0 0 2024-03-23T08:23:19.918 INFO:tasks.workunit.client.1.smithi193.stdout:1/302: rmdir da/dd/df/d2e 39 2024-03-23T08:23:19.922 INFO:tasks.workunit.client.1.smithi193.stdout:9/334: creat d2/d7/d31/d3c/f6e x:0 0 0 2024-03-23T08:23:19.922 INFO:tasks.workunit.client.1.smithi193.stdout:4/292: dwrite d11/d19/d22/f25 [4194304,4194304] 0 2024-03-23T08:23:19.924 INFO:tasks.workunit.client.1.smithi193.stdout:2/339: symlink d0/d28/d33/l5f 0 2024-03-23T08:23:19.926 INFO:tasks.workunit.client.1.smithi193.stdout:5/311: dwrite d0/d1c/d3b/d4b/f61 [0,4194304] 0 2024-03-23T08:23:19.927 INFO:tasks.workunit.client.1.smithi193.stdout:4/293: creat d11/d2d/f68 x:0 0 0 2024-03-23T08:23:19.929 INFO:tasks.workunit.client.1.smithi193.stdout:6/249: rename d0/ld to d0/d8/l4a 0 2024-03-23T08:23:19.931 INFO:tasks.workunit.client.1.smithi193.stdout:9/335: creat d2/d7/f6f x:0 0 0 2024-03-23T08:23:19.933 INFO:tasks.workunit.client.1.smithi193.stdout:5/312: symlink d0/d1c/d3b/l70 0 2024-03-23T08:23:19.936 INFO:tasks.workunit.client.1.smithi193.stdout:8/273: rename df/d4b/d4d/c24 to df/d10/d3e/d54/c60 0 2024-03-23T08:23:19.939 INFO:tasks.workunit.client.1.smithi193.stdout:1/303: dwrite da/d13/d3f/f56 [0,4194304] 0 2024-03-23T08:23:19.939 INFO:tasks.workunit.client.1.smithi193.stdout:4/294: unlink d11/d1a/d33/d42/d52/l51 0 2024-03-23T08:23:19.940 INFO:tasks.workunit.client.1.smithi193.stdout:8/274: write df/f33 [643001,87750] 0 2024-03-23T08:23:19.940 INFO:tasks.workunit.client.1.smithi193.stdout:5/313: symlink d0/d5a/l71 0 2024-03-23T08:23:19.943 INFO:tasks.workunit.client.1.smithi193.stdout:9/336: mkdir d2/d7/db/d17/d33/d70 0 2024-03-23T08:23:19.943 INFO:tasks.workunit.client.1.smithi193.stdout:2/340: getdents d0/d1b/d14/d25/d2a 0 2024-03-23T08:23:19.945 INFO:tasks.workunit.client.1.smithi193.stdout:6/250: getdents d0/d8/d12/d21 0 2024-03-23T08:23:19.948 INFO:tasks.workunit.client.1.smithi193.stdout:8/275: rename le to df/d4b/d4d/l61 0 2024-03-23T08:23:19.949 INFO:tasks.workunit.client.1.smithi193.stdout:0/310: dread db/d11/d14/d21/d33/f68 [0,4194304] 0 2024-03-23T08:23:19.949 INFO:tasks.workunit.client.1.smithi193.stdout:6/251: symlink d0/db/df/d11/d3b/l4b 0 2024-03-23T08:23:19.951 INFO:tasks.workunit.client.1.smithi193.stdout:4/295: rename d11/d1a/d33/d42/c21 to d11/d19/c69 0 2024-03-23T08:23:19.953 INFO:tasks.workunit.client.1.smithi193.stdout:9/337: mkdir d2/d7/db/d1b/d43/d71 0 2024-03-23T08:23:19.956 INFO:tasks.workunit.client.1.smithi193.stdout:8/276: mkdir df/d62 0 2024-03-23T08:23:19.956 INFO:tasks.workunit.client.1.smithi193.stdout:6/252: dread d0/db/f33 [0,4194304] 0 2024-03-23T08:23:19.956 INFO:tasks.workunit.client.1.smithi193.stdout:4/296: rmdir d11/d2d 39 2024-03-23T08:23:19.958 INFO:tasks.workunit.client.1.smithi193.stdout:9/338: mknod d2/d7/db/d32/d6b/c72 0 2024-03-23T08:23:19.959 INFO:tasks.workunit.client.1.smithi193.stdout:6/253: mknod d0/d8/d12/c4c 0 2024-03-23T08:23:19.960 INFO:tasks.workunit.client.1.smithi193.stdout:1/304: rename da/dd/df/d2e/l3a to da/dd/l69 0 2024-03-23T08:23:19.961 INFO:tasks.workunit.client.1.smithi193.stdout:6/254: chown d0/db/l1a 816853147 1 2024-03-23T08:23:19.961 INFO:tasks.workunit.client.1.smithi193.stdout:0/311: rename db/d11/d14/d24/d2a to db/d11/d14/d24/d2a/d4d/d6c 22 2024-03-23T08:23:19.962 INFO:tasks.workunit.client.1.smithi193.stdout:0/312: fdatasync db/f1e 0 2024-03-23T08:23:19.963 INFO:tasks.workunit.client.1.smithi193.stdout:6/255: chown d0/db/f33 89641663 1 2024-03-23T08:23:19.964 INFO:tasks.workunit.client.1.smithi193.stdout:5/314: dread d0/f7 [0,4194304] 0 2024-03-23T08:23:19.964 INFO:tasks.workunit.client.1.smithi193.stdout:4/297: link d11/d19/d22/f25 d11/d19/d22/f6a 0 2024-03-23T08:23:19.965 INFO:tasks.workunit.client.1.smithi193.stdout:1/305: write da/d13/d3f/f56 [2103423,103222] 0 2024-03-23T08:23:19.969 INFO:tasks.workunit.client.1.smithi193.stdout:2/341: dread d0/f2d [0,4194304] 0 2024-03-23T08:23:19.970 INFO:tasks.workunit.client.1.smithi193.stdout:0/313: symlink db/d11/d14/d24/d47/l6d 0 2024-03-23T08:23:19.971 INFO:tasks.workunit.client.1.smithi193.stdout:1/306: fsync da/dd/f14 0 2024-03-23T08:23:19.976 INFO:tasks.workunit.client.1.smithi193.stdout:6/256: rename d0/d8/c2f to d0/db/df/c4d 0 2024-03-23T08:23:19.976 INFO:tasks.workunit.client.1.smithi193.stdout:9/339: dwrite d2/d7/f36 [4194304,4194304] 0 2024-03-23T08:23:19.976 INFO:tasks.workunit.client.1.smithi193.stdout:9/340: write d2/d7/d31/f4f [4697333,19055] 0 2024-03-23T08:23:19.977 INFO:tasks.workunit.client.1.smithi193.stdout:1/307: fdatasync da/d13/d3f/d5a/f61 0 2024-03-23T08:23:19.978 INFO:tasks.workunit.client.1.smithi193.stdout:2/342: write d0/d1b/d14/d25/d2a/d4e/f5b [59330,71675] 0 2024-03-23T08:23:19.979 INFO:tasks.workunit.client.1.smithi193.stdout:6/257: fdatasync d0/db/df/f25 0 2024-03-23T08:23:19.980 INFO:tasks.workunit.client.1.smithi193.stdout:1/308: chown da/c10 0 1 2024-03-23T08:23:19.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:19 smithi070 ceph-mon[39699]: pgmap v19: 65 pgs: 65 active+clean; 635 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 3.9 MiB/s rd, 66 MiB/s wr, 424 op/s 2024-03-23T08:23:19.988 INFO:tasks.workunit.client.1.smithi193.stdout:0/314: dwrite db/d11/f12 [0,4194304] 0 2024-03-23T08:23:19.989 INFO:tasks.workunit.client.1.smithi193.stdout:1/309: creat da/dd/df/d2e/f6a x:0 0 0 2024-03-23T08:23:19.991 INFO:tasks.workunit.client.1.smithi193.stdout:2/343: rename d0/d1b/d49 to d0/d1b/d14/d25/d2a/d50/d60 0 2024-03-23T08:23:19.992 INFO:tasks.workunit.client.1.smithi193.stdout:4/298: dread d11/d1a/d33/d44/f4a [0,4194304] 0 2024-03-23T08:23:19.992 INFO:tasks.workunit.client.1.smithi193.stdout:9/341: dread d2/f4 [0,4194304] 0 2024-03-23T08:23:19.993 INFO:tasks.workunit.client.1.smithi193.stdout:0/315: fdatasync db/d11/d14/d24/f5f 0 2024-03-23T08:23:19.993 INFO:tasks.workunit.client.1.smithi193.stdout:1/310: chown da/dd/l69 46717858 1 2024-03-23T08:23:19.993 INFO:tasks.workunit.client.1.smithi193.stdout:5/315: dread d0/f27 [0,4194304] 0 2024-03-23T08:23:19.994 INFO:tasks.workunit.client.1.smithi193.stdout:6/258: dread d0/db/df/d11/f30 [0,4194304] 0 2024-03-23T08:23:19.995 INFO:tasks.workunit.client.1.smithi193.stdout:2/344: symlink d0/d37/l61 0 2024-03-23T08:23:19.997 INFO:tasks.workunit.client.1.smithi193.stdout:6/259: chown d0/db/c26 159405 1 2024-03-23T08:23:19.997 INFO:tasks.workunit.client.1.smithi193.stdout:0/316: fsync db/d11/d14/d21/f41 0 2024-03-23T08:23:19.997 INFO:tasks.workunit.client.1.smithi193.stdout:5/316: mkdir d0/d2f/d72 0 2024-03-23T08:23:19.998 INFO:tasks.workunit.client.1.smithi193.stdout:4/299: truncate d11/d1a/d33/d42/f4d 528705 0 2024-03-23T08:23:19.998 INFO:tasks.workunit.client.1.smithi193.stdout:9/342: mkdir d2/d7/d31/d5f/d73 0 2024-03-23T08:23:19.999 INFO:tasks.workunit.client.1.smithi193.stdout:0/317: stat db/d11/d14/d24/f4a 0 2024-03-23T08:23:20.001 INFO:tasks.workunit.client.1.smithi193.stdout:9/343: write d2/d7/db/f3b [2311980,33329] 0 2024-03-23T08:23:20.001 INFO:tasks.workunit.client.1.smithi193.stdout:6/260: write d0/db/df/d11/d36/f41 [778307,2053] 0 2024-03-23T08:23:20.003 INFO:tasks.workunit.client.1.smithi193.stdout:5/317: write d0/d1c/d3b/f3c [961129,111864] 0 2024-03-23T08:23:20.004 INFO:tasks.workunit.client.1.smithi193.stdout:0/318: stat db/d11/c35 0 2024-03-23T08:23:20.005 INFO:tasks.workunit.client.1.smithi193.stdout:0/319: stat db/d11/c35 0 2024-03-23T08:23:20.006 INFO:tasks.workunit.client.1.smithi193.stdout:9/344: fsync d2/d7/f6f 0 2024-03-23T08:23:20.007 INFO:tasks.workunit.client.1.smithi193.stdout:5/318: stat d0/d1f/f5b 0 2024-03-23T08:23:20.011 INFO:tasks.workunit.client.1.smithi193.stdout:0/320: dread - db/d11/d14/d21/d33/d48/f5d zero size 2024-03-23T08:23:20.011 INFO:tasks.workunit.client.1.smithi193.stdout:2/345: dwrite d0/d1b/d14/f40 [0,4194304] 0 2024-03-23T08:23:20.012 INFO:tasks.workunit.client.1.smithi193.stdout:5/319: chown d0/d32/l33 6276061 1 2024-03-23T08:23:20.014 INFO:tasks.workunit.client.1.smithi193.stdout:9/345: write d2/d7/db/d17/f54 [221584,18355] 0 2024-03-23T08:23:20.016 INFO:tasks.workunit.client.1.smithi193.stdout:2/346: creat d0/d28/f62 x:0 0 0 2024-03-23T08:23:20.018 INFO:tasks.workunit.client.1.smithi193.stdout:5/320: rename d0/d32 to d0/d2f/d72/d73 0 2024-03-23T08:23:20.019 INFO:tasks.workunit.client.1.smithi193.stdout:0/321: write db/d11/d14/d21/d33/f43 [1598778,18134] 0 2024-03-23T08:23:20.020 INFO:tasks.workunit.client.1.smithi193.stdout:4/300: dwrite d11/d2d/d3a/f5e [0,4194304] 0 2024-03-23T08:23:20.022 INFO:tasks.workunit.client.1.smithi193.stdout:1/311: dread da/d13/d3f/f5e [0,4194304] 0 2024-03-23T08:23:20.024 INFO:tasks.workunit.client.1.smithi193.stdout:9/346: creat d2/d7/db/d1b/d43/f74 x:0 0 0 2024-03-23T08:23:20.024 INFO:tasks.workunit.client.1.smithi193.stdout:1/312: write f8 [191593,123241] 0 2024-03-23T08:23:20.027 INFO:tasks.workunit.client.1.smithi193.stdout:9/347: chown d2/d7/f6f 273161695 1 2024-03-23T08:23:20.029 INFO:tasks.workunit.client.1.smithi193.stdout:0/322: symlink db/d11/d14/d24/d47/d59/l6e 0 2024-03-23T08:23:20.029 INFO:tasks.workunit.client.1.smithi193.stdout:2/347: dread d0/f1d [0,4194304] 0 2024-03-23T08:23:20.031 INFO:tasks.workunit.client.1.smithi193.stdout:5/321: link d0/f17 d0/d2f/d72/d73/d45/f74 0 2024-03-23T08:23:20.033 INFO:tasks.workunit.client.1.smithi193.stdout:6/261: dread d0/db/df/d11/f2e [0,4194304] 0 2024-03-23T08:23:20.034 INFO:tasks.workunit.client.1.smithi193.stdout:0/323: creat db/f6f x:0 0 0 2024-03-23T08:23:20.034 INFO:tasks.workunit.client.1.smithi193.stdout:6/262: chown d0/l18 334 1 2024-03-23T08:23:20.034 INFO:tasks.workunit.client.1.smithi193.stdout:9/348: getdents d2/d7/db/d17 0 2024-03-23T08:23:20.036 INFO:tasks.workunit.client.1.smithi193.stdout:5/322: link d0/d1c/f24 d0/d5a/f75 0 2024-03-23T08:23:20.038 INFO:tasks.workunit.client.1.smithi193.stdout:9/349: stat d2/d7/db/d17/c55 0 2024-03-23T08:23:20.038 INFO:tasks.workunit.client.1.smithi193.stdout:5/323: write d0/f43 [1276725,22782] 0 2024-03-23T08:23:20.040 INFO:tasks.workunit.client.1.smithi193.stdout:0/324: read db/d11/d14/d21/f27 [5550920,22464] 0 2024-03-23T08:23:20.040 INFO:tasks.workunit.client.1.smithi193.stdout:6/263: read d0/fe [4135373,45894] 0 2024-03-23T08:23:20.048 INFO:tasks.workunit.client.1.smithi193.stdout:0/325: mknod db/d11/d14/d24/d2a/c70 0 2024-03-23T08:23:20.052 INFO:tasks.workunit.client.1.smithi193.stdout:4/301: dwrite fd [0,4194304] 0 2024-03-23T08:23:20.053 INFO:tasks.workunit.client.1.smithi193.stdout:1/313: dwrite da/d13/d3f/f5e [0,4194304] 0 2024-03-23T08:23:20.053 INFO:tasks.workunit.client.1.smithi193.stdout:2/348: dread d0/fc [4194304,4194304] 0 2024-03-23T08:23:20.055 INFO:tasks.workunit.client.1.smithi193.stdout:3/219: sync 2024-03-23T08:23:20.055 INFO:tasks.workunit.client.1.smithi193.stdout:7/269: sync 2024-03-23T08:23:20.063 INFO:tasks.workunit.client.1.smithi193.stdout:1/314: mkdir da/dd/df/d2e/d6b 0 2024-03-23T08:23:20.064 INFO:tasks.workunit.client.1.smithi193.stdout:9/350: dwrite d2/d7/f36 [4194304,4194304] 0 2024-03-23T08:23:20.067 INFO:tasks.workunit.client.1.smithi193.stdout:7/270: mkdir d6/d9/d1f/d5f 0 2024-03-23T08:23:20.071 INFO:tasks.workunit.client.1.smithi193.stdout:9/351: stat d2/d7/d31/f4f 0 2024-03-23T08:23:20.071 INFO:tasks.workunit.client.1.smithi193.stdout:2/349: mkdir d0/d1b/d14/d25/d2a/d4e/d63 0 2024-03-23T08:23:20.071 INFO:tasks.workunit.client.1.smithi193.stdout:5/324: dread d0/d2f/d72/d73/d45/f66 [0,4194304] 0 2024-03-23T08:23:20.073 INFO:tasks.workunit.client.1.smithi193.stdout:3/220: chown d1/de/d13/d21/d25/d2b/d3c/d4a 825047 1 2024-03-23T08:23:20.074 INFO:tasks.workunit.client.1.smithi193.stdout:0/326: dwrite fa [0,4194304] 0 2024-03-23T08:23:20.074 INFO:tasks.workunit.client.1.smithi193.stdout:6/264: dwrite d0/db/df/d11/d36/f41 [0,4194304] 0 2024-03-23T08:23:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:19 smithi193 ceph-mon[48370]: pgmap v19: 65 pgs: 65 active+clean; 635 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 3.9 MiB/s rd, 66 MiB/s wr, 424 op/s 2024-03-23T08:23:20.081 INFO:tasks.workunit.client.1.smithi193.stdout:2/350: creat d0/d1b/f64 x:0 0 0 2024-03-23T08:23:20.081 INFO:tasks.workunit.client.1.smithi193.stdout:0/327: mkdir db/d11/d14/d3b/d71 0 2024-03-23T08:23:20.082 INFO:tasks.workunit.client.1.smithi193.stdout:4/302: dwrite f4 [4194304,4194304] 0 2024-03-23T08:23:20.083 INFO:tasks.workunit.client.1.smithi193.stdout:6/265: truncate d0/d8/f1d 4555356 0 2024-03-23T08:23:20.083 INFO:tasks.workunit.client.1.smithi193.stdout:3/221: write d1/de/d13/d21/f31 [3570030,70072] 0 2024-03-23T08:23:20.085 INFO:tasks.workunit.client.1.smithi193.stdout:9/352: getdents d2/d7/d31/d5f/d73 0 2024-03-23T08:23:20.086 INFO:tasks.workunit.client.1.smithi193.stdout:5/325: rename d0/d1c/d3b/d4b/f61 to d0/d2f/f76 0 2024-03-23T08:23:20.087 INFO:tasks.workunit.client.1.smithi193.stdout:0/328: fdatasync db/f6f 0 2024-03-23T08:23:20.088 INFO:tasks.workunit.client.1.smithi193.stdout:1/315: dwrite da/f4e [0,4194304] 0 2024-03-23T08:23:20.089 INFO:tasks.workunit.client.1.smithi193.stdout:2/351: mknod d0/d37/d48/c65 0 2024-03-23T08:23:20.091 INFO:tasks.workunit.client.1.smithi193.stdout:3/222: creat d1/f51 x:0 0 0 2024-03-23T08:23:20.093 INFO:tasks.workunit.client.1.smithi193.stdout:9/353: write d2/d7/db/d32/f5a [3864360,37040] 0 2024-03-23T08:23:20.095 INFO:tasks.workunit.client.1.smithi193.stdout:6/266: creat d0/db/f4e x:0 0 0 2024-03-23T08:23:20.096 INFO:tasks.workunit.client.1.smithi193.stdout:2/352: fdatasync d0/d1b/d3b/f58 0 2024-03-23T08:23:20.096 INFO:tasks.workunit.client.1.smithi193.stdout:3/223: rmdir d1/de 39 2024-03-23T08:23:20.097 INFO:tasks.workunit.client.1.smithi193.stdout:4/303: getdents d11/d2d/d3a 0 2024-03-23T08:23:20.099 INFO:tasks.workunit.client.1.smithi193.stdout:6/267: fdatasync d0/db/df/d11/d36/f41 0 2024-03-23T08:23:20.101 INFO:tasks.workunit.client.1.smithi193.stdout:0/329: link db/d11/d14/f1f db/d11/d14/d24/d47/d59/f72 0 2024-03-23T08:23:20.102 INFO:tasks.workunit.client.1.smithi193.stdout:7/271: dread d6/d9/d1f/d23/d33/d38/d3b/f48 [0,4194304] 0 2024-03-23T08:23:20.103 INFO:tasks.workunit.client.1.smithi193.stdout:1/316: truncate da/dd/df/f2d 1416575 0 2024-03-23T08:23:20.104 INFO:tasks.workunit.client.1.smithi193.stdout:1/317: readlink da/dd/l5f 0 2024-03-23T08:23:20.106 INFO:tasks.workunit.client.1.smithi193.stdout:9/354: mkdir d2/d7/db/d17/d4b/d59/d75 0 2024-03-23T08:23:20.107 INFO:tasks.workunit.client.1.smithi193.stdout:1/318: write da/dd/df/f1d [193657,94367] 0 2024-03-23T08:23:20.109 INFO:tasks.workunit.client.1.smithi193.stdout:4/304: truncate d11/d19/d22/f6a 5801741 0 2024-03-23T08:23:20.110 INFO:tasks.workunit.client.1.smithi193.stdout:7/272: link d6/f14 d6/d9/d1f/d3e/d42/f60 0 2024-03-23T08:23:20.110 INFO:tasks.workunit.client.1.smithi193.stdout:0/330: fdatasync db/d11/d14/d21/d33/d48/f54 0 2024-03-23T08:23:20.112 INFO:tasks.workunit.client.1.smithi193.stdout:4/305: readlink d11/d2d/d3a/l66 0 2024-03-23T08:23:20.112 INFO:tasks.workunit.client.1.smithi193.stdout:3/224: dread d1/de/d13/d21/d25/d2b/f35 [0,4194304] 0 2024-03-23T08:23:20.114 INFO:tasks.workunit.client.1.smithi193.stdout:2/353: rename d0/d1b/d14/d25/d2a to d0/d1b/d14/d66 0 2024-03-23T08:23:20.117 INFO:tasks.workunit.client.1.smithi193.stdout:2/354: fsync d0/d37/d48/f5e 0 2024-03-23T08:23:20.119 INFO:tasks.workunit.client.1.smithi193.stdout:3/225: write d1/de/d13/d21/d50/d44/f4f [250868,84054] 0 2024-03-23T08:23:20.120 INFO:tasks.workunit.client.1.smithi193.stdout:7/273: creat d6/d9/d1f/d23/d33/d38/d46/f61 x:0 0 0 2024-03-23T08:23:20.125 INFO:tasks.workunit.client.1.smithi193.stdout:2/355: chown d0/d28/f47 11737382 1 2024-03-23T08:23:20.125 INFO:tasks.workunit.client.1.smithi193.stdout:9/355: dwrite d2/d7/db/d17/d33/f64 [0,4194304] 0 2024-03-23T08:23:20.127 INFO:tasks.workunit.client.1.smithi193.stdout:7/274: fdatasync d6/d9/d1f/d23/d33/f34 0 2024-03-23T08:23:20.127 INFO:tasks.workunit.client.1.smithi193.stdout:4/306: truncate d11/d19/d22/f31 7950573 0 2024-03-23T08:23:20.128 INFO:tasks.workunit.client.1.smithi193.stdout:3/226: mknod d1/d22/c52 0 2024-03-23T08:23:20.129 INFO:tasks.workunit.client.1.smithi193.stdout:1/319: dread f1 [0,4194304] 0 2024-03-23T08:23:20.129 INFO:tasks.workunit.client.1.smithi193.stdout:0/331: link db/d11/d14/d24/d2a/c70 db/d11/d14/c73 0 2024-03-23T08:23:20.130 INFO:tasks.workunit.client.1.smithi193.stdout:6/268: dread d0/d8/d12/f1b [0,4194304] 0 2024-03-23T08:23:20.130 INFO:tasks.workunit.client.1.smithi193.stdout:4/307: chown d11/d2d/d58 202329 1 2024-03-23T08:23:20.132 INFO:tasks.workunit.client.1.smithi193.stdout:9/356: dread d2/d7/db/d17/f54 [0,4194304] 0 2024-03-23T08:23:20.133 INFO:tasks.workunit.client.1.smithi193.stdout:1/320: truncate da/d13/f49 569194 0 2024-03-23T08:23:20.135 INFO:tasks.workunit.client.1.smithi193.stdout:4/308: creat d11/d19/d22/f6b x:0 0 0 2024-03-23T08:23:20.136 INFO:tasks.workunit.client.1.smithi193.stdout:0/332: creat db/d11/d14/d24/d47/f74 x:0 0 0 2024-03-23T08:23:20.136 INFO:tasks.workunit.client.1.smithi193.stdout:3/227: dread d1/de/d13/d21/d25/d2b/f35 [0,4194304] 0 2024-03-23T08:23:20.160 INFO:tasks.workunit.client.1.smithi193.stdout:8/277: sync 2024-03-23T08:23:20.174 INFO:tasks.workunit.client.1.smithi193.stdout:4/309: write d11/d1a/f1c [13132767,107835] 0 2024-03-23T08:23:20.174 INFO:tasks.workunit.client.1.smithi193.stdout:1/321: creat da/dd/df/d2e/d6b/f6c x:0 0 0 2024-03-23T08:23:20.174 INFO:tasks.workunit.client.1.smithi193.stdout:2/356: truncate d0/f2d 5850324 0 2024-03-23T08:23:20.176 INFO:tasks.workunit.client.1.smithi193.stdout:0/333: symlink db/d11/d14/d3b/l75 0 2024-03-23T08:23:20.177 INFO:tasks.workunit.client.1.smithi193.stdout:7/275: dwrite d6/d9/f19 [4194304,4194304] 0 2024-03-23T08:23:20.177 INFO:tasks.workunit.client.1.smithi193.stdout:8/278: mkdir df/d63 0 2024-03-23T08:23:20.178 INFO:tasks.workunit.client.1.smithi193.stdout:3/228: dread - d1/de/d13/d21/d25/d34/f47 zero size 2024-03-23T08:23:20.179 INFO:tasks.workunit.client.1.smithi193.stdout:4/310: write fd [4766013,56511] 0 2024-03-23T08:23:20.179 INFO:tasks.workunit.client.1.smithi193.stdout:1/322: fdatasync da/dd/df/d2e/f48 0 2024-03-23T08:23:20.181 INFO:tasks.workunit.client.1.smithi193.stdout:7/276: chown d6/d9/d1f/d5f 476739 1 2024-03-23T08:23:20.182 INFO:tasks.workunit.client.1.smithi193.stdout:8/279: unlink c9 0 2024-03-23T08:23:20.183 INFO:tasks.workunit.client.1.smithi193.stdout:6/269: dwrite d0/fe [4194304,4194304] 0 2024-03-23T08:23:20.185 INFO:tasks.workunit.client.1.smithi193.stdout:2/357: symlink d0/d1b/d14/d66/d4e/l67 0 2024-03-23T08:23:20.187 INFO:tasks.workunit.client.1.smithi193.stdout:1/323: mknod da/d13/c6d 0 2024-03-23T08:23:20.194 INFO:tasks.workunit.client.1.smithi193.stdout:1/324: creat da/dd/df/d52/f6e x:0 0 0 2024-03-23T08:23:20.195 INFO:tasks.workunit.client.1.smithi193.stdout:0/334: rename db/d11/d14/d24/d2a/f62 to db/d52/f76 0 2024-03-23T08:23:20.196 INFO:tasks.workunit.client.1.smithi193.stdout:4/311: dwrite d11/d19/d22/f2b [0,4194304] 0 2024-03-23T08:23:20.196 INFO:tasks.workunit.client.1.smithi193.stdout:7/277: symlink d6/d9/d1f/d23/d33/l62 0 2024-03-23T08:23:20.199 INFO:tasks.workunit.client.1.smithi193.stdout:6/270: dwrite d0/db/f20 [0,4194304] 0 2024-03-23T08:23:20.199 INFO:tasks.workunit.client.1.smithi193.stdout:0/335: read db/d11/d14/d21/d33/f68 [499374,8467] 0 2024-03-23T08:23:20.200 INFO:tasks.workunit.client.1.smithi193.stdout:8/280: mknod df/d4b/d4d/c64 0 2024-03-23T08:23:20.200 INFO:tasks.workunit.client.1.smithi193.stdout:4/312: fsync d11/d19/f20 0 2024-03-23T08:23:20.204 INFO:tasks.workunit.client.1.smithi193.stdout:8/281: read - df/d46/f47 zero size 2024-03-23T08:23:20.212 INFO:tasks.workunit.client.1.smithi193.stdout:1/325: dread da/dd/df/d2e/f48 [0,4194304] 0 2024-03-23T08:23:20.212 INFO:tasks.workunit.client.1.smithi193.stdout:3/229: dwrite d1/d22/f2d [0,4194304] 0 2024-03-23T08:23:20.214 INFO:tasks.workunit.client.1.smithi193.stdout:7/278: dwrite d6/fc [8388608,4194304] 0 2024-03-23T08:23:20.216 INFO:tasks.workunit.client.1.smithi193.stdout:1/326: chown da/f3c 518527257 1 2024-03-23T08:23:20.216 INFO:tasks.workunit.client.1.smithi193.stdout:0/336: mknod db/d52/c77 0 2024-03-23T08:23:20.217 INFO:tasks.workunit.client.1.smithi193.stdout:4/313: creat d11/d1a/d33/d42/d52/f6c x:0 0 0 2024-03-23T08:23:20.219 INFO:tasks.workunit.client.1.smithi193.stdout:7/279: chown d6/d9/d1f/d23/f2e 5666789 1 2024-03-23T08:23:20.227 INFO:tasks.workunit.client.1.smithi193.stdout:2/358: getdents d0/d1b/d14/d66 0 2024-03-23T08:23:20.227 INFO:tasks.workunit.client.1.smithi193.stdout:4/314: write d11/d1a/d33/d42/f29 [243238,128191] 0 2024-03-23T08:23:20.227 INFO:tasks.workunit.client.1.smithi193.stdout:7/280: mknod d6/d9/d1f/d23/d33/d3c/d41/c63 0 2024-03-23T08:23:20.227 INFO:tasks.workunit.client.1.smithi193.stdout:1/327: rename da/d13/d3f/f41 to da/d13/f6f 0 2024-03-23T08:23:20.227 INFO:tasks.workunit.client.1.smithi193.stdout:3/230: getdents d1/de/d28 0 2024-03-23T08:23:20.227 INFO:tasks.workunit.client.1.smithi193.stdout:8/282: link df/d4b/d4d/d2a/l4c df/d62/l65 0 2024-03-23T08:23:20.228 INFO:tasks.workunit.client.1.smithi193.stdout:7/281: write d6/d9/d1f/f50 [1250330,115587] 0 2024-03-23T08:23:20.228 INFO:tasks.workunit.client.1.smithi193.stdout:6/271: getdents d0 0 2024-03-23T08:23:20.229 INFO:tasks.workunit.client.1.smithi193.stdout:0/337: creat db/d11/d14/d21/f78 x:0 0 0 2024-03-23T08:23:20.229 INFO:tasks.workunit.client.1.smithi193.stdout:5/326: sync 2024-03-23T08:23:20.230 INFO:tasks.workunit.client.1.smithi193.stdout:1/328: symlink da/d13/d3f/d5a/l70 0 2024-03-23T08:23:20.231 INFO:tasks.workunit.client.1.smithi193.stdout:8/283: creat df/d4b/f66 x:0 0 0 2024-03-23T08:23:20.232 INFO:tasks.workunit.client.1.smithi193.stdout:3/231: mknod d1/de/d13/d21/c53 0 2024-03-23T08:23:20.232 INFO:tasks.workunit.client.1.smithi193.stdout:7/282: truncate d6/d9/d1f/d23/d33/d38/f44 103509 0 2024-03-23T08:23:20.233 INFO:tasks.workunit.client.1.smithi193.stdout:4/315: symlink d11/d1a/d33/l6d 0 2024-03-23T08:23:20.233 INFO:tasks.workunit.client.1.smithi193.stdout:8/284: chown df/d4b/d4d/d2a/c2e 57 1 2024-03-23T08:23:20.236 INFO:tasks.workunit.client.1.smithi193.stdout:6/272: creat d0/db/df/f4f x:0 0 0 2024-03-23T08:23:20.237 INFO:tasks.workunit.client.1.smithi193.stdout:8/285: truncate df/f21 481908 0 2024-03-23T08:23:20.237 INFO:tasks.workunit.client.1.smithi193.stdout:3/232: creat d1/de/d13/d21/d50/d44/f54 x:0 0 0 2024-03-23T08:23:20.238 INFO:tasks.workunit.client.1.smithi193.stdout:3/233: stat d1/de 0 2024-03-23T08:23:20.239 INFO:tasks.workunit.client.1.smithi193.stdout:6/273: stat d0/d8/d12/d21/d2a/l3c 0 2024-03-23T08:23:20.240 INFO:tasks.workunit.client.1.smithi193.stdout:7/283: rename d6/d9/d1f/l28 to d6/d9/d1f/d23/d33/d38/d46/l64 0 2024-03-23T08:23:20.241 INFO:tasks.workunit.client.1.smithi193.stdout:8/286: write df/f48 [865209,29916] 0 2024-03-23T08:23:20.244 INFO:tasks.workunit.client.1.smithi193.stdout:3/234: creat d1/d22/f55 x:0 0 0 2024-03-23T08:23:20.245 INFO:tasks.workunit.client.1.smithi193.stdout:8/287: mkdir df/d10/d67 0 2024-03-23T08:23:20.245 INFO:tasks.workunit.client.1.smithi193.stdout:3/235: stat d1/f1e 0 2024-03-23T08:23:20.247 INFO:tasks.workunit.client.1.smithi193.stdout:0/338: dwrite db/d11/d14/d21/f45 [0,4194304] 0 2024-03-23T08:23:20.247 INFO:tasks.workunit.client.1.smithi193.stdout:7/284: write d6/d9/d1f/d23/d33/d3c/f4b [3497790,129419] 0 2024-03-23T08:23:20.249 INFO:tasks.workunit.client.1.smithi193.stdout:5/327: dread d0/f14 [4194304,4194304] 0 2024-03-23T08:23:20.250 INFO:tasks.workunit.client.1.smithi193.stdout:3/236: dread - d1/de/d13/d21/d50/d44/f54 zero size 2024-03-23T08:23:20.251 INFO:tasks.workunit.client.1.smithi193.stdout:2/359: truncate d0/fc 1177574 0 2024-03-23T08:23:20.252 INFO:tasks.workunit.client.1.smithi193.stdout:0/339: write db/d11/d14/d24/d47/f74 [16982,33552] 0 2024-03-23T08:23:20.253 INFO:tasks.workunit.client.1.smithi193.stdout:8/288: read df/f34 [937197,81322] 0 2024-03-23T08:23:20.253 INFO:tasks.workunit.client.1.smithi193.stdout:2/360: stat d0/d1b/f8 0 2024-03-23T08:23:20.255 INFO:tasks.workunit.client.1.smithi193.stdout:5/328: creat d0/d2f/d72/d73/d45/f77 x:0 0 0 2024-03-23T08:23:20.256 INFO:tasks.workunit.client.1.smithi193.stdout:3/237: unlink d1/c4e 0 2024-03-23T08:23:20.257 INFO:tasks.workunit.client.1.smithi193.stdout:8/289: rmdir df/d10/d3e/d4e 39 2024-03-23T08:23:20.259 INFO:tasks.workunit.client.1.smithi193.stdout:9/357: sync 2024-03-23T08:23:20.259 INFO:tasks.workunit.client.1.smithi193.stdout:2/361: mknod d0/d1b/d19/c68 0 2024-03-23T08:23:20.262 INFO:tasks.workunit.client.1.smithi193.stdout:5/329: rmdir d0/d2f/d72/d73 39 2024-03-23T08:23:20.262 INFO:tasks.workunit.client.1.smithi193.stdout:2/362: write d0/d1b/f13 [2002427,41780] 0 2024-03-23T08:23:20.264 INFO:tasks.workunit.client.1.smithi193.stdout:4/316: dread d11/d19/f2f [0,4194304] 0 2024-03-23T08:23:20.265 INFO:tasks.workunit.client.1.smithi193.stdout:3/238: rename d1/fa to d1/de/d13/d21/d25/f56 0 2024-03-23T08:23:20.266 INFO:tasks.workunit.client.1.smithi193.stdout:9/358: rmdir d2 39 2024-03-23T08:23:20.266 INFO:tasks.workunit.client.1.smithi193.stdout:6/274: write d0/db/df/f25 [2220858,124703] 0 2024-03-23T08:23:20.266 INFO:tasks.workunit.client.1.smithi193.stdout:1/329: dwrite da/f3c [4194304,4194304] 0 2024-03-23T08:23:20.268 INFO:tasks.workunit.client.1.smithi193.stdout:0/340: getdents db/d11/d14/d3b/d71 0 2024-03-23T08:23:20.268 INFO:tasks.workunit.client.1.smithi193.stdout:2/363: write d0/d28/f43 [3817078,97624] 0 2024-03-23T08:23:20.272 INFO:tasks.workunit.client.1.smithi193.stdout:5/330: creat d0/d1f/f78 x:0 0 0 2024-03-23T08:23:20.272 INFO:tasks.workunit.client.1.smithi193.stdout:1/330: write da/d13/f49 [231313,54851] 0 2024-03-23T08:23:20.272 INFO:tasks.workunit.client.1.smithi193.stdout:0/341: readlink db/d11/d14/l18 0 2024-03-23T08:23:20.273 INFO:tasks.workunit.client.1.smithi193.stdout:3/239: truncate d1/de/d13/d21/f31 5120214 0 2024-03-23T08:23:20.274 INFO:tasks.workunit.client.1.smithi193.stdout:7/285: truncate d6/d9/d1f/d23/d33/d3c/f4b 918382 0 2024-03-23T08:23:20.276 INFO:tasks.workunit.client.1.smithi193.stdout:2/364: truncate d0/d28/f44 395427 0 2024-03-23T08:23:20.277 INFO:tasks.workunit.client.1.smithi193.stdout:4/317: creat d11/d1a/d33/d44/f6e x:0 0 0 2024-03-23T08:23:20.278 INFO:tasks.workunit.client.1.smithi193.stdout:3/240: write d1/fc [3217706,21745] 0 2024-03-23T08:23:20.278 INFO:tasks.workunit.client.1.smithi193.stdout:1/331: read da/d13/f49 [503653,93767] 0 2024-03-23T08:23:20.278 INFO:tasks.workunit.client.1.smithi193.stdout:9/359: fsync d2/d7/d31/d3c/f6e 0 2024-03-23T08:23:20.279 INFO:tasks.workunit.client.1.smithi193.stdout:0/342: write db/d11/d14/d21/d33/f43 [2764586,62909] 0 2024-03-23T08:23:20.283 INFO:tasks.workunit.client.1.smithi193.stdout:3/241: chown d1/de/d13/d21/d25 4269571 1 2024-03-23T08:23:20.284 INFO:tasks.workunit.client.1.smithi193.stdout:5/331: rename d0/d1c/c23 to d0/d5a/c79 0 2024-03-23T08:23:20.284 INFO:tasks.workunit.client.1.smithi193.stdout:0/343: read db/d11/d14/d21/f26 [155536,61255] 0 2024-03-23T08:23:20.284 INFO:tasks.workunit.client.1.smithi193.stdout:9/360: chown d2/fc 54 1 2024-03-23T08:23:20.286 INFO:tasks.workunit.client.1.smithi193.stdout:8/290: getdents df/d46 0 2024-03-23T08:23:20.289 INFO:tasks.workunit.client.1.smithi193.stdout:3/242: unlink d1/l41 0 2024-03-23T08:23:20.290 INFO:tasks.workunit.client.1.smithi193.stdout:7/286: mkdir d6/d9/d1f/d3e/d65 0 2024-03-23T08:23:20.291 INFO:tasks.workunit.client.1.smithi193.stdout:1/332: mknod da/dd/c71 0 2024-03-23T08:23:20.294 INFO:tasks.workunit.client.1.smithi193.stdout:2/365: dwrite d0/d37/d48/f5e [0,4194304] 0 2024-03-23T08:23:20.295 INFO:tasks.workunit.client.1.smithi193.stdout:9/361: rename d2/d7/db/d17/d1f/l21 to d2/d7/db/d1b/d43/l76 0 2024-03-23T08:23:20.296 INFO:tasks.workunit.client.1.smithi193.stdout:7/287: write d6/f32 [5195738,67737] 0 2024-03-23T08:23:20.299 INFO:tasks.workunit.client.1.smithi193.stdout:3/243: write d1/de/d13/d21/d25/d34/f47 [125027,112584] 0 2024-03-23T08:23:20.299 INFO:tasks.workunit.client.1.smithi193.stdout:8/291: link df/d10/f3b df/d63/f68 0 2024-03-23T08:23:20.299 INFO:tasks.workunit.client.1.smithi193.stdout:1/333: chown da/d13/d3f/d5a/f65 56286157 1 2024-03-23T08:23:20.301 INFO:tasks.workunit.client.1.smithi193.stdout:6/275: truncate d0/d8/f1d 3905954 0 2024-03-23T08:23:20.301 INFO:tasks.workunit.client.1.smithi193.stdout:2/366: mknod d0/d37/d48/c69 0 2024-03-23T08:23:20.304 INFO:tasks.workunit.client.1.smithi193.stdout:9/362: write d2/d7/d31/d3c/f6e [72492,83738] 0 2024-03-23T08:23:20.306 INFO:tasks.workunit.client.1.smithi193.stdout:4/318: dread d11/d1a/f1c [8388608,4194304] 0 2024-03-23T08:23:20.307 INFO:tasks.workunit.client.1.smithi193.stdout:3/244: dread - d1/d22/f30 zero size 2024-03-23T08:23:20.308 INFO:tasks.workunit.client.1.smithi193.stdout:0/344: dwrite db/d11/f40 [4194304,4194304] 0 2024-03-23T08:23:20.308 INFO:tasks.workunit.client.1.smithi193.stdout:6/276: readlink d0/db/df/d11/d3b/l4b 0 2024-03-23T08:23:20.308 INFO:tasks.workunit.client.1.smithi193.stdout:8/292: dread df/d4b/d4d/f22 [0,4194304] 0 2024-03-23T08:23:20.309 INFO:tasks.workunit.client.1.smithi193.stdout:7/288: read d6/d9/d1f/d23/d33/d3c/f4b [339470,1039] 0 2024-03-23T08:23:20.310 INFO:tasks.workunit.client.1.smithi193.stdout:4/319: chown d11/d2d/f4b 451856 1 2024-03-23T08:23:20.311 INFO:tasks.workunit.client.1.smithi193.stdout:9/363: mkdir d2/d7/db/d1b/d5c/d77 0 2024-03-23T08:23:20.313 INFO:tasks.workunit.client.1.smithi193.stdout:3/245: mknod d1/de/d3d/c57 0 2024-03-23T08:23:20.314 INFO:tasks.workunit.client.1.smithi193.stdout:6/277: symlink d0/d8/l50 0 2024-03-23T08:23:20.314 INFO:tasks.workunit.client.1.smithi193.stdout:4/320: chown d11/d1a/d33/c40 4371 1 2024-03-23T08:23:20.317 INFO:tasks.workunit.client.1.smithi193.stdout:3/246: dread - d1/d22/f55 zero size 2024-03-23T08:23:20.318 INFO:tasks.workunit.client.1.smithi193.stdout:4/321: fdatasync d11/d48/d50/d49/f5c 0 2024-03-23T08:23:20.319 INFO:tasks.workunit.client.1.smithi193.stdout:3/247: dread - d1/d22/f30 zero size 2024-03-23T08:23:20.321 INFO:tasks.workunit.client.1.smithi193.stdout:0/345: dwrite db/d11/d14/d21/f2c [0,4194304] 0 2024-03-23T08:23:20.322 INFO:tasks.workunit.client.1.smithi193.stdout:3/248: readlink d1/de/d13/d21/d25/l42 0 2024-03-23T08:23:20.322 INFO:tasks.workunit.client.1.smithi193.stdout:8/293: dwrite df/d10/d19/f4f [0,4194304] 0 2024-03-23T08:23:20.323 INFO:tasks.workunit.client.1.smithi193.stdout:8/294: read df/f56 [431398,36462] 0 2024-03-23T08:23:20.448 INFO:tasks.workunit.client.1.smithi193.stdout:5/332: truncate d0/f4d 696849 0 2024-03-23T08:23:20.448 INFO:tasks.workunit.client.1.smithi193.stdout:2/367: dread d0/d1b/f21 [0,4194304] 0 2024-03-23T08:23:20.451 INFO:tasks.workunit.client.1.smithi193.stdout:6/278: mkdir d0/db/df/d11/d3b/d51 0 2024-03-23T08:23:20.452 INFO:tasks.workunit.client.1.smithi193.stdout:6/279: fsync d0/d8/d12/f1e 0 2024-03-23T08:23:20.453 INFO:tasks.workunit.client.1.smithi193.stdout:6/280: dread - d0/db/df/d11/f37 zero size 2024-03-23T08:23:20.456 INFO:tasks.workunit.client.1.smithi193.stdout:4/322: rename fd to d11/d2d/d3a/f6f 0 2024-03-23T08:23:20.457 INFO:tasks.workunit.client.1.smithi193.stdout:3/249: mkdir d1/de/d13/d21/d50/d44/d58 0 2024-03-23T08:23:20.458 INFO:tasks.workunit.client.1.smithi193.stdout:1/334: write da/dd/f14 [1240486,120580] 0 2024-03-23T08:23:20.459 INFO:tasks.workunit.client.1.smithi193.stdout:8/295: unlink df/d4b/d4d/d2a/c2e 0 2024-03-23T08:23:20.460 INFO:tasks.workunit.client.1.smithi193.stdout:5/333: mkdir d0/d1c/d7a 0 2024-03-23T08:23:20.460 INFO:tasks.workunit.client.1.smithi193.stdout:8/296: dread - df/f2c zero size 2024-03-23T08:23:20.461 INFO:tasks.workunit.client.1.smithi193.stdout:2/368: mknod d0/d28/c6a 0 2024-03-23T08:23:20.461 INFO:tasks.workunit.client.1.smithi193.stdout:5/334: chown d0/d1c/d3b/d4b/d55/f63 391435408 1 2024-03-23T08:23:20.462 INFO:tasks.workunit.client.1.smithi193.stdout:9/364: mkdir d2/d7/d31/d5f/d73/d78 0 2024-03-23T08:23:20.463 INFO:tasks.workunit.client.1.smithi193.stdout:2/369: dread d0/d1b/d14/f45 [0,4194304] 0 2024-03-23T08:23:20.466 INFO:tasks.workunit.client.1.smithi193.stdout:0/346: rename fa to db/d11/d14/d24/d47/d59/f79 0 2024-03-23T08:23:20.466 INFO:tasks.workunit.client.1.smithi193.stdout:7/289: truncate d6/d9/d1f/f2a 3975358 0 2024-03-23T08:23:20.467 INFO:tasks.workunit.client.1.smithi193.stdout:0/347: read db/d11/f12 [4115005,46207] 0 2024-03-23T08:23:20.468 INFO:tasks.workunit.client.1.smithi193.stdout:4/323: mkdir d11/d1a/d33/d42/d52/d70 0 2024-03-23T08:23:20.470 INFO:tasks.workunit.client.1.smithi193.stdout:1/335: rmdir da/dd/df/d52 39 2024-03-23T08:23:20.475 INFO:tasks.workunit.client.1.smithi193.stdout:9/365: mkdir d2/d7/d31/d3c/d79 0 2024-03-23T08:23:20.477 INFO:tasks.workunit.client.1.smithi193.stdout:1/336: dwrite da/dd/df/f1d [0,4194304] 0 2024-03-23T08:23:20.478 INFO:tasks.workunit.client.1.smithi193.stdout:3/250: dwrite d1/de/d13/d21/d25/d2b/f35 [0,4194304] 0 2024-03-23T08:23:20.480 INFO:tasks.workunit.client.1.smithi193.stdout:1/337: dread - da/dd/df/d2e/d6b/f6c zero size 2024-03-23T08:23:20.483 INFO:tasks.workunit.client.1.smithi193.stdout:6/281: rename d0/db/df/d11/f2e to d0/d8/d12/f52 0 2024-03-23T08:23:20.483 INFO:tasks.workunit.client.1.smithi193.stdout:7/290: mkdir d6/d9/d1f/d23/d33/d38/d66 0 2024-03-23T08:23:20.485 INFO:tasks.workunit.client.1.smithi193.stdout:0/348: creat db/d11/d14/d24/d2a/d4d/f7a x:0 0 0 2024-03-23T08:23:20.486 INFO:tasks.workunit.client.1.smithi193.stdout:4/324: unlink d11/d1a/l47 0 2024-03-23T08:23:20.488 INFO:tasks.workunit.client.1.smithi193.stdout:9/366: dwrite d2/d7/f1d [0,4194304] 0 2024-03-23T08:23:20.490 INFO:tasks.workunit.client.1.smithi193.stdout:4/325: write d11/d48/d50/d49/f67 [230804,88797] 0 2024-03-23T08:23:20.494 INFO:tasks.workunit.client.1.smithi193.stdout:5/335: symlink d0/d2f/d72/d73/d57/l7b 0 2024-03-23T08:23:20.495 INFO:tasks.workunit.client.1.smithi193.stdout:3/251: mkdir d1/de/d13/d21/d59 0 2024-03-23T08:23:20.495 INFO:tasks.workunit.client.1.smithi193.stdout:6/282: dwrite d0/db/df/f25 [0,4194304] 0 2024-03-23T08:23:20.497 INFO:tasks.workunit.client.1.smithi193.stdout:1/338: symlink da/dd/df/d2e/l72 0 2024-03-23T08:23:20.498 INFO:tasks.workunit.client.1.smithi193.stdout:0/349: rename db/d11 to db/d11/d14/d24/d2a/d4b/d7b 22 2024-03-23T08:23:20.498 INFO:tasks.workunit.client.1.smithi193.stdout:5/336: chown d0/d2f/d72/d73/d45/f77 459 1 2024-03-23T08:23:20.499 INFO:tasks.workunit.client.1.smithi193.stdout:8/297: link f3 df/d4b/d4d/f69 0 2024-03-23T08:23:20.499 INFO:tasks.workunit.client.1.smithi193.stdout:9/367: mkdir d2/d7/db/d7a 0 2024-03-23T08:23:20.500 INFO:tasks.workunit.client.1.smithi193.stdout:2/370: link d0/d37/d48/c69 d0/d1b/d14/d66/d50/c6b 0 2024-03-23T08:23:20.500 INFO:tasks.workunit.client.1.smithi193.stdout:0/350: readlink db/d11/d14/d24/d47/d59/l6e 0 2024-03-23T08:23:20.501 INFO:tasks.workunit.client.1.smithi193.stdout:8/298: chown cb 191 1 2024-03-23T08:23:20.501 INFO:tasks.workunit.client.1.smithi193.stdout:1/339: rmdir da/dd/df 39 2024-03-23T08:23:20.501 INFO:tasks.workunit.client.1.smithi193.stdout:9/368: readlink d2/d7/l16 0 2024-03-23T08:23:20.503 INFO:tasks.workunit.client.1.smithi193.stdout:5/337: creat d0/d2f/d72/d73/d57/d6d/f7c x:0 0 0 2024-03-23T08:23:20.503 INFO:tasks.workunit.client.1.smithi193.stdout:3/252: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/d5a 0 2024-03-23T08:23:20.504 INFO:tasks.workunit.client.1.smithi193.stdout:4/326: getdents d11/d1a/d33/d42 0 2024-03-23T08:23:20.504 INFO:tasks.workunit.client.1.smithi193.stdout:8/299: mknod df/d10/c6a 0 2024-03-23T08:23:20.505 INFO:tasks.workunit.client.1.smithi193.stdout:5/338: write d0/d1f/f5b [771643,83255] 0 2024-03-23T08:23:20.505 INFO:tasks.workunit.client.1.smithi193.stdout:4/327: chown d11/d1a/d65 5932 1 2024-03-23T08:23:20.506 INFO:tasks.workunit.client.1.smithi193.stdout:3/253: chown d1/de/d3d/c57 14589 1 2024-03-23T08:23:20.507 INFO:tasks.workunit.client.1.smithi193.stdout:3/254: fsync d1/de/d13/d21/d25/d2b/f4d 0 2024-03-23T08:23:20.507 INFO:tasks.workunit.client.1.smithi193.stdout:2/371: creat d0/d1b/d14/d25/f6c x:0 0 0 2024-03-23T08:23:20.509 INFO:tasks.workunit.client.1.smithi193.stdout:0/351: creat db/d11/d14/d24/d47/d59/f7c x:0 0 0 2024-03-23T08:23:20.510 INFO:tasks.workunit.client.1.smithi193.stdout:8/300: chown df/d10/d3e/d4e 1834958 1 2024-03-23T08:23:20.513 INFO:tasks.workunit.client.1.smithi193.stdout:9/369: creat d2/d7/db/d17/d33/d70/f7b x:0 0 0 2024-03-23T08:23:20.513 INFO:tasks.workunit.client.1.smithi193.stdout:2/372: symlink d0/d1b/d14/d66/l6d 0 2024-03-23T08:23:20.514 INFO:tasks.workunit.client.1.smithi193.stdout:4/328: dwrite fb [0,4194304] 0 2024-03-23T08:23:20.516 INFO:tasks.workunit.client.1.smithi193.stdout:4/329: fdatasync d11/d1a/d33/d42/f59 0 2024-03-23T08:23:20.517 INFO:tasks.workunit.client.1.smithi193.stdout:5/339: truncate d0/f17 531850 0 2024-03-23T08:23:20.522 INFO:tasks.workunit.client.1.smithi193.stdout:2/373: mknod d0/d37/d48/c6e 0 2024-03-23T08:23:20.524 INFO:tasks.workunit.client.1.smithi193.stdout:9/370: symlink d2/d7/db/d7a/l7c 0 2024-03-23T08:23:20.527 INFO:tasks.workunit.client.1.smithi193.stdout:8/301: creat df/d10/d3e/f6b x:0 0 0 2024-03-23T08:23:20.528 INFO:tasks.workunit.client.1.smithi193.stdout:6/283: fdatasync d0/d8/f1d 0 2024-03-23T08:23:20.529 INFO:tasks.workunit.client.1.smithi193.stdout:3/255: dwrite d1/de/d13/d21/f23 [0,4194304] 0 2024-03-23T08:23:20.536 INFO:tasks.workunit.client.1.smithi193.stdout:6/284: fsync d0/d8/d12/d21/f29 0 2024-03-23T08:23:20.538 INFO:tasks.workunit.client.1.smithi193.stdout:4/330: dwrite d11/d1a/d33/d42/f59 [0,4194304] 0 2024-03-23T08:23:20.539 INFO:tasks.workunit.client.1.smithi193.stdout:3/256: truncate d1/de/f11 4495369 0 2024-03-23T08:23:20.540 INFO:tasks.workunit.client.1.smithi193.stdout:1/340: write da/dd/df/f2d [993837,83093] 0 2024-03-23T08:23:20.544 INFO:tasks.workunit.client.1.smithi193.stdout:9/371: getdents d2/d7/db/d17/d1f 0 2024-03-23T08:23:20.545 INFO:tasks.workunit.client.1.smithi193.stdout:7/291: chown d6/d9/d1f/f2a 7507 1 2024-03-23T08:23:20.545 INFO:tasks.workunit.client.1.smithi193.stdout:8/302: write df/f1f [1796084,1904] 0 2024-03-23T08:23:20.546 INFO:tasks.workunit.client.1.smithi193.stdout:1/341: rmdir da/dd/df 39 2024-03-23T08:23:20.548 INFO:tasks.workunit.client.1.smithi193.stdout:4/331: truncate d11/d2d/f68 320262 0 2024-03-23T08:23:20.549 INFO:tasks.workunit.client.1.smithi193.stdout:3/257: creat d1/de/d13/f5b x:0 0 0 2024-03-23T08:23:20.550 INFO:tasks.workunit.client.1.smithi193.stdout:2/374: dwrite d0/d1b/d14/f41 [0,4194304] 0 2024-03-23T08:23:20.550 INFO:tasks.workunit.client.1.smithi193.stdout:9/372: readlink d2/d7/d31/d3c/l47 0 2024-03-23T08:23:20.551 INFO:tasks.workunit.client.1.smithi193.stdout:5/340: dread d0/ff [0,4194304] 0 2024-03-23T08:23:20.551 INFO:tasks.workunit.client.1.smithi193.stdout:7/292: truncate d6/d9/f36 399778 0 2024-03-23T08:23:20.551 INFO:tasks.workunit.client.1.smithi193.stdout:3/258: dread - d1/f20 zero size 2024-03-23T08:23:20.552 INFO:tasks.workunit.client.1.smithi193.stdout:4/332: chown d11/d2d/f4b 2582846 1 2024-03-23T08:23:20.555 INFO:tasks.workunit.client.1.smithi193.stdout:9/373: creat d2/f7d x:0 0 0 2024-03-23T08:23:20.555 INFO:tasks.workunit.client.1.smithi193.stdout:3/259: stat d1/d22 0 2024-03-23T08:23:20.556 INFO:tasks.workunit.client.1.smithi193.stdout:7/293: chown d6/d9/f21 4 1 2024-03-23T08:23:20.561 INFO:tasks.workunit.client.1.smithi193.stdout:3/260: rename d1/de/d3d/c57 to d1/d22/c5c 0 2024-03-23T08:23:20.563 INFO:tasks.workunit.client.1.smithi193.stdout:3/261: write d1/de/d13/d21/d27/f1f [276569,25999] 0 2024-03-23T08:23:20.563 INFO:tasks.workunit.client.1.smithi193.stdout:5/341: truncate d0/f7 4179167 0 2024-03-23T08:23:20.565 INFO:tasks.workunit.client.1.smithi193.stdout:1/342: link da/dd/df/d2e/c3e da/c73 0 2024-03-23T08:23:20.565 INFO:tasks.workunit.client.1.smithi193.stdout:7/294: write d6/d9/d1f/d23/d33/d38/d3b/f5a [610082,60762] 0 2024-03-23T08:23:20.567 INFO:tasks.workunit.client.1.smithi193.stdout:5/342: read - d0/d1c/d3b/d4b/d55/f59 zero size 2024-03-23T08:23:20.568 INFO:tasks.workunit.client.1.smithi193.stdout:3/262: write d1/de/d13/f24 [3607733,54492] 0 2024-03-23T08:23:20.570 INFO:tasks.workunit.client.1.smithi193.stdout:1/343: write da/dd/df/f2d [1625934,13467] 0 2024-03-23T08:23:20.574 INFO:tasks.workunit.client.1.smithi193.stdout:9/374: symlink d2/d7/db/d1b/l7e 0 2024-03-23T08:23:20.574 INFO:tasks.workunit.client.1.smithi193.stdout:3/263: truncate d1/de/d13/d21/f23 5748230 0 2024-03-23T08:23:20.575 INFO:tasks.workunit.client.1.smithi193.stdout:2/375: dread d0/d37/f4c [0,4194304] 0 2024-03-23T08:23:20.576 INFO:tasks.workunit.client.1.smithi193.stdout:5/343: getdents d0/d1c/d7a 0 2024-03-23T08:23:20.579 INFO:tasks.workunit.client.1.smithi193.stdout:9/375: truncate d2/d7/d31/f40 875979 0 2024-03-23T08:23:20.579 INFO:tasks.workunit.client.1.smithi193.stdout:3/264: mknod d1/de/d13/d21/d27/c5d 0 2024-03-23T08:23:20.579 INFO:tasks.workunit.client.1.smithi193.stdout:4/333: dread d11/d1a/f1c [0,4194304] 0 2024-03-23T08:23:20.580 INFO:tasks.workunit.client.1.smithi193.stdout:5/344: creat d0/d1c/d7a/f7d x:0 0 0 2024-03-23T08:23:20.581 INFO:tasks.workunit.client.1.smithi193.stdout:1/344: dread da/dd/df/d52/f53 [0,4194304] 0 2024-03-23T08:23:20.583 INFO:tasks.workunit.client.1.smithi193.stdout:3/265: truncate d1/de/d13/f5b 267067 0 2024-03-23T08:23:20.584 INFO:tasks.workunit.client.1.smithi193.stdout:4/334: creat d11/d1a/d33/d42/f71 x:0 0 0 2024-03-23T08:23:20.585 INFO:tasks.workunit.client.1.smithi193.stdout:1/345: mknod da/d13/c74 0 2024-03-23T08:23:20.586 INFO:tasks.workunit.client.1.smithi193.stdout:5/345: symlink d0/d2f/d72/l7e 0 2024-03-23T08:23:20.586 INFO:tasks.workunit.client.1.smithi193.stdout:9/376: mknod d2/d7/db/d1b/d43/d71/c7f 0 2024-03-23T08:23:20.588 INFO:tasks.workunit.client.1.smithi193.stdout:2/376: dwrite d0/d1b/f13 [0,4194304] 0 2024-03-23T08:23:20.589 INFO:tasks.workunit.client.1.smithi193.stdout:1/346: fsync da/dd/df/d52/f6e 0 2024-03-23T08:23:20.589 INFO:tasks.workunit.client.1.smithi193.stdout:9/377: readlink d2/l18 0 2024-03-23T08:23:20.589 INFO:tasks.workunit.client.1.smithi193.stdout:5/346: dread - d0/d1c/d3b/d4b/d55/f59 zero size 2024-03-23T08:23:20.590 INFO:tasks.workunit.client.1.smithi193.stdout:7/295: dread d6/fc [12582912,4194304] 0 2024-03-23T08:23:20.593 INFO:tasks.workunit.client.1.smithi193.stdout:1/347: read da/dd/df/f25 [1977108,50431] 0 2024-03-23T08:23:20.595 INFO:tasks.workunit.client.1.smithi193.stdout:9/378: write d2/d7/db/d32/d6b/f6c [534569,45541] 0 2024-03-23T08:23:20.595 INFO:tasks.workunit.client.1.smithi193.stdout:2/377: read d0/d1b/d14/d25/f3a [1543188,7344] 0 2024-03-23T08:23:20.598 INFO:tasks.workunit.client.1.smithi193.stdout:4/335: dread d11/d1a/d33/d44/f4a [0,4194304] 0 2024-03-23T08:23:20.598 INFO:tasks.workunit.client.1.smithi193.stdout:7/296: symlink d6/d9/d1f/d23/d33/d3c/l67 0 2024-03-23T08:23:20.600 INFO:tasks.workunit.client.1.smithi193.stdout:3/266: dwrite d1/de/d13/d21/d27/f1c [0,4194304] 0 2024-03-23T08:23:20.600 INFO:tasks.workunit.client.1.smithi193.stdout:5/347: link d0/d1f/f22 d0/d1c/d3b/f7f 0 2024-03-23T08:23:20.602 INFO:tasks.workunit.client.1.smithi193.stdout:2/378: rename d0/d1b to d0/d1b/d14/d66/d50/d60/d6f 22 2024-03-23T08:23:20.603 INFO:tasks.workunit.client.1.smithi193.stdout:3/267: chown d1/de/d13/d21/d27/f1c 3341610 1 2024-03-23T08:23:20.603 INFO:tasks.workunit.client.1.smithi193.stdout:9/379: creat d2/d7/db/d1b/d43/f80 x:0 0 0 2024-03-23T08:23:20.607 INFO:tasks.workunit.client.1.smithi193.stdout:4/336: rename d11/d48/d50/d49/f67 to d11/d1a/f72 0 2024-03-23T08:23:20.611 INFO:tasks.workunit.client.1.smithi193.stdout:1/348: dread da/d13/f34 [0,4194304] 0 2024-03-23T08:23:20.621 INFO:tasks.workunit.client.1.smithi193.stdout:2/379: dwrite d0/d1b/d3b/f42 [0,4194304] 0 2024-03-23T08:23:20.626 INFO:tasks.workunit.client.1.smithi193.stdout:7/297: dwrite d6/d9/d1f/d23/f2e [0,4194304] 0 2024-03-23T08:23:20.628 INFO:tasks.workunit.client.1.smithi193.stdout:4/337: write d11/d1a/d33/d44/f4a [921107,61148] 0 2024-03-23T08:23:20.628 INFO:tasks.workunit.client.1.smithi193.stdout:5/348: dwrite d0/d1c/f1e [0,4194304] 0 2024-03-23T08:23:20.630 INFO:tasks.workunit.client.1.smithi193.stdout:3/268: dwrite d1/de/d13/d21/d27/f1f [4194304,4194304] 0 2024-03-23T08:23:20.631 INFO:tasks.workunit.client.1.smithi193.stdout:4/338: fdatasync f5 0 2024-03-23T08:23:20.633 INFO:tasks.workunit.client.1.smithi193.stdout:9/380: dread d2/d7/d31/f4f [0,4194304] 0 2024-03-23T08:23:20.635 INFO:tasks.workunit.client.1.smithi193.stdout:7/298: rmdir d6/d9/d1f/d23/d33/d38/d3b 39 2024-03-23T08:23:20.638 INFO:tasks.workunit.client.1.smithi193.stdout:5/349: read d0/f11 [1343994,62972] 0 2024-03-23T08:23:20.638 INFO:tasks.workunit.client.1.smithi193.stdout:3/269: mknod d1/de/d13/c5e 0 2024-03-23T08:23:20.639 INFO:tasks.workunit.client.1.smithi193.stdout:4/339: creat d11/d27/f73 x:0 0 0 2024-03-23T08:23:20.642 INFO:tasks.workunit.client.1.smithi193.stdout:3/270: fsync d1/de/d13/d21/d25/d2b/f4d 0 2024-03-23T08:23:20.643 INFO:tasks.workunit.client.1.smithi193.stdout:2/380: dwrite d0/d1b/d14/d25/f29 [0,4194304] 0 2024-03-23T08:23:20.646 INFO:tasks.workunit.client.1.smithi193.stdout:2/381: chown d0/d28/d33/l5f 494118 1 2024-03-23T08:23:20.646 INFO:tasks.workunit.client.1.smithi193.stdout:4/340: unlink l7 0 2024-03-23T08:23:20.648 INFO:tasks.workunit.client.1.smithi193.stdout:7/299: stat d6/d9/d1f/d23/d33/d3c/d41/f4a 0 2024-03-23T08:23:20.649 INFO:tasks.workunit.client.1.smithi193.stdout:5/350: creat d0/d1c/d3b/d4b/d55/f80 x:0 0 0 2024-03-23T08:23:20.649 INFO:tasks.workunit.client.1.smithi193.stdout:9/381: dwrite d2/d7/f1e [0,4194304] 0 2024-03-23T08:23:20.649 INFO:tasks.workunit.client.1.smithi193.stdout:3/271: symlink d1/de/d3d/l5f 0 2024-03-23T08:23:20.652 INFO:tasks.workunit.client.1.smithi193.stdout:3/272: readlink d1/l7 0 2024-03-23T08:23:20.652 INFO:tasks.workunit.client.1.smithi193.stdout:2/382: creat d0/d28/d33/f70 x:0 0 0 2024-03-23T08:23:20.653 INFO:tasks.workunit.client.1.smithi193.stdout:7/300: write d6/d9/f19 [3496397,40278] 0 2024-03-23T08:23:20.655 INFO:tasks.workunit.client.1.smithi193.stdout:4/341: creat d11/d1a/d33/d42/d52/d70/f74 x:0 0 0 2024-03-23T08:23:20.655 INFO:tasks.workunit.client.1.smithi193.stdout:0/352: sync 2024-03-23T08:23:20.660 INFO:tasks.workunit.client.1.smithi193.stdout:4/342: write d11/d19/d22/f2b [1908395,9255] 0 2024-03-23T08:23:20.662 INFO:tasks.workunit.client.1.smithi193.stdout:7/301: creat d6/d9/d1f/d23/d33/d38/f68 x:0 0 0 2024-03-23T08:23:20.664 INFO:tasks.workunit.client.1.smithi193.stdout:3/273: rename d1/de/d13/d21/d50/d44/d58 to d1/de/d13/d21/d59/d60 0 2024-03-23T08:23:20.668 INFO:tasks.workunit.client.1.smithi193.stdout:1/349: dread da/f59 [0,4194304] 0 2024-03-23T08:23:20.668 INFO:tasks.workunit.client.1.smithi193.stdout:4/343: creat d11/d19/d22/f75 x:0 0 0 2024-03-23T08:23:20.668 INFO:tasks.workunit.client.1.smithi193.stdout:5/351: dwrite d0/d1c/d3b/d4b/d55/f59 [0,4194304] 0 2024-03-23T08:23:20.669 INFO:tasks.workunit.client.1.smithi193.stdout:2/383: dread d0/d1b/d14/f41 [0,4194304] 0 2024-03-23T08:23:20.669 INFO:tasks.workunit.client.1.smithi193.stdout:3/274: mknod d1/d22/c61 0 2024-03-23T08:23:20.670 INFO:tasks.workunit.client.1.smithi193.stdout:7/302: chown d6/d9/d1f/d23/d33/d38/d46/f61 22367 1 2024-03-23T08:23:20.672 INFO:tasks.workunit.client.1.smithi193.stdout:5/352: mkdir d0/d1c/d3b/d81 0 2024-03-23T08:23:20.673 INFO:tasks.workunit.client.1.smithi193.stdout:4/344: creat d11/d2d/d3a/f76 x:0 0 0 2024-03-23T08:23:20.676 INFO:tasks.workunit.client.1.smithi193.stdout:1/350: write da/d13/f34 [1875081,83427] 0 2024-03-23T08:23:20.678 INFO:tasks.workunit.client.1.smithi193.stdout:4/345: dread - d11/d2d/d3a/f76 zero size 2024-03-23T08:23:20.682 INFO:tasks.workunit.client.1.smithi193.stdout:0/353: truncate db/d11/d14/d21/f22 450603 0 2024-03-23T08:23:20.686 INFO:tasks.workunit.client.1.smithi193.stdout:4/346: mkdir d11/d27/d77 0 2024-03-23T08:23:20.688 INFO:tasks.workunit.client.1.smithi193.stdout:2/384: truncate d0/d28/f43 342035 0 2024-03-23T08:23:20.694 INFO:tasks.workunit.client.1.smithi193.stdout:7/303: dwrite d6/d9/f31 [4194304,4194304] 0 2024-03-23T08:23:20.697 INFO:tasks.workunit.client.1.smithi193.stdout:0/354: write db/d11/d14/d51/f56 [80323,86762] 0 2024-03-23T08:23:20.697 INFO:tasks.workunit.client.1.smithi193.stdout:8/303: sync 2024-03-23T08:23:20.697 INFO:tasks.workunit.client.1.smithi193.stdout:6/285: sync 2024-03-23T08:23:20.699 INFO:tasks.workunit.client.1.smithi193.stdout:2/385: fsync d0/d37/d48/f55 0 2024-03-23T08:23:20.700 INFO:tasks.workunit.client.1.smithi193.stdout:3/275: dwrite d1/de/d13/d21/d25/d2b/f39 [4194304,4194304] 0 2024-03-23T08:23:20.701 INFO:tasks.workunit.client.1.smithi193.stdout:9/382: dread d2/d7/db/d17/d33/f45 [0,4194304] 0 2024-03-23T08:23:20.704 INFO:tasks.workunit.client.1.smithi193.stdout:5/353: dwrite d0/f3e [0,4194304] 0 2024-03-23T08:23:20.704 INFO:tasks.workunit.client.1.smithi193.stdout:7/304: dread - d6/d9/d1f/d3e/f5e zero size 2024-03-23T08:23:20.707 INFO:tasks.workunit.client.1.smithi193.stdout:6/286: creat d0/d8/d3a/f53 x:0 0 0 2024-03-23T08:23:20.715 INFO:tasks.workunit.client.1.smithi193.stdout:0/355: dread - db/d11/d14/d24/d47/d59/f7c zero size 2024-03-23T08:23:20.716 INFO:tasks.workunit.client.1.smithi193.stdout:8/304: creat df/d10/d67/f6c x:0 0 0 2024-03-23T08:23:20.717 INFO:tasks.workunit.client.1.smithi193.stdout:1/351: dwrite da/d13/d24/f58 [0,4194304] 0 2024-03-23T08:23:20.717 INFO:tasks.workunit.client.1.smithi193.stdout:4/347: dread d11/d1a/d33/d42/f28 [0,4194304] 0 2024-03-23T08:23:20.718 INFO:tasks.workunit.client.1.smithi193.stdout:3/276: chown d1/de/d13/d21/d50/d44/f54 386377425 1 2024-03-23T08:23:20.722 INFO:tasks.workunit.client.1.smithi193.stdout:7/305: mkdir d6/d9/d1f/d23/d69 0 2024-03-23T08:23:20.726 INFO:tasks.workunit.client.1.smithi193.stdout:6/287: mkdir d0/db/df/d11/d3b/d51/d54 0 2024-03-23T08:23:20.726 INFO:tasks.workunit.client.1.smithi193.stdout:8/305: read df/d10/f3b [520366,109571] 0 2024-03-23T08:23:20.732 INFO:tasks.workunit.client.1.smithi193.stdout:9/383: dwrite d2/d7/f37 [0,4194304] 0 2024-03-23T08:23:20.733 INFO:tasks.workunit.client.1.smithi193.stdout:8/306: dread - df/d46/f47 zero size 2024-03-23T08:23:20.733 INFO:tasks.workunit.client.1.smithi193.stdout:1/352: mkdir da/dd/df/d75 0 2024-03-23T08:23:20.736 INFO:tasks.workunit.client.1.smithi193.stdout:2/386: dwrite d0/d1b/f8 [0,4194304] 0 2024-03-23T08:23:20.742 INFO:tasks.workunit.client.1.smithi193.stdout:5/354: dwrite d0/f3e [0,4194304] 0 2024-03-23T08:23:20.742 INFO:tasks.workunit.client.1.smithi193.stdout:7/306: chown d6/d9/d1f/d23/d33/d38/d3b/f48 11 1 2024-03-23T08:23:20.743 INFO:tasks.workunit.client.1.smithi193.stdout:8/307: dread - df/d4b/f66 zero size 2024-03-23T08:23:20.746 INFO:tasks.workunit.client.1.smithi193.stdout:1/353: rmdir da/d13/d3f 39 2024-03-23T08:23:20.753 INFO:tasks.workunit.client.1.smithi193.stdout:3/277: dread d1/de/d13/d21/d27/f1c [0,4194304] 0 2024-03-23T08:23:20.754 INFO:tasks.workunit.client.1.smithi193.stdout:8/308: symlink df/d10/d3e/d4e/l6d 0 2024-03-23T08:23:20.756 INFO:tasks.workunit.client.1.smithi193.stdout:1/354: write da/d13/d24/f43 [1655550,129497] 0 2024-03-23T08:23:20.756 INFO:tasks.workunit.client.1.smithi193.stdout:6/288: dwrite d0/d8/d3a/f45 [0,4194304] 0 2024-03-23T08:23:20.757 INFO:tasks.workunit.client.1.smithi193.stdout:7/307: link d6/d9/d1f/f50 d6/d9/f6a 0 2024-03-23T08:23:20.761 INFO:tasks.workunit.client.1.smithi193.stdout:4/348: dwrite d11/d1a/d33/d42/d52/f6c [0,4194304] 0 2024-03-23T08:23:20.764 INFO:tasks.workunit.client.1.smithi193.stdout:3/278: chown d1/de/d13/d21/d25/d2b/d3c/d4a/d5a 1 1 2024-03-23T08:23:20.767 INFO:tasks.workunit.client.1.smithi193.stdout:9/384: dread d2/d7/db/d17/d33/f45 [0,4194304] 0 2024-03-23T08:23:20.768 INFO:tasks.workunit.client.1.smithi193.stdout:5/355: write d0/fa [3852554,91567] 0 2024-03-23T08:23:20.769 INFO:tasks.workunit.client.1.smithi193.stdout:0/356: dread db/d11/d14/d24/d47/d59/f79 [0,4194304] 0 2024-03-23T08:23:20.771 INFO:tasks.workunit.client.1.smithi193.stdout:4/349: dread - d11/d19/d22/f6b zero size 2024-03-23T08:23:20.774 INFO:tasks.workunit.client.1.smithi193.stdout:8/309: rename df/d4b/d4d/c32 to df/d4b/d4d/c6e 0 2024-03-23T08:23:20.776 INFO:tasks.workunit.client.1.smithi193.stdout:9/385: chown d2/d7/db/d17/d1f/f22 30045526 1 2024-03-23T08:23:20.777 INFO:tasks.workunit.client.1.smithi193.stdout:2/387: dwrite d0/fc [0,4194304] 0 2024-03-23T08:23:20.778 INFO:tasks.workunit.client.1.smithi193.stdout:5/356: write d0/d1c/f65 [181924,79721] 0 2024-03-23T08:23:20.782 INFO:tasks.workunit.client.1.smithi193.stdout:0/357: write db/d11/d14/d24/f4a [4502736,86845] 0 2024-03-23T08:23:20.783 INFO:tasks.workunit.client.1.smithi193.stdout:4/350: truncate f5 7886012 0 2024-03-23T08:23:20.785 INFO:tasks.workunit.client.1.smithi193.stdout:6/289: dwrite d0/db/f33 [0,4194304] 0 2024-03-23T08:23:20.785 INFO:tasks.workunit.client.1.smithi193.stdout:1/355: dwrite da/d13/d24/f43 [0,4194304] 0 2024-03-23T08:23:20.785 INFO:tasks.workunit.client.1.smithi193.stdout:2/388: chown d0/d1b/d14/f40 86 1 2024-03-23T08:23:20.793 INFO:tasks.workunit.client.1.smithi193.stdout:9/386: chown d2/d7/db/d17/c55 33496725 1 2024-03-23T08:23:20.794 INFO:tasks.workunit.client.1.smithi193.stdout:4/351: chown d11/d2d/f68 100383 1 2024-03-23T08:23:20.796 INFO:tasks.workunit.client.1.smithi193.stdout:8/310: stat df/d4b/d4d/c6e 0 2024-03-23T08:23:20.799 INFO:tasks.workunit.client.1.smithi193.stdout:9/387: chown d2/d7/db/d1b/d5c/c61 1748933 1 2024-03-23T08:23:20.799 INFO:tasks.workunit.client.1.smithi193.stdout:7/308: dread d6/d9/f19 [0,4194304] 0 2024-03-23T08:23:20.800 INFO:tasks.workunit.client.1.smithi193.stdout:1/356: dread da/d13/d24/f45 [0,4194304] 0 2024-03-23T08:23:20.803 INFO:tasks.workunit.client.1.smithi193.stdout:5/357: creat d0/d1c/d3b/d81/f82 x:0 0 0 2024-03-23T08:23:20.803 INFO:tasks.workunit.client.1.smithi193.stdout:7/309: stat d6/l1a 0 2024-03-23T08:23:20.803 INFO:tasks.workunit.client.1.smithi193.stdout:5/358: stat d0/d2f/d72/d73 0 2024-03-23T08:23:20.805 INFO:tasks.workunit.client.1.smithi193.stdout:8/311: mknod df/d4b/d4d/d2a/c6f 0 2024-03-23T08:23:20.807 INFO:tasks.workunit.client.1.smithi193.stdout:3/279: dread d1/fc [0,4194304] 0 2024-03-23T08:23:20.811 INFO:tasks.workunit.client.1.smithi193.stdout:8/312: truncate df/d4b/d4d/f45 320870 0 2024-03-23T08:23:20.812 INFO:tasks.workunit.client.1.smithi193.stdout:1/357: dread da/d13/d24/f58 [0,4194304] 0 2024-03-23T08:23:20.813 INFO:tasks.workunit.client.1.smithi193.stdout:9/388: rename d2/d7/db/d17/d33/f62 to d2/d7/db/d1b/f81 0 2024-03-23T08:23:20.816 INFO:tasks.workunit.client.1.smithi193.stdout:6/290: truncate d0/d8/d12/d21/f29 1418051 0 2024-03-23T08:23:20.817 INFO:tasks.workunit.client.1.smithi193.stdout:3/280: creat d1/de/d13/f62 x:0 0 0 2024-03-23T08:23:20.817 INFO:tasks.workunit.client.1.smithi193.stdout:7/310: dwrite d6/d9/d1f/d23/d33/d38/f68 [0,4194304] 0 2024-03-23T08:23:20.817 INFO:tasks.workunit.client.1.smithi193.stdout:1/358: write da/dd/df/f17 [666205,35859] 0 2024-03-23T08:23:20.818 INFO:tasks.workunit.client.1.smithi193.stdout:2/389: getdents d0 0 2024-03-23T08:23:20.825 INFO:tasks.workunit.client.1.smithi193.stdout:0/358: dread db/d11/d14/d21/f27 [0,4194304] 0 2024-03-23T08:23:20.826 INFO:tasks.workunit.client.1.smithi193.stdout:9/389: rmdir d2/d7/db/d17 39 2024-03-23T08:23:20.826 INFO:tasks.workunit.client.1.smithi193.stdout:2/390: fsync d0/d1b/d14/d25/f5a 0 2024-03-23T08:23:20.828 INFO:tasks.workunit.client.1.smithi193.stdout:7/311: mkdir d6/d9/d1f/d4d/d6b 0 2024-03-23T08:23:20.831 INFO:tasks.workunit.client.1.smithi193.stdout:0/359: fsync db/d11/d14/d21/f2c 0 2024-03-23T08:23:20.832 INFO:tasks.workunit.client.1.smithi193.stdout:8/313: dwrite df/d10/d3e/f6b [0,4194304] 0 2024-03-23T08:23:20.832 INFO:tasks.workunit.client.1.smithi193.stdout:9/390: write d2/d7/f11 [2731696,42392] 0 2024-03-23T08:23:20.834 INFO:tasks.workunit.client.1.smithi193.stdout:3/281: creat d1/de/d13/d21/d27/f63 x:0 0 0 2024-03-23T08:23:20.835 INFO:tasks.workunit.client.1.smithi193.stdout:5/359: write d0/d1c/d3b/f7f [1343273,68721] 0 2024-03-23T08:23:20.838 INFO:tasks.workunit.client.1.smithi193.stdout:2/391: rmdir d0/d28/d33 39 2024-03-23T08:23:20.840 INFO:tasks.workunit.client.1.smithi193.stdout:4/352: dwrite f4 [4194304,4194304] 0 2024-03-23T08:23:20.840 INFO:tasks.workunit.client.1.smithi193.stdout:3/282: unlink d1/l7 0 2024-03-23T08:23:20.842 INFO:tasks.workunit.client.1.smithi193.stdout:5/360: mknod d0/d2f/c83 0 2024-03-23T08:23:20.844 INFO:tasks.workunit.client.1.smithi193.stdout:4/353: readlink d11/d2d/l34 0 2024-03-23T08:23:20.844 INFO:tasks.workunit.client.1.smithi193.stdout:0/360: dread db/d11/d14/d24/d47/f74 [0,4194304] 0 2024-03-23T08:23:20.845 INFO:tasks.workunit.client.1.smithi193.stdout:9/391: readlink d2/d7/db/d1b/d43/l76 0 2024-03-23T08:23:20.845 INFO:tasks.workunit.client.1.smithi193.stdout:6/291: dwrite d0/db/df/d11/d36/f41 [0,4194304] 0 2024-03-23T08:23:20.849 INFO:tasks.workunit.client.1.smithi193.stdout:9/392: readlink d2/l18 0 2024-03-23T08:23:20.851 INFO:tasks.workunit.client.1.smithi193.stdout:3/283: creat d1/d22/f64 x:0 0 0 2024-03-23T08:23:20.851 INFO:tasks.workunit.client.1.smithi193.stdout:1/359: getdents da/d13/d3f/d5a 0 2024-03-23T08:23:20.855 INFO:tasks.workunit.client.1.smithi193.stdout:5/361: creat d0/d1c/d3b/d4b/f84 x:0 0 0 2024-03-23T08:23:20.858 INFO:tasks.workunit.client.1.smithi193.stdout:1/360: mkdir da/d13/d3f/d5a/d76 0 2024-03-23T08:23:20.861 INFO:tasks.workunit.client.1.smithi193.stdout:0/361: link db/d11/d14/d24/d2a/d4d/f7a db/d11/d14/d24/f7d 0 2024-03-23T08:23:20.861 INFO:tasks.workunit.client.1.smithi193.stdout:3/284: mknod d1/c65 0 2024-03-23T08:23:20.863 INFO:tasks.workunit.client.1.smithi193.stdout:5/362: mkdir d0/d1c/d3b/d85 0 2024-03-23T08:23:20.863 INFO:tasks.workunit.client.1.smithi193.stdout:8/314: dread f0 [4194304,4194304] 0 2024-03-23T08:23:20.866 INFO:tasks.workunit.client.1.smithi193.stdout:2/392: dwrite d0/d1b/d14/d66/d50/f52 [0,4194304] 0 2024-03-23T08:23:20.866 INFO:tasks.workunit.client.1.smithi193.stdout:5/363: stat d0/d2f/d72/d73/d57/c52 0 2024-03-23T08:23:20.869 INFO:tasks.workunit.client.1.smithi193.stdout:9/393: dwrite d2/d7/d31/d3c/f6e [0,4194304] 0 2024-03-23T08:23:20.870 INFO:tasks.workunit.client.1.smithi193.stdout:2/393: creat d0/f71 x:0 0 0 2024-03-23T08:23:20.871 INFO:tasks.workunit.client.1.smithi193.stdout:0/362: rename db/d52/c77 to db/d11/c7e 0 2024-03-23T08:23:20.871 INFO:tasks.workunit.client.1.smithi193.stdout:3/285: dread d1/de/d13/d21/d25/d34/f47 [0,4194304] 0 2024-03-23T08:23:20.873 INFO:tasks.workunit.client.1.smithi193.stdout:5/364: mkdir d0/d1c/d7a/d86 0 2024-03-23T08:23:20.876 INFO:tasks.workunit.client.1.smithi193.stdout:4/354: write d11/d19/d22/f25 [4433637,29755] 0 2024-03-23T08:23:20.876 INFO:tasks.workunit.client.1.smithi193.stdout:1/361: rename da/dd/df/d2e/d6b to da/d13/d3f/d5a/d76/d77 0 2024-03-23T08:23:20.882 INFO:tasks.workunit.client.1.smithi193.stdout:8/315: dwrite df/d46/f47 [0,4194304] 0 2024-03-23T08:23:20.886 INFO:tasks.workunit.client.1.smithi193.stdout:3/286: mknod d1/de/d13/c66 0 2024-03-23T08:23:20.886 INFO:tasks.workunit.client.1.smithi193.stdout:5/365: unlink d0/f17 0 2024-03-23T08:23:20.892 INFO:tasks.workunit.client.1.smithi193.stdout:3/287: dread - d1/d22/f64 zero size 2024-03-23T08:23:20.895 INFO:tasks.workunit.client.1.smithi193.stdout:3/288: write d1/de/d13/f62 [293111,61426] 0 2024-03-23T08:23:20.895 INFO:tasks.workunit.client.1.smithi193.stdout:8/316: creat df/d10/f70 x:0 0 0 2024-03-23T08:23:20.895 INFO:tasks.workunit.client.1.smithi193.stdout:1/362: creat da/dd/f78 x:0 0 0 2024-03-23T08:23:20.896 INFO:tasks.workunit.client.1.smithi193.stdout:2/394: write d0/f71 [87949,5071] 0 2024-03-23T08:23:20.898 INFO:tasks.workunit.client.1.smithi193.stdout:5/366: rename d0/d1c/d3b/f53 to d0/d2f/d72/d73/d57/d6d/f87 0 2024-03-23T08:23:20.900 INFO:tasks.workunit.client.1.smithi193.stdout:8/317: chown df/d4b/d4d/f69 105118 1 2024-03-23T08:23:20.901 INFO:tasks.workunit.client.1.smithi193.stdout:2/395: write d0/d28/d33/f70 [693715,23893] 0 2024-03-23T08:23:20.904 INFO:tasks.workunit.client.1.smithi193.stdout:1/363: link da/dd/f39 da/d13/d24/f79 0 2024-03-23T08:23:20.904 INFO:tasks.workunit.client.1.smithi193.stdout:3/289: mknod d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/c67 0 2024-03-23T08:23:20.906 INFO:tasks.workunit.client.1.smithi193.stdout:3/290: chown d1/c5 141 1 2024-03-23T08:23:20.908 INFO:tasks.workunit.client.1.smithi193.stdout:1/364: write da/dd/df/f25 [2882994,110304] 0 2024-03-23T08:23:20.913 INFO:tasks.workunit.client.1.smithi193.stdout:5/367: dwrite d0/d1c/f21 [0,4194304] 0 2024-03-23T08:23:20.913 INFO:tasks.workunit.client.1.smithi193.stdout:3/291: readlink d1/de/d13/d21/d25/d2b/d3c/l3e 0 2024-03-23T08:23:20.916 INFO:tasks.workunit.client.1.smithi193.stdout:1/365: unlink da/dd/df/d2e/f6a 0 2024-03-23T08:23:20.920 INFO:tasks.workunit.client.1.smithi193.stdout:5/368: symlink d0/d2f/d72/d73/d57/l88 0 2024-03-23T08:23:20.923 INFO:tasks.workunit.client.1.smithi193.stdout:9/394: dread d2/d7/db/d32/f5a [4194304,4194304] 0 2024-03-23T08:23:20.923 INFO:tasks.workunit.client.1.smithi193.stdout:3/292: chown d1/de/d13/d21/d25/d2b/f35 79 1 2024-03-23T08:23:20.927 INFO:tasks.workunit.client.1.smithi193.stdout:3/293: rename d1/d22 to d1/d22/d68 22 2024-03-23T08:23:20.928 INFO:tasks.workunit.client.1.smithi193.stdout:2/396: dwrite d0/d1b/d3b/f46 [0,4194304] 0 2024-03-23T08:23:20.928 INFO:tasks.workunit.client.1.smithi193.stdout:1/366: write da/dd/df/d2e/f48 [1448008,72166] 0 2024-03-23T08:23:20.931 INFO:tasks.workunit.client.1.smithi193.stdout:4/355: dread d11/d19/d22/f6a [0,4194304] 0 2024-03-23T08:23:20.933 INFO:tasks.workunit.client.1.smithi193.stdout:0/363: dread db/d11/d14/d21/d33/f68 [0,4194304] 0 2024-03-23T08:23:20.934 INFO:tasks.workunit.client.1.smithi193.stdout:2/397: mkdir d0/d28/d33/d72 0 2024-03-23T08:23:20.938 INFO:tasks.workunit.client.1.smithi193.stdout:4/356: chown d11/d2d/d3a/l5a 78 1 2024-03-23T08:23:20.947 INFO:tasks.workunit.client.1.smithi193.stdout:5/369: dwrite d0/d1f/f41 [0,4194304] 0 2024-03-23T08:23:20.947 INFO:tasks.workunit.client.1.smithi193.stdout:2/398: mknod d0/d1b/d19/c73 0 2024-03-23T08:23:20.947 INFO:tasks.workunit.client.1.smithi193.stdout:0/364: creat db/d11/d14/d3b/d71/f7f x:0 0 0 2024-03-23T08:23:20.947 INFO:tasks.workunit.client.1.smithi193.stdout:4/357: mknod d11/d2d/d58/c78 0 2024-03-23T08:23:20.948 INFO:tasks.workunit.client.1.smithi193.stdout:9/395: dwrite d2/d7/f6f [0,4194304] 0 2024-03-23T08:23:20.952 INFO:tasks.workunit.client.1.smithi193.stdout:1/367: dwrite da/dd/df/d2e/f48 [0,4194304] 0 2024-03-23T08:23:20.961 INFO:tasks.workunit.client.1.smithi193.stdout:4/358: mknod d11/d2d/d58/c79 0 2024-03-23T08:23:20.961 INFO:tasks.workunit.client.1.smithi193.stdout:5/370: truncate d0/f7 422578 0 2024-03-23T08:23:20.961 INFO:tasks.workunit.client.1.smithi193.stdout:0/365: creat db/d11/d14/d21/d33/f80 x:0 0 0 2024-03-23T08:23:20.961 INFO:tasks.workunit.client.1.smithi193.stdout:2/399: creat d0/d1b/f74 x:0 0 0 2024-03-23T08:23:20.961 INFO:tasks.workunit.client.1.smithi193.stdout:1/368: creat da/dd/df/d52/f7a x:0 0 0 2024-03-23T08:23:20.961 INFO:tasks.workunit.client.1.smithi193.stdout:5/371: fdatasync d0/f14 0 2024-03-23T08:23:20.962 INFO:tasks.workunit.client.1.smithi193.stdout:1/369: dread - da/dd/df/f3b zero size 2024-03-23T08:23:20.962 INFO:tasks.workunit.client.1.smithi193.stdout:3/294: dread d1/de/d13/d21/d27/f1f [0,4194304] 0 2024-03-23T08:23:20.966 INFO:tasks.workunit.client.1.smithi193.stdout:4/359: creat d11/f7a x:0 0 0 2024-03-23T08:23:20.967 INFO:tasks.workunit.client.1.smithi193.stdout:5/372: chown d0/d5a/c79 2 1 2024-03-23T08:23:20.967 INFO:tasks.workunit.client.1.smithi193.stdout:4/360: dread - d11/d48/d50/f61 zero size 2024-03-23T08:23:20.967 INFO:tasks.workunit.client.1.smithi193.stdout:9/396: link d2/d7/c3e d2/d7/d31/d5f/c82 0 2024-03-23T08:23:20.968 INFO:tasks.workunit.client.1.smithi193.stdout:3/295: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/l69 0 2024-03-23T08:23:20.968 INFO:tasks.workunit.client.1.smithi193.stdout:4/361: write d11/d2d/f4b [669251,92423] 0 2024-03-23T08:23:20.970 INFO:tasks.workunit.client.1.smithi193.stdout:3/296: stat d1/de 0 2024-03-23T08:23:20.970 INFO:tasks.workunit.client.1.smithi193.stdout:1/370: link da/d13/d3f/f44 da/d13/d24/f7b 0 2024-03-23T08:23:20.971 INFO:tasks.workunit.client.1.smithi193.stdout:3/297: readlink d1/de/d13/d21/d25/d2b/d3c/l43 0 2024-03-23T08:23:20.972 INFO:tasks.workunit.client.1.smithi193.stdout:9/397: rename d2/d7/db/f26 to d2/d7/d31/d5f/f83 0 2024-03-23T08:23:20.972 INFO:tasks.workunit.client.1.smithi193.stdout:3/298: write d1/f20 [735825,74609] 0 2024-03-23T08:23:20.974 INFO:tasks.workunit.client.1.smithi193.stdout:3/299: chown d1/de/d3d 850 1 2024-03-23T08:23:20.975 INFO:tasks.workunit.client.1.smithi193.stdout:1/371: mkdir da/dd/df/d7c 0 2024-03-23T08:23:20.976 INFO:tasks.workunit.client.1.smithi193.stdout:3/300: creat d1/de/d13/d21/d25/d2b/d3c/f6a x:0 0 0 2024-03-23T08:23:20.980 INFO:tasks.workunit.client.1.smithi193.stdout:0/366: dread db/d11/d14/d21/f26 [0,4194304] 0 2024-03-23T08:23:20.981 INFO:tasks.workunit.client.1.smithi193.stdout:1/372: link da/dd/df/d52/f53 da/d13/f7d 0 2024-03-23T08:23:20.983 INFO:tasks.workunit.client.1.smithi193.stdout:3/301: link d1/de/d13/d21/d27/f1f d1/de/d13/d21/d59/d60/f6b 0 2024-03-23T08:23:20.984 INFO:tasks.workunit.client.1.smithi193.stdout:3/302: write d1/de/d13/f62 [717975,46024] 0 2024-03-23T08:23:20.984 INFO:tasks.workunit.client.1.smithi193.stdout:0/367: rename db/d52/f76 to db/d11/d14/d24/d2a/d4d/f81 0 2024-03-23T08:23:20.988 INFO:tasks.workunit.client.1.smithi193.stdout:1/373: dwrite da/d13/d3f/f56 [0,4194304] 0 2024-03-23T08:23:20.988 INFO:tasks.workunit.client.1.smithi193.stdout:0/368: write db/d11/d14/d21/f49 [3917891,49152] 0 2024-03-23T08:23:20.991 INFO:tasks.workunit.client.1.smithi193.stdout:3/303: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/f6c x:0 0 0 2024-03-23T08:23:20.998 INFO:tasks.workunit.client.1.smithi193.stdout:1/374: dread da/dd/df/d52/f53 [0,4194304] 0 2024-03-23T08:23:20.998 INFO:tasks.workunit.client.1.smithi193.stdout:4/362: dwrite d11/d19/d22/f26 [0,4194304] 0 2024-03-23T08:23:21.001 INFO:tasks.workunit.client.1.smithi193.stdout:4/363: readlink d11/d48/d50/l43 0 2024-03-23T08:23:21.001 INFO:tasks.workunit.client.1.smithi193.stdout:1/375: mkdir da/dd/df/d7e 0 2024-03-23T08:23:21.002 INFO:tasks.workunit.client.1.smithi193.stdout:0/369: dwrite db/d11/d14/d21/f2c [0,4194304] 0 2024-03-23T08:23:21.004 INFO:tasks.workunit.client.1.smithi193.stdout:4/364: symlink d11/l7b 0 2024-03-23T08:23:21.006 INFO:tasks.workunit.client.1.smithi193.stdout:3/304: dwrite d1/de/d13/d21/d25/d2b/f35 [0,4194304] 0 2024-03-23T08:23:21.007 INFO:tasks.workunit.client.1.smithi193.stdout:1/376: fdatasync da/dd/df/d52/f53 0 2024-03-23T08:23:21.013 INFO:tasks.workunit.client.1.smithi193.stdout:1/377: stat da/c10 0 2024-03-23T08:23:21.013 INFO:tasks.workunit.client.1.smithi193.stdout:0/370: link db/d11/d14/d21/d33/l3c db/d11/d14/d51/l82 0 2024-03-23T08:23:21.014 INFO:tasks.workunit.client.1.smithi193.stdout:4/365: getdents d11/d2d 0 2024-03-23T08:23:21.015 INFO:tasks.workunit.client.1.smithi193.stdout:3/305: mknod d1/de/d28/c6d 0 2024-03-23T08:23:21.015 INFO:tasks.workunit.client.1.smithi193.stdout:4/366: chown d11/d48/d50 438189726 1 2024-03-23T08:23:21.016 INFO:tasks.workunit.client.1.smithi193.stdout:3/306: chown d1/de/d13/d21/c53 45109 1 2024-03-23T08:23:21.018 INFO:tasks.workunit.client.1.smithi193.stdout:3/307: chown d1/de/d13/f24 291 1 2024-03-23T08:23:21.018 INFO:tasks.workunit.client.1.smithi193.stdout:4/367: mknod d11/d1a/d33/d42/c7c 0 2024-03-23T08:23:21.019 INFO:tasks.workunit.client.1.smithi193.stdout:4/368: readlink d11/d2d/d3a/l5a 0 2024-03-23T08:23:21.020 INFO:tasks.workunit.client.1.smithi193.stdout:3/308: chown d1/de/d13/d21/d27/c36 0 1 2024-03-23T08:23:21.020 INFO:tasks.workunit.client.1.smithi193.stdout:4/369: chown d11/d19/d22/f6b 248106807 1 2024-03-23T08:23:21.021 INFO:tasks.workunit.client.1.smithi193.stdout:0/371: dwrite db/d11/d14/d21/f41 [0,4194304] 0 2024-03-23T08:23:21.022 INFO:tasks.workunit.client.1.smithi193.stdout:3/309: mkdir d1/de/d3d/d6e 0 2024-03-23T08:23:21.024 INFO:tasks.workunit.client.1.smithi193.stdout:3/310: chown d1/de/d13/d21/d25/d2b/d3c/l43 484149332 1 2024-03-23T08:23:21.026 INFO:tasks.workunit.client.1.smithi193.stdout:4/370: dread d11/d19/d22/f6a [0,4194304] 0 2024-03-23T08:23:21.028 INFO:tasks.workunit.client.1.smithi193.stdout:3/311: unlink d1/de/d13/d21/d27/c36 0 2024-03-23T08:23:21.028 INFO:tasks.workunit.client.1.smithi193.stdout:4/371: chown d11/d19/c69 26314 1 2024-03-23T08:23:21.034 INFO:tasks.workunit.client.1.smithi193.stdout:4/372: dwrite d11/d48/d50/f5b [0,4194304] 0 2024-03-23T08:23:21.086 INFO:tasks.workunit.client.1.smithi193.stdout:8/318: sync 2024-03-23T08:23:21.086 INFO:tasks.workunit.client.1.smithi193.stdout:6/292: sync 2024-03-23T08:23:21.086 INFO:tasks.workunit.client.1.smithi193.stdout:7/312: sync 2024-03-23T08:23:21.087 INFO:tasks.workunit.client.1.smithi193.stdout:3/312: rename d1/de/d3d to d1/de/d13/d21/d50/d6f 0 2024-03-23T08:23:21.088 INFO:tasks.workunit.client.1.smithi193.stdout:6/293: chown d0/d8/d12/c4c 1761840 1 2024-03-23T08:23:21.088 INFO:tasks.workunit.client.1.smithi193.stdout:4/373: link d11/d19/d22/f45 d11/d1a/d33/f7d 0 2024-03-23T08:23:21.089 INFO:tasks.workunit.client.1.smithi193.stdout:8/319: creat df/d4b/f71 x:0 0 0 2024-03-23T08:23:21.089 INFO:tasks.workunit.client.1.smithi193.stdout:6/294: fdatasync d0/db/df/d11/d3b/f49 0 2024-03-23T08:23:21.090 INFO:tasks.workunit.client.1.smithi193.stdout:3/313: creat d1/de/f70 x:0 0 0 2024-03-23T08:23:21.090 INFO:tasks.workunit.client.1.smithi193.stdout:8/320: stat df/f34 0 2024-03-23T08:23:21.092 INFO:tasks.workunit.client.1.smithi193.stdout:4/374: rename d11/d19/d22/f6a to d11/d27/d77/f7e 0 2024-03-23T08:23:21.093 INFO:tasks.workunit.client.1.smithi193.stdout:3/314: truncate d1/d22/f30 787838 0 2024-03-23T08:23:21.093 INFO:tasks.workunit.client.1.smithi193.stdout:6/295: dread d0/d8/f48 [0,4194304] 0 2024-03-23T08:23:21.095 INFO:tasks.workunit.client.1.smithi193.stdout:7/313: dread d6/d9/d1f/d23/d33/d38/f68 [0,4194304] 0 2024-03-23T08:23:21.095 INFO:tasks.workunit.client.1.smithi193.stdout:8/321: dread df/f18 [0,4194304] 0 2024-03-23T08:23:21.095 INFO:tasks.workunit.client.1.smithi193.stdout:3/315: creat d1/de/f71 x:0 0 0 2024-03-23T08:23:21.096 INFO:tasks.workunit.client.1.smithi193.stdout:4/375: mkdir d11/d1a/d33/d7f 0 2024-03-23T08:23:21.096 INFO:tasks.workunit.client.1.smithi193.stdout:8/322: chown df/d4b 324720156 1 2024-03-23T08:23:21.096 INFO:tasks.workunit.client.1.smithi193.stdout:6/296: mknod d0/db/df/d11/d3b/d51/c55 0 2024-03-23T08:23:21.099 INFO:tasks.workunit.client.1.smithi193.stdout:7/314: rmdir d6/d9 39 2024-03-23T08:23:21.100 INFO:tasks.workunit.client.1.smithi193.stdout:8/323: dread df/f21 [0,4194304] 0 2024-03-23T08:23:21.101 INFO:tasks.workunit.client.1.smithi193.stdout:4/376: mkdir d11/d2d/d58/d80 0 2024-03-23T08:23:21.104 INFO:tasks.workunit.client.1.smithi193.stdout:8/324: creat df/d10/d19/f72 x:0 0 0 2024-03-23T08:23:21.105 INFO:tasks.workunit.client.1.smithi193.stdout:3/316: creat d1/de/d13/d21/d50/d6f/d6e/f72 x:0 0 0 2024-03-23T08:23:21.106 INFO:tasks.workunit.client.1.smithi193.stdout:7/315: dwrite d6/f32 [4194304,4194304] 0 2024-03-23T08:23:21.106 INFO:tasks.workunit.client.1.smithi193.stdout:4/377: rename d11/f7a to d11/d1a/d33/d7f/f81 0 2024-03-23T08:23:21.106 INFO:tasks.workunit.client.1.smithi193.stdout:8/325: chown df/d63/f68 25931169 1 2024-03-23T08:23:21.109 INFO:tasks.workunit.client.1.smithi193.stdout:4/378: rmdir d11/d1a/d33/d44 39 2024-03-23T08:23:21.110 INFO:tasks.workunit.client.1.smithi193.stdout:8/326: dread df/d63/f68 [0,4194304] 0 2024-03-23T08:23:21.111 INFO:tasks.workunit.client.1.smithi193.stdout:7/316: mkdir d6/d9/d1f/d23/d33/d38/d3b/d6c 0 2024-03-23T08:23:21.113 INFO:tasks.workunit.client.1.smithi193.stdout:3/317: dread d1/f20 [0,4194304] 0 2024-03-23T08:23:21.113 INFO:tasks.workunit.client.1.smithi193.stdout:7/317: write d6/f39 [463689,67401] 0 2024-03-23T08:23:21.114 INFO:tasks.workunit.client.1.smithi193.stdout:4/379: read - d11/d1a/d33/d44/f6e zero size 2024-03-23T08:23:21.116 INFO:tasks.workunit.client.1.smithi193.stdout:6/297: dread d0/d8/f1d [0,4194304] 0 2024-03-23T08:23:21.116 INFO:tasks.workunit.client.1.smithi193.stdout:7/318: chown d6/d9/d1f/d23/d33/d3c/d41/f4a 29410 1 2024-03-23T08:23:21.116 INFO:tasks.workunit.client.1.smithi193.stdout:4/380: write d11/d19/d22/f26 [3052570,113782] 0 2024-03-23T08:23:21.117 INFO:tasks.workunit.client.1.smithi193.stdout:8/327: rename df/d10/d3e/d54/c60 to df/d62/c73 0 2024-03-23T08:23:21.118 INFO:tasks.workunit.client.1.smithi193.stdout:3/318: fsync d1/de/d13/f33 0 2024-03-23T08:23:21.122 INFO:tasks.workunit.client.1.smithi193.stdout:4/381: creat d11/d1a/d33/d44/f82 x:0 0 0 2024-03-23T08:23:21.123 INFO:tasks.workunit.client.1.smithi193.stdout:8/328: symlink df/d4b/d4d/d2a/l74 0 2024-03-23T08:23:21.123 INFO:tasks.workunit.client.1.smithi193.stdout:8/329: chown df/d4b/d4d 1 1 2024-03-23T08:23:21.125 INFO:tasks.workunit.client.1.smithi193.stdout:6/298: unlink d0/d8/l4a 0 2024-03-23T08:23:21.127 INFO:tasks.workunit.client.1.smithi193.stdout:4/382: rmdir d11/d48 39 2024-03-23T08:23:21.129 INFO:tasks.workunit.client.1.smithi193.stdout:7/319: creat d6/d9/d1f/d23/d33/f6d x:0 0 0 2024-03-23T08:23:21.134 INFO:tasks.workunit.client.1.smithi193.stdout:4/383: link d11/d19/d22/f2b d11/f83 0 2024-03-23T08:23:21.136 INFO:tasks.workunit.client.1.smithi193.stdout:3/319: dwrite d1/de/f16 [0,4194304] 0 2024-03-23T08:23:21.136 INFO:tasks.workunit.client.1.smithi193.stdout:8/330: dwrite df/d10/d19/f4f [0,4194304] 0 2024-03-23T08:23:21.138 INFO:tasks.workunit.client.1.smithi193.stdout:6/299: dwrite d0/db/f20 [0,4194304] 0 2024-03-23T08:23:21.142 INFO:tasks.workunit.client.1.smithi193.stdout:7/320: getdents d6/d9/d1f/d23 0 2024-03-23T08:23:21.142 INFO:tasks.workunit.client.1.smithi193.stdout:6/300: read - d0/db/df/d11/d36/f44 zero size 2024-03-23T08:23:21.143 INFO:tasks.workunit.client.1.smithi193.stdout:8/331: write df/f38 [633899,51012] 0 2024-03-23T08:23:21.148 INFO:tasks.workunit.client.1.smithi193.stdout:7/321: symlink d6/l6e 0 2024-03-23T08:23:21.148 INFO:tasks.workunit.client.1.smithi193.stdout:8/332: chown df/d4b/d4d/f5c 1143462102 1 2024-03-23T08:23:21.150 INFO:tasks.workunit.client.1.smithi193.stdout:6/301: mkdir d0/db/df/d11/d56 0 2024-03-23T08:23:21.154 INFO:tasks.workunit.client.1.smithi193.stdout:8/333: link df/d4b/d4d/c64 df/d10/c75 0 2024-03-23T08:23:21.156 INFO:tasks.workunit.client.1.smithi193.stdout:8/334: unlink df/f38 0 2024-03-23T08:23:21.159 INFO:tasks.workunit.client.1.smithi193.stdout:4/384: dread f5 [0,4194304] 0 2024-03-23T08:23:21.159 INFO:tasks.workunit.client.1.smithi193.stdout:4/385: dread - d11/d27/f73 zero size 2024-03-23T08:23:21.161 INFO:tasks.workunit.client.1.smithi193.stdout:4/386: fsync d11/d48/d50/d49/f4f 0 2024-03-23T08:23:21.164 INFO:tasks.workunit.client.1.smithi193.stdout:6/302: dread d0/d8/d12/f1b [0,4194304] 0 2024-03-23T08:23:21.164 INFO:tasks.workunit.client.1.smithi193.stdout:4/387: getdents d11/d48/d50/d49 0 2024-03-23T08:23:21.166 INFO:tasks.workunit.client.1.smithi193.stdout:4/388: mkdir d11/d19/d84 0 2024-03-23T08:23:21.167 INFO:tasks.workunit.client.1.smithi193.stdout:4/389: stat d11/d2d/d58/d80 0 2024-03-23T08:23:21.169 INFO:tasks.workunit.client.1.smithi193.stdout:4/390: mkdir d11/d27/d85 0 2024-03-23T08:23:21.169 INFO:tasks.workunit.client.1.smithi193.stdout:6/303: link d0/d8/d12/d21/c3f d0/db/df/d11/c57 0 2024-03-23T08:23:21.170 INFO:tasks.workunit.client.1.smithi193.stdout:2/400: sync 2024-03-23T08:23:21.172 INFO:tasks.workunit.client.1.smithi193.stdout:6/304: stat d0/d8/d3a/f45 0 2024-03-23T08:23:21.172 INFO:tasks.workunit.client.1.smithi193.stdout:8/335: truncate df/d10/d19/f4f 3399018 0 2024-03-23T08:23:21.173 INFO:tasks.workunit.client.1.smithi193.stdout:4/391: write d11/d2d/d3a/f76 [1024874,43541] 0 2024-03-23T08:23:21.174 INFO:tasks.workunit.client.1.smithi193.stdout:3/320: dwrite d1/de/d13/d21/d59/d60/f6b [4194304,4194304] 0 2024-03-23T08:23:21.177 INFO:tasks.workunit.client.1.smithi193.stdout:2/401: mknod d0/d28/d33/d72/c75 0 2024-03-23T08:23:21.177 INFO:tasks.workunit.client.1.smithi193.stdout:3/321: fsync d1/de/f70 0 2024-03-23T08:23:21.178 INFO:tasks.workunit.client.1.smithi193.stdout:7/322: dwrite d6/d9/d1f/d3e/d42/f60 [0,4194304] 0 2024-03-23T08:23:21.179 INFO:tasks.workunit.client.1.smithi193.stdout:2/402: chown d0/d1b/d19 1576 1 2024-03-23T08:23:21.181 INFO:tasks.workunit.client.1.smithi193.stdout:4/392: write d11/d1a/f72 [697556,33568] 0 2024-03-23T08:23:21.181 INFO:tasks.workunit.client.1.smithi193.stdout:6/305: symlink d0/db/df/d11/d3b/l58 0 2024-03-23T08:23:21.183 INFO:tasks.workunit.client.1.smithi193.stdout:7/323: creat d6/d9/f6f x:0 0 0 2024-03-23T08:23:21.185 INFO:tasks.workunit.client.1.smithi193.stdout:3/322: mknod d1/de/d13/d21/d59/d60/c73 0 2024-03-23T08:23:21.187 INFO:tasks.workunit.client.1.smithi193.stdout:3/323: read d1/de/d13/d21/d27/f1f [7055433,19257] 0 2024-03-23T08:23:21.190 INFO:tasks.workunit.client.1.smithi193.stdout:4/393: creat d11/d2d/d58/f86 x:0 0 0 2024-03-23T08:23:21.192 INFO:tasks.workunit.client.1.smithi193.stdout:3/324: mknod d1/d22/c74 0 2024-03-23T08:23:21.193 INFO:tasks.workunit.client.1.smithi193.stdout:8/336: dwrite df/d4b/d4d/f1e [0,4194304] 0 2024-03-23T08:23:21.195 INFO:tasks.workunit.client.1.smithi193.stdout:2/403: rmdir d0/d1b/d14/d66/d4e/d63 0 2024-03-23T08:23:21.197 INFO:tasks.workunit.client.1.smithi193.stdout:4/394: creat d11/d1a/d33/d42/d52/f87 x:0 0 0 2024-03-23T08:23:21.198 INFO:tasks.workunit.client.1.smithi193.stdout:3/325: rename d1/de/d13/d21/l2c to d1/de/d13/d21/d25/d34/l75 0 2024-03-23T08:23:21.199 INFO:tasks.workunit.client.1.smithi193.stdout:9/398: sync 2024-03-23T08:23:21.199 INFO:tasks.workunit.client.1.smithi193.stdout:5/373: sync 2024-03-23T08:23:21.199 INFO:tasks.workunit.client.1.smithi193.stdout:1/378: sync 2024-03-23T08:23:21.199 INFO:tasks.workunit.client.1.smithi193.stdout:0/372: sync 2024-03-23T08:23:21.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/404: mknod d0/d1b/d14/d56/c76 0 2024-03-23T08:23:21.201 INFO:tasks.workunit.client.1.smithi193.stdout:8/337: dread df/f34 [0,4194304] 0 2024-03-23T08:23:21.203 INFO:tasks.workunit.client.1.smithi193.stdout:1/379: chown da/d13/d24/f43 39 1 2024-03-23T08:23:21.206 INFO:tasks.workunit.client.1.smithi193.stdout:4/395: fsync d11/d1a/d33/d42/f59 0 2024-03-23T08:23:21.207 INFO:tasks.workunit.client.1.smithi193.stdout:3/326: read d1/de/d13/d21/d25/d34/f47 [88601,95285] 0 2024-03-23T08:23:21.211 INFO:tasks.workunit.client.1.smithi193.stdout:0/373: creat db/d11/d14/d24/d2a/f83 x:0 0 0 2024-03-23T08:23:21.212 INFO:tasks.workunit.client.1.smithi193.stdout:2/405: mkdir d0/d1b/d77 0 2024-03-23T08:23:21.214 INFO:tasks.workunit.client.1.smithi193.stdout:9/399: creat d2/d7/db/d1b/d43/f84 x:0 0 0 2024-03-23T08:23:21.216 INFO:tasks.workunit.client.1.smithi193.stdout:7/324: dwrite d6/d9/f16 [0,4194304] 0 2024-03-23T08:23:21.217 INFO:tasks.workunit.client.1.smithi193.stdout:1/380: read da/d13/d3f/f44 [167633,91214] 0 2024-03-23T08:23:21.217 INFO:tasks.workunit.client.1.smithi193.stdout:6/306: dwrite d0/d8/f1d [4194304,4194304] 0 2024-03-23T08:23:21.222 INFO:tasks.workunit.client.1.smithi193.stdout:4/396: truncate d11/d1a/d33/d42/d52/f87 722061 0 2024-03-23T08:23:21.222 INFO:tasks.workunit.client.1.smithi193.stdout:6/307: stat d0/db/df/d11/f37 0 2024-03-23T08:23:21.224 INFO:tasks.workunit.client.1.smithi193.stdout:3/327: mknod d1/de/d13/d21/d25/d2b/d3c/c76 0 2024-03-23T08:23:21.226 INFO:tasks.workunit.client.1.smithi193.stdout:9/400: creat d2/d7/db/d1b/f85 x:0 0 0 2024-03-23T08:23:21.226 INFO:tasks.workunit.client.1.smithi193.stdout:4/397: stat d11/d1a/d33/d44/f82 0 2024-03-23T08:23:21.227 INFO:tasks.workunit.client.1.smithi193.stdout:5/374: link d0/d1f/f51 d0/d2f/d72/d73/d45/d6e/f89 0 2024-03-23T08:23:21.227 INFO:tasks.workunit.client.1.smithi193.stdout:8/338: dwrite df/d4b/f66 [0,4194304] 0 2024-03-23T08:23:21.227 INFO:tasks.workunit.client.1.smithi193.stdout:1/381: dread da/dd/f39 [0,4194304] 0 2024-03-23T08:23:21.230 INFO:tasks.workunit.client.1.smithi193.stdout:6/308: write d0/d8/d12/f1b [4847722,125055] 0 2024-03-23T08:23:21.231 INFO:tasks.workunit.client.1.smithi193.stdout:1/382: chown da/d13/d3f/d5a/d76/d77/f6c 0 1 2024-03-23T08:23:21.231 INFO:tasks.workunit.client.1.smithi193.stdout:7/325: mkdir d6/d9/d1f/d3e/d65/d70 0 2024-03-23T08:23:21.232 INFO:tasks.workunit.client.1.smithi193.stdout:1/383: readlink da/l33 0 2024-03-23T08:23:21.232 INFO:tasks.workunit.client.1.smithi193.stdout:0/374: link db/d11/d14/d24/d2a/f83 db/d11/d14/d24/d2a/d4b/f84 0 2024-03-23T08:23:21.233 INFO:tasks.workunit.client.1.smithi193.stdout:4/398: mkdir d11/d19/d88 0 2024-03-23T08:23:21.236 INFO:tasks.workunit.client.1.smithi193.stdout:2/406: rename d0/d37/d48/c65 to d0/c78 0 2024-03-23T08:23:21.236 INFO:tasks.workunit.client.1.smithi193.stdout:9/401: dread d2/d7/d31/d3c/f6e [0,4194304] 0 2024-03-23T08:23:21.236 INFO:tasks.workunit.client.1.smithi193.stdout:7/326: creat d6/d9/d1f/d3e/d42/f71 x:0 0 0 2024-03-23T08:23:21.238 INFO:tasks.workunit.client.1.smithi193.stdout:1/384: unlink da/d13/c23 0 2024-03-23T08:23:21.238 INFO:tasks.workunit.client.1.smithi193.stdout:9/402: chown d2/d7/db/d32/f5a 110 1 2024-03-23T08:23:21.238 INFO:tasks.workunit.client.1.smithi193.stdout:2/407: dread d0/f71 [0,4194304] 0 2024-03-23T08:23:21.239 INFO:tasks.workunit.client.1.smithi193.stdout:7/327: dread d6/d9/f36 [0,4194304] 0 2024-03-23T08:23:21.240 INFO:tasks.workunit.client.1.smithi193.stdout:8/339: mkdir df/d63/d76 0 2024-03-23T08:23:21.241 INFO:tasks.workunit.client.1.smithi193.stdout:7/328: dread d6/d9/f15 [4194304,4194304] 0 2024-03-23T08:23:21.246 INFO:tasks.workunit.client.1.smithi193.stdout:0/375: mkdir db/d11/d14/d21/d33/d48/d85 0 2024-03-23T08:23:21.246 INFO:tasks.workunit.client.1.smithi193.stdout:0/376: read - db/d52/f5c zero size 2024-03-23T08:23:21.246 INFO:tasks.workunit.client.1.smithi193.stdout:6/309: mknod d0/db/df/c59 0 2024-03-23T08:23:21.246 INFO:tasks.workunit.client.1.smithi193.stdout:3/328: rename d1/de/c46 to d1/de/d13/d21/d25/d34/c77 0 2024-03-23T08:23:21.249 INFO:tasks.workunit.client.1.smithi193.stdout:9/403: chown d2/d7/db/d17/d33/c39 30111 1 2024-03-23T08:23:21.250 INFO:tasks.workunit.client.1.smithi193.stdout:9/404: write d2/d7/db/d1b/d43/f84 [414106,18945] 0 2024-03-23T08:23:21.250 INFO:tasks.workunit.client.1.smithi193.stdout:5/375: creat d0/d2f/d72/d73/d45/f8a x:0 0 0 2024-03-23T08:23:21.251 INFO:tasks.workunit.client.1.smithi193.stdout:8/340: mknod df/d10/d3e/d54/c77 0 2024-03-23T08:23:21.252 INFO:tasks.workunit.client.1.smithi193.stdout:8/341: fdatasync df/f18 0 2024-03-23T08:23:21.252 INFO:tasks.workunit.client.1.smithi193.stdout:7/329: creat d6/d9/d4e/f72 x:0 0 0 2024-03-23T08:23:21.252 INFO:tasks.workunit.client.1.smithi193.stdout:5/376: write d0/d1f/f49 [3413390,73558] 0 2024-03-23T08:23:21.254 INFO:tasks.workunit.client.1.smithi193.stdout:5/377: chown d0/d1f/f41 94422826 1 2024-03-23T08:23:21.255 INFO:tasks.workunit.client.1.smithi193.stdout:4/399: rename d11/d2d/d3a/l5a to d11/d2d/l89 0 2024-03-23T08:23:21.256 INFO:tasks.workunit.client.1.smithi193.stdout:3/329: mkdir d1/de/d78 0 2024-03-23T08:23:21.257 INFO:tasks.workunit.client.1.smithi193.stdout:1/385: mkdir da/dd/df/d7e/d7f 0 2024-03-23T08:23:21.257 INFO:tasks.workunit.client.1.smithi193.stdout:5/378: chown d0/d2f/d72/d73/l33 63 1 2024-03-23T08:23:21.259 INFO:tasks.workunit.client.1.smithi193.stdout:1/386: write da/dd/df/f1d [3424593,60440] 0 2024-03-23T08:23:21.259 INFO:tasks.workunit.client.1.smithi193.stdout:5/379: write d0/f12 [314248,106558] 0 2024-03-23T08:23:21.261 INFO:tasks.workunit.client.1.smithi193.stdout:9/405: dread d2/d7/d31/d5f/f83 [0,4194304] 0 2024-03-23T08:23:21.261 INFO:tasks.workunit.client.1.smithi193.stdout:7/330: link d6/d9/d1f/d23/f2e d6/d9/d1f/d3e/d65/d70/f73 0 2024-03-23T08:23:21.262 INFO:tasks.workunit.client.1.smithi193.stdout:4/400: creat d11/d27/d85/f8a x:0 0 0 2024-03-23T08:23:21.263 INFO:tasks.workunit.client.1.smithi193.stdout:0/377: rename db/d11/d14/d51/l82 to db/d11/d14/d24/l86 0 2024-03-23T08:23:21.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/310: dread d0/d8/d12/f1e [0,4194304] 0 2024-03-23T08:23:21.265 INFO:tasks.workunit.client.1.smithi193.stdout:0/378: stat db/d11/d14/d21/f26 0 2024-03-23T08:23:21.269 INFO:tasks.workunit.client.1.smithi193.stdout:4/401: dread d11/d2d/d3a/f76 [0,4194304] 0 2024-03-23T08:23:21.269 INFO:tasks.workunit.client.1.smithi193.stdout:6/311: dread d0/db/f33 [0,4194304] 0 2024-03-23T08:23:21.271 INFO:tasks.workunit.client.1.smithi193.stdout:2/408: rename d0/d1b/f64 to d0/d1b/d3b/f79 0 2024-03-23T08:23:21.272 INFO:tasks.workunit.client.1.smithi193.stdout:8/342: link df/d4b/d4d/d2a/c6f df/c78 0 2024-03-23T08:23:21.272 INFO:tasks.workunit.client.1.smithi193.stdout:9/406: dread d2/d7/db/d32/f3f [0,4194304] 0 2024-03-23T08:23:21.273 INFO:tasks.workunit.client.1.smithi193.stdout:2/409: write d0/d1b/d14/d25/f6c [915638,16969] 0 2024-03-23T08:23:21.275 INFO:tasks.workunit.client.1.smithi193.stdout:9/407: truncate d2/d7/d31/d5f/f68 110753 0 2024-03-23T08:23:21.275 INFO:tasks.workunit.client.1.smithi193.stdout:2/410: read d0/d1b/d19/f1a [3061447,68396] 0 2024-03-23T08:23:21.276 INFO:tasks.workunit.client.1.smithi193.stdout:7/331: symlink d6/l74 0 2024-03-23T08:23:21.277 INFO:tasks.workunit.client.1.smithi193.stdout:4/402: mkdir d11/d2d/d58/d8b 0 2024-03-23T08:23:21.279 INFO:tasks.workunit.client.1.smithi193.stdout:3/330: rename d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/f6c to d1/de/d13/d21/d50/d44/f79 0 2024-03-23T08:23:21.280 INFO:tasks.workunit.client.1.smithi193.stdout:6/312: dread d0/d8/f48 [0,4194304] 0 2024-03-23T08:23:21.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/343: mknod df/d10/d67/c79 0 2024-03-23T08:23:21.282 INFO:tasks.workunit.client.1.smithi193.stdout:1/387: rmdir da/dd 39 2024-03-23T08:23:21.282 INFO:tasks.workunit.client.1.smithi193.stdout:0/379: dwrite db/d11/d14/d24/d47/d59/f79 [0,4194304] 0 2024-03-23T08:23:21.284 INFO:tasks.workunit.client.1.smithi193.stdout:3/331: fsync d1/de/d13/d21/d50/d6f/d6e/f72 0 2024-03-23T08:23:21.286 INFO:tasks.workunit.client.1.smithi193.stdout:9/408: unlink l1 0 2024-03-23T08:23:21.291 INFO:tasks.workunit.client.1.smithi193.stdout:5/380: rename d0/d2f/d72/d73/d45/d6e/f89 to d0/d2f/f8b 0 2024-03-23T08:23:21.291 INFO:tasks.workunit.client.1.smithi193.stdout:6/313: creat d0/db/df/d11/f5a x:0 0 0 2024-03-23T08:23:21.293 INFO:tasks.workunit.client.1.smithi193.stdout:2/411: dread d0/d1b/f8 [0,4194304] 0 2024-03-23T08:23:21.293 INFO:tasks.workunit.client.1.smithi193.stdout:6/314: dread - d0/db/df/f4f zero size 2024-03-23T08:23:21.295 INFO:tasks.workunit.client.1.smithi193.stdout:3/332: mknod d1/de/c7a 0 2024-03-23T08:23:21.301 INFO:tasks.workunit.client.1.smithi193.stdout:9/409: mknod d2/d7/db/d32/c86 0 2024-03-23T08:23:21.301 INFO:tasks.workunit.client.1.smithi193.stdout:8/344: symlink df/d4b/d4d/l7a 0 2024-03-23T08:23:21.302 INFO:tasks.workunit.client.1.smithi193.stdout:2/412: rmdir d0/d1b/d14/d66/d50 39 2024-03-23T08:23:21.302 INFO:tasks.workunit.client.1.smithi193.stdout:1/388: truncate da/dd/df/d52/f68 347192 0 2024-03-23T08:23:21.303 INFO:tasks.workunit.client.1.smithi193.stdout:3/333: read d1/f1e [2156460,65445] 0 2024-03-23T08:23:21.303 INFO:tasks.workunit.client.1.smithi193.stdout:4/403: creat d11/f8c x:0 0 0 2024-03-23T08:23:21.303 INFO:tasks.workunit.client.1.smithi193.stdout:0/380: creat db/f87 x:0 0 0 2024-03-23T08:23:21.305 INFO:tasks.workunit.client.1.smithi193.stdout:5/381: dwrite d0/d1f/f78 [0,4194304] 0 2024-03-23T08:23:21.307 INFO:tasks.workunit.client.1.smithi193.stdout:7/332: dread d6/d9/f1b [4194304,4194304] 0 2024-03-23T08:23:21.307 INFO:tasks.workunit.client.1.smithi193.stdout:8/345: fdatasync df/d10/d19/f72 0 2024-03-23T08:23:21.309 INFO:tasks.workunit.client.1.smithi193.stdout:3/334: chown d1/de/d28 63807294 1 2024-03-23T08:23:21.311 INFO:tasks.workunit.client.1.smithi193.stdout:9/410: dread d2/d7/f15 [0,4194304] 0 2024-03-23T08:23:21.311 INFO:tasks.workunit.client.1.smithi193.stdout:4/404: mkdir d11/d8d 0 2024-03-23T08:23:21.313 INFO:tasks.workunit.client.1.smithi193.stdout:1/389: write da/dd/df/d52/f7a [235389,28996] 0 2024-03-23T08:23:21.314 INFO:tasks.workunit.client.1.smithi193.stdout:0/381: rmdir db/d11/d14/d24/d47 39 2024-03-23T08:23:21.318 INFO:tasks.workunit.client.1.smithi193.stdout:5/382: fsync d0/d2f/d72/d73/d45/f56 0 2024-03-23T08:23:21.318 INFO:tasks.workunit.client.1.smithi193.stdout:6/315: dwrite d0/db/df/d11/f5a [0,4194304] 0 2024-03-23T08:23:21.320 INFO:tasks.workunit.client.1.smithi193.stdout:8/346: write df/d10/d19/f4f [2112287,52511] 0 2024-03-23T08:23:21.321 INFO:tasks.workunit.client.1.smithi193.stdout:9/411: creat d2/d7/db/d7a/f87 x:0 0 0 2024-03-23T08:23:21.322 INFO:tasks.workunit.client.1.smithi193.stdout:4/405: mkdir d11/d8e 0 2024-03-23T08:23:21.323 INFO:tasks.workunit.client.1.smithi193.stdout:3/335: symlink d1/de/d78/l7b 0 2024-03-23T08:23:21.325 INFO:tasks.workunit.client.1.smithi193.stdout:0/382: unlink db/d11/d14/d21/d33/c42 0 2024-03-23T08:23:21.325 INFO:tasks.workunit.client.1.smithi193.stdout:1/390: mknod da/c80 0 2024-03-23T08:23:21.325 INFO:tasks.workunit.client.1.smithi193.stdout:5/383: rename d0/d5a/c79 to d0/d2f/c8c 0 2024-03-23T08:23:21.327 INFO:tasks.workunit.client.1.smithi193.stdout:7/333: truncate d6/d9/d1f/d3e/d65/d70/f73 1915453 0 2024-03-23T08:23:21.328 INFO:tasks.workunit.client.1.smithi193.stdout:9/412: chown d2/d7/db/d17/c55 7872470 1 2024-03-23T08:23:21.329 INFO:tasks.workunit.client.1.smithi193.stdout:2/413: dwrite d0/f2d [4194304,4194304] 0 2024-03-23T08:23:21.329 INFO:tasks.workunit.client.1.smithi193.stdout:8/347: rename df/d4b/d4d/d2a/f49 to df/d10/f7b 0 2024-03-23T08:23:21.333 INFO:tasks.workunit.client.1.smithi193.stdout:8/348: stat df/d4b/f66 0 2024-03-23T08:23:21.335 INFO:tasks.workunit.client.1.smithi193.stdout:7/334: creat d6/d9/d1f/d3e/d65/f75 x:0 0 0 2024-03-23T08:23:21.335 INFO:tasks.workunit.client.1.smithi193.stdout:0/383: mkdir db/d11/d14/d21/d33/d69/d88 0 2024-03-23T08:23:21.338 INFO:tasks.workunit.client.1.smithi193.stdout:7/335: readlink d6/d9/d1f/d23/d33/l54 0 2024-03-23T08:23:21.340 INFO:tasks.workunit.client.1.smithi193.stdout:2/414: rename d0/d28/f44 to d0/d1b/d3b/f7a 0 2024-03-23T08:23:21.340 INFO:tasks.workunit.client.1.smithi193.stdout:8/349: chown df/d10/d3e/d54/c77 88586258 1 2024-03-23T08:23:21.346 INFO:tasks.workunit.client.1.smithi193.stdout:7/336: write d6/f8 [4551760,123785] 0 2024-03-23T08:23:21.347 INFO:tasks.workunit.client.1.smithi193.stdout:3/336: dwrite d1/de/d13/d21/d25/d2b/f39 [0,4194304] 0 2024-03-23T08:23:21.348 INFO:tasks.workunit.client.1.smithi193.stdout:9/413: dwrite d2/d7/f36 [4194304,4194304] 0 2024-03-23T08:23:21.349 INFO:tasks.workunit.client.1.smithi193.stdout:4/406: truncate d11/d19/d22/f26 179252 0 2024-03-23T08:23:21.351 INFO:tasks.workunit.client.1.smithi193.stdout:2/415: mkdir d0/d1b/d14/d66/d7b 0 2024-03-23T08:23:21.352 INFO:tasks.workunit.client.1.smithi193.stdout:9/414: readlink d2/d7/l16 0 2024-03-23T08:23:21.354 INFO:tasks.workunit.client.1.smithi193.stdout:4/407: stat d11/d48/l55 0 2024-03-23T08:23:21.358 INFO:tasks.workunit.client.1.smithi193.stdout:3/337: mkdir d1/de/d13/d21/d27/d7c 0 2024-03-23T08:23:21.362 INFO:tasks.workunit.client.1.smithi193.stdout:7/337: mknod d6/d9/d1f/d3e/d65/d70/c76 0 2024-03-23T08:23:21.362 INFO:tasks.workunit.client.1.smithi193.stdout:9/415: fdatasync d2/d7/db/d1b/d43/f74 0 2024-03-23T08:23:21.365 INFO:tasks.workunit.client.1.smithi193.stdout:4/408: getdents d11/d48/d50/d49 0 2024-03-23T08:23:21.366 INFO:tasks.workunit.client.1.smithi193.stdout:8/350: dwrite df/d10/f70 [0,4194304] 0 2024-03-23T08:23:21.368 INFO:tasks.workunit.client.1.smithi193.stdout:2/416: dread d0/d1b/d14/d25/f6c [0,4194304] 0 2024-03-23T08:23:21.368 INFO:tasks.workunit.client.1.smithi193.stdout:5/384: dwrite d0/d1f/f51 [0,4194304] 0 2024-03-23T08:23:21.373 INFO:tasks.workunit.client.1.smithi193.stdout:5/385: fdatasync d0/d1f/f5b 0 2024-03-23T08:23:21.376 INFO:tasks.workunit.client.1.smithi193.stdout:9/416: symlink d2/l88 0 2024-03-23T08:23:21.378 INFO:tasks.workunit.client.1.smithi193.stdout:3/338: read - d1/de/d13/d21/d27/f63 zero size 2024-03-23T08:23:21.378 INFO:tasks.workunit.client.1.smithi193.stdout:0/384: dwrite db/d11/d14/d24/f7d [0,4194304] 0 2024-03-23T08:23:21.379 INFO:tasks.workunit.client.1.smithi193.stdout:2/417: write d0/d1b/d14/d66/d50/f52 [451429,40855] 0 2024-03-23T08:23:21.380 INFO:tasks.workunit.client.1.smithi193.stdout:3/339: stat d1/de/c12 0 2024-03-23T08:23:21.381 INFO:tasks.workunit.client.1.smithi193.stdout:4/409: mkdir d11/d1a/d33/d8f 0 2024-03-23T08:23:21.381 INFO:tasks.workunit.client.1.smithi193.stdout:8/351: symlink df/d4b/d4d/l7c 0 2024-03-23T08:23:21.386 INFO:tasks.workunit.client.1.smithi193.stdout:0/385: dread - db/d11/d14/d21/d33/d48/f5d zero size 2024-03-23T08:23:21.387 INFO:tasks.workunit.client.1.smithi193.stdout:3/340: symlink d1/de/d13/d21/d25/d34/l7d 0 2024-03-23T08:23:21.388 INFO:tasks.workunit.client.1.smithi193.stdout:9/417: read d2/d7/db/d17/d1f/f22 [3549331,68712] 0 2024-03-23T08:23:21.390 INFO:tasks.workunit.client.1.smithi193.stdout:0/386: symlink db/d11/d14/d3b/l89 0 2024-03-23T08:23:21.391 INFO:tasks.workunit.client.1.smithi193.stdout:2/418: getdents d0/d1b/d19 0 2024-03-23T08:23:21.393 INFO:tasks.workunit.client.1.smithi193.stdout:3/341: readlink d1/de/d13/d21/d25/d2b/d3c/l3e 0 2024-03-23T08:23:21.397 INFO:tasks.workunit.client.1.smithi193.stdout:2/419: write d0/d1b/d3b/f46 [2357804,388] 0 2024-03-23T08:23:21.399 INFO:tasks.workunit.client.1.smithi193.stdout:3/342: readlink d1/de/d13/d21/d25/d34/l49 0 2024-03-23T08:23:21.400 INFO:tasks.workunit.client.1.smithi193.stdout:8/352: dwrite df/d10/f29 [0,4194304] 0 2024-03-23T08:23:21.400 INFO:tasks.workunit.client.1.smithi193.stdout:0/387: write db/d11/d14/d21/f26 [4188270,46750] 0 2024-03-23T08:23:21.404 INFO:tasks.workunit.client.1.smithi193.stdout:4/410: dwrite d11/d2d/f4b [0,4194304] 0 2024-03-23T08:23:21.411 INFO:tasks.workunit.client.1.smithi193.stdout:5/386: dread d0/f2a [0,4194304] 0 2024-03-23T08:23:21.411 INFO:tasks.workunit.client.1.smithi193.stdout:8/353: creat df/d10/d67/f7d x:0 0 0 2024-03-23T08:23:21.411 INFO:tasks.workunit.client.1.smithi193.stdout:4/411: symlink d11/d48/d50/l90 0 2024-03-23T08:23:21.411 INFO:tasks.workunit.client.1.smithi193.stdout:2/420: write d0/d1b/d14/d25/f5a [221974,23764] 0 2024-03-23T08:23:21.416 INFO:tasks.workunit.client.1.smithi193.stdout:4/412: chown l10 833231 1 2024-03-23T08:23:21.416 INFO:tasks.workunit.client.1.smithi193.stdout:7/338: dread d6/d9/d1f/f30 [4194304,4194304] 0 2024-03-23T08:23:21.418 INFO:tasks.workunit.client.1.smithi193.stdout:2/421: read d0/f7 [3194923,76752] 0 2024-03-23T08:23:21.418 INFO:tasks.workunit.client.1.smithi193.stdout:0/388: symlink db/d11/d14/d21/l8a 0 2024-03-23T08:23:21.419 INFO:tasks.workunit.client.1.smithi193.stdout:8/354: fsync df/d4b/d4d/f22 0 2024-03-23T08:23:21.421 INFO:tasks.workunit.client.1.smithi193.stdout:4/413: creat d11/d8d/f91 x:0 0 0 2024-03-23T08:23:21.421 INFO:tasks.workunit.client.1.smithi193.stdout:3/343: dwrite d1/de/d13/f5b [0,4194304] 0 2024-03-23T08:23:21.422 INFO:tasks.workunit.client.1.smithi193.stdout:2/422: rmdir d0/d1b/d14/d66 39 2024-03-23T08:23:21.423 INFO:tasks.workunit.client.1.smithi193.stdout:7/339: creat d6/d9/d1f/d23/d33/d38/d3b/d6c/f77 x:0 0 0 2024-03-23T08:23:21.424 INFO:tasks.workunit.client.1.smithi193.stdout:8/355: chown df/d4b/d4d/c41 3111 1 2024-03-23T08:23:21.425 INFO:tasks.workunit.client.1.smithi193.stdout:2/423: mkdir d0/d1b/d14/d56/d7c 0 2024-03-23T08:23:21.426 INFO:tasks.workunit.client.1.smithi193.stdout:3/344: creat d1/de/d13/d21/d25/f7e x:0 0 0 2024-03-23T08:23:21.426 INFO:tasks.workunit.client.1.smithi193.stdout:8/356: chown df/d4b/d4d/d2a/l58 5181750 1 2024-03-23T08:23:21.427 INFO:tasks.workunit.client.1.smithi193.stdout:3/345: fsync d1/de/f16 0 2024-03-23T08:23:21.434 INFO:tasks.workunit.client.1.smithi193.stdout:3/346: fsync d1/de/f11 0 2024-03-23T08:23:21.434 INFO:tasks.workunit.client.1.smithi193.stdout:4/414: link d11/d2d/d3a/f6f d11/f92 0 2024-03-23T08:23:21.435 INFO:tasks.workunit.client.1.smithi193.stdout:2/424: dwrite d0/d1b/d3b/f42 [0,4194304] 0 2024-03-23T08:23:21.435 INFO:tasks.workunit.client.1.smithi193.stdout:8/357: symlink df/d10/d3e/l7e 0 2024-03-23T08:23:21.437 INFO:tasks.workunit.client.1.smithi193.stdout:5/387: dwrite d0/d5a/f75 [0,4194304] 0 2024-03-23T08:23:21.439 INFO:tasks.workunit.client.1.smithi193.stdout:8/358: readlink df/d4b/d4d/d2a/l36 0 2024-03-23T08:23:21.441 INFO:tasks.workunit.client.1.smithi193.stdout:3/347: creat d1/de/d13/d21/d25/d2b/f7f x:0 0 0 2024-03-23T08:23:21.445 INFO:tasks.workunit.client.1.smithi193.stdout:5/388: rename d0/f27 to d0/d1c/d3b/d4b/f8d 0 2024-03-23T08:23:21.446 INFO:tasks.workunit.client.1.smithi193.stdout:7/340: truncate d6/d9/d1f/f30 6490443 0 2024-03-23T08:23:21.447 INFO:tasks.workunit.client.1.smithi193.stdout:3/348: mknod d1/de/c80 0 2024-03-23T08:23:21.449 INFO:tasks.workunit.client.1.smithi193.stdout:5/389: readlink d0/l6 0 2024-03-23T08:23:21.449 INFO:tasks.workunit.client.1.smithi193.stdout:2/425: write d0/d1b/d14/f45 [71290,27071] 0 2024-03-23T08:23:21.449 INFO:tasks.workunit.client.1.smithi193.stdout:7/341: truncate d6/d9/d1f/d3e/d65/f75 600538 0 2024-03-23T08:23:21.450 INFO:tasks.workunit.client.1.smithi193.stdout:4/415: write d11/d1a/d33/f7d [2464893,52157] 0 2024-03-23T08:23:21.452 INFO:tasks.workunit.client.1.smithi193.stdout:5/390: read - d0/d2f/d72/d73/d57/d6d/f7c zero size 2024-03-23T08:23:21.452 INFO:tasks.workunit.client.1.smithi193.stdout:4/416: fsync d11/d19/f2f 0 2024-03-23T08:23:21.455 INFO:tasks.workunit.client.1.smithi193.stdout:3/349: dread d1/de/f16 [0,4194304] 0 2024-03-23T08:23:21.458 INFO:tasks.workunit.client.1.smithi193.stdout:5/391: mkdir d0/d2f/d72/d73/d57/d6d/d8e 0 2024-03-23T08:23:21.459 INFO:tasks.workunit.client.1.smithi193.stdout:8/359: truncate df/d46/f47 2586584 0 2024-03-23T08:23:21.460 INFO:tasks.workunit.client.1.smithi193.stdout:5/392: stat d0/f14 0 2024-03-23T08:23:21.462 INFO:tasks.workunit.client.1.smithi193.stdout:2/426: getdents d0/d1b 0 2024-03-23T08:23:21.464 INFO:tasks.workunit.client.1.smithi193.stdout:5/393: write d0/d1f/f78 [610734,87354] 0 2024-03-23T08:23:21.465 INFO:tasks.workunit.client.1.smithi193.stdout:8/360: write df/d10/f3f [937596,63826] 0 2024-03-23T08:23:21.466 INFO:tasks.workunit.client.1.smithi193.stdout:7/342: dread d6/d9/d1f/d23/f2e [0,4194304] 0 2024-03-23T08:23:21.469 INFO:tasks.workunit.client.1.smithi193.stdout:4/417: dwrite d11/d48/d50/f5b [0,4194304] 0 2024-03-23T08:23:21.469 INFO:tasks.workunit.client.1.smithi193.stdout:5/394: fsync d0/d1f/f78 0 2024-03-23T08:23:21.469 INFO:tasks.workunit.client.1.smithi193.stdout:2/427: write d0/d1b/d3b/f46 [4045384,97185] 0 2024-03-23T08:23:21.471 INFO:tasks.workunit.client.1.smithi193.stdout:7/343: chown d6/d9/d1f/d23/d33/d38/d3b 3 1 2024-03-23T08:23:21.473 INFO:tasks.workunit.client.1.smithi193.stdout:3/350: dwrite d1/de/d13/d21/d27/f1f [4194304,4194304] 0 2024-03-23T08:23:21.474 INFO:tasks.workunit.client.1.smithi193.stdout:4/418: chown d11/d1a/d33/d44/c46 0 1 2024-03-23T08:23:21.475 INFO:tasks.workunit.client.1.smithi193.stdout:2/428: write d0/d1b/d3b/f7a [842654,10519] 0 2024-03-23T08:23:21.478 INFO:tasks.workunit.client.1.smithi193.stdout:7/344: creat d6/d9/f78 x:0 0 0 2024-03-23T08:23:21.479 INFO:tasks.workunit.client.1.smithi193.stdout:3/351: creat d1/de/d13/d21/f81 x:0 0 0 2024-03-23T08:23:21.480 INFO:tasks.workunit.client.1.smithi193.stdout:4/419: rmdir d11/d48/d50/d49 39 2024-03-23T08:23:21.480 INFO:tasks.workunit.client.1.smithi193.stdout:5/395: dread d0/d5a/f75 [0,4194304] 0 2024-03-23T08:23:21.485 INFO:tasks.workunit.client.1.smithi193.stdout:5/396: creat d0/d1c/d7a/f8f x:0 0 0 2024-03-23T08:23:21.486 INFO:tasks.workunit.client.1.smithi193.stdout:7/345: rename d6/d9/f35 to d6/d9/d1f/d23/d33/d3c/d41/f79 0 2024-03-23T08:23:21.490 INFO:tasks.workunit.client.1.smithi193.stdout:3/352: getdents d1/de 0 2024-03-23T08:23:21.490 INFO:tasks.workunit.client.1.smithi193.stdout:2/429: truncate d0/d1b/f21 671575 0 2024-03-23T08:23:21.490 INFO:tasks.workunit.client.1.smithi193.stdout:7/346: dread d6/d9/d1f/d3e/d65/d70/f73 [0,4194304] 0 2024-03-23T08:23:21.491 INFO:tasks.workunit.client.1.smithi193.stdout:8/361: write df/d46/f47 [822814,69029] 0 2024-03-23T08:23:21.493 INFO:tasks.workunit.client.1.smithi193.stdout:7/347: chown d6/d9/d1f/d23/d33/d38/d3b 62356468 1 2024-03-23T08:23:21.493 INFO:tasks.workunit.client.1.smithi193.stdout:5/397: link d0/d1c/c5e d0/d1c/d3b/d81/c90 0 2024-03-23T08:23:21.496 INFO:tasks.workunit.client.1.smithi193.stdout:8/362: stat df/d4b/d4d/d2a/c6f 0 2024-03-23T08:23:21.502 INFO:tasks.workunit.client.1.smithi193.stdout:4/420: dread d11/d19/d22/f45 [0,4194304] 0 2024-03-23T08:23:21.502 INFO:tasks.workunit.client.1.smithi193.stdout:7/348: link d6/d9/fd d6/d9/d1f/d3e/d42/f7a 0 2024-03-23T08:23:21.502 INFO:tasks.workunit.client.1.smithi193.stdout:2/430: dwrite d0/d1b/d3b/f7a [0,4194304] 0 2024-03-23T08:23:21.504 INFO:tasks.workunit.client.1.smithi193.stdout:3/353: truncate d1/de/d13/d21/d25/d2b/f39 8059873 0 2024-03-23T08:23:21.506 INFO:tasks.workunit.client.1.smithi193.stdout:5/398: dwrite d0/d2f/d72/d73/f6c [0,4194304] 0 2024-03-23T08:23:21.507 INFO:tasks.workunit.client.1.smithi193.stdout:2/431: symlink d0/d1b/d14/l7d 0 2024-03-23T08:23:21.507 INFO:tasks.workunit.client.1.smithi193.stdout:4/421: mknod d11/d19/d88/c93 0 2024-03-23T08:23:21.510 INFO:tasks.workunit.client.1.smithi193.stdout:3/354: creat d1/de/d13/d21/f82 x:0 0 0 2024-03-23T08:23:21.512 INFO:tasks.workunit.client.1.smithi193.stdout:4/422: write d11/d1a/d33/d44/f4a [4318980,131066] 0 2024-03-23T08:23:21.512 INFO:tasks.workunit.client.1.smithi193.stdout:5/399: dread d0/d1c/f65 [0,4194304] 0 2024-03-23T08:23:21.514 INFO:tasks.workunit.client.1.smithi193.stdout:3/355: symlink d1/de/d13/l83 0 2024-03-23T08:23:21.515 INFO:tasks.workunit.client.1.smithi193.stdout:2/432: getdents d0/d28/d33 0 2024-03-23T08:23:21.515 INFO:tasks.workunit.client.1.smithi193.stdout:3/356: dread - d1/de/f71 zero size 2024-03-23T08:23:21.515 INFO:tasks.workunit.client.1.smithi193.stdout:7/349: truncate d6/d9/d1f/d23/d33/d3c/d41/f4a 533366 0 2024-03-23T08:23:21.516 INFO:tasks.workunit.client.1.smithi193.stdout:3/357: chown d1/de/c80 40 1 2024-03-23T08:23:21.517 INFO:tasks.workunit.client.1.smithi193.stdout:4/423: creat d11/d19/f94 x:0 0 0 2024-03-23T08:23:21.517 INFO:tasks.workunit.client.1.smithi193.stdout:5/400: mkdir d0/d2f/d72/d73/d45/d6e/d91 0 2024-03-23T08:23:21.517 INFO:tasks.workunit.client.1.smithi193.stdout:3/358: readlink d1/de/d13/d21/d25/d34/l49 0 2024-03-23T08:23:21.518 INFO:tasks.workunit.client.1.smithi193.stdout:5/401: fdatasync d0/d1c/f5c 0 2024-03-23T08:23:21.518 INFO:tasks.workunit.client.1.smithi193.stdout:7/350: mknod d6/d9/d1f/d4d/c7b 0 2024-03-23T08:23:21.519 INFO:tasks.workunit.client.1.smithi193.stdout:2/433: mkdir d0/d1b/d14/d56/d7c/d7e 0 2024-03-23T08:23:21.520 INFO:tasks.workunit.client.1.smithi193.stdout:4/424: truncate d11/d1a/d33/d44/f6e 844152 0 2024-03-23T08:23:21.520 INFO:tasks.workunit.client.1.smithi193.stdout:7/351: mkdir d6/d9/d1f/d4d/d7c 0 2024-03-23T08:23:21.521 INFO:tasks.workunit.client.1.smithi193.stdout:3/359: creat d1/f84 x:0 0 0 2024-03-23T08:23:21.521 INFO:tasks.workunit.client.1.smithi193.stdout:2/434: rmdir d0/d1b/d14/d56 39 2024-03-23T08:23:21.522 INFO:tasks.workunit.client.1.smithi193.stdout:4/425: read d11/d2d/f68 [53569,113951] 0 2024-03-23T08:23:21.522 INFO:tasks.workunit.client.1.smithi193.stdout:3/360: read - d1/f51 zero size 2024-03-23T08:23:21.522 INFO:tasks.workunit.client.1.smithi193.stdout:7/352: fdatasync d6/d9/d1f/d23/d33/f34 0 2024-03-23T08:23:21.523 INFO:tasks.workunit.client.1.smithi193.stdout:5/402: truncate d0/d2f/d72/d73/d57/d6d/f87 167285 0 2024-03-23T08:23:21.524 INFO:tasks.workunit.client.1.smithi193.stdout:4/426: write d11/d1a/d33/f39 [4721420,39316] 0 2024-03-23T08:23:21.524 INFO:tasks.workunit.client.1.smithi193.stdout:6/316: sync 2024-03-23T08:23:21.526 INFO:tasks.workunit.client.1.smithi193.stdout:4/427: readlink d11/d48/d50/l3d 0 2024-03-23T08:23:21.527 INFO:tasks.workunit.client.1.smithi193.stdout:6/317: dread - d0/d8/d3a/f53 zero size 2024-03-23T08:23:21.527 INFO:tasks.workunit.client.1.smithi193.stdout:2/435: dread d0/d1b/d3b/f58 [0,4194304] 0 2024-03-23T08:23:21.527 INFO:tasks.workunit.client.1.smithi193.stdout:7/353: unlink d6/d9/d1f/d23/d33/d3c/d41/d47/l5c 0 2024-03-23T08:23:21.528 INFO:tasks.workunit.client.1.smithi193.stdout:7/354: chown d6/d9/f6a 1441399 1 2024-03-23T08:23:21.529 INFO:tasks.workunit.client.1.smithi193.stdout:2/436: chown d0/d1b/c22 3398205 1 2024-03-23T08:23:21.529 INFO:tasks.workunit.client.1.smithi193.stdout:5/403: link d0/d1f/f78 d0/d2f/d72/d73/d57/d6d/d8e/f92 0 2024-03-23T08:23:21.531 INFO:tasks.workunit.client.1.smithi193.stdout:5/404: fdatasync d0/d2f/f38 0 2024-03-23T08:23:21.531 INFO:tasks.workunit.client.1.smithi193.stdout:7/355: mkdir d6/d9/d1f/d3e/d65/d7d 0 2024-03-23T08:23:21.532 INFO:tasks.workunit.client.1.smithi193.stdout:5/405: readlink d0/l3f 0 2024-03-23T08:23:21.533 INFO:tasks.workunit.client.1.smithi193.stdout:7/356: fsync d6/f2d 0 2024-03-23T08:23:21.533 INFO:tasks.workunit.client.1.smithi193.stdout:6/318: truncate d0/d8/d12/d21/f29 210327 0 2024-03-23T08:23:21.533 INFO:tasks.workunit.client.1.smithi193.stdout:5/406: dread - d0/d1f/f2e zero size 2024-03-23T08:23:21.535 INFO:tasks.workunit.client.1.smithi193.stdout:2/437: link d0/d37/f4c d0/d1b/d19/f7f 0 2024-03-23T08:23:21.535 INFO:tasks.workunit.client.1.smithi193.stdout:4/428: dwrite d11/d1a/d33/f7d [0,4194304] 0 2024-03-23T08:23:21.536 INFO:tasks.workunit.client.1.smithi193.stdout:2/438: read - d0/d1b/f74 zero size 2024-03-23T08:23:21.537 INFO:tasks.workunit.client.1.smithi193.stdout:4/429: write d11/d27/f73 [681439,92110] 0 2024-03-23T08:23:21.538 INFO:tasks.workunit.client.1.smithi193.stdout:7/357: mknod d6/d9/d1f/d23/d33/d38/d3b/c7e 0 2024-03-23T08:23:21.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/439: chown d0/fc 0 1 2024-03-23T08:23:21.559 INFO:tasks.workunit.client.1.smithi193.stdout:4/430: mkdir d11/d95 0 2024-03-23T08:23:21.559 INFO:tasks.workunit.client.1.smithi193.stdout:6/319: link d0/db/df/d11/d36/c3e d0/d8/d12/d21/c5b 0 2024-03-23T08:23:21.561 INFO:tasks.workunit.client.1.smithi193.stdout:5/407: dwrite d0/f12 [4194304,4194304] 0 2024-03-23T08:23:21.562 INFO:tasks.workunit.client.1.smithi193.stdout:2/440: truncate d0/f1d 323933 0 2024-03-23T08:23:21.564 INFO:tasks.workunit.client.1.smithi193.stdout:5/408: mkdir d0/d1f/d93 0 2024-03-23T08:23:21.567 INFO:tasks.workunit.client.1.smithi193.stdout:2/441: write d0/d1b/d3b/f58 [1030554,23477] 0 2024-03-23T08:23:21.567 INFO:tasks.workunit.client.1.smithi193.stdout:4/431: rename d11/d2d/d58/c79 to d11/d2d/d58/d8b/c96 0 2024-03-23T08:23:21.570 INFO:tasks.workunit.client.1.smithi193.stdout:7/358: dwrite d6/d9/d1f/d3e/d42/f59 [0,4194304] 0 2024-03-23T08:23:21.573 INFO:tasks.workunit.client.1.smithi193.stdout:4/432: dread - d11/d1a/d33/d7f/f81 zero size 2024-03-23T08:23:21.573 INFO:tasks.workunit.client.1.smithi193.stdout:2/442: truncate d0/d1b/d14/d25/f3a 4449199 0 2024-03-23T08:23:21.574 INFO:tasks.workunit.client.1.smithi193.stdout:7/359: dread - d6/d9/d1f/d23/d33/d38/d46/f61 zero size 2024-03-23T08:23:21.574 INFO:tasks.workunit.client.1.smithi193.stdout:5/409: link d0/d1c/f3a d0/d1c/d3b/f94 0 2024-03-23T08:23:21.574 INFO:tasks.workunit.client.1.smithi193.stdout:4/433: fdatasync f4 0 2024-03-23T08:23:21.579 INFO:tasks.workunit.client.1.smithi193.stdout:2/443: rename d0/d1b/d3b/f54 to d0/d1b/d19/f80 0 2024-03-23T08:23:21.580 INFO:tasks.workunit.client.1.smithi193.stdout:4/434: creat d11/d8d/f97 x:0 0 0 2024-03-23T08:23:21.580 INFO:tasks.workunit.client.1.smithi193.stdout:4/435: readlink d11/d48/d50/l90 0 2024-03-23T08:23:21.581 INFO:tasks.workunit.client.1.smithi193.stdout:7/360: write d6/fc [5429328,117675] 0 2024-03-23T08:23:21.582 INFO:tasks.workunit.client.1.smithi193.stdout:5/410: truncate d0/f7 698282 0 2024-03-23T08:23:21.584 INFO:tasks.workunit.client.1.smithi193.stdout:2/444: mknod d0/d28/d33/c81 0 2024-03-23T08:23:21.586 INFO:tasks.workunit.client.1.smithi193.stdout:7/361: write d6/d9/d1f/d23/d33/d38/f68 [1557121,27800] 0 2024-03-23T08:23:21.586 INFO:tasks.workunit.client.1.smithi193.stdout:4/436: write d11/d48/d50/d49/f4f [814744,126951] 0 2024-03-23T08:23:21.588 INFO:tasks.workunit.client.1.smithi193.stdout:2/445: rmdir d0/d28/d33/d72 39 2024-03-23T08:23:21.590 INFO:tasks.workunit.client.1.smithi193.stdout:2/446: rmdir d0 39 2024-03-23T08:23:21.592 INFO:tasks.workunit.client.1.smithi193.stdout:7/362: dread - d6/d9/d1f/d23/d33/d38/d3b/d6c/f77 zero size 2024-03-23T08:23:21.592 INFO:tasks.workunit.client.1.smithi193.stdout:5/411: dwrite d0/f12 [4194304,4194304] 0 2024-03-23T08:23:21.593 INFO:tasks.workunit.client.1.smithi193.stdout:4/437: dread d11/d1a/d33/d42/f4d [0,4194304] 0 2024-03-23T08:23:21.594 INFO:tasks.workunit.client.1.smithi193.stdout:4/438: write fb [2037801,102419] 0 2024-03-23T08:23:21.595 INFO:tasks.workunit.client.1.smithi193.stdout:7/363: write d6/d9/d1f/d3e/f5e [578144,4158] 0 2024-03-23T08:23:21.596 INFO:tasks.workunit.client.1.smithi193.stdout:5/412: write d0/d1c/d3b/d4b/d55/f59 [3634964,103157] 0 2024-03-23T08:23:21.600 INFO:tasks.workunit.client.1.smithi193.stdout:2/447: creat d0/d28/f82 x:0 0 0 2024-03-23T08:23:21.602 INFO:tasks.workunit.client.1.smithi193.stdout:7/364: creat d6/d9/d1f/d3e/d65/d70/f7f x:0 0 0 2024-03-23T08:23:21.603 INFO:tasks.workunit.client.1.smithi193.stdout:5/413: creat d0/d1c/d3b/d4b/f95 x:0 0 0 2024-03-23T08:23:21.604 INFO:tasks.workunit.client.1.smithi193.stdout:6/320: dwrite d0/db/f33 [0,4194304] 0 2024-03-23T08:23:21.604 INFO:tasks.workunit.client.1.smithi193.stdout:2/448: creat d0/d1b/d14/d25/f83 x:0 0 0 2024-03-23T08:23:21.606 INFO:tasks.workunit.client.1.smithi193.stdout:4/439: dread d11/d2d/f4b [0,4194304] 0 2024-03-23T08:23:21.608 INFO:tasks.workunit.client.1.smithi193.stdout:2/449: mknod d0/d37/d48/c84 0 2024-03-23T08:23:21.610 INFO:tasks.workunit.client.1.smithi193.stdout:7/365: creat d6/d9/d1f/d4d/d6b/f80 x:0 0 0 2024-03-23T08:23:21.614 INFO:tasks.workunit.client.1.smithi193.stdout:6/321: dwrite d0/db/df/f25 [0,4194304] 0 2024-03-23T08:23:21.615 INFO:tasks.workunit.client.1.smithi193.stdout:6/322: chown d0/d8/d12/f1e 125961 1 2024-03-23T08:23:21.618 INFO:tasks.workunit.client.1.smithi193.stdout:7/366: mkdir d6/d9/d4e/d81 0 2024-03-23T08:23:21.618 INFO:tasks.workunit.client.1.smithi193.stdout:2/450: rmdir d0/d1b/d14/d56 39 2024-03-23T08:23:21.620 INFO:tasks.workunit.client.1.smithi193.stdout:5/414: rename d0/f14 to d0/d2f/d72/d73/d45/f96 0 2024-03-23T08:23:21.621 INFO:tasks.workunit.client.1.smithi193.stdout:6/323: creat d0/db/df/d11/d56/f5c x:0 0 0 2024-03-23T08:23:21.622 INFO:tasks.workunit.client.1.smithi193.stdout:6/324: chown d0/d8/d12 4 1 2024-03-23T08:23:21.624 INFO:tasks.workunit.client.1.smithi193.stdout:5/415: unlink d0/d1c/d3b/d4b/f8d 0 2024-03-23T08:23:21.625 INFO:tasks.workunit.client.1.smithi193.stdout:7/367: dwrite d6/d9/d1f/d23/d33/d38/d46/f5b [0,4194304] 0 2024-03-23T08:23:21.627 INFO:tasks.workunit.client.1.smithi193.stdout:6/325: symlink d0/d8/d12/d21/l5d 0 2024-03-23T08:23:21.627 INFO:tasks.workunit.client.1.smithi193.stdout:7/368: fsync d6/d9/d1f/d23/d33/f34 0 2024-03-23T08:23:21.628 INFO:tasks.workunit.client.1.smithi193.stdout:4/440: rename d11/d2d/c41 to d11/d1a/d33/c98 0 2024-03-23T08:23:21.629 INFO:tasks.workunit.client.1.smithi193.stdout:7/369: chown d6/d9/l1d 2785 1 2024-03-23T08:23:21.629 INFO:tasks.workunit.client.1.smithi193.stdout:6/326: fsync d0/db/df/d11/d36/f41 0 2024-03-23T08:23:21.630 INFO:tasks.workunit.client.1.smithi193.stdout:2/451: rename d0/d1b/d14/c1c to d0/d1b/d14/c85 0 2024-03-23T08:23:21.630 INFO:tasks.workunit.client.1.smithi193.stdout:9/418: sync 2024-03-23T08:23:21.630 INFO:tasks.workunit.client.1.smithi193.stdout:1/391: sync 2024-03-23T08:23:21.631 INFO:tasks.workunit.client.1.smithi193.stdout:7/370: chown d6/d9/d1f/d4d/d7c 7 1 2024-03-23T08:23:21.633 INFO:tasks.workunit.client.1.smithi193.stdout:4/441: readlink d11/d27/l57 0 2024-03-23T08:23:21.634 INFO:tasks.workunit.client.1.smithi193.stdout:9/419: mknod d2/d7/db/d1b/d5c/c89 0 2024-03-23T08:23:21.635 INFO:tasks.workunit.client.1.smithi193.stdout:1/392: symlink da/dd/l81 0 2024-03-23T08:23:21.636 INFO:tasks.workunit.client.1.smithi193.stdout:4/442: dread - d11/d1a/d33/d44/f82 zero size 2024-03-23T08:23:21.636 INFO:tasks.workunit.client.1.smithi193.stdout:6/327: symlink d0/db/df/l5e 0 2024-03-23T08:23:21.637 INFO:tasks.workunit.client.1.smithi193.stdout:2/452: creat d0/d1b/d14/d56/d7c/f86 x:0 0 0 2024-03-23T08:23:21.638 INFO:tasks.workunit.client.1.smithi193.stdout:9/420: write d2/d7/db/f3b [2930091,11640] 0 2024-03-23T08:23:21.639 INFO:tasks.workunit.client.1.smithi193.stdout:5/416: dwrite d0/f11 [0,4194304] 0 2024-03-23T08:23:21.639 INFO:tasks.workunit.client.1.smithi193.stdout:1/393: mkdir da/dd/df/d52/d82 0 2024-03-23T08:23:21.641 INFO:tasks.workunit.client.1.smithi193.stdout:1/394: readlink da/dd/l5f 0 2024-03-23T08:23:21.643 INFO:tasks.workunit.client.1.smithi193.stdout:5/417: mkdir d0/d1c/d3b/d85/d97 0 2024-03-23T08:23:21.644 INFO:tasks.workunit.client.1.smithi193.stdout:9/421: mknod d2/d7/db/d17/d33/d70/c8a 0 2024-03-23T08:23:21.644 INFO:tasks.workunit.client.1.smithi193.stdout:2/453: mknod d0/d28/d33/c87 0 2024-03-23T08:23:21.645 INFO:tasks.workunit.client.1.smithi193.stdout:1/395: rmdir da/dd/df/d52 39 2024-03-23T08:23:21.646 INFO:tasks.workunit.client.1.smithi193.stdout:5/418: write d0/f3e [3897289,96864] 0 2024-03-23T08:23:21.650 INFO:tasks.workunit.client.1.smithi193.stdout:1/396: read da/dd/f14 [747184,28614] 0 2024-03-23T08:23:21.651 INFO:tasks.workunit.client.1.smithi193.stdout:5/419: creat d0/d1c/d3b/f98 x:0 0 0 2024-03-23T08:23:21.651 INFO:tasks.workunit.client.1.smithi193.stdout:9/422: mkdir d2/d7/db/d17/d4b/d8b 0 2024-03-23T08:23:21.653 INFO:tasks.workunit.client.1.smithi193.stdout:4/443: dwrite d11/d27/d77/f7e [0,4194304] 0 2024-03-23T08:23:21.655 INFO:tasks.workunit.client.1.smithi193.stdout:5/420: write d0/d1c/d3b/f98 [835569,13265] 0 2024-03-23T08:23:21.655 INFO:tasks.workunit.client.1.smithi193.stdout:9/423: dread - d2/d7/db/d7a/f87 zero size 2024-03-23T08:23:21.664 INFO:tasks.workunit.client.1.smithi193.stdout:1/397: creat da/dd/df/d52/d82/f83 x:0 0 0 2024-03-23T08:23:21.664 INFO:tasks.workunit.client.1.smithi193.stdout:2/454: dwrite d0/d1b/d19/f5c [0,4194304] 0 2024-03-23T08:23:21.664 INFO:tasks.workunit.client.1.smithi193.stdout:9/424: chown d2/d7/db/d17/d1f/c66 182 1 2024-03-23T08:23:21.664 INFO:tasks.workunit.client.1.smithi193.stdout:5/421: mkdir d0/d2f/d72/d99 0 2024-03-23T08:23:21.667 INFO:tasks.workunit.client.1.smithi193.stdout:1/398: creat da/dd/df/d7e/f84 x:0 0 0 2024-03-23T08:23:21.669 INFO:tasks.workunit.client.1.smithi193.stdout:5/422: symlink d0/d1c/d7a/l9a 0 2024-03-23T08:23:21.670 INFO:tasks.workunit.client.1.smithi193.stdout:6/328: dread d0/fe [0,4194304] 0 2024-03-23T08:23:21.671 INFO:tasks.workunit.client.1.smithi193.stdout:1/399: read da/dd/f62 [1158257,47721] 0 2024-03-23T08:23:21.671 INFO:tasks.workunit.client.1.smithi193.stdout:2/455: rename d0/d37/d48/f5e to d0/d1b/d14/d56/d7c/d7e/f88 0 2024-03-23T08:23:21.671 INFO:tasks.workunit.client.1.smithi193.stdout:4/444: dwrite d11/d1a/d33/f39 [4194304,4194304] 0 2024-03-23T08:23:21.672 INFO:tasks.workunit.client.1.smithi193.stdout:5/423: read d0/fa [4873216,88249] 0 2024-03-23T08:23:21.674 INFO:tasks.workunit.client.1.smithi193.stdout:5/424: readlink d0/l3f 0 2024-03-23T08:23:21.674 INFO:tasks.workunit.client.1.smithi193.stdout:2/456: read - d0/d1b/f74 zero size 2024-03-23T08:23:21.676 INFO:tasks.workunit.client.1.smithi193.stdout:5/425: write d0/d1f/f22 [796124,3526] 0 2024-03-23T08:23:21.678 INFO:tasks.workunit.client.1.smithi193.stdout:5/426: chown d0/f1d 19 1 2024-03-23T08:23:21.678 INFO:tasks.workunit.client.1.smithi193.stdout:6/329: unlink d0/db/df/d11/c47 0 2024-03-23T08:23:21.679 INFO:tasks.workunit.client.1.smithi193.stdout:4/445: mkdir d11/d19/d88/d99 0 2024-03-23T08:23:21.679 INFO:tasks.workunit.client.1.smithi193.stdout:2/457: creat d0/d1b/d14/d25/f89 x:0 0 0 2024-03-23T08:23:21.681 INFO:tasks.workunit.client.1.smithi193.stdout:6/330: mknod d0/d24/c5f 0 2024-03-23T08:23:21.683 INFO:tasks.workunit.client.1.smithi193.stdout:4/446: creat d11/d95/f9a x:0 0 0 2024-03-23T08:23:21.684 INFO:tasks.workunit.client.1.smithi193.stdout:6/331: stat d0/d8/l22 0 2024-03-23T08:23:21.685 INFO:tasks.workunit.client.1.smithi193.stdout:4/447: chown d11/f8c 87464 1 2024-03-23T08:23:21.688 INFO:tasks.workunit.client.1.smithi193.stdout:4/448: rmdir d11/d19/d88 39 2024-03-23T08:23:21.689 INFO:tasks.workunit.client.1.smithi193.stdout:2/458: dread d0/f2d [4194304,4194304] 0 2024-03-23T08:23:21.691 INFO:tasks.workunit.client.1.smithi193.stdout:6/332: link d0/db/df/d11/f37 d0/d8/d12/f60 0 2024-03-23T08:23:21.694 INFO:tasks.workunit.client.1.smithi193.stdout:9/425: dwrite d2/d7/db/d32/f3f [0,4194304] 0 2024-03-23T08:23:21.695 INFO:tasks.workunit.client.1.smithi193.stdout:1/400: dread da/f59 [0,4194304] 0 2024-03-23T08:23:21.696 INFO:tasks.workunit.client.1.smithi193.stdout:6/333: unlink d0/d8/f19 0 2024-03-23T08:23:21.696 INFO:tasks.workunit.client.1.smithi193.stdout:3/361: sync 2024-03-23T08:23:21.696 INFO:tasks.workunit.client.1.smithi193.stdout:0/389: sync 2024-03-23T08:23:21.696 INFO:tasks.workunit.client.1.smithi193.stdout:8/363: sync 2024-03-23T08:23:21.697 INFO:tasks.workunit.client.1.smithi193.stdout:4/449: mknod d11/d2d/d58/d80/c9b 0 2024-03-23T08:23:21.698 INFO:tasks.workunit.client.1.smithi193.stdout:3/362: write d1/f51 [290385,19916] 0 2024-03-23T08:23:21.699 INFO:tasks.workunit.client.1.smithi193.stdout:8/364: write df/f33 [351949,92159] 0 2024-03-23T08:23:21.699 INFO:tasks.workunit.client.1.smithi193.stdout:5/427: dread d0/d2f/d72/d73/d45/f96 [4194304,4194304] 0 2024-03-23T08:23:21.699 INFO:tasks.workunit.client.1.smithi193.stdout:2/459: rename d0/d28/f82 to d0/d37/f8a 0 2024-03-23T08:23:21.699 INFO:tasks.workunit.client.1.smithi193.stdout:6/334: chown d0/d8/d12/f1b 11 1 2024-03-23T08:23:21.701 INFO:tasks.workunit.client.1.smithi193.stdout:0/390: chown db/d11/d14/d24/d2a/d4d/f66 470333 1 2024-03-23T08:23:21.702 INFO:tasks.workunit.client.1.smithi193.stdout:5/428: fsync d0/d1f/f51 0 2024-03-23T08:23:21.702 INFO:tasks.workunit.client.1.smithi193.stdout:2/460: dread - d0/d37/d48/f4a zero size 2024-03-23T08:23:21.702 INFO:tasks.workunit.client.1.smithi193.stdout:9/426: unlink d2/d7/f56 0 2024-03-23T08:23:21.703 INFO:tasks.workunit.client.1.smithi193.stdout:5/429: readlink d0/d1c/l2c 0 2024-03-23T08:23:21.703 INFO:tasks.workunit.client.1.smithi193.stdout:2/461: stat d0/d37/d48/c6e 0 2024-03-23T08:23:21.705 INFO:tasks.workunit.client.1.smithi193.stdout:8/365: fsync f3 0 2024-03-23T08:23:21.707 INFO:tasks.workunit.client.1.smithi193.stdout:1/401: rename da/dd/l55 to da/dd/df/d2e/l85 0 2024-03-23T08:23:21.708 INFO:tasks.workunit.client.1.smithi193.stdout:0/391: symlink db/d11/d14/d21/d33/l8b 0 2024-03-23T08:23:21.712 INFO:tasks.workunit.client.1.smithi193.stdout:9/427: dread d2/d7/db/d32/f3f [0,4194304] 0 2024-03-23T08:23:21.713 INFO:tasks.workunit.client.1.smithi193.stdout:4/450: dwrite d11/d2d/f4b [4194304,4194304] 0 2024-03-23T08:23:21.714 INFO:tasks.workunit.client.1.smithi193.stdout:8/366: mkdir df/d10/d67/d7f 0 2024-03-23T08:23:21.714 INFO:tasks.workunit.client.1.smithi193.stdout:9/428: fsync d2/d7/db/d17/d33/f5d 0 2024-03-23T08:23:21.717 INFO:tasks.workunit.client.1.smithi193.stdout:6/335: rename d0/d8 to d0/db/df/d11/d36/d61 0 2024-03-23T08:23:21.720 INFO:tasks.workunit.client.1.smithi193.stdout:2/462: mknod d0/d1b/d77/c8b 0 2024-03-23T08:23:21.722 INFO:tasks.workunit.client.1.smithi193.stdout:9/429: dwrite d2/d7/db/d1b/d43/f74 [0,4194304] 0 2024-03-23T08:23:21.724 INFO:tasks.workunit.client.1.smithi193.stdout:4/451: symlink d11/d1a/d33/d42/d52/d70/l9c 0 2024-03-23T08:23:21.725 INFO:tasks.workunit.client.1.smithi193.stdout:8/367: rmdir df/d10/d3e/d4e 39 2024-03-23T08:23:21.729 INFO:tasks.workunit.client.1.smithi193.stdout:4/452: dread - d11/d1a/d33/d42/f71 zero size 2024-03-23T08:23:21.729 INFO:tasks.workunit.client.1.smithi193.stdout:8/368: mknod df/d10/d67/c80 0 2024-03-23T08:23:21.731 INFO:tasks.workunit.client.1.smithi193.stdout:1/402: getdents da/dd 0 2024-03-23T08:23:21.731 INFO:tasks.workunit.client.1.smithi193.stdout:9/430: getdents d2/d7/db/d17/d4b/d59/d75 0 2024-03-23T08:23:21.731 INFO:tasks.workunit.client.1.smithi193.stdout:4/453: mkdir d11/d2d/d58/d80/d9d 0 2024-03-23T08:23:21.731 INFO:tasks.workunit.client.1.smithi193.stdout:8/369: truncate df/f33 1570708 0 2024-03-23T08:23:21.732 INFO:tasks.workunit.client.1.smithi193.stdout:2/463: creat d0/d28/f8c x:0 0 0 2024-03-23T08:23:21.734 INFO:tasks.workunit.client.1.smithi193.stdout:1/403: write da/d13/d3f/f54 [418165,120681] 0 2024-03-23T08:23:21.734 INFO:tasks.workunit.client.1.smithi193.stdout:8/370: creat df/d63/f81 x:0 0 0 2024-03-23T08:23:21.735 INFO:tasks.workunit.client.1.smithi193.stdout:5/430: truncate d0/d1c/f21 1975331 0 2024-03-23T08:23:21.736 INFO:tasks.workunit.client.1.smithi193.stdout:8/371: readlink df/d10/l57 0 2024-03-23T08:23:21.737 INFO:tasks.workunit.client.1.smithi193.stdout:4/454: rename d11/l3e to d11/d2d/d58/l9e 0 2024-03-23T08:23:21.738 INFO:tasks.workunit.client.1.smithi193.stdout:8/372: write df/d4b/d4d/f5c [784860,6443] 0 2024-03-23T08:23:21.740 INFO:tasks.workunit.client.1.smithi193.stdout:6/336: creat d0/db/df/d11/d36/d61/d12/f62 x:0 0 0 2024-03-23T08:23:21.740 INFO:tasks.workunit.client.1.smithi193.stdout:1/404: mkdir da/dd/df/d7e/d86 0 2024-03-23T08:23:21.741 INFO:tasks.workunit.client.1.smithi193.stdout:2/464: rename d0/d1b/d14/c26 to d0/d1b/d14/d25/c8d 0 2024-03-23T08:23:21.742 INFO:tasks.workunit.client.1.smithi193.stdout:1/405: fdatasync f5 0 2024-03-23T08:23:21.742 INFO:tasks.workunit.client.1.smithi193.stdout:8/373: truncate df/d10/d19/f26 2179249 0 2024-03-23T08:23:21.742 INFO:tasks.workunit.client.1.smithi193.stdout:4/455: mkdir d11/d2d/d58/d80/d9f 0 2024-03-23T08:23:21.743 INFO:tasks.workunit.client.1.smithi193.stdout:6/337: write d0/db/df/f4f [773454,73931] 0 2024-03-23T08:23:21.744 INFO:tasks.workunit.client.1.smithi193.stdout:5/431: dread d0/d1c/d3b/f3c [0,4194304] 0 2024-03-23T08:23:21.748 INFO:tasks.workunit.client.1.smithi193.stdout:8/374: write df/d10/f2f [950013,82900] 0 2024-03-23T08:23:21.748 INFO:tasks.workunit.client.1.smithi193.stdout:5/432: dread - d0/d1c/f5c zero size 2024-03-23T08:23:21.754 INFO:tasks.workunit.client.1.smithi193.stdout:6/338: fsync d0/db/df/d11/d36/d61/d12/d21/f29 0 2024-03-23T08:23:21.754 INFO:tasks.workunit.client.1.smithi193.stdout:0/392: dwrite db/d11/d14/d24/d47/f74 [0,4194304] 0 2024-03-23T08:23:21.757 INFO:tasks.workunit.client.1.smithi193.stdout:1/406: write f1 [2983786,45103] 0 2024-03-23T08:23:21.758 INFO:tasks.workunit.client.1.smithi193.stdout:2/465: rename d0/d1b/f74 to d0/d1b/d14/d66/f8e 0 2024-03-23T08:23:21.759 INFO:tasks.workunit.client.1.smithi193.stdout:9/431: dread d2/d7/db/f13 [0,4194304] 0 2024-03-23T08:23:21.760 INFO:tasks.workunit.client.1.smithi193.stdout:4/456: dwrite d11/d48/d50/d49/f5c [4194304,4194304] 0 2024-03-23T08:23:21.760 INFO:tasks.workunit.client.1.smithi193.stdout:0/393: chown db/d11/d14/d24 230 1 2024-03-23T08:23:21.763 INFO:tasks.workunit.client.1.smithi193.stdout:7/371: sync 2024-03-23T08:23:21.764 INFO:tasks.workunit.client.1.smithi193.stdout:5/433: rename d0/d1c/f1e to d0/d2f/d72/d99/f9b 0 2024-03-23T08:23:21.765 INFO:tasks.workunit.client.1.smithi193.stdout:1/407: truncate da/dd/f14 1679020 0 2024-03-23T08:23:21.765 INFO:tasks.workunit.client.1.smithi193.stdout:9/432: symlink d2/d7/db/d32/l8c 0 2024-03-23T08:23:21.766 INFO:tasks.workunit.client.1.smithi193.stdout:9/433: fsync d2/d7/f36 0 2024-03-23T08:23:21.767 INFO:tasks.workunit.client.1.smithi193.stdout:5/434: chown d0/d1c/d3b/d81/c90 0 1 2024-03-23T08:23:21.769 INFO:tasks.workunit.client.1.smithi193.stdout:5/435: chown d0/d2f/d72/d73/d57/c52 4 1 2024-03-23T08:23:21.769 INFO:tasks.workunit.client.1.smithi193.stdout:1/408: creat da/d13/d3f/f87 x:0 0 0 2024-03-23T08:23:21.770 INFO:tasks.workunit.client.1.smithi193.stdout:5/436: dread - d0/d1c/d7a/f7d zero size 2024-03-23T08:23:21.771 INFO:tasks.workunit.client.1.smithi193.stdout:0/394: rename db/d11/d14/d21/d33/d48/d65 to db/d11/d14/d24/d47/d59/d8c 0 2024-03-23T08:23:21.772 INFO:tasks.workunit.client.1.smithi193.stdout:9/434: write d2/d7/db/d32/f3f [4010480,111437] 0 2024-03-23T08:23:21.775 INFO:tasks.workunit.client.1.smithi193.stdout:2/466: mkdir d0/d1b/d14/d66/d50/d8f 0 2024-03-23T08:23:21.775 INFO:tasks.workunit.client.1.smithi193.stdout:6/339: truncate d0/db/f33 2752974 0 2024-03-23T08:23:21.779 INFO:tasks.workunit.client.1.smithi193.stdout:1/409: symlink da/l88 0 2024-03-23T08:23:21.824 INFO:tasks.workunit.client.1.smithi193.stdout:0/395: dread db/d11/d14/f1f [0,4194304] 0 2024-03-23T08:23:21.830 INFO:tasks.workunit.client.1.smithi193.stdout:5/437: truncate d0/d2f/d72/d73/d45/f77 824217 0 2024-03-23T08:23:21.833 INFO:tasks.workunit.client.1.smithi193.stdout:7/372: dread d6/d9/d1f/f30 [0,4194304] 0 2024-03-23T08:23:21.834 INFO:tasks.workunit.client.1.smithi193.stdout:1/410: creat da/dd/df/d7e/d86/f89 x:0 0 0 2024-03-23T08:23:21.836 INFO:tasks.workunit.client.1.smithi193.stdout:6/340: dwrite d0/db/df/d11/d36/d61/d12/f1b [4194304,4194304] 0 2024-03-23T08:23:21.836 INFO:tasks.workunit.client.1.smithi193.stdout:2/467: getdents d0/d1b/d77 0 2024-03-23T08:23:21.837 INFO:tasks.workunit.client.1.smithi193.stdout:0/396: link db/f6f db/d11/d14/d24/f8d 0 2024-03-23T08:23:21.837 INFO:tasks.workunit.client.1.smithi193.stdout:7/373: write d6/d9/d1f/d23/d33/d38/f68 [1315800,128618] 0 2024-03-23T08:23:21.840 INFO:tasks.workunit.client.1.smithi193.stdout:1/411: dread da/d13/d24/f7b [0,4194304] 0 2024-03-23T08:23:21.841 INFO:tasks.workunit.client.1.smithi193.stdout:6/341: mknod d0/db/df/d11/d36/d61/d3a/c63 0 2024-03-23T08:23:21.846 INFO:tasks.workunit.client.1.smithi193.stdout:9/435: dread d2/d7/db/d1b/f30 [0,4194304] 0 2024-03-23T08:23:21.846 INFO:tasks.workunit.client.1.smithi193.stdout:7/374: truncate d6/d9/fd 1627957 0 2024-03-23T08:23:21.846 INFO:tasks.workunit.client.1.smithi193.stdout:1/412: dread da/dd/df/d52/f7a [0,4194304] 0 2024-03-23T08:23:21.846 INFO:tasks.workunit.client.1.smithi193.stdout:6/342: chown d0/db/df/d11/d36/d61/d12/d21/d2a/l3c 5 1 2024-03-23T08:23:21.848 INFO:tasks.workunit.client.1.smithi193.stdout:9/436: chown d2/d7/db/d32/l8c 1 1 2024-03-23T08:23:21.848 INFO:tasks.workunit.client.1.smithi193.stdout:6/343: chown d0/db/df/d11/f37 94047 1 2024-03-23T08:23:21.848 INFO:tasks.workunit.client.1.smithi193.stdout:0/397: dread db/d11/d14/d24/f7d [0,4194304] 0 2024-03-23T08:23:21.849 INFO:tasks.workunit.client.1.smithi193.stdout:6/344: fdatasync d0/db/df/d11/f30 0 2024-03-23T08:23:21.849 INFO:tasks.workunit.client.1.smithi193.stdout:2/468: dread d0/d28/d33/f70 [0,4194304] 0 2024-03-23T08:23:21.850 INFO:tasks.workunit.client.1.smithi193.stdout:1/413: creat da/dd/df/d7e/d7f/f8a x:0 0 0 2024-03-23T08:23:21.852 INFO:tasks.workunit.client.1.smithi193.stdout:2/469: chown d0/d1b/d14/d25/l27 1272 1 2024-03-23T08:23:21.853 INFO:tasks.workunit.client.1.smithi193.stdout:6/345: mknod d0/db/df/d11/d36/d61/d12/d21/c64 0 2024-03-23T08:23:21.853 INFO:tasks.workunit.client.1.smithi193.stdout:3/363: sync 2024-03-23T08:23:21.857 INFO:tasks.workunit.client.1.smithi193.stdout:0/398: truncate db/d11/d14/d24/f5f 1577897 0 2024-03-23T08:23:21.858 INFO:tasks.workunit.client.1.smithi193.stdout:3/364: dread d1/de/d13/f62 [0,4194304] 0 2024-03-23T08:23:21.858 INFO:tasks.workunit.client.1.smithi193.stdout:6/346: mknod d0/db/df/d11/c65 0 2024-03-23T08:23:21.861 INFO:tasks.workunit.client.1.smithi193.stdout:6/347: chown d0/db/f4e 51 1 2024-03-23T08:23:21.861 INFO:tasks.workunit.client.1.smithi193.stdout:2/470: fdatasync d0/f1d 0 2024-03-23T08:23:21.861 INFO:tasks.workunit.client.1.smithi193.stdout:3/365: chown d1/de/d28/c6d 935169 1 2024-03-23T08:23:21.862 INFO:tasks.workunit.client.1.smithi193.stdout:1/414: link da/d13/d3f/f44 da/d13/d3f/d5a/f8b 0 2024-03-23T08:23:21.864 INFO:tasks.workunit.client.1.smithi193.stdout:0/399: dwrite db/d11/d14/d21/f45 [4194304,4194304] 0 2024-03-23T08:23:21.865 INFO:tasks.workunit.client.1.smithi193.stdout:2/471: rmdir d0/d1b/d14/d56 39 2024-03-23T08:23:21.914 INFO:tasks.workunit.client.1.smithi193.stdout:1/415: creat da/d13/d3f/d5a/f8c x:0 0 0 2024-03-23T08:23:21.917 INFO:tasks.workunit.client.1.smithi193.stdout:0/400: symlink db/d11/d14/d21/d33/d69/d88/l8e 0 2024-03-23T08:23:21.917 INFO:tasks.workunit.client.1.smithi193.stdout:1/416: mkdir da/dd/d8d 0 2024-03-23T08:23:21.918 INFO:tasks.workunit.client.1.smithi193.stdout:2/472: creat d0/d1b/d14/d66/d50/d8f/f90 x:0 0 0 2024-03-23T08:23:21.923 INFO:tasks.workunit.client.1.smithi193.stdout:2/473: unlink d0/d1b/d14/d56/d7c/f86 0 2024-03-23T08:23:21.925 INFO:tasks.workunit.client.1.smithi193.stdout:3/366: dread d1/de/f11 [0,4194304] 0 2024-03-23T08:23:21.926 INFO:tasks.workunit.client.1.smithi193.stdout:2/474: unlink d0/d1b/d14/c5d 0 2024-03-23T08:23:21.927 INFO:tasks.workunit.client.1.smithi193.stdout:6/348: dread d0/db/df/d11/d36/d61/d3a/f45 [0,4194304] 0 2024-03-23T08:23:21.928 INFO:tasks.workunit.client.1.smithi193.stdout:3/367: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/d85 0 2024-03-23T08:23:21.930 INFO:tasks.workunit.client.1.smithi193.stdout:6/349: fdatasync d0/f40 0 2024-03-23T08:23:21.931 INFO:tasks.workunit.client.1.smithi193.stdout:1/417: write da/dd/df/f1d [2948061,18511] 0 2024-03-23T08:23:21.932 INFO:tasks.workunit.client.1.smithi193.stdout:3/368: mknod d1/de/d13/d21/d50/d44/c86 0 2024-03-23T08:23:21.935 INFO:tasks.workunit.client.1.smithi193.stdout:0/401: dread db/d11/d14/d24/d2a/f2e [0,4194304] 0 2024-03-23T08:23:21.936 INFO:tasks.workunit.client.1.smithi193.stdout:1/418: dread - da/dd/df/d7e/f84 zero size 2024-03-23T08:23:21.938 INFO:tasks.workunit.client.1.smithi193.stdout:2/475: dwrite d0/d37/d48/f4a [0,4194304] 0 2024-03-23T08:23:21.939 INFO:tasks.workunit.client.1.smithi193.stdout:3/369: creat d1/de/d13/d21/d27/d7c/f87 x:0 0 0 2024-03-23T08:23:21.940 INFO:tasks.workunit.client.1.smithi193.stdout:0/402: creat db/f8f x:0 0 0 2024-03-23T08:23:21.942 INFO:tasks.workunit.client.1.smithi193.stdout:6/350: dwrite d0/db/f20 [0,4194304] 0 2024-03-23T08:23:21.944 INFO:tasks.workunit.client.1.smithi193.stdout:6/351: fdatasync d0/db/df/d11/d36/f44 0 2024-03-23T08:23:21.945 INFO:tasks.workunit.client.1.smithi193.stdout:2/476: creat d0/d1b/d14/d66/d50/f91 x:0 0 0 2024-03-23T08:23:21.948 INFO:tasks.workunit.client.1.smithi193.stdout:2/477: stat d0/c78 0 2024-03-23T08:23:21.948 INFO:tasks.workunit.client.1.smithi193.stdout:6/352: creat d0/db/df/d11/d3b/f66 x:0 0 0 2024-03-23T08:23:21.951 INFO:tasks.workunit.client.1.smithi193.stdout:0/403: dread db/d11/d14/d24/d2a/f2e [0,4194304] 0 2024-03-23T08:23:21.952 INFO:tasks.workunit.client.1.smithi193.stdout:8/375: sync 2024-03-23T08:23:21.953 INFO:tasks.workunit.client.1.smithi193.stdout:2/478: rmdir d0/d1b/d14/d66/d7b 0 2024-03-23T08:23:21.953 INFO:tasks.workunit.client.1.smithi193.stdout:6/353: write d0/db/df/d11/d36/d61/d3a/f53 [72945,44199] 0 2024-03-23T08:23:21.956 INFO:tasks.workunit.client.1.smithi193.stdout:0/404: write db/d11/d14/d24/d47/d59/f79 [6434438,23210] 0 2024-03-23T08:23:21.956 INFO:tasks.workunit.client.1.smithi193.stdout:2/479: chown d0/d1b/d14/d66/d50/d60 250 1 2024-03-23T08:23:21.957 INFO:tasks.workunit.client.1.smithi193.stdout:3/370: dwrite d1/de/d13/d21/d50/d44/f79 [0,4194304] 0 2024-03-23T08:23:21.957 INFO:tasks.workunit.client.1.smithi193.stdout:2/480: chown d0/d1b/d14 237 1 2024-03-23T08:23:21.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:21 smithi070 ceph-mon[39699]: pgmap v20: 65 pgs: 65 active+clean; 635 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 3.2 MiB/s rd, 64 MiB/s wr, 263 op/s 2024-03-23T08:23:22.008 INFO:tasks.workunit.client.1.smithi193.stdout:7/375: sync 2024-03-23T08:23:22.008 INFO:tasks.workunit.client.1.smithi193.stdout:4/457: sync 2024-03-23T08:23:22.008 INFO:tasks.workunit.client.1.smithi193.stdout:5/438: sync 2024-03-23T08:23:22.040 INFO:tasks.workunit.client.1.smithi193.stdout:8/376: rmdir df/d10/d67 39 2024-03-23T08:23:22.041 INFO:tasks.workunit.client.1.smithi193.stdout:6/354: mknod d0/c67 0 2024-03-23T08:23:22.041 INFO:tasks.workunit.client.1.smithi193.stdout:8/377: read df/d4b/d4d/f5c [492352,1840] 0 2024-03-23T08:23:22.042 INFO:tasks.workunit.client.1.smithi193.stdout:0/405: mknod db/d11/d14/d21/d33/d48/c90 0 2024-03-23T08:23:22.043 INFO:tasks.workunit.client.1.smithi193.stdout:3/371: creat d1/de/d13/f88 x:0 0 0 2024-03-23T08:23:22.043 INFO:tasks.workunit.client.1.smithi193.stdout:8/378: truncate df/d4b/f71 443231 0 2024-03-23T08:23:22.044 INFO:tasks.workunit.client.1.smithi193.stdout:7/376: fdatasync d6/d9/f6a 0 2024-03-23T08:23:22.046 INFO:tasks.workunit.client.1.smithi193.stdout:0/406: chown db/d11/d14/d21/d33/d48/f5d 940 1 2024-03-23T08:23:22.046 INFO:tasks.workunit.client.1.smithi193.stdout:3/372: stat d1/de/d13/d21/d25/d2b/d3c/d4a/d85 0 2024-03-23T08:23:22.047 INFO:tasks.workunit.client.1.smithi193.stdout:4/458: creat d11/d27/d77/fa0 x:0 0 0 2024-03-23T08:23:22.047 INFO:tasks.workunit.client.1.smithi193.stdout:5/439: symlink d0/d1c/d3b/d81/l9c 0 2024-03-23T08:23:22.049 INFO:tasks.workunit.client.1.smithi193.stdout:6/355: dread d0/db/df/d11/d36/d61/d12/d21/f29 [0,4194304] 0 2024-03-23T08:23:22.050 INFO:tasks.workunit.client.1.smithi193.stdout:2/481: write d0/d1b/f8 [6208241,8722] 0 2024-03-23T08:23:22.052 INFO:tasks.workunit.client.1.smithi193.stdout:6/356: chown d0/db/df/f4f 1164239778 1 2024-03-23T08:23:22.052 INFO:tasks.workunit.client.1.smithi193.stdout:7/377: rmdir d6/d9/d4e 39 2024-03-23T08:23:22.053 INFO:tasks.workunit.client.1.smithi193.stdout:3/373: creat d1/de/f89 x:0 0 0 2024-03-23T08:23:22.054 INFO:tasks.workunit.client.1.smithi193.stdout:8/379: mknod df/c82 0 2024-03-23T08:23:22.056 INFO:tasks.workunit.client.1.smithi193.stdout:8/380: dread - df/d4b/d4d/d2a/f3c zero size 2024-03-23T08:23:22.060 INFO:tasks.workunit.client.1.smithi193.stdout:7/378: read - d6/d9/d1f/d23/d33/f6d zero size 2024-03-23T08:23:22.060 INFO:tasks.workunit.client.1.smithi193.stdout:3/374: symlink d1/de/d28/l8a 0 2024-03-23T08:23:22.060 INFO:tasks.workunit.client.1.smithi193.stdout:2/482: truncate d0/fd 1165464 0 2024-03-23T08:23:22.063 INFO:tasks.workunit.client.1.smithi193.stdout:4/459: truncate d11/d1a/d33/f7d 11180 0 2024-03-23T08:23:22.063 INFO:tasks.workunit.client.1.smithi193.stdout:3/375: rmdir d1/de/d13/d21/d50 39 2024-03-23T08:23:22.065 INFO:tasks.workunit.client.1.smithi193.stdout:2/483: mkdir d0/d1b/d14/d66/d50/d92 0 2024-03-23T08:23:22.066 INFO:tasks.workunit.client.1.smithi193.stdout:7/379: unlink d6/d9/d1f/d23/d33/f34 0 2024-03-23T08:23:22.067 INFO:tasks.workunit.client.1.smithi193.stdout:4/460: mknod d11/d1a/d33/d44/ca1 0 2024-03-23T08:23:22.068 INFO:tasks.workunit.client.1.smithi193.stdout:0/407: dread db/d11/d14/d21/f41 [0,4194304] 0 2024-03-23T08:23:22.071 INFO:tasks.workunit.client.1.smithi193.stdout:3/376: readlink d1/de/d13/d21/d50/l4c 0 2024-03-23T08:23:22.072 INFO:tasks.workunit.client.1.smithi193.stdout:8/381: truncate df/f33 649422 0 2024-03-23T08:23:22.073 INFO:tasks.workunit.client.1.smithi193.stdout:3/377: readlink d1/de/d13/d21/d25/d34/l7d 0 2024-03-23T08:23:22.074 INFO:tasks.workunit.client.1.smithi193.stdout:9/437: sync 2024-03-23T08:23:22.074 INFO:tasks.workunit.client.1.smithi193.stdout:1/419: sync 2024-03-23T08:23:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:21 smithi193 ceph-mon[48370]: pgmap v20: 65 pgs: 65 active+clean; 635 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 3.2 MiB/s rd, 64 MiB/s wr, 263 op/s 2024-03-23T08:23:22.075 INFO:tasks.workunit.client.1.smithi193.stdout:5/440: dread d0/d1f/f78 [0,4194304] 0 2024-03-23T08:23:22.076 INFO:tasks.workunit.client.1.smithi193.stdout:2/484: read d0/d1b/f13 [2256031,82429] 0 2024-03-23T08:23:22.077 INFO:tasks.workunit.client.1.smithi193.stdout:0/408: rename db/d11/c35 to db/d11/d14/d24/d2a/d4b/c91 0 2024-03-23T08:23:22.077 INFO:tasks.workunit.client.1.smithi193.stdout:9/438: chown d2/d7/f1e 88767 1 2024-03-23T08:23:22.077 INFO:tasks.workunit.client.1.smithi193.stdout:7/380: mkdir d6/d9/d1f/d23/d33/d38/d46/d82 0 2024-03-23T08:23:22.079 INFO:tasks.workunit.client.1.smithi193.stdout:8/382: mkdir df/d10/d83 0 2024-03-23T08:23:22.079 INFO:tasks.workunit.client.1.smithi193.stdout:2/485: chown d0/d37/f8a 7592 1 2024-03-23T08:23:22.081 INFO:tasks.workunit.client.1.smithi193.stdout:1/420: dread - da/dd/f64 zero size 2024-03-23T08:23:22.081 INFO:tasks.workunit.client.1.smithi193.stdout:5/441: symlink d0/d5a/l9d 0 2024-03-23T08:23:22.081 INFO:tasks.workunit.client.1.smithi193.stdout:9/439: dread - d2/d7/db/d17/d33/d70/f7b zero size 2024-03-23T08:23:22.082 INFO:tasks.workunit.client.1.smithi193.stdout:6/357: dread d0/db/df/d11/d36/d61/f1d [0,4194304] 0 2024-03-23T08:23:22.082 INFO:tasks.workunit.client.1.smithi193.stdout:4/461: truncate d11/d1a/d33/d42/f4d 1523251 0 2024-03-23T08:23:22.086 INFO:tasks.workunit.client.1.smithi193.stdout:0/409: mknod db/d11/d14/d3b/d71/c92 0 2024-03-23T08:23:22.088 INFO:tasks.workunit.client.1.smithi193.stdout:7/381: rmdir d6/d9/d1f/d23/d33/d38/d46 39 2024-03-23T08:23:22.093 INFO:tasks.workunit.client.1.smithi193.stdout:9/440: chown d2/d7/db/d1b/l29 231817017 1 2024-03-23T08:23:22.094 INFO:tasks.workunit.client.1.smithi193.stdout:2/486: fdatasync d0/d28/f43 0 2024-03-23T08:23:22.095 INFO:tasks.workunit.client.1.smithi193.stdout:5/442: dread d0/d1f/f5b [0,4194304] 0 2024-03-23T08:23:22.096 INFO:tasks.workunit.client.1.smithi193.stdout:1/421: write da/d13/f6f [766573,93261] 0 2024-03-23T08:23:22.098 INFO:tasks.workunit.client.1.smithi193.stdout:7/382: creat d6/d9/d1f/d3e/d65/d70/f83 x:0 0 0 2024-03-23T08:23:22.100 INFO:tasks.workunit.client.1.smithi193.stdout:4/462: creat d11/d2d/d58/d80/d9f/fa2 x:0 0 0 2024-03-23T08:23:22.101 INFO:tasks.workunit.client.1.smithi193.stdout:8/383: dwrite df/d63/f81 [0,4194304] 0 2024-03-23T08:23:22.102 INFO:tasks.workunit.client.1.smithi193.stdout:0/410: unlink db/d11/d14/d21/d33/d69/d88/l8e 0 2024-03-23T08:23:22.106 INFO:tasks.workunit.client.1.smithi193.stdout:9/441: write d2/d7/db/d17/d1f/f42 [586134,31631] 0 2024-03-23T08:23:22.108 INFO:tasks.workunit.client.1.smithi193.stdout:4/463: read - d11/d95/f9a zero size 2024-03-23T08:23:22.109 INFO:tasks.workunit.client.1.smithi193.stdout:5/443: creat d0/d1c/d3b/d81/f9e x:0 0 0 2024-03-23T08:23:22.110 INFO:tasks.workunit.client.1.smithi193.stdout:2/487: creat d0/d1b/d14/d66/d50/f93 x:0 0 0 2024-03-23T08:23:22.110 INFO:tasks.workunit.client.1.smithi193.stdout:6/358: dwrite d0/db/df/d11/d36/f44 [0,4194304] 0 2024-03-23T08:23:22.112 INFO:tasks.workunit.client.1.smithi193.stdout:4/464: chown d11/d19/f2f 852 1 2024-03-23T08:23:22.112 INFO:tasks.workunit.client.1.smithi193.stdout:8/384: rmdir df/d46 39 2024-03-23T08:23:22.114 INFO:tasks.workunit.client.1.smithi193.stdout:7/383: mknod d6/d9/d1f/c84 0 2024-03-23T08:23:22.117 INFO:tasks.workunit.client.1.smithi193.stdout:2/488: stat d0/d28/l36 0 2024-03-23T08:23:22.119 INFO:tasks.workunit.client.1.smithi193.stdout:3/378: dread d1/fc [0,4194304] 0 2024-03-23T08:23:22.120 INFO:tasks.workunit.client.1.smithi193.stdout:4/465: creat d11/d8d/fa3 x:0 0 0 2024-03-23T08:23:22.122 INFO:tasks.workunit.client.1.smithi193.stdout:6/359: write d0/db/df/d11/d36/f41 [917587,100397] 0 2024-03-23T08:23:22.124 INFO:tasks.workunit.client.1.smithi193.stdout:9/442: getdents d2/d7/db/d32/d6b 0 2024-03-23T08:23:22.124 INFO:tasks.workunit.client.1.smithi193.stdout:8/385: getdents df/d10/d83 0 2024-03-23T08:23:22.125 INFO:tasks.workunit.client.1.smithi193.stdout:5/444: mkdir d0/d1c/d7a/d86/d9f 0 2024-03-23T08:23:22.125 INFO:tasks.workunit.client.1.smithi193.stdout:0/411: dread db/d11/d14/d24/d2a/f2e [0,4194304] 0 2024-03-23T08:23:22.126 INFO:tasks.workunit.client.1.smithi193.stdout:2/489: mknod d0/d1b/d14/d66/d4e/c94 0 2024-03-23T08:23:22.126 INFO:tasks.workunit.client.1.smithi193.stdout:9/443: stat d2/d7/db/d1b/d5c 0 2024-03-23T08:23:22.127 INFO:tasks.workunit.client.1.smithi193.stdout:4/466: write d11/d2d/d3a/f76 [1715632,127182] 0 2024-03-23T08:23:22.128 INFO:tasks.workunit.client.1.smithi193.stdout:0/412: stat db/d11/f12 0 2024-03-23T08:23:22.128 INFO:tasks.workunit.client.1.smithi193.stdout:8/386: write df/d4b/d4d/f1e [3675927,18182] 0 2024-03-23T08:23:22.131 INFO:tasks.workunit.client.1.smithi193.stdout:1/422: dread da/d13/d24/f43 [0,4194304] 0 2024-03-23T08:23:22.133 INFO:tasks.workunit.client.1.smithi193.stdout:8/387: write df/f18 [1041740,52818] 0 2024-03-23T08:23:22.133 INFO:tasks.workunit.client.1.smithi193.stdout:2/490: truncate d0/d1b/d14/d25/f89 763202 0 2024-03-23T08:23:22.135 INFO:tasks.workunit.client.1.smithi193.stdout:7/384: write d6/d9/d1f/f30 [6464556,61295] 0 2024-03-23T08:23:22.135 INFO:tasks.workunit.client.1.smithi193.stdout:1/423: fsync da/dd/df/d7e/f84 0 2024-03-23T08:23:22.137 INFO:tasks.workunit.client.1.smithi193.stdout:0/413: chown db/d11/d14/d21/c57 2506 1 2024-03-23T08:23:22.138 INFO:tasks.workunit.client.1.smithi193.stdout:3/379: getdents d1/de/d13/d21/d50/d44 0 2024-03-23T08:23:22.138 INFO:tasks.workunit.client.1.smithi193.stdout:7/385: dread - d6/d9/f6f zero size 2024-03-23T08:23:22.138 INFO:tasks.workunit.client.1.smithi193.stdout:6/360: dwrite d0/db/df/d11/d36/f41 [0,4194304] 0 2024-03-23T08:23:22.139 INFO:tasks.workunit.client.1.smithi193.stdout:8/388: mkdir df/d63/d84 0 2024-03-23T08:23:22.140 INFO:tasks.workunit.client.1.smithi193.stdout:9/444: truncate d2/d7/f1d 2989942 0 2024-03-23T08:23:22.140 INFO:tasks.workunit.client.1.smithi193.stdout:5/445: creat d0/d2f/d72/d73/d45/fa0 x:0 0 0 2024-03-23T08:23:22.141 INFO:tasks.workunit.client.1.smithi193.stdout:3/380: dread - d1/de/f70 zero size 2024-03-23T08:23:22.142 INFO:tasks.workunit.client.1.smithi193.stdout:6/361: dread - d0/db/df/d11/d3b/f66 zero size 2024-03-23T08:23:22.143 INFO:tasks.workunit.client.1.smithi193.stdout:2/491: write d0/d1b/f13 [4427000,62043] 0 2024-03-23T08:23:22.144 INFO:tasks.workunit.client.1.smithi193.stdout:4/467: truncate d11/d1a/d33/d42/f59 3264338 0 2024-03-23T08:23:22.144 INFO:tasks.workunit.client.1.smithi193.stdout:7/386: write d6/d9/fa [941289,91765] 0 2024-03-23T08:23:22.144 INFO:tasks.workunit.client.1.smithi193.stdout:1/424: mkdir da/d13/d3f/d5a/d8e 0 2024-03-23T08:23:22.147 INFO:tasks.workunit.client.1.smithi193.stdout:0/414: truncate db/d11/d14/d24/d2a/d4d/f66 630146 0 2024-03-23T08:23:22.149 INFO:tasks.workunit.client.1.smithi193.stdout:8/389: creat df/d10/d3e/d54/f85 x:0 0 0 2024-03-23T08:23:22.149 INFO:tasks.workunit.client.1.smithi193.stdout:5/446: write d0/d2f/d72/d73/d45/f77 [1058587,45888] 0 2024-03-23T08:23:22.149 INFO:tasks.workunit.client.1.smithi193.stdout:3/381: dread - d1/de/d13/d21/d25/d2b/f4d zero size 2024-03-23T08:23:22.151 INFO:tasks.workunit.client.1.smithi193.stdout:4/468: unlink d11/d19/c23 0 2024-03-23T08:23:22.152 INFO:tasks.workunit.client.1.smithi193.stdout:6/362: creat d0/db/df/d11/d36/f68 x:0 0 0 2024-03-23T08:23:22.153 INFO:tasks.workunit.client.1.smithi193.stdout:1/425: write da/dd/df/f17 [1602828,39305] 0 2024-03-23T08:23:22.156 INFO:tasks.workunit.client.1.smithi193.stdout:0/415: mkdir db/d11/d14/d24/d2a/d4d/d93 0 2024-03-23T08:23:22.157 INFO:tasks.workunit.client.1.smithi193.stdout:5/447: mknod d0/d1c/d3b/d85/ca1 0 2024-03-23T08:23:22.159 INFO:tasks.workunit.client.1.smithi193.stdout:7/387: truncate d6/d9/d1f/f2a 2273343 0 2024-03-23T08:23:22.159 INFO:tasks.workunit.client.1.smithi193.stdout:6/363: readlink d0/db/df/d11/d36/d61/l22 0 2024-03-23T08:23:22.161 INFO:tasks.workunit.client.1.smithi193.stdout:1/426: write da/d13/f34 [2978139,96928] 0 2024-03-23T08:23:22.161 INFO:tasks.workunit.client.1.smithi193.stdout:4/469: write d11/d1a/d33/d44/f54 [948696,120360] 0 2024-03-23T08:23:22.161 INFO:tasks.workunit.client.1.smithi193.stdout:3/382: mknod d1/de/d13/d21/d25/d34/c8b 0 2024-03-23T08:23:22.164 INFO:tasks.workunit.client.1.smithi193.stdout:2/492: rename d0/d1b/d3b/f42 to d0/d1b/d14/f95 0 2024-03-23T08:23:22.164 INFO:tasks.workunit.client.1.smithi193.stdout:9/445: dwrite d2/d7/db/d17/f54 [0,4194304] 0 2024-03-23T08:23:22.167 INFO:tasks.workunit.client.1.smithi193.stdout:3/383: readlink d1/de/d13/d21/d25/d34/l7d 0 2024-03-23T08:23:22.169 INFO:tasks.workunit.client.1.smithi193.stdout:2/493: dread - d0/d1b/d3b/f79 zero size 2024-03-23T08:23:22.170 INFO:tasks.workunit.client.1.smithi193.stdout:5/448: mkdir d0/d2f/d72/d73/d45/da2 0 2024-03-23T08:23:22.171 INFO:tasks.workunit.client.1.smithi193.stdout:0/416: truncate db/d11/f28 359616 0 2024-03-23T08:23:22.172 INFO:tasks.workunit.client.1.smithi193.stdout:3/384: write d1/f51 [347083,61238] 0 2024-03-23T08:23:22.172 INFO:tasks.workunit.client.1.smithi193.stdout:7/388: mknod d6/d9/d1f/d23/d33/c85 0 2024-03-23T08:23:22.175 INFO:tasks.workunit.client.1.smithi193.stdout:4/470: mknod d11/d19/d84/ca4 0 2024-03-23T08:23:22.176 INFO:tasks.workunit.client.1.smithi193.stdout:2/494: write d0/f2d [2231597,39767] 0 2024-03-23T08:23:22.176 INFO:tasks.workunit.client.1.smithi193.stdout:4/471: chown d11 12092 1 2024-03-23T08:23:22.177 INFO:tasks.workunit.client.1.smithi193.stdout:5/449: creat d0/d1f/d93/fa3 x:0 0 0 2024-03-23T08:23:22.179 INFO:tasks.workunit.client.1.smithi193.stdout:1/427: rename da/d13/d3f/f56 to da/d13/d3f/d5a/d76/f8f 0 2024-03-23T08:23:22.180 INFO:tasks.workunit.client.1.smithi193.stdout:8/390: dwrite df/f33 [0,4194304] 0 2024-03-23T08:23:22.180 INFO:tasks.workunit.client.1.smithi193.stdout:0/417: fsync db/d11/d14/d21/d33/f43 0 2024-03-23T08:23:22.181 INFO:tasks.workunit.client.1.smithi193.stdout:4/472: chown d11/d48/d50 1416 1 2024-03-23T08:23:22.183 INFO:tasks.workunit.client.1.smithi193.stdout:7/389: symlink d6/d9/d1f/d23/d33/d3c/l86 0 2024-03-23T08:23:22.186 INFO:tasks.workunit.client.1.smithi193.stdout:9/446: dwrite d2/f7d [0,4194304] 0 2024-03-23T08:23:22.186 INFO:tasks.workunit.client.1.smithi193.stdout:3/385: unlink d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/l69 0 2024-03-23T08:23:22.187 INFO:tasks.workunit.client.1.smithi193.stdout:5/450: chown d0/d1c/d3b/d4b/d55/f63 1 1 2024-03-23T08:23:22.192 INFO:tasks.workunit.client.1.smithi193.stdout:2/495: creat d0/d1b/d14/d56/f96 x:0 0 0 2024-03-23T08:23:22.192 INFO:tasks.workunit.client.1.smithi193.stdout:4/473: write d11/d48/d50/f61 [196699,72705] 0 2024-03-23T08:23:22.192 INFO:tasks.workunit.client.1.smithi193.stdout:8/391: chown df/d4b/d4d/d2a/l58 0 1 2024-03-23T08:23:22.193 INFO:tasks.workunit.client.1.smithi193.stdout:0/418: symlink db/l94 0 2024-03-23T08:23:22.196 INFO:tasks.workunit.client.1.smithi193.stdout:0/419: dread - db/d11/d14/d3b/d71/f7f zero size 2024-03-23T08:23:22.196 INFO:tasks.workunit.client.1.smithi193.stdout:9/447: mkdir d2/d7/db/d17/d4b/d8d 0 2024-03-23T08:23:22.197 INFO:tasks.workunit.client.1.smithi193.stdout:8/392: readlink df/d46/l5e 0 2024-03-23T08:23:22.197 INFO:tasks.workunit.client.1.smithi193.stdout:7/390: dwrite d6/f14 [0,4194304] 0 2024-03-23T08:23:22.199 INFO:tasks.workunit.client.1.smithi193.stdout:9/448: write d2/d7/db/d32/f67 [1070312,112123] 0 2024-03-23T08:23:22.200 INFO:tasks.workunit.client.1.smithi193.stdout:5/451: symlink d0/d1c/d7a/d86/d9f/la4 0 2024-03-23T08:23:22.203 INFO:tasks.workunit.client.1.smithi193.stdout:1/428: creat da/d13/f90 x:0 0 0 2024-03-23T08:23:22.204 INFO:tasks.workunit.client.1.smithi193.stdout:0/420: rmdir db/d52 39 2024-03-23T08:23:22.205 INFO:tasks.workunit.client.1.smithi193.stdout:2/496: creat d0/d28/d33/d72/f97 x:0 0 0 2024-03-23T08:23:22.207 INFO:tasks.workunit.client.1.smithi193.stdout:1/429: write da/dd/df/d7e/d86/f89 [524644,97506] 0 2024-03-23T08:23:22.207 INFO:tasks.workunit.client.1.smithi193.stdout:4/474: link d11/d8d/fa3 d11/d48/d50/fa5 0 2024-03-23T08:23:22.209 INFO:tasks.workunit.client.1.smithi193.stdout:2/497: readlink d0/d1b/lf 0 2024-03-23T08:23:22.211 INFO:tasks.workunit.client.1.smithi193.stdout:0/421: creat db/d11/d14/f95 x:0 0 0 2024-03-23T08:23:22.212 INFO:tasks.workunit.client.1.smithi193.stdout:8/393: creat df/f86 x:0 0 0 2024-03-23T08:23:22.212 INFO:tasks.workunit.client.1.smithi193.stdout:9/449: creat d2/d7/f8e x:0 0 0 2024-03-23T08:23:22.213 INFO:tasks.workunit.client.1.smithi193.stdout:5/452: rename d0/d1c/d3b/d4b/d55/f63 to d0/d2f/d72/d73/d57/d6d/d8e/fa5 0 2024-03-23T08:23:22.213 INFO:tasks.workunit.client.1.smithi193.stdout:4/475: symlink d11/d2d/d58/d80/la6 0 2024-03-23T08:23:22.216 INFO:tasks.workunit.client.1.smithi193.stdout:1/430: rmdir da/dd/df/d7e 39 2024-03-23T08:23:22.217 INFO:tasks.workunit.client.1.smithi193.stdout:0/422: mkdir db/d11/d14/d24/d2a/d96 0 2024-03-23T08:23:22.218 INFO:tasks.workunit.client.1.smithi193.stdout:9/450: readlink d2/d7/db/d17/l49 0 2024-03-23T08:23:22.219 INFO:tasks.workunit.client.1.smithi193.stdout:7/391: dwrite d6/d9/d1f/d23/d33/d38/d3b/f55 [0,4194304] 0 2024-03-23T08:23:22.220 INFO:tasks.workunit.client.1.smithi193.stdout:5/453: dread d0/d2f/d72/d73/d57/d6d/f87 [0,4194304] 0 2024-03-23T08:23:22.222 INFO:tasks.workunit.client.1.smithi193.stdout:8/394: creat df/d4b/d4d/f87 x:0 0 0 2024-03-23T08:23:22.226 INFO:tasks.workunit.client.1.smithi193.stdout:1/431: creat da/d13/d3f/d5a/d8e/f91 x:0 0 0 2024-03-23T08:23:22.229 INFO:tasks.workunit.client.1.smithi193.stdout:2/498: link d0/d28/d33/c87 d0/d28/d33/d72/c98 0 2024-03-23T08:23:22.230 INFO:tasks.workunit.client.1.smithi193.stdout:9/451: creat d2/f8f x:0 0 0 2024-03-23T08:23:22.231 INFO:tasks.workunit.client.1.smithi193.stdout:0/423: creat db/d11/d14/d24/f97 x:0 0 0 2024-03-23T08:23:22.233 INFO:tasks.workunit.client.1.smithi193.stdout:9/452: stat d2/d7/d31/d5f/d73 0 2024-03-23T08:23:22.234 INFO:tasks.workunit.client.1.smithi193.stdout:0/424: write db/f87 [711490,115378] 0 2024-03-23T08:23:22.234 INFO:tasks.workunit.client.1.smithi193.stdout:3/386: dwrite d1/de/d13/d21/d25/d2b/f39 [0,4194304] 0 2024-03-23T08:23:22.235 INFO:tasks.workunit.client.1.smithi193.stdout:8/395: dwrite df/f33 [0,4194304] 0 2024-03-23T08:23:22.240 INFO:tasks.workunit.client.1.smithi193.stdout:7/392: write d6/d9/f6a [1261514,78423] 0 2024-03-23T08:23:22.241 INFO:tasks.workunit.client.1.smithi193.stdout:4/476: dread d11/f83 [0,4194304] 0 2024-03-23T08:23:22.241 INFO:tasks.workunit.client.1.smithi193.stdout:3/387: readlink d1/de/d13/d21/d25/l42 0 2024-03-23T08:23:22.242 INFO:tasks.workunit.client.1.smithi193.stdout:0/425: mkdir db/d11/d14/d21/d33/d69/d88/d98 0 2024-03-23T08:23:22.242 INFO:tasks.workunit.client.1.smithi193.stdout:7/393: chown d6/d9/d1f/d23/d33/d38 2897308 1 2024-03-23T08:23:22.244 INFO:tasks.workunit.client.1.smithi193.stdout:8/396: creat df/d10/d3e/d4e/f88 x:0 0 0 2024-03-23T08:23:22.245 INFO:tasks.workunit.client.1.smithi193.stdout:7/394: write d6/d9/d1f/d23/d33/f6d [78837,34781] 0 2024-03-23T08:23:22.246 INFO:tasks.workunit.client.1.smithi193.stdout:8/397: dread - df/d10/d19/f72 zero size 2024-03-23T08:23:22.246 INFO:tasks.workunit.client.1.smithi193.stdout:0/426: creat db/d11/d14/d51/f99 x:0 0 0 2024-03-23T08:23:22.247 INFO:tasks.workunit.client.1.smithi193.stdout:9/453: link d2/d7/db/d1b/d43/d71/c7f d2/d7/d31/d3c/d79/c90 0 2024-03-23T08:23:22.248 INFO:tasks.workunit.client.1.smithi193.stdout:7/395: symlink d6/d9/d1f/d4d/d6b/l87 0 2024-03-23T08:23:22.249 INFO:tasks.workunit.client.1.smithi193.stdout:3/388: getdents d1/de/d13/d21/d25/d2b/d3c/d4a/d85 0 2024-03-23T08:23:22.250 INFO:tasks.workunit.client.1.smithi193.stdout:8/398: write df/d10/d3e/d4e/f5b [92491,36885] 0 2024-03-23T08:23:22.250 INFO:tasks.workunit.client.1.smithi193.stdout:0/427: mknod db/d11/d14/d21/d33/d69/c9a 0 2024-03-23T08:23:22.251 INFO:tasks.workunit.client.1.smithi193.stdout:9/454: creat d2/d7/d31/d3c/f91 x:0 0 0 2024-03-23T08:23:22.251 INFO:tasks.workunit.client.1.smithi193.stdout:7/396: dread - d6/d9/d1f/d3e/d65/d70/f83 zero size 2024-03-23T08:23:22.252 INFO:tasks.workunit.client.1.smithi193.stdout:3/389: write d1/de/d13/f24 [4506218,9256] 0 2024-03-23T08:23:22.253 INFO:tasks.workunit.client.1.smithi193.stdout:0/428: fdatasync db/d11/d14/d24/d47/d59/f72 0 2024-03-23T08:23:22.254 INFO:tasks.workunit.client.1.smithi193.stdout:9/455: mknod d2/d7/db/d1b/d43/d71/c92 0 2024-03-23T08:23:22.256 INFO:tasks.workunit.client.1.smithi193.stdout:0/429: unlink db/d11/d14/d21/f45 0 2024-03-23T08:23:22.257 INFO:tasks.workunit.client.1.smithi193.stdout:9/456: write d2/d7/f11 [4555238,90623] 0 2024-03-23T08:23:22.257 INFO:tasks.workunit.client.1.smithi193.stdout:3/390: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/d85/l8c 0 2024-03-23T08:23:22.259 INFO:tasks.workunit.client.1.smithi193.stdout:0/430: read - db/d52/f5c zero size 2024-03-23T08:23:22.261 INFO:tasks.workunit.client.1.smithi193.stdout:9/457: creat d2/d7/db/d32/d6b/f93 x:0 0 0 2024-03-23T08:23:22.262 INFO:tasks.workunit.client.1.smithi193.stdout:3/391: write d1/de/d13/d21/f82 [330621,6522] 0 2024-03-23T08:23:22.263 INFO:tasks.workunit.client.1.smithi193.stdout:9/458: stat d2/d7/d31/c4a 0 2024-03-23T08:23:22.263 INFO:tasks.workunit.client.1.smithi193.stdout:7/397: dwrite d6/d9/d1f/d3e/d42/f71 [0,4194304] 0 2024-03-23T08:23:22.264 INFO:tasks.workunit.client.1.smithi193.stdout:3/392: write d1/f2f [565357,43262] 0 2024-03-23T08:23:22.266 INFO:tasks.workunit.client.1.smithi193.stdout:7/398: chown d6/d9/d4e/d81 2 1 2024-03-23T08:23:22.266 INFO:tasks.workunit.client.1.smithi193.stdout:9/459: read d2/d7/db/d17/d33/f45 [3920286,107171] 0 2024-03-23T08:23:22.267 INFO:tasks.workunit.client.1.smithi193.stdout:3/393: symlink d1/de/d13/d21/d25/d2b/d3c/l8d 0 2024-03-23T08:23:22.269 INFO:tasks.workunit.client.1.smithi193.stdout:9/460: write d2/f25 [437409,76066] 0 2024-03-23T08:23:22.272 INFO:tasks.workunit.client.1.smithi193.stdout:8/399: dread f3 [0,4194304] 0 2024-03-23T08:23:22.361 INFO:tasks.workunit.client.1.smithi193.stdout:6/364: sync 2024-03-23T08:23:22.382 INFO:tasks.workunit.client.1.smithi193.stdout:7/399: mkdir d6/d9/d1f/d3e/d88 0 2024-03-23T08:23:22.383 INFO:tasks.workunit.client.1.smithi193.stdout:9/461: write d2/d7/d31/d5f/f83 [1719613,44037] 0 2024-03-23T08:23:22.384 INFO:tasks.workunit.client.1.smithi193.stdout:8/400: unlink df/d10/f27 0 2024-03-23T08:23:22.397 INFO:tasks.workunit.client.1.smithi193.stdout:5/454: truncate d0/d2f/d72/d73/d45/f96 5398999 0 2024-03-23T08:23:22.398 INFO:tasks.workunit.client.1.smithi193.stdout:2/499: write d0/fd [1770146,52617] 0 2024-03-23T08:23:22.400 INFO:tasks.workunit.client.1.smithi193.stdout:9/462: creat d2/d7/db/d1b/d5c/f94 x:0 0 0 2024-03-23T08:23:22.401 INFO:tasks.workunit.client.1.smithi193.stdout:6/365: mkdir d0/d24/d69 0 2024-03-23T08:23:22.402 INFO:tasks.workunit.client.1.smithi193.stdout:8/401: creat df/d10/d3e/d54/f89 x:0 0 0 2024-03-23T08:23:22.402 INFO:tasks.workunit.client.1.smithi193.stdout:3/394: link d1/de/d13/c66 d1/de/d28/c8e 0 2024-03-23T08:23:22.405 INFO:tasks.workunit.client.1.smithi193.stdout:5/455: symlink d0/d5a/la6 0 2024-03-23T08:23:22.405 INFO:tasks.workunit.client.1.smithi193.stdout:8/402: write df/d10/d3e/f6b [256184,111859] 0 2024-03-23T08:23:22.406 INFO:tasks.workunit.client.1.smithi193.stdout:9/463: mkdir d2/d7/db/d7a/d95 0 2024-03-23T08:23:22.409 INFO:tasks.workunit.client.1.smithi193.stdout:9/464: write d2/f7d [3362757,90320] 0 2024-03-23T08:23:22.412 INFO:tasks.workunit.client.1.smithi193.stdout:6/366: dwrite d0/db/f4e [0,4194304] 0 2024-03-23T08:23:22.412 INFO:tasks.workunit.client.1.smithi193.stdout:9/465: chown d2/d7/db/f3b 5 1 2024-03-23T08:23:22.414 INFO:tasks.workunit.client.1.smithi193.stdout:6/367: dread - d0/db/df/d11/d3b/f49 zero size 2024-03-23T08:23:22.417 INFO:tasks.workunit.client.1.smithi193.stdout:2/500: dread d0/fc [0,4194304] 0 2024-03-23T08:23:22.421 INFO:tasks.workunit.client.1.smithi193.stdout:9/466: dread d2/d7/d31/d5f/f83 [0,4194304] 0 2024-03-23T08:23:22.422 INFO:tasks.workunit.client.1.smithi193.stdout:9/467: write d2/d7/f36 [7072121,18113] 0 2024-03-23T08:23:22.441 INFO:tasks.workunit.client.1.smithi193.stdout:4/477: sync 2024-03-23T08:23:22.441 INFO:tasks.workunit.client.1.smithi193.stdout:1/432: sync 2024-03-23T08:23:22.443 INFO:tasks.workunit.client.1.smithi193.stdout:7/400: rmdir d6/d9/d1f/d3e/d88 0 2024-03-23T08:23:22.447 INFO:tasks.workunit.client.1.smithi193.stdout:0/431: truncate db/d11/d14/d21/f26 1416903 0 2024-03-23T08:23:22.453 INFO:tasks.workunit.client.1.smithi193.stdout:2/501: chown d0/l11 50269 1 2024-03-23T08:23:22.453 INFO:tasks.workunit.client.1.smithi193.stdout:9/468: symlink d2/d7/db/d1b/l96 0 2024-03-23T08:23:22.454 INFO:tasks.workunit.client.1.smithi193.stdout:7/401: dwrite d6/d9/d1f/d3e/d65/f75 [0,4194304] 0 2024-03-23T08:23:22.456 INFO:tasks.workunit.client.1.smithi193.stdout:9/469: stat d2/d7/db/d17/d1f/l27 0 2024-03-23T08:23:22.457 INFO:tasks.workunit.client.1.smithi193.stdout:1/433: symlink da/dd/df/d52/d82/l92 0 2024-03-23T08:23:22.459 INFO:tasks.workunit.client.1.smithi193.stdout:8/403: creat df/d63/d84/f8a x:0 0 0 2024-03-23T08:23:22.460 INFO:tasks.workunit.client.1.smithi193.stdout:9/470: fdatasync d2/d7/db/f13 0 2024-03-23T08:23:22.462 INFO:tasks.workunit.client.1.smithi193.stdout:8/404: write df/d10/d3e/d4e/f5b [198211,58077] 0 2024-03-23T08:23:22.464 INFO:tasks.workunit.client.1.smithi193.stdout:8/405: dread - df/d4b/d4d/f87 zero size 2024-03-23T08:23:22.464 INFO:tasks.workunit.client.1.smithi193.stdout:2/502: rmdir d0/d1b/d14/d66/d50 39 2024-03-23T08:23:22.469 INFO:tasks.workunit.client.1.smithi193.stdout:4/478: symlink d11/d48/la7 0 2024-03-23T08:23:22.470 INFO:tasks.workunit.client.1.smithi193.stdout:7/402: creat d6/d9/d1f/d23/d33/d38/d3b/f89 x:0 0 0 2024-03-23T08:23:22.473 INFO:tasks.workunit.client.1.smithi193.stdout:3/395: getdents d1/de/d13/d21/d25/d2b 0 2024-03-23T08:23:22.474 INFO:tasks.workunit.client.1.smithi193.stdout:5/456: link d0/fe d0/d2f/d72/d73/d57/d6d/d8e/fa7 0 2024-03-23T08:23:22.474 INFO:tasks.workunit.client.1.smithi193.stdout:2/503: mkdir d0/d1b/d14/d56/d99 0 2024-03-23T08:23:22.477 INFO:tasks.workunit.client.1.smithi193.stdout:7/403: creat d6/d9/d1f/d5f/f8a x:0 0 0 2024-03-23T08:23:22.477 INFO:tasks.workunit.client.1.smithi193.stdout:0/432: dwrite db/d11/d14/d21/d33/f68 [0,4194304] 0 2024-03-23T08:23:22.482 INFO:tasks.workunit.client.1.smithi193.stdout:5/457: chown d0/l46 1772404500 1 2024-03-23T08:23:22.485 INFO:tasks.workunit.client.1.smithi193.stdout:2/504: dread d0/d1b/d14/f45 [0,4194304] 0 2024-03-23T08:23:22.489 INFO:tasks.workunit.client.1.smithi193.stdout:3/396: creat d1/de/d13/d21/d25/d34/f8f x:0 0 0 2024-03-23T08:23:22.490 INFO:tasks.workunit.client.1.smithi193.stdout:1/434: dwrite da/dd/df/d7e/d7f/f8a [0,4194304] 0 2024-03-23T08:23:22.497 INFO:tasks.workunit.client.1.smithi193.stdout:1/435: fsync da/dd/f64 0 2024-03-23T08:23:22.497 INFO:tasks.workunit.client.1.smithi193.stdout:0/433: mkdir db/d11/d14/d21/d33/d69/d9b 0 2024-03-23T08:23:22.497 INFO:tasks.workunit.client.1.smithi193.stdout:5/458: creat d0/d1c/d7a/d86/d9f/fa8 x:0 0 0 2024-03-23T08:23:22.502 INFO:tasks.workunit.client.1.smithi193.stdout:0/434: dread - db/f6f zero size 2024-03-23T08:23:22.502 INFO:tasks.workunit.client.1.smithi193.stdout:7/404: dread d6/d9/d1f/d3e/d42/f60 [0,4194304] 0 2024-03-23T08:23:22.505 INFO:tasks.workunit.client.1.smithi193.stdout:2/505: creat d0/d1b/d14/d66/d50/d92/f9a x:0 0 0 2024-03-23T08:23:22.505 INFO:tasks.workunit.client.1.smithi193.stdout:4/479: dwrite d11/d1a/d33/d42/d52/f6c [0,4194304] 0 2024-03-23T08:23:22.505 INFO:tasks.workunit.client.1.smithi193.stdout:1/436: creat da/dd/df/d2e/f93 x:0 0 0 2024-03-23T08:23:22.507 INFO:tasks.workunit.client.1.smithi193.stdout:5/459: symlink d0/d1f/la9 0 2024-03-23T08:23:22.508 INFO:tasks.workunit.client.1.smithi193.stdout:5/460: write d0/d1f/f49 [1235464,20163] 0 2024-03-23T08:23:22.509 INFO:tasks.workunit.client.1.smithi193.stdout:5/461: chown d0/d1f/d93 7822254 1 2024-03-23T08:23:22.510 INFO:tasks.workunit.client.1.smithi193.stdout:4/480: mkdir d11/d2d/da8 0 2024-03-23T08:23:22.513 INFO:tasks.workunit.client.1.smithi193.stdout:1/437: getdents da/dd/d8d 0 2024-03-23T08:23:22.520 INFO:tasks.workunit.client.1.smithi193.stdout:1/438: creat da/d13/f94 x:0 0 0 2024-03-23T08:23:22.524 INFO:tasks.workunit.client.1.smithi193.stdout:1/439: mkdir da/dd/df/d7e/d86/d95 0 2024-03-23T08:23:22.527 INFO:tasks.workunit.client.1.smithi193.stdout:5/462: dread d0/d2f/d72/d99/f9b [0,4194304] 0 2024-03-23T08:23:22.529 INFO:tasks.workunit.client.1.smithi193.stdout:5/463: truncate d0/d1c/d3b/d4b/f84 781790 0 2024-03-23T08:23:22.533 INFO:tasks.workunit.client.1.smithi193.stdout:5/464: dread d0/f43 [0,4194304] 0 2024-03-23T08:23:22.536 INFO:tasks.workunit.client.1.smithi193.stdout:5/465: read d0/d1f/f41 [2366251,22187] 0 2024-03-23T08:23:22.536 INFO:tasks.workunit.client.1.smithi193.stdout:4/481: mkdir d11/da9 0 2024-03-23T08:23:22.538 INFO:tasks.workunit.client.1.smithi193.stdout:1/440: dread da/d13/d24/f58 [0,4194304] 0 2024-03-23T08:23:22.538 INFO:tasks.workunit.client.1.smithi193.stdout:3/397: dwrite d1/f1e [4194304,4194304] 0 2024-03-23T08:23:22.539 INFO:tasks.workunit.client.1.smithi193.stdout:6/368: unlink d0/db/c32 0 2024-03-23T08:23:22.540 INFO:tasks.workunit.client.1.smithi193.stdout:5/466: write d0/d1c/f20 [17430,121129] 0 2024-03-23T08:23:22.544 INFO:tasks.workunit.client.1.smithi193.stdout:0/435: mkdir db/d11/d14/d9c 0 2024-03-23T08:23:22.545 INFO:tasks.workunit.client.1.smithi193.stdout:6/369: unlink d0/db/df/d11/d36/d61/d12/f62 0 2024-03-23T08:23:22.545 INFO:tasks.workunit.client.1.smithi193.stdout:1/441: read da/d13/f34 [2713961,119053] 0 2024-03-23T08:23:22.547 INFO:tasks.workunit.client.1.smithi193.stdout:0/436: write db/d11/d14/d24/d2a/d4d/f50 [1034120,12059] 0 2024-03-23T08:23:22.547 INFO:tasks.workunit.client.1.smithi193.stdout:8/406: rmdir df/d10 39 2024-03-23T08:23:22.548 INFO:tasks.workunit.client.1.smithi193.stdout:2/506: write d0/d37/f4c [3916947,60382] 0 2024-03-23T08:23:22.548 INFO:tasks.workunit.client.1.smithi193.stdout:6/370: truncate d0/db/df/f4f 1589560 0 2024-03-23T08:23:22.549 INFO:tasks.workunit.client.1.smithi193.stdout:0/437: stat db/cc 0 2024-03-23T08:23:22.551 INFO:tasks.workunit.client.1.smithi193.stdout:5/467: dwrite d0/f2a [0,4194304] 0 2024-03-23T08:23:22.556 INFO:tasks.workunit.client.1.smithi193.stdout:6/371: symlink d0/db/df/l6a 0 2024-03-23T08:23:22.556 INFO:tasks.workunit.client.1.smithi193.stdout:1/442: link da/dd/df/d2e/l72 da/d13/d3f/d5a/l96 0 2024-03-23T08:23:22.558 INFO:tasks.workunit.client.1.smithi193.stdout:1/443: write da/d13/d3f/f87 [375928,52019] 0 2024-03-23T08:23:22.563 INFO:tasks.workunit.client.1.smithi193.stdout:5/468: dwrite d0/d2f/d72/d73/d45/f77 [0,4194304] 0 2024-03-23T08:23:22.564 INFO:tasks.workunit.client.1.smithi193.stdout:0/438: rmdir db/d11/d14/d21/d33/d69/d88/d98 0 2024-03-23T08:23:22.565 INFO:tasks.workunit.client.1.smithi193.stdout:5/469: write d0/d2f/d72/d73/d57/d6d/f7c [186102,68183] 0 2024-03-23T08:23:22.565 INFO:tasks.workunit.client.1.smithi193.stdout:1/444: fdatasync da/dd/df/f29 0 2024-03-23T08:23:22.568 INFO:tasks.workunit.client.1.smithi193.stdout:0/439: truncate db/d11/d14/d51/f99 489864 0 2024-03-23T08:23:22.570 INFO:tasks.workunit.client.1.smithi193.stdout:8/407: truncate df/d4b/d4d/d2a/f2b 2737918 0 2024-03-23T08:23:22.570 INFO:tasks.workunit.client.1.smithi193.stdout:0/440: write db/d11/d14/f1f [1505579,41549] 0 2024-03-23T08:23:22.571 INFO:tasks.workunit.client.1.smithi193.stdout:6/372: getdents d0/db 0 2024-03-23T08:23:22.571 INFO:tasks.workunit.client.1.smithi193.stdout:5/470: mkdir d0/d2f/d72/d73/d45/daa 0 2024-03-23T08:23:22.576 INFO:tasks.workunit.client.1.smithi193.stdout:2/507: dwrite d0/f71 [0,4194304] 0 2024-03-23T08:23:22.577 INFO:tasks.workunit.client.1.smithi193.stdout:0/441: mknod db/d11/d14/d3b/c9d 0 2024-03-23T08:23:22.580 INFO:tasks.workunit.client.1.smithi193.stdout:2/508: rmdir d0/d1b/d19 39 2024-03-23T08:23:22.580 INFO:tasks.workunit.client.1.smithi193.stdout:2/509: readlink d0/d28/d33/l5f 0 2024-03-23T08:23:22.581 INFO:tasks.workunit.client.1.smithi193.stdout:0/442: mknod db/d11/d14/d21/c9e 0 2024-03-23T08:23:22.582 INFO:tasks.workunit.client.1.smithi193.stdout:5/471: dread d0/f15 [0,4194304] 0 2024-03-23T08:23:22.583 INFO:tasks.workunit.client.1.smithi193.stdout:1/445: write da/f59 [4060658,6831] 0 2024-03-23T08:23:22.587 INFO:tasks.workunit.client.1.smithi193.stdout:0/443: symlink db/d11/l9f 0 2024-03-23T08:23:22.588 INFO:tasks.workunit.client.1.smithi193.stdout:8/408: truncate df/f48 22841 0 2024-03-23T08:23:22.589 INFO:tasks.workunit.client.1.smithi193.stdout:2/510: mkdir d0/d1b/d77/d9b 0 2024-03-23T08:23:22.590 INFO:tasks.workunit.client.1.smithi193.stdout:6/373: write d0/db/df/d11/d36/d61/d12/f60 [1039353,129804] 0 2024-03-23T08:23:22.591 INFO:tasks.workunit.client.1.smithi193.stdout:5/472: truncate d0/d1f/f5b 672182 0 2024-03-23T08:23:22.595 INFO:tasks.workunit.client.1.smithi193.stdout:0/444: write db/d11/d14/d21/f41 [1694229,16032] 0 2024-03-23T08:23:22.595 INFO:tasks.workunit.client.1.smithi193.stdout:2/511: write d0/d28/d33/f70 [347299,37475] 0 2024-03-23T08:23:22.597 INFO:tasks.workunit.client.1.smithi193.stdout:1/446: dwrite da/d13/d3f/d5a/f67 [0,4194304] 0 2024-03-23T08:23:22.597 INFO:tasks.workunit.client.1.smithi193.stdout:5/473: dread - d0/d1c/f5c zero size 2024-03-23T08:23:22.601 INFO:tasks.workunit.client.1.smithi193.stdout:2/512: fsync d0/d1b/d3b/f58 0 2024-03-23T08:23:22.602 INFO:tasks.workunit.client.1.smithi193.stdout:0/445: readlink db/d11/d14/d24/l86 0 2024-03-23T08:23:22.607 INFO:tasks.workunit.client.1.smithi193.stdout:8/409: getdents df/d10/d19 0 2024-03-23T08:23:22.609 INFO:tasks.workunit.client.1.smithi193.stdout:0/446: rmdir db/d11/d14/d51 39 2024-03-23T08:23:22.610 INFO:tasks.workunit.client.1.smithi193.stdout:5/474: rename d0/d2f/d72/d73/d45/f74 to d0/d2f/d72/d73/d57/d6d/d8e/fab 0 2024-03-23T08:23:22.612 INFO:tasks.workunit.client.1.smithi193.stdout:8/410: mkdir df/d62/d8b 0 2024-03-23T08:23:22.616 INFO:tasks.workunit.client.1.smithi193.stdout:1/447: dwrite da/dd/df/d2e/f93 [0,4194304] 0 2024-03-23T08:23:22.620 INFO:tasks.workunit.client.1.smithi193.stdout:5/475: symlink d0/d2f/d72/d73/d45/d6e/lac 0 2024-03-23T08:23:22.620 INFO:tasks.workunit.client.1.smithi193.stdout:6/374: dread d0/db/df/f25 [0,4194304] 0 2024-03-23T08:23:22.621 INFO:tasks.workunit.client.1.smithi193.stdout:2/513: dwrite d0/f71 [4194304,4194304] 0 2024-03-23T08:23:22.623 INFO:tasks.workunit.client.1.smithi193.stdout:1/448: chown da/d13/d3f/d5a/d76 0 1 2024-03-23T08:23:22.624 INFO:tasks.workunit.client.1.smithi193.stdout:9/471: sync 2024-03-23T08:23:22.630 INFO:tasks.workunit.client.1.smithi193.stdout:2/514: write d0/d28/d33/d72/f97 [737686,99391] 0 2024-03-23T08:23:22.630 INFO:tasks.workunit.client.1.smithi193.stdout:5/476: creat d0/d1c/d3b/d85/d97/fad x:0 0 0 2024-03-23T08:23:22.632 INFO:tasks.workunit.client.1.smithi193.stdout:1/449: creat da/d13/d24/f97 x:0 0 0 2024-03-23T08:23:22.634 INFO:tasks.workunit.client.1.smithi193.stdout:2/515: read d0/fc [328578,59648] 0 2024-03-23T08:23:22.634 INFO:tasks.workunit.client.1.smithi193.stdout:6/375: link d0/db/f4e d0/db/df/d11/d36/d61/d3a/f6b 0 2024-03-23T08:23:22.634 INFO:tasks.workunit.client.1.smithi193.stdout:8/411: dwrite df/d4b/d4d/f87 [0,4194304] 0 2024-03-23T08:23:22.634 INFO:tasks.workunit.client.1.smithi193.stdout:9/472: mkdir d2/d7/db/d17/d4b/d59/d97 0 2024-03-23T08:23:22.635 INFO:tasks.workunit.client.1.smithi193.stdout:1/450: dread - da/d13/f94 zero size 2024-03-23T08:23:22.635 INFO:tasks.workunit.client.1.smithi193.stdout:0/447: dwrite db/d11/d14/d3b/d71/f7f [0,4194304] 0 2024-03-23T08:23:22.637 INFO:tasks.workunit.client.1.smithi193.stdout:8/412: readlink df/d10/l57 0 2024-03-23T08:23:22.639 INFO:tasks.workunit.client.1.smithi193.stdout:6/376: mkdir d0/db/df/d11/d3b/d6c 0 2024-03-23T08:23:22.641 INFO:tasks.workunit.client.1.smithi193.stdout:2/516: read - d0/d1b/d14/d66/d50/d8f/f90 zero size 2024-03-23T08:23:22.644 INFO:tasks.workunit.client.1.smithi193.stdout:8/413: truncate df/d4b/d4d/f25 994110 0 2024-03-23T08:23:22.645 INFO:tasks.workunit.client.1.smithi193.stdout:0/448: dread - db/d11/d14/d24/f25 zero size 2024-03-23T08:23:22.646 INFO:tasks.workunit.client.1.smithi193.stdout:1/451: mkdir da/dd/df/d75/d98 0 2024-03-23T08:23:22.646 INFO:tasks.workunit.client.1.smithi193.stdout:9/473: dread d2/d7/d31/d5f/f83 [0,4194304] 0 2024-03-23T08:23:22.649 INFO:tasks.workunit.client.1.smithi193.stdout:0/449: symlink db/d11/d14/d24/d2a/la0 0 2024-03-23T08:23:22.650 INFO:tasks.workunit.client.1.smithi193.stdout:9/474: read d2/f28 [3319704,24139] 0 2024-03-23T08:23:22.651 INFO:tasks.workunit.client.1.smithi193.stdout:6/377: creat d0/db/df/d11/d36/d61/f6d x:0 0 0 2024-03-23T08:23:22.656 INFO:tasks.workunit.client.1.smithi193.stdout:6/378: creat d0/db/df/d11/d36/f6e x:0 0 0 2024-03-23T08:23:22.656 INFO:tasks.workunit.client.1.smithi193.stdout:8/414: dwrite df/d4b/d4d/d2a/f3c [0,4194304] 0 2024-03-23T08:23:22.657 INFO:tasks.workunit.client.1.smithi193.stdout:6/379: read - d0/db/df/d11/d3b/f49 zero size 2024-03-23T08:23:22.657 INFO:tasks.workunit.client.1.smithi193.stdout:5/477: dread d0/d2f/d72/d73/f48 [0,4194304] 0 2024-03-23T08:23:22.658 INFO:tasks.workunit.client.1.smithi193.stdout:0/450: getdents db/d11/d14/d21/d33/d48 0 2024-03-23T08:23:22.659 INFO:tasks.workunit.client.1.smithi193.stdout:9/475: link d2/d7/db/d1b/f4c d2/f98 0 2024-03-23T08:23:22.660 INFO:tasks.workunit.client.1.smithi193.stdout:8/415: creat df/d10/d19/f8c x:0 0 0 2024-03-23T08:23:22.661 INFO:tasks.workunit.client.1.smithi193.stdout:5/478: chown d0/d1c/d3b/f3c 3885 1 2024-03-23T08:23:22.662 INFO:tasks.workunit.client.1.smithi193.stdout:9/476: stat d2/d7/db/d17/d4b/d59 0 2024-03-23T08:23:22.662 INFO:tasks.workunit.client.1.smithi193.stdout:0/451: chown db/d11/d14/d24/d47/d59/l6e 1989 1 2024-03-23T08:23:22.663 INFO:tasks.workunit.client.1.smithi193.stdout:0/452: write db/d11/d14/d24/f39 [3890345,21434] 0 2024-03-23T08:23:22.664 INFO:tasks.workunit.client.1.smithi193.stdout:9/477: write d2/d7/db/d17/d33/f63 [672057,21124] 0 2024-03-23T08:23:22.665 INFO:tasks.workunit.client.1.smithi193.stdout:0/453: fdatasync db/d11/d14/d24/d2a/d4d/f66 0 2024-03-23T08:23:22.667 INFO:tasks.workunit.client.1.smithi193.stdout:9/478: symlink d2/d7/db/d17/d1f/l99 0 2024-03-23T08:23:22.670 INFO:tasks.workunit.client.1.smithi193.stdout:6/380: dread d0/db/f33 [0,4194304] 0 2024-03-23T08:23:22.671 INFO:tasks.workunit.client.1.smithi193.stdout:9/479: dread d2/d7/db/d1b/d43/f84 [0,4194304] 0 2024-03-23T08:23:22.674 INFO:tasks.workunit.client.1.smithi193.stdout:9/480: unlink d2/d7/db/d17/d33/c5e 0 2024-03-23T08:23:22.675 INFO:tasks.workunit.client.1.smithi193.stdout:9/481: rmdir d2/d7/d31/d3c/d79 39 2024-03-23T08:23:22.675 INFO:tasks.workunit.client.1.smithi193.stdout:6/381: link d0/db/df/d11/d36/d61/l50 d0/db/df/d11/d36/l6f 0 2024-03-23T08:23:22.677 INFO:tasks.workunit.client.1.smithi193.stdout:0/454: dread db/d11/d14/d24/f39 [0,4194304] 0 2024-03-23T08:23:22.679 INFO:tasks.workunit.client.1.smithi193.stdout:9/482: rename d2/d7/db/d1b/d5c/d77 to d2/d7/db/d1b/d43/d9a 0 2024-03-23T08:23:22.679 INFO:tasks.workunit.client.1.smithi193.stdout:0/455: creat db/d11/d14/d24/d2a/d4d/fa1 x:0 0 0 2024-03-23T08:23:22.681 INFO:tasks.workunit.client.1.smithi193.stdout:6/382: rename d0/db/df/l6a to d0/db/df/d11/d36/l70 0 2024-03-23T08:23:22.681 INFO:tasks.workunit.client.1.smithi193.stdout:0/456: write db/d11/d14/d24/d47/f74 [1983124,122727] 0 2024-03-23T08:23:22.682 INFO:tasks.workunit.client.1.smithi193.stdout:9/483: creat d2/d7/d31/d5f/d73/d78/f9b x:0 0 0 2024-03-23T08:23:22.683 INFO:tasks.workunit.client.1.smithi193.stdout:0/457: readlink db/l2f 0 2024-03-23T08:23:22.684 INFO:tasks.workunit.client.1.smithi193.stdout:6/383: symlink d0/db/df/d11/d56/l71 0 2024-03-23T08:23:22.686 INFO:tasks.workunit.client.1.smithi193.stdout:9/484: mkdir d2/d7/d31/d5f/d9c 0 2024-03-23T08:23:22.688 INFO:tasks.workunit.client.1.smithi193.stdout:6/384: write d0/db/df/d11/d56/f5c [965116,49468] 0 2024-03-23T08:23:22.689 INFO:tasks.workunit.client.1.smithi193.stdout:0/458: getdents db/d11/d14/d21 0 2024-03-23T08:23:22.691 INFO:tasks.workunit.client.1.smithi193.stdout:2/517: truncate d0/d37/d48/f4a 550945 0 2024-03-23T08:23:22.692 INFO:tasks.workunit.client.1.smithi193.stdout:9/485: truncate d2/d7/d31/d5f/d73/d78/f9b 224403 0 2024-03-23T08:23:22.695 INFO:tasks.workunit.client.1.smithi193.stdout:9/486: unlink d2/d7/db/f13 0 2024-03-23T08:23:22.696 INFO:tasks.workunit.client.1.smithi193.stdout:2/518: dread d0/fc [0,4194304] 0 2024-03-23T08:23:22.698 INFO:tasks.workunit.client.1.smithi193.stdout:8/416: getdents df/d10/d19 0 2024-03-23T08:23:22.704 INFO:tasks.workunit.client.1.smithi193.stdout:2/519: rename d0/d1b/c9 to d0/d1b/c9c 0 2024-03-23T08:23:22.705 INFO:tasks.workunit.client.1.smithi193.stdout:6/385: dread d0/db/df/d11/d36/d61/d12/f1b [0,4194304] 0 2024-03-23T08:23:22.705 INFO:tasks.workunit.client.1.smithi193.stdout:5/479: dwrite d0/d1c/d3b/f94 [0,4194304] 0 2024-03-23T08:23:22.708 INFO:tasks.workunit.client.1.smithi193.stdout:8/417: link df/d10/c75 df/d63/d76/c8d 0 2024-03-23T08:23:22.709 INFO:tasks.workunit.client.1.smithi193.stdout:6/386: fsync d0/db/df/d11/f30 0 2024-03-23T08:23:22.714 INFO:tasks.workunit.client.1.smithi193.stdout:5/480: write d0/d2f/d72/d73/d57/d6d/d8e/fa5 [184083,34905] 0 2024-03-23T08:23:22.715 INFO:tasks.workunit.client.1.smithi193.stdout:6/387: mkdir d0/d27/d72 0 2024-03-23T08:23:22.721 INFO:tasks.workunit.client.1.smithi193.stdout:8/418: dwrite df/d10/f3f [0,4194304] 0 2024-03-23T08:23:22.721 INFO:tasks.workunit.client.1.smithi193.stdout:2/520: truncate d0/d1b/d14/f41 3660421 0 2024-03-23T08:23:22.721 INFO:tasks.workunit.client.1.smithi193.stdout:9/487: dread d2/d7/f11 [0,4194304] 0 2024-03-23T08:23:22.723 INFO:tasks.workunit.client.1.smithi193.stdout:6/388: creat d0/db/df/d11/d36/d61/d12/d21/f73 x:0 0 0 2024-03-23T08:23:22.723 INFO:tasks.workunit.client.1.smithi193.stdout:2/521: truncate d0/fc 5185036 0 2024-03-23T08:23:22.729 INFO:tasks.workunit.client.1.smithi193.stdout:2/522: symlink d0/d1b/d77/l9d 0 2024-03-23T08:23:22.729 INFO:tasks.workunit.client.1.smithi193.stdout:6/389: chown d0/db/df/d11/d36/d61/d3a/f45 90 1 2024-03-23T08:23:22.733 INFO:tasks.workunit.client.1.smithi193.stdout:5/481: dwrite d0/fe [0,4194304] 0 2024-03-23T08:23:22.733 INFO:tasks.workunit.client.1.smithi193.stdout:2/523: write d0/d28/f8c [496796,62643] 0 2024-03-23T08:23:22.734 INFO:tasks.workunit.client.1.smithi193.stdout:6/390: symlink d0/db/df/d11/d36/l74 0 2024-03-23T08:23:22.735 INFO:tasks.workunit.client.1.smithi193.stdout:9/488: dwrite d2/d7/f37 [0,4194304] 0 2024-03-23T08:23:22.738 INFO:tasks.workunit.client.1.smithi193.stdout:5/482: symlink d0/d2f/d72/d99/lae 0 2024-03-23T08:23:22.742 INFO:tasks.workunit.client.1.smithi193.stdout:7/405: sync 2024-03-23T08:23:22.744 INFO:tasks.workunit.client.1.smithi193.stdout:2/524: rename d0/fc to d0/d1b/f9e 0 2024-03-23T08:23:22.746 INFO:tasks.workunit.client.1.smithi193.stdout:9/489: dwrite d2/d7/f37 [0,4194304] 0 2024-03-23T08:23:22.749 INFO:tasks.workunit.client.1.smithi193.stdout:7/406: truncate d6/d9/d1f/d3e/d65/d70/f83 985224 0 2024-03-23T08:23:22.749 INFO:tasks.workunit.client.1.smithi193.stdout:9/490: stat d2/d7/db/d1b/d5c/c89 0 2024-03-23T08:23:22.749 INFO:tasks.workunit.client.1.smithi193.stdout:2/525: mkdir d0/d28/d9f 0 2024-03-23T08:23:22.750 INFO:tasks.workunit.client.1.smithi193.stdout:8/419: truncate f0 276446 0 2024-03-23T08:23:22.751 INFO:tasks.workunit.client.1.smithi193.stdout:2/526: chown d0/d37/d48/f55 8278 1 2024-03-23T08:23:22.753 INFO:tasks.workunit.client.1.smithi193.stdout:5/483: getdents d0/d1c 0 2024-03-23T08:23:22.754 INFO:tasks.workunit.client.1.smithi193.stdout:9/491: unlink d2/d7/db/d32/d6b/f93 0 2024-03-23T08:23:22.755 INFO:tasks.workunit.client.1.smithi193.stdout:9/492: readlink d2/d7/db/d1b/l7e 0 2024-03-23T08:23:22.757 INFO:tasks.workunit.client.1.smithi193.stdout:5/484: unlink d0/d1c/f20 0 2024-03-23T08:23:22.758 INFO:tasks.workunit.client.1.smithi193.stdout:5/485: dread - d0/d1c/d3b/d4b/d55/f80 zero size 2024-03-23T08:23:22.759 INFO:tasks.workunit.client.1.smithi193.stdout:9/493: write d2/d7/f11 [4046353,15981] 0 2024-03-23T08:23:22.761 INFO:tasks.workunit.client.1.smithi193.stdout:5/486: creat d0/d1c/d3b/d85/d97/faf x:0 0 0 2024-03-23T08:23:22.762 INFO:tasks.workunit.client.1.smithi193.stdout:2/527: creat d0/d1b/d14/d66/fa0 x:0 0 0 2024-03-23T08:23:22.765 INFO:tasks.workunit.client.1.smithi193.stdout:5/487: rename d0/d1c/f65 to d0/d1c/fb0 0 2024-03-23T08:23:22.766 INFO:tasks.workunit.client.1.smithi193.stdout:9/494: mkdir d2/d7/db/d17/d4b/d59/d97/d9d 0 2024-03-23T08:23:22.771 INFO:tasks.workunit.client.1.smithi193.stdout:2/528: dwrite d0/d1b/d14/d66/d50/f93 [0,4194304] 0 2024-03-23T08:23:22.772 INFO:tasks.workunit.client.1.smithi193.stdout:7/407: truncate d6/d9/f19 3112048 0 2024-03-23T08:23:22.776 INFO:tasks.workunit.client.1.smithi193.stdout:8/420: write f0 [820310,109201] 0 2024-03-23T08:23:22.776 INFO:tasks.workunit.client.1.smithi193.stdout:5/488: dwrite d0/d1f/f51 [0,4194304] 0 2024-03-23T08:23:22.777 INFO:tasks.workunit.client.1.smithi193.stdout:2/529: mkdir d0/d1b/d14/da1 0 2024-03-23T08:23:22.785 INFO:tasks.workunit.client.1.smithi193.stdout:5/489: rename d0/d2f to d0/d1c/d7a/db1 0 2024-03-23T08:23:22.790 INFO:tasks.workunit.client.1.smithi193.stdout:7/408: dread d6/d9/d1f/f50 [0,4194304] 0 2024-03-23T08:23:22.793 INFO:tasks.workunit.client.1.smithi193.stdout:9/495: dwrite d2/fc [0,4194304] 0 2024-03-23T08:23:22.795 INFO:tasks.workunit.client.1.smithi193.stdout:2/530: dwrite d0/d1b/d14/d66/d50/f52 [0,4194304] 0 2024-03-23T08:23:22.796 INFO:tasks.workunit.client.1.smithi193.stdout:7/409: creat d6/d9/d1f/d23/d33/d3c/f8b x:0 0 0 2024-03-23T08:23:22.797 INFO:tasks.workunit.client.1.smithi193.stdout:7/410: chown d6/l6e 53 1 2024-03-23T08:23:22.800 INFO:tasks.workunit.client.1.smithi193.stdout:9/496: rename d2/d7/db/d17/d1f/f42 to d2/d7/db/d17/d4b/f9e 0 2024-03-23T08:23:22.801 INFO:tasks.workunit.client.1.smithi193.stdout:9/497: stat d2/f98 0 2024-03-23T08:23:22.802 INFO:tasks.workunit.client.1.smithi193.stdout:5/490: dwrite d0/d1c/d7a/db1/d72/d73/d45/f66 [0,4194304] 0 2024-03-23T08:23:22.803 INFO:tasks.workunit.client.1.smithi193.stdout:8/421: dread df/d10/f7b [0,4194304] 0 2024-03-23T08:23:22.803 INFO:tasks.workunit.client.1.smithi193.stdout:7/411: write d6/d9/d1f/d23/d33/d38/f44 [1026844,108998] 0 2024-03-23T08:23:22.841 INFO:tasks.workunit.client.1.smithi193.stdout:5/491: chown d0/d5a/l6a 393798 1 2024-03-23T08:23:22.842 INFO:tasks.workunit.client.1.smithi193.stdout:8/422: write df/d4b/f71 [266490,37516] 0 2024-03-23T08:23:22.842 INFO:tasks.workunit.client.1.smithi193.stdout:9/498: rename d2/f28 to d2/d7/db/d7a/f9f 0 2024-03-23T08:23:22.844 INFO:tasks.workunit.client.1.smithi193.stdout:8/423: write df/d4b/d4d/f1e [5131617,93925] 0 2024-03-23T08:23:22.847 INFO:tasks.workunit.client.1.smithi193.stdout:7/412: read d6/d9/d1f/d23/d33/d38/d46/f5b [730261,122436] 0 2024-03-23T08:23:22.848 INFO:tasks.workunit.client.1.smithi193.stdout:9/499: rename d2/d7/db/d7a/l7c to d2/d7/d31/d5f/d9c/la0 0 2024-03-23T08:23:22.848 INFO:tasks.workunit.client.1.smithi193.stdout:8/424: mknod df/d63/d84/c8e 0 2024-03-23T08:23:22.850 INFO:tasks.workunit.client.1.smithi193.stdout:8/425: readlink df/d4b/d4d/d2a/l74 0 2024-03-23T08:23:22.852 INFO:tasks.workunit.client.1.smithi193.stdout:5/492: dwrite d0/d1f/d93/fa3 [0,4194304] 0 2024-03-23T08:23:22.853 INFO:tasks.workunit.client.1.smithi193.stdout:5/493: chown d0/d1c/d7a/db1/d72/d73/d45 133402047 1 2024-03-23T08:23:22.856 INFO:tasks.workunit.client.1.smithi193.stdout:5/494: read d0/d1c/d7a/db1/d72/d73/d45/f77 [1884647,128268] 0 2024-03-23T08:23:22.857 INFO:tasks.workunit.client.1.smithi193.stdout:8/426: mknod df/d10/d67/d7f/c8f 0 2024-03-23T08:23:22.860 INFO:tasks.workunit.client.1.smithi193.stdout:8/427: creat df/d46/f90 x:0 0 0 2024-03-23T08:23:22.860 INFO:tasks.workunit.client.1.smithi193.stdout:9/500: dwrite d2/d7/db/d17/d33/f45 [0,4194304] 0 2024-03-23T08:23:22.861 INFO:tasks.workunit.client.1.smithi193.stdout:5/495: symlink d0/d1c/d3b/d85/d97/lb2 0 2024-03-23T08:23:22.863 INFO:tasks.workunit.client.1.smithi193.stdout:8/428: mknod df/d63/d84/c91 0 2024-03-23T08:23:22.865 INFO:tasks.workunit.client.1.smithi193.stdout:5/496: dread - d0/d1c/d7a/d86/d9f/fa8 zero size 2024-03-23T08:23:22.865 INFO:tasks.workunit.client.1.smithi193.stdout:8/429: chown df/d4b/d4d/f1e 18332 1 2024-03-23T08:23:22.866 INFO:tasks.workunit.client.1.smithi193.stdout:9/501: rename d2/d7/db/d17/d1f/c65 to d2/d7/db/d1b/d43/ca1 0 2024-03-23T08:23:22.866 INFO:tasks.workunit.client.1.smithi193.stdout:3/398: sync 2024-03-23T08:23:22.866 INFO:tasks.workunit.client.1.smithi193.stdout:4/482: sync 2024-03-23T08:23:22.867 INFO:tasks.workunit.client.1.smithi193.stdout:5/497: chown d0/d1c/d3b/d81/l9c 97981859 1 2024-03-23T08:23:22.870 INFO:tasks.workunit.client.1.smithi193.stdout:4/483: creat d11/d1a/d33/d44/faa x:0 0 0 2024-03-23T08:23:22.872 INFO:tasks.workunit.client.1.smithi193.stdout:9/502: rename d2/d7/db/d17/c55 to d2/d7/db/d32/ca2 0 2024-03-23T08:23:22.872 INFO:tasks.workunit.client.1.smithi193.stdout:9/503: stat d2/d7/db/d1b/f85 0 2024-03-23T08:23:22.873 INFO:tasks.workunit.client.1.smithi193.stdout:8/430: getdents df/d63/d76 0 2024-03-23T08:23:22.875 INFO:tasks.workunit.client.1.smithi193.stdout:3/399: mknod d1/de/d13/d21/d25/d2b/d3c/d40/c90 0 2024-03-23T08:23:22.875 INFO:tasks.workunit.client.1.smithi193.stdout:5/498: dread d0/f7 [0,4194304] 0 2024-03-23T08:23:22.876 INFO:tasks.workunit.client.1.smithi193.stdout:8/431: write df/f34 [527702,101091] 0 2024-03-23T08:23:22.876 INFO:tasks.workunit.client.1.smithi193.stdout:4/484: creat d11/d48/d50/fab x:0 0 0 2024-03-23T08:23:22.877 INFO:tasks.workunit.client.1.smithi193.stdout:9/504: link d2/d7/f2b d2/d7/db/d7a/d95/fa3 0 2024-03-23T08:23:22.878 INFO:tasks.workunit.client.1.smithi193.stdout:8/432: chown df/d10/d19/f72 0 1 2024-03-23T08:23:22.880 INFO:tasks.workunit.client.1.smithi193.stdout:4/485: mkdir d11/d8e/dac 0 2024-03-23T08:23:22.882 INFO:tasks.workunit.client.1.smithi193.stdout:9/505: mknod d2/d7/db/d1b/d5c/ca4 0 2024-03-23T08:23:22.882 INFO:tasks.workunit.client.1.smithi193.stdout:8/433: mknod df/d63/d84/c92 0 2024-03-23T08:23:22.882 INFO:tasks.workunit.client.1.smithi193.stdout:4/486: dread - d11/d27/d85/f8a zero size 2024-03-23T08:23:22.885 INFO:tasks.workunit.client.1.smithi193.stdout:8/434: mknod df/d62/c93 0 2024-03-23T08:23:22.886 INFO:tasks.workunit.client.1.smithi193.stdout:7/413: dread d6/f8 [0,4194304] 0 2024-03-23T08:23:22.886 INFO:tasks.workunit.client.1.smithi193.stdout:9/506: unlink d2/d7/db/d32/l69 0 2024-03-23T08:23:22.887 INFO:tasks.workunit.client.1.smithi193.stdout:3/400: dwrite d1/de/d13/d21/d50/d6f/d6e/f72 [0,4194304] 0 2024-03-23T08:23:22.888 INFO:tasks.workunit.client.1.smithi193.stdout:4/487: unlink d11/d48/d50/l90 0 2024-03-23T08:23:22.889 INFO:tasks.workunit.client.1.smithi193.stdout:7/414: dread - d6/d9/d1f/d23/d33/d38/d3b/f89 zero size 2024-03-23T08:23:22.890 INFO:tasks.workunit.client.1.smithi193.stdout:3/401: truncate d1/f84 276024 0 2024-03-23T08:23:22.891 INFO:tasks.workunit.client.1.smithi193.stdout:5/499: write d0/d1c/f21 [922686,33453] 0 2024-03-23T08:23:22.892 INFO:tasks.workunit.client.1.smithi193.stdout:6/391: sync 2024-03-23T08:23:22.892 INFO:tasks.workunit.client.1.smithi193.stdout:1/452: sync 2024-03-23T08:23:22.893 INFO:tasks.workunit.client.1.smithi193.stdout:0/459: sync 2024-03-23T08:23:22.896 INFO:tasks.workunit.client.1.smithi193.stdout:1/453: write f5 [324783,79637] 0 2024-03-23T08:23:22.898 INFO:tasks.workunit.client.1.smithi193.stdout:0/460: mknod db/d52/ca2 0 2024-03-23T08:23:22.899 INFO:tasks.workunit.client.1.smithi193.stdout:5/500: creat d0/d1f/fb3 x:0 0 0 2024-03-23T08:23:22.900 INFO:tasks.workunit.client.1.smithi193.stdout:6/392: read - d0/db/df/d11/d3b/f49 zero size 2024-03-23T08:23:22.900 INFO:tasks.workunit.client.1.smithi193.stdout:3/402: dread d1/f2f [0,4194304] 0 2024-03-23T08:23:22.903 INFO:tasks.workunit.client.1.smithi193.stdout:8/435: dwrite df/d46/f47 [0,4194304] 0 2024-03-23T08:23:22.905 INFO:tasks.workunit.client.1.smithi193.stdout:0/461: stat db/d11/d14/c73 0 2024-03-23T08:23:22.909 INFO:tasks.workunit.client.1.smithi193.stdout:4/488: write d11/d19/d22/f31 [7090881,36] 0 2024-03-23T08:23:22.911 INFO:tasks.workunit.client.1.smithi193.stdout:7/415: dwrite d6/d9/d1f/d23/f45 [0,4194304] 0 2024-03-23T08:23:22.913 INFO:tasks.workunit.client.1.smithi193.stdout:6/393: write d0/fe [9061467,126269] 0 2024-03-23T08:23:22.913 INFO:tasks.workunit.client.1.smithi193.stdout:5/501: creat d0/d1c/d7a/db1/d72/d73/d57/fb4 x:0 0 0 2024-03-23T08:23:22.913 INFO:tasks.workunit.client.1.smithi193.stdout:0/462: read - db/d11/d14/d24/f97 zero size 2024-03-23T08:23:22.920 INFO:tasks.workunit.client.1.smithi193.stdout:8/436: read df/f21 [46913,57200] 0 2024-03-23T08:23:22.920 INFO:tasks.workunit.client.1.smithi193.stdout:0/463: dread - db/d11/d14/d24/d2a/f83 zero size 2024-03-23T08:23:22.920 INFO:tasks.workunit.client.1.smithi193.stdout:4/489: fdatasync d11/d1a/d33/f7d 0 2024-03-23T08:23:22.925 INFO:tasks.workunit.client.1.smithi193.stdout:9/507: dread d2/d7/db/f3b [0,4194304] 0 2024-03-23T08:23:22.926 INFO:tasks.workunit.client.1.smithi193.stdout:3/403: dwrite d1/de/d13/d21/f81 [0,4194304] 0 2024-03-23T08:23:22.929 INFO:tasks.workunit.client.1.smithi193.stdout:8/437: symlink df/d10/d19/l94 0 2024-03-23T08:23:22.931 INFO:tasks.workunit.client.1.smithi193.stdout:1/454: dread da/f12 [0,4194304] 0 2024-03-23T08:23:22.933 INFO:tasks.workunit.client.1.smithi193.stdout:9/508: fsync d2/d7/db/d1b/f81 0 2024-03-23T08:23:22.934 INFO:tasks.workunit.client.1.smithi193.stdout:4/490: symlink d11/d48/d50/lad 0 2024-03-23T08:23:22.935 INFO:tasks.workunit.client.1.smithi193.stdout:8/438: write df/d10/d19/f4f [3173434,75004] 0 2024-03-23T08:23:22.936 INFO:tasks.workunit.client.1.smithi193.stdout:1/455: write da/f3c [4702899,33881] 0 2024-03-23T08:23:22.937 INFO:tasks.workunit.client.1.smithi193.stdout:0/464: rename db/d11/c7e to db/d11/d14/d51/ca3 0 2024-03-23T08:23:22.937 INFO:tasks.workunit.client.1.smithi193.stdout:4/491: chown d11/d27/d85 135513 1 2024-03-23T08:23:22.938 INFO:tasks.workunit.client.1.smithi193.stdout:7/416: truncate d6/d9/f31 2744453 0 2024-03-23T08:23:22.938 INFO:tasks.workunit.client.1.smithi193.stdout:6/394: dread d0/db/df/d11/d36/d61/d3a/f6b [0,4194304] 0 2024-03-23T08:23:22.939 INFO:tasks.workunit.client.1.smithi193.stdout:4/492: readlink d11/d48/la7 0 2024-03-23T08:23:22.940 INFO:tasks.workunit.client.1.smithi193.stdout:2/531: sync 2024-03-23T08:23:22.940 INFO:tasks.workunit.client.1.smithi193.stdout:8/439: dread - df/d10/d3e/d54/f89 zero size 2024-03-23T08:23:22.940 INFO:tasks.workunit.client.1.smithi193.stdout:3/404: link d1/de/d13/d21/d25/d2b/f4d d1/de/d13/d21/d25/d34/f91 0 2024-03-23T08:23:22.941 INFO:tasks.workunit.client.1.smithi193.stdout:1/456: mknod da/d13/d3f/c99 0 2024-03-23T08:23:22.941 INFO:tasks.workunit.client.1.smithi193.stdout:5/502: dread d0/d1c/d7a/db1/d72/d73/d45/f77 [0,4194304] 0 2024-03-23T08:23:22.941 INFO:tasks.workunit.client.1.smithi193.stdout:4/493: stat d11/l15 0 2024-03-23T08:23:22.942 INFO:tasks.workunit.client.1.smithi193.stdout:7/417: unlink d6/d9/f6a 0 2024-03-23T08:23:22.943 INFO:tasks.workunit.client.1.smithi193.stdout:4/494: fdatasync d11/f8c 0 2024-03-23T08:23:22.944 INFO:tasks.workunit.client.1.smithi193.stdout:0/465: symlink db/d11/d14/d21/d33/la4 0 2024-03-23T08:23:22.945 INFO:tasks.workunit.client.1.smithi193.stdout:2/532: mknod d0/d1b/d14/d66/d4e/ca2 0 2024-03-23T08:23:22.946 INFO:tasks.workunit.client.1.smithi193.stdout:8/440: chown df/d4b/d4d/d2a/c6f 486682 1 2024-03-23T08:23:22.946 INFO:tasks.workunit.client.1.smithi193.stdout:4/495: dread - d11/d8d/f97 zero size 2024-03-23T08:23:22.946 INFO:tasks.workunit.client.1.smithi193.stdout:6/395: mkdir d0/db/df/d11/d36/d61/d75 0 2024-03-23T08:23:22.948 INFO:tasks.workunit.client.1.smithi193.stdout:8/441: chown df/d10/d3e 1 1 2024-03-23T08:23:22.948 INFO:tasks.workunit.client.1.smithi193.stdout:3/405: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/l92 0 2024-03-23T08:23:22.948 INFO:tasks.workunit.client.1.smithi193.stdout:9/509: link d2/d7/db/d1b/l29 d2/d7/db/d7a/d95/la5 0 2024-03-23T08:23:22.949 INFO:tasks.workunit.client.1.smithi193.stdout:1/457: mknod da/dd/df/d75/d98/c9a 0 2024-03-23T08:23:22.950 INFO:tasks.workunit.client.1.smithi193.stdout:0/466: mknod db/d11/d14/d3b/d71/ca5 0 2024-03-23T08:23:22.950 INFO:tasks.workunit.client.1.smithi193.stdout:6/396: write d0/db/df/d11/d3b/f49 [950326,14841] 0 2024-03-23T08:23:22.950 INFO:tasks.workunit.client.1.smithi193.stdout:2/533: chown d0/d1b/d14/d66/d50/d92/f9a 114163 1 2024-03-23T08:23:22.950 INFO:tasks.workunit.client.1.smithi193.stdout:3/406: chown d1/d22/c74 1 1 2024-03-23T08:23:22.950 INFO:tasks.workunit.client.1.smithi193.stdout:4/496: mknod d11/d1a/d33/d7f/cae 0 2024-03-23T08:23:22.953 INFO:tasks.workunit.client.1.smithi193.stdout:5/503: getdents d0 0 2024-03-23T08:23:22.955 INFO:tasks.workunit.client.1.smithi193.stdout:9/510: mknod d2/d7/d31/d5f/d73/d78/ca6 0 2024-03-23T08:23:22.955 INFO:tasks.workunit.client.1.smithi193.stdout:2/534: fsync d0/d1b/d3b/f7a 0 2024-03-23T08:23:22.955 INFO:tasks.workunit.client.1.smithi193.stdout:6/397: write d0/f40 [4333969,29569] 0 2024-03-23T08:23:22.956 INFO:tasks.workunit.client.1.smithi193.stdout:8/442: dread df/d4b/f71 [0,4194304] 0 2024-03-23T08:23:22.956 INFO:tasks.workunit.client.1.smithi193.stdout:7/418: getdents d6/d9/d1f/d23/d33/d38/d3b/d6c 0 2024-03-23T08:23:22.958 INFO:tasks.workunit.client.1.smithi193.stdout:3/407: creat d1/de/d13/d21/d25/d2b/d3c/d4a/f93 x:0 0 0 2024-03-23T08:23:22.959 INFO:tasks.workunit.client.1.smithi193.stdout:7/419: write d6/d9/f16 [504326,56175] 0 2024-03-23T08:23:22.960 INFO:tasks.workunit.client.1.smithi193.stdout:6/398: write d0/db/df/d11/f37 [2038288,78057] 0 2024-03-23T08:23:22.960 INFO:tasks.workunit.client.1.smithi193.stdout:1/458: rename da/c80 to da/d13/d24/c9b 0 2024-03-23T08:23:22.961 INFO:tasks.workunit.client.1.smithi193.stdout:4/497: mknod d11/d1a/d33/d7f/caf 0 2024-03-23T08:23:22.964 INFO:tasks.workunit.client.1.smithi193.stdout:2/535: unlink d0/d1b/d14/c16 0 2024-03-23T08:23:22.965 INFO:tasks.workunit.client.1.smithi193.stdout:4/498: stat d11/d19/d22/f31 0 2024-03-23T08:23:22.966 INFO:tasks.workunit.client.1.smithi193.stdout:4/499: chown d11/d1a/d33/d44/ca1 31 1 2024-03-23T08:23:22.967 INFO:tasks.workunit.client.1.smithi193.stdout:4/500: chown d11/d1a/d33/d44/f4a 350363 1 2024-03-23T08:23:22.967 INFO:tasks.workunit.client.1.smithi193.stdout:4/501: readlink l10 0 2024-03-23T08:23:22.968 INFO:tasks.workunit.client.1.smithi193.stdout:3/408: mkdir d1/de/d13/d21/d50/d6f/d6e/d94 0 2024-03-23T08:23:22.969 INFO:tasks.workunit.client.1.smithi193.stdout:3/409: write d1/de/d13/d21/d25/f7e [610915,124356] 0 2024-03-23T08:23:22.969 INFO:tasks.workunit.client.1.smithi193.stdout:5/504: rename d0/d1c/f21 to d0/d1c/d7a/db1/d72/d99/fb5 0 2024-03-23T08:23:22.971 INFO:tasks.workunit.client.1.smithi193.stdout:5/505: stat d0/d1c/d3b/d85/d97/fad 0 2024-03-23T08:23:22.972 INFO:tasks.workunit.client.1.smithi193.stdout:5/506: readlink d0/d1c/d7a/d86/d9f/la4 0 2024-03-23T08:23:22.972 INFO:tasks.workunit.client.1.smithi193.stdout:8/443: symlink df/d4b/d4d/l95 0 2024-03-23T08:23:22.972 INFO:tasks.workunit.client.1.smithi193.stdout:8/444: chown ca 1 1 2024-03-23T08:23:22.973 INFO:tasks.workunit.client.1.smithi193.stdout:9/511: creat d2/d7/db/d7a/d95/fa7 x:0 0 0 2024-03-23T08:23:22.973 INFO:tasks.workunit.client.1.smithi193.stdout:5/507: stat d0/d1c/d7a/db1/d72/d73/d45/fa0 0 2024-03-23T08:23:22.974 INFO:tasks.workunit.client.1.smithi193.stdout:2/536: mkdir d0/d1b/d14/d56/da3 0 2024-03-23T08:23:22.974 INFO:tasks.workunit.client.1.smithi193.stdout:5/508: truncate d0/f1a 1333536 0 2024-03-23T08:23:22.974 INFO:tasks.workunit.client.1.smithi193.stdout:4/502: mkdir d11/d1a/db0 0 2024-03-23T08:23:22.975 INFO:tasks.workunit.client.1.smithi193.stdout:4/503: stat d11/d1a/d33/d44/f4a 0 2024-03-23T08:23:22.977 INFO:tasks.workunit.client.1.smithi193.stdout:6/399: rename d0/db/df/d11/d36/d61/d3a/f53 to d0/db/df/d11/d3b/f76 0 2024-03-23T08:23:22.978 INFO:tasks.workunit.client.1.smithi193.stdout:7/420: creat d6/d9/d1f/d23/d33/d38/d46/d82/f8c x:0 0 0 2024-03-23T08:23:22.979 INFO:tasks.workunit.client.1.smithi193.stdout:3/410: mknod d1/de/d13/d21/d25/c95 0 2024-03-23T08:23:22.981 INFO:tasks.workunit.client.1.smithi193.stdout:8/445: chown df/d10/c23 0 1 2024-03-23T08:23:22.982 INFO:tasks.workunit.client.1.smithi193.stdout:2/537: dwrite d0/d37/d48/f55 [0,4194304] 0 2024-03-23T08:23:22.983 INFO:tasks.workunit.client.1.smithi193.stdout:4/504: mkdir d11/d1a/d33/d42/d52/db1 0 2024-03-23T08:23:22.985 INFO:tasks.workunit.client.1.smithi193.stdout:3/411: mkdir d1/de/d13/d96 0 2024-03-23T08:23:22.985 INFO:tasks.workunit.client.1.smithi193.stdout:6/400: rename d0/fe to d0/db/df/d11/f77 0 2024-03-23T08:23:22.989 INFO:tasks.workunit.client.1.smithi193.stdout:7/421: creat d6/d9/d1f/d23/d33/d38/d3b/d6c/f8d x:0 0 0 2024-03-23T08:23:22.991 INFO:tasks.workunit.client.1.smithi193.stdout:1/459: getdents da/d13/d24 0 2024-03-23T08:23:22.991 INFO:tasks.workunit.client.1.smithi193.stdout:5/509: link d0/d5a/c5d d0/d1c/d7a/db1/d72/d73/d57/d6d/cb6 0 2024-03-23T08:23:22.991 INFO:tasks.workunit.client.1.smithi193.stdout:2/538: read d0/d1b/d14/d25/f5a [38499,38305] 0 2024-03-23T08:23:22.992 INFO:tasks.workunit.client.1.smithi193.stdout:6/401: write d0/db/df/d11/d36/f6e [249485,90227] 0 2024-03-23T08:23:22.993 INFO:tasks.workunit.client.1.smithi193.stdout:3/412: stat d1/de/d13/d21/d25/d2b/d3c 0 2024-03-23T08:23:22.996 INFO:tasks.workunit.client.1.smithi193.stdout:9/512: dwrite d2/d7/db/d7a/f9f [4194304,4194304] 0 2024-03-23T08:23:23.001 INFO:tasks.workunit.client.1.smithi193.stdout:6/402: mknod d0/d27/c78 0 2024-03-23T08:23:23.003 INFO:tasks.workunit.client.1.smithi193.stdout:1/460: mknod da/c9c 0 2024-03-23T08:23:23.003 INFO:tasks.workunit.client.1.smithi193.stdout:4/505: dwrite d11/d2d/d58/f86 [0,4194304] 0 2024-03-23T08:23:23.003 INFO:tasks.workunit.client.1.smithi193.stdout:2/539: symlink d0/d1b/d14/d66/d50/d8f/la4 0 2024-03-23T08:23:23.006 INFO:tasks.workunit.client.1.smithi193.stdout:3/413: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/d85/d97 0 2024-03-23T08:23:23.008 INFO:tasks.workunit.client.1.smithi193.stdout:5/510: dwrite d0/fe [0,4194304] 0 2024-03-23T08:23:23.008 INFO:tasks.workunit.client.1.smithi193.stdout:8/446: write df/f48 [651435,117148] 0 2024-03-23T08:23:23.011 INFO:tasks.workunit.client.1.smithi193.stdout:9/513: dread d2/d7/db/d17/d4b/f9e [0,4194304] 0 2024-03-23T08:23:23.012 INFO:tasks.workunit.client.1.smithi193.stdout:1/461: creat da/d13/d3f/d5a/f9d x:0 0 0 2024-03-23T08:23:23.016 INFO:tasks.workunit.client.1.smithi193.stdout:2/540: truncate d0/d28/f62 566196 0 2024-03-23T08:23:23.016 INFO:tasks.workunit.client.1.smithi193.stdout:6/403: rename d0/db/df/d11/f77 to d0/db/df/d11/d36/f79 0 2024-03-23T08:23:23.017 INFO:tasks.workunit.client.1.smithi193.stdout:3/414: truncate d1/de/d13/d21/d25/d2b/d3c/d4a/f93 674712 0 2024-03-23T08:23:23.018 INFO:tasks.workunit.client.1.smithi193.stdout:1/462: chown da/d13/f7d 16 1 2024-03-23T08:23:23.018 INFO:tasks.workunit.client.1.smithi193.stdout:9/514: creat d2/d7/d31/d5f/d73/fa8 x:0 0 0 2024-03-23T08:23:23.019 INFO:tasks.workunit.client.1.smithi193.stdout:5/511: write d0/d1c/fb0 [1106503,26873] 0 2024-03-23T08:23:23.020 INFO:tasks.workunit.client.1.smithi193.stdout:8/447: creat df/d62/d8b/f96 x:0 0 0 2024-03-23T08:23:23.021 INFO:tasks.workunit.client.1.smithi193.stdout:8/448: read - df/f5a zero size 2024-03-23T08:23:23.022 INFO:tasks.workunit.client.1.smithi193.stdout:1/463: fsync da/dd/df/d52/f7a 0 2024-03-23T08:23:23.023 INFO:tasks.workunit.client.1.smithi193.stdout:3/415: write d1/de/d13/d21/d25/d2b/f35 [2194494,58930] 0 2024-03-23T08:23:23.024 INFO:tasks.workunit.client.1.smithi193.stdout:8/449: symlink df/d4b/l97 0 2024-03-23T08:23:23.025 INFO:tasks.workunit.client.1.smithi193.stdout:5/512: creat d0/d1f/fb7 x:0 0 0 2024-03-23T08:23:23.026 INFO:tasks.workunit.client.1.smithi193.stdout:9/515: mkdir d2/d7/db/d7a/d95/da9 0 2024-03-23T08:23:23.030 INFO:tasks.workunit.client.1.smithi193.stdout:1/464: rmdir da/dd/df/d52/d82 39 2024-03-23T08:23:23.031 INFO:tasks.workunit.client.1.smithi193.stdout:3/416: chown d1/de/d13/d21/d25/d2b/f39 989 1 2024-03-23T08:23:23.032 INFO:tasks.workunit.client.1.smithi193.stdout:9/516: dread d2/d7/d31/f40 [0,4194304] 0 2024-03-23T08:23:23.034 INFO:tasks.workunit.client.1.smithi193.stdout:1/465: fsync da/dd/df/d52/f68 0 2024-03-23T08:23:23.034 INFO:tasks.workunit.client.1.smithi193.stdout:5/513: creat d0/d1c/d7a/db1/d72/d73/d57/d6d/fb8 x:0 0 0 2024-03-23T08:23:23.034 INFO:tasks.workunit.client.1.smithi193.stdout:4/506: dread d11/d48/f5d [0,4194304] 0 2024-03-23T08:23:23.040 INFO:tasks.workunit.client.1.smithi193.stdout:5/514: fsync d0/f4d 0 2024-03-23T08:23:23.040 INFO:tasks.workunit.client.1.smithi193.stdout:3/417: readlink d1/de/d13/d21/d27/l38 0 2024-03-23T08:23:23.041 INFO:tasks.workunit.client.1.smithi193.stdout:6/404: dwrite d0/db/df/d11/d36/d61/d12/d21/f73 [0,4194304] 0 2024-03-23T08:23:23.041 INFO:tasks.workunit.client.1.smithi193.stdout:4/507: fsync d11/d2d/d3a/f5e 0 2024-03-23T08:23:23.043 INFO:tasks.workunit.client.1.smithi193.stdout:1/466: fsync da/d13/d24/f43 0 2024-03-23T08:23:23.048 INFO:tasks.workunit.client.1.smithi193.stdout:5/515: mknod d0/d1c/cb9 0 2024-03-23T08:23:23.049 INFO:tasks.workunit.client.1.smithi193.stdout:2/541: dwrite d0/f1d [0,4194304] 0 2024-03-23T08:23:23.049 INFO:tasks.workunit.client.1.smithi193.stdout:4/508: rename d11/da9 to d11/d1a/d33/d42/db2 0 2024-03-23T08:23:23.050 INFO:tasks.workunit.client.1.smithi193.stdout:8/450: dwrite df/d4b/d4d/f1e [0,4194304] 0 2024-03-23T08:23:23.052 INFO:tasks.workunit.client.1.smithi193.stdout:4/509: fsync f5 0 2024-03-23T08:23:23.059 INFO:tasks.workunit.client.1.smithi193.stdout:6/405: fdatasync d0/db/df/d11/d36/f79 0 2024-03-23T08:23:23.059 INFO:tasks.workunit.client.1.smithi193.stdout:1/467: mknod da/dd/c9e 0 2024-03-23T08:23:23.060 INFO:tasks.workunit.client.1.smithi193.stdout:2/542: write d0/d28/f62 [1000142,125123] 0 2024-03-23T08:23:23.061 INFO:tasks.workunit.client.1.smithi193.stdout:2/543: fsync d0/d1b/d14/d56/f96 0 2024-03-23T08:23:23.063 INFO:tasks.workunit.client.1.smithi193.stdout:6/406: chown d0/c67 11133 1 2024-03-23T08:23:23.064 INFO:tasks.workunit.client.1.smithi193.stdout:8/451: link df/d4b/d4d/d2a/l58 df/d10/d67/l98 0 2024-03-23T08:23:23.064 INFO:tasks.workunit.client.1.smithi193.stdout:5/516: write d0/d5a/f75 [4405346,99879] 0 2024-03-23T08:23:23.065 INFO:tasks.workunit.client.1.smithi193.stdout:9/517: dwrite d2/d7/db/d17/d33/f45 [4194304,4194304] 0 2024-03-23T08:23:23.068 INFO:tasks.workunit.client.1.smithi193.stdout:1/468: creat da/dd/df/d52/d82/f9f x:0 0 0 2024-03-23T08:23:23.068 INFO:tasks.workunit.client.1.smithi193.stdout:8/452: write df/d10/d3e/d54/f89 [180005,65706] 0 2024-03-23T08:23:23.069 INFO:tasks.workunit.client.1.smithi193.stdout:6/407: write d0/db/df/d11/d36/f79 [8525099,86757] 0 2024-03-23T08:23:23.070 INFO:tasks.workunit.client.1.smithi193.stdout:9/518: symlink d2/d7/db/d7a/laa 0 2024-03-23T08:23:23.071 INFO:tasks.workunit.client.1.smithi193.stdout:5/517: chown d0/d1c/d7a/db1/d72/d73/d45/da2 1627 1 2024-03-23T08:23:23.071 INFO:tasks.workunit.client.1.smithi193.stdout:8/453: readlink df/d10/d3e/d4e/l6d 0 2024-03-23T08:23:23.073 INFO:tasks.workunit.client.1.smithi193.stdout:4/510: dread d11/d19/d22/f26 [0,4194304] 0 2024-03-23T08:23:23.076 INFO:tasks.workunit.client.1.smithi193.stdout:1/469: chown da/dd/df/d52/d82/f9f 1257730 1 2024-03-23T08:23:23.080 INFO:tasks.workunit.client.1.smithi193.stdout:8/454: creat df/d4b/d4d/f99 x:0 0 0 2024-03-23T08:23:23.080 INFO:tasks.workunit.client.1.smithi193.stdout:6/408: mkdir d0/db/df/d11/d36/d61/d3a/d7a 0 2024-03-23T08:23:23.082 INFO:tasks.workunit.client.1.smithi193.stdout:5/518: chown d0/d1c/d7a/db1/d72/d73/d57/d6d/d8e/f92 102122257 1 2024-03-23T08:23:23.083 INFO:tasks.workunit.client.1.smithi193.stdout:1/470: fdatasync da/d13/d3f/d5a/d76/f8f 0 2024-03-23T08:23:23.087 INFO:tasks.workunit.client.1.smithi193.stdout:2/544: dwrite d0/d1b/d14/d66/d50/d92/f9a [0,4194304] 0 2024-03-23T08:23:23.091 INFO:tasks.workunit.client.1.smithi193.stdout:6/409: creat d0/d24/d69/f7b x:0 0 0 2024-03-23T08:23:23.091 INFO:tasks.workunit.client.1.smithi193.stdout:9/519: dwrite d2/f25 [0,4194304] 0 2024-03-23T08:23:23.092 INFO:tasks.workunit.client.1.smithi193.stdout:5/519: mknod d0/d1f/d93/cba 0 2024-03-23T08:23:23.092 INFO:tasks.workunit.client.1.smithi193.stdout:8/455: link ca df/d63/d84/c9a 0 2024-03-23T08:23:23.094 INFO:tasks.workunit.client.1.smithi193.stdout:1/471: symlink da/d13/d3f/d5a/d76/la0 0 2024-03-23T08:23:23.095 INFO:tasks.workunit.client.1.smithi193.stdout:2/545: fdatasync d0/d28/d33/d72/f97 0 2024-03-23T08:23:23.095 INFO:tasks.workunit.client.1.smithi193.stdout:9/520: unlink d2/d7/d31/c3a 0 2024-03-23T08:23:23.096 INFO:tasks.workunit.client.1.smithi193.stdout:4/511: dwrite d11/d19/d22/f6b [0,4194304] 0 2024-03-23T08:23:23.098 INFO:tasks.workunit.client.1.smithi193.stdout:5/520: creat d0/d1f/fbb x:0 0 0 2024-03-23T08:23:23.100 INFO:tasks.workunit.client.1.smithi193.stdout:8/456: mknod df/d63/d84/c9b 0 2024-03-23T08:23:23.105 INFO:tasks.workunit.client.1.smithi193.stdout:9/521: mknod d2/d7/db/d17/d33/cab 0 2024-03-23T08:23:23.105 INFO:tasks.workunit.client.1.smithi193.stdout:1/472: write da/d13/d24/f79 [416908,10332] 0 2024-03-23T08:23:23.106 INFO:tasks.workunit.client.1.smithi193.stdout:5/521: mkdir d0/d1c/d3b/dbc 0 2024-03-23T08:23:23.107 INFO:tasks.workunit.client.1.smithi193.stdout:1/473: read da/f12 [3982214,5867] 0 2024-03-23T08:23:23.108 INFO:tasks.workunit.client.1.smithi193.stdout:4/512: mkdir d11/d48/d50/db3 0 2024-03-23T08:23:23.109 INFO:tasks.workunit.client.1.smithi193.stdout:5/522: fdatasync d0/d1c/d7a/db1/f67 0 2024-03-23T08:23:23.111 INFO:tasks.workunit.client.1.smithi193.stdout:8/457: getdents df/d63 0 2024-03-23T08:23:23.112 INFO:tasks.workunit.client.1.smithi193.stdout:2/546: dwrite d0/d1b/d14/d25/f29 [0,4194304] 0 2024-03-23T08:23:23.113 INFO:tasks.workunit.client.1.smithi193.stdout:1/474: mknod da/d13/ca1 0 2024-03-23T08:23:23.113 INFO:tasks.workunit.client.1.smithi193.stdout:4/513: write d11/d48/d50/f5b [3384602,20236] 0 2024-03-23T08:23:23.118 INFO:tasks.workunit.client.1.smithi193.stdout:9/522: rename d2/d7/db/d1b/f85 to d2/d7/db/d7a/d95/da9/fac 0 2024-03-23T08:23:23.120 INFO:tasks.workunit.client.1.smithi193.stdout:8/458: write df/d4b/d4d/f87 [2636086,96652] 0 2024-03-23T08:23:23.121 INFO:tasks.workunit.client.1.smithi193.stdout:4/514: mknod d11/cb4 0 2024-03-23T08:23:23.122 INFO:tasks.workunit.client.1.smithi193.stdout:2/547: creat d0/d28/d33/d72/fa5 x:0 0 0 2024-03-23T08:23:23.123 INFO:tasks.workunit.client.1.smithi193.stdout:6/410: dwrite d0/db/df/d11/d36/d61/d3a/f45 [0,4194304] 0 2024-03-23T08:23:23.124 INFO:tasks.workunit.client.1.smithi193.stdout:1/475: symlink da/dd/df/d7e/la2 0 2024-03-23T08:23:23.126 INFO:tasks.workunit.client.1.smithi193.stdout:1/476: chown da/l47 5386 1 2024-03-23T08:23:23.127 INFO:tasks.workunit.client.1.smithi193.stdout:9/523: mknod d2/d7/db/d32/d6b/cad 0 2024-03-23T08:23:23.130 INFO:tasks.workunit.client.1.smithi193.stdout:5/523: dwrite d0/d1f/fb7 [0,4194304] 0 2024-03-23T08:23:23.134 INFO:tasks.workunit.client.1.smithi193.stdout:8/459: mkdir df/d4b/d4d/d9c 0 2024-03-23T08:23:23.137 INFO:tasks.workunit.client.1.smithi193.stdout:4/515: rename d11/d48/d50/l3d to d11/d1a/d33/d42/d52/d70/lb5 0 2024-03-23T08:23:23.137 INFO:tasks.workunit.client.1.smithi193.stdout:8/460: fdatasync df/f34 0 2024-03-23T08:23:23.138 INFO:tasks.workunit.client.1.smithi193.stdout:8/461: readlink df/d4b/d4d/l5d 0 2024-03-23T08:23:23.138 INFO:tasks.workunit.client.1.smithi193.stdout:1/477: dwrite da/d13/d3f/d5a/d76/f8f [0,4194304] 0 2024-03-23T08:23:23.139 INFO:tasks.workunit.client.1.smithi193.stdout:2/548: rename d0/d1b/d77/c8b to d0/d1b/d19/ca6 0 2024-03-23T08:23:23.141 INFO:tasks.workunit.client.1.smithi193.stdout:1/478: chown da/dd/df/f3b 848252935 1 2024-03-23T08:23:23.142 INFO:tasks.workunit.client.1.smithi193.stdout:7/422: sync 2024-03-23T08:23:23.142 INFO:tasks.workunit.client.1.smithi193.stdout:5/524: link d0/d1c/d7a/db1/c83 d0/d1c/d7a/db1/d72/d73/d45/da2/cbd 0 2024-03-23T08:23:23.142 INFO:tasks.workunit.client.1.smithi193.stdout:0/467: sync 2024-03-23T08:23:23.145 INFO:tasks.workunit.client.1.smithi193.stdout:5/525: chown d0/c6b 812322805 1 2024-03-23T08:23:23.148 INFO:tasks.workunit.client.1.smithi193.stdout:1/479: dread da/dd/df/d2e/f4f [0,4194304] 0 2024-03-23T08:23:23.149 INFO:tasks.workunit.client.1.smithi193.stdout:9/524: rename d2/d7/db/d17/d33/c35 to d2/d7/d31/d5f/d73/cae 0 2024-03-23T08:23:23.151 INFO:tasks.workunit.client.1.smithi193.stdout:9/525: chown d2/d7/db/l2c 26737 1 2024-03-23T08:23:23.152 INFO:tasks.workunit.client.1.smithi193.stdout:0/468: dwrite db/d11/d14/d3b/d71/f7f [0,4194304] 0 2024-03-23T08:23:23.152 INFO:tasks.workunit.client.1.smithi193.stdout:8/462: link df/d10/d19/f8c df/d10/d3e/d4e/f9d 0 2024-03-23T08:23:23.153 INFO:tasks.workunit.client.1.smithi193.stdout:2/549: mkdir d0/d1b/da7 0 2024-03-23T08:23:23.155 INFO:tasks.workunit.client.1.smithi193.stdout:6/411: write d0/db/f33 [1633343,66190] 0 2024-03-23T08:23:23.155 INFO:tasks.workunit.client.1.smithi193.stdout:8/463: stat df/d4b/d4d/d2a/c2d 0 2024-03-23T08:23:23.157 INFO:tasks.workunit.client.1.smithi193.stdout:5/526: mknod d0/d1c/d3b/d81/cbe 0 2024-03-23T08:23:23.158 INFO:tasks.workunit.client.1.smithi193.stdout:7/423: mkdir d6/d9/d1f/d4d/d7c/d8e 0 2024-03-23T08:23:23.161 INFO:tasks.workunit.client.1.smithi193.stdout:4/516: truncate d11/d48/f5d 2955088 0 2024-03-23T08:23:23.163 INFO:tasks.workunit.client.1.smithi193.stdout:9/526: readlink d2/d7/db/d7a/d95/la5 0 2024-03-23T08:23:23.163 INFO:tasks.workunit.client.1.smithi193.stdout:1/480: dread da/d13/d3f/d5a/d76/f8f [0,4194304] 0 2024-03-23T08:23:23.163 INFO:tasks.workunit.client.1.smithi193.stdout:7/424: write d6/d9/d1f/d23/d33/d38/f44 [1346062,117950] 0 2024-03-23T08:23:23.165 INFO:tasks.workunit.client.1.smithi193.stdout:0/469: unlink db/c17 0 2024-03-23T08:23:23.168 INFO:tasks.workunit.client.1.smithi193.stdout:0/470: chown db/d11/d14/d24/d2a/d4b/c91 4 1 2024-03-23T08:23:23.171 INFO:tasks.workunit.client.1.smithi193.stdout:2/550: rmdir d0/d1b/d14/d56/d99 0 2024-03-23T08:23:23.171 INFO:tasks.workunit.client.1.smithi193.stdout:0/471: truncate db/d11/d14/d24/d2a/d4d/f66 1121758 0 2024-03-23T08:23:23.172 INFO:tasks.workunit.client.1.smithi193.stdout:0/472: dread - db/d11/d14/d24/d2a/d4b/f84 zero size 2024-03-23T08:23:23.174 INFO:tasks.workunit.client.1.smithi193.stdout:0/473: fsync db/d11/d14/d3b/d71/f7f 0 2024-03-23T08:23:23.174 INFO:tasks.workunit.client.1.smithi193.stdout:3/418: sync 2024-03-23T08:23:23.174 INFO:tasks.workunit.client.1.smithi193.stdout:6/412: rename d0/d24 to d0/db/df/d11/d36/d61/d7c 0 2024-03-23T08:23:23.175 INFO:tasks.workunit.client.1.smithi193.stdout:4/517: symlink d11/d2d/d58/d80/d9d/lb6 0 2024-03-23T08:23:23.177 INFO:tasks.workunit.client.1.smithi193.stdout:3/419: write d1/de/d13/d21/d59/d60/f6b [1731414,68731] 0 2024-03-23T08:23:23.178 INFO:tasks.workunit.client.1.smithi193.stdout:2/551: chown d0/d37/d48/f4a 4 1 2024-03-23T08:23:23.181 INFO:tasks.workunit.client.1.smithi193.stdout:9/527: link d2/d7/db/d17/d33/f45 d2/d7/db/d1b/faf 0 2024-03-23T08:23:23.181 INFO:tasks.workunit.client.1.smithi193.stdout:9/528: write d2/fc [1290553,55815] 0 2024-03-23T08:23:23.185 INFO:tasks.workunit.client.1.smithi193.stdout:5/527: truncate d0/d1c/d3b/f98 494177 0 2024-03-23T08:23:23.187 INFO:tasks.workunit.client.1.smithi193.stdout:3/420: rmdir d1/de/d13/d21/d50/d6f 39 2024-03-23T08:23:23.189 INFO:tasks.workunit.client.1.smithi193.stdout:5/528: stat d0/d1c/d7a/db1/d72/d73/d57/d6d/c6f 0 2024-03-23T08:23:23.189 INFO:tasks.workunit.client.1.smithi193.stdout:8/464: dwrite df/d4b/d4d/d2a/f2b [0,4194304] 0 2024-03-23T08:23:23.189 INFO:tasks.workunit.client.1.smithi193.stdout:2/552: write d0/d1b/d14/d25/f6c [449253,63227] 0 2024-03-23T08:23:23.193 INFO:tasks.workunit.client.1.smithi193.stdout:7/425: link d6/d9/d1f/d3e/d42/l5d d6/d9/d1f/d5f/l8f 0 2024-03-23T08:23:23.193 INFO:tasks.workunit.client.1.smithi193.stdout:2/553: read - d0/d1b/d14/d66/d50/f91 zero size 2024-03-23T08:23:23.198 INFO:tasks.workunit.client.1.smithi193.stdout:6/413: creat d0/db/df/d11/d3b/d6c/f7d x:0 0 0 2024-03-23T08:23:23.199 INFO:tasks.workunit.client.1.smithi193.stdout:1/481: truncate da/dd/df/d2e/f93 3202287 0 2024-03-23T08:23:23.200 INFO:tasks.workunit.client.1.smithi193.stdout:1/482: stat da/dd/df/d2e/c51 0 2024-03-23T08:23:23.200 INFO:tasks.workunit.client.1.smithi193.stdout:3/421: rmdir d1/de/d13/d21/d27/d7c 39 2024-03-23T08:23:23.204 INFO:tasks.workunit.client.1.smithi193.stdout:5/529: mkdir d0/d5a/dbf 0 2024-03-23T08:23:23.204 INFO:tasks.workunit.client.1.smithi193.stdout:1/483: dread da/d13/d3f/f87 [0,4194304] 0 2024-03-23T08:23:23.206 INFO:tasks.workunit.client.1.smithi193.stdout:5/530: write d0/d1c/d7a/db1/d72/d73/f44 [1994877,29834] 0 2024-03-23T08:23:23.206 INFO:tasks.workunit.client.1.smithi193.stdout:8/465: creat df/d62/d8b/f9e x:0 0 0 2024-03-23T08:23:23.207 INFO:tasks.workunit.client.1.smithi193.stdout:8/466: chown df/c82 585952 1 2024-03-23T08:23:23.209 INFO:tasks.workunit.client.1.smithi193.stdout:2/554: write d0/d1b/f13 [3959672,10695] 0 2024-03-23T08:23:23.210 INFO:tasks.workunit.client.1.smithi193.stdout:7/426: symlink d6/d9/d1f/d4d/d6b/l90 0 2024-03-23T08:23:23.211 INFO:tasks.workunit.client.1.smithi193.stdout:0/474: link db/d11/d14/d24/f39 db/d11/fa6 0 2024-03-23T08:23:23.214 INFO:tasks.workunit.client.1.smithi193.stdout:5/531: dread d0/f43 [0,4194304] 0 2024-03-23T08:23:23.215 INFO:tasks.workunit.client.1.smithi193.stdout:5/532: fsync d0/d1f/fb7 0 2024-03-23T08:23:23.279 INFO:tasks.workunit.client.1.smithi193.stdout:4/518: creat d11/d19/fb7 x:0 0 0 2024-03-23T08:23:23.280 INFO:tasks.workunit.client.1.smithi193.stdout:6/414: creat d0/db/df/d11/d36/d61/d12/d21/d2a/f7e x:0 0 0 2024-03-23T08:23:23.281 INFO:tasks.workunit.client.1.smithi193.stdout:3/422: dread d1/de/d13/d21/d27/f1c [0,4194304] 0 2024-03-23T08:23:23.282 INFO:tasks.workunit.client.1.smithi193.stdout:3/423: stat d1/de/d13/d21/d27/c5d 0 2024-03-23T08:23:23.283 INFO:tasks.workunit.client.1.smithi193.stdout:9/529: rename d2/d7/db/d17/d33/d70/c8a to d2/d7/d31/d3c/cb0 0 2024-03-23T08:23:23.291 INFO:tasks.workunit.client.1.smithi193.stdout:5/533: mkdir d0/d1f/d93/dc0 0 2024-03-23T08:23:23.291 INFO:tasks.workunit.client.1.smithi193.stdout:5/534: readlink d0/d5a/l6a 0 2024-03-23T08:23:23.294 INFO:tasks.workunit.client.1.smithi193.stdout:6/415: rename d0/db/df/d11/d36/d61/d12/d21/c39 to d0/db/df/d11/c7f 0 2024-03-23T08:23:23.294 INFO:tasks.workunit.client.1.smithi193.stdout:9/530: read d2/d7/d31/f40 [446065,19985] 0 2024-03-23T08:23:23.297 INFO:tasks.workunit.client.1.smithi193.stdout:0/475: truncate db/d11/d14/f95 1006397 0 2024-03-23T08:23:23.299 INFO:tasks.workunit.client.1.smithi193.stdout:8/467: write f3 [1280260,104592] 0 2024-03-23T08:23:23.300 INFO:tasks.workunit.client.1.smithi193.stdout:9/531: creat d2/d7/d31/d5f/fb1 x:0 0 0 2024-03-23T08:23:23.301 INFO:tasks.workunit.client.1.smithi193.stdout:2/555: creat d0/d28/fa8 x:0 0 0 2024-03-23T08:23:23.301 INFO:tasks.workunit.client.1.smithi193.stdout:7/427: creat d6/d9/d1f/d23/d33/d38/f91 x:0 0 0 2024-03-23T08:23:23.303 INFO:tasks.workunit.client.1.smithi193.stdout:5/535: symlink d0/d1c/d7a/db1/d72/d73/d45/da2/lc1 0 2024-03-23T08:23:23.303 INFO:tasks.workunit.client.1.smithi193.stdout:7/428: write d6/d9/f16 [1962956,6105] 0 2024-03-23T08:23:23.304 INFO:tasks.workunit.client.1.smithi193.stdout:6/416: mknod d0/db/df/d11/d36/d61/c80 0 2024-03-23T08:23:23.304 INFO:tasks.workunit.client.1.smithi193.stdout:3/424: rmdir d1/de/d13/d21/d25/d2b/d3c/d4a/d85/d97 0 2024-03-23T08:23:23.306 INFO:tasks.workunit.client.1.smithi193.stdout:9/532: mkdir d2/d7/db/d17/d33/db2 0 2024-03-23T08:23:23.308 INFO:tasks.workunit.client.1.smithi193.stdout:4/519: getdents d11/d2d 0 2024-03-23T08:23:23.308 INFO:tasks.workunit.client.1.smithi193.stdout:9/533: write d2/f7d [3721597,32039] 0 2024-03-23T08:23:23.310 INFO:tasks.workunit.client.1.smithi193.stdout:7/429: unlink d6/l1a 0 2024-03-23T08:23:23.311 INFO:tasks.workunit.client.1.smithi193.stdout:5/536: dwrite d0/d1f/fb3 [0,4194304] 0 2024-03-23T08:23:23.315 INFO:tasks.workunit.client.1.smithi193.stdout:8/468: creat df/d4b/d4d/d9c/f9f x:0 0 0 2024-03-23T08:23:23.315 INFO:tasks.workunit.client.1.smithi193.stdout:3/425: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/d85/l98 0 2024-03-23T08:23:23.317 INFO:tasks.workunit.client.1.smithi193.stdout:3/426: dread - d1/de/d13/d21/d25/d2b/f7f zero size 2024-03-23T08:23:23.318 INFO:tasks.workunit.client.1.smithi193.stdout:3/427: chown d1/d22/c61 63 1 2024-03-23T08:23:23.318 INFO:tasks.workunit.client.1.smithi193.stdout:4/520: unlink d11/d27/c62 0 2024-03-23T08:23:23.318 INFO:tasks.workunit.client.1.smithi193.stdout:3/428: chown d1/de/d13/f5b 54376376 1 2024-03-23T08:23:23.319 INFO:tasks.workunit.client.1.smithi193.stdout:4/521: fsync d11/d1a/d33/d42/d52/f87 0 2024-03-23T08:23:23.320 INFO:tasks.workunit.client.1.smithi193.stdout:4/522: chown d11/d2d/d58 806017113 1 2024-03-23T08:23:23.320 INFO:tasks.workunit.client.1.smithi193.stdout:0/476: creat db/d11/d14/d21/d33/d48/d85/fa7 x:0 0 0 2024-03-23T08:23:23.321 INFO:tasks.workunit.client.1.smithi193.stdout:7/430: mkdir d6/d9/d1f/d4d/d7c/d92 0 2024-03-23T08:23:23.324 INFO:tasks.workunit.client.1.smithi193.stdout:6/417: rename d0/db to d0/d27/d81 0 2024-03-23T08:23:23.326 INFO:tasks.workunit.client.1.smithi193.stdout:2/556: getdents d0/d1b/d14/d66/d50/d8f 0 2024-03-23T08:23:23.327 INFO:tasks.workunit.client.1.smithi193.stdout:3/429: dwrite d1/de/d13/d21/d25/d2b/d3c/d4a/f93 [0,4194304] 0 2024-03-23T08:23:23.330 INFO:tasks.workunit.client.1.smithi193.stdout:7/431: rmdir d6/d9/d1f/d23/d33/d3c 39 2024-03-23T08:23:23.331 INFO:tasks.workunit.client.1.smithi193.stdout:7/432: dread - d6/d9/d1f/d3e/d65/d70/f7f zero size 2024-03-23T08:23:23.332 INFO:tasks.workunit.client.1.smithi193.stdout:5/537: rename d0/d1c/c5e to d0/d1c/d3b/d4b/cc2 0 2024-03-23T08:23:23.332 INFO:tasks.workunit.client.1.smithi193.stdout:4/523: getdents d11/d48/d50/d49 0 2024-03-23T08:23:23.333 INFO:tasks.workunit.client.1.smithi193.stdout:5/538: fdatasync d0/d1c/d3b/f94 0 2024-03-23T08:23:23.334 INFO:tasks.workunit.client.1.smithi193.stdout:0/477: mknod db/d11/d14/d21/d33/d69/d9b/ca8 0 2024-03-23T08:23:23.334 INFO:tasks.workunit.client.1.smithi193.stdout:9/534: getdents d2/d7/db/d17/d1f 0 2024-03-23T08:23:23.336 INFO:tasks.workunit.client.1.smithi193.stdout:3/430: rename d1/d22 to d1/de/d13/d96/d99 0 2024-03-23T08:23:23.340 INFO:tasks.workunit.client.1.smithi193.stdout:7/433: creat d6/d9/d4e/d81/f93 x:0 0 0 2024-03-23T08:23:23.340 INFO:tasks.workunit.client.1.smithi193.stdout:9/535: fsync d2/d7/db/f3b 0 2024-03-23T08:23:23.340 INFO:tasks.workunit.client.1.smithi193.stdout:3/431: read d1/de/d13/d96/d99/f2d [3818218,104285] 0 2024-03-23T08:23:23.341 INFO:tasks.workunit.client.1.smithi193.stdout:5/539: creat d0/d1c/d7a/db1/d72/d99/fc3 x:0 0 0 2024-03-23T08:23:23.341 INFO:tasks.workunit.client.1.smithi193.stdout:6/418: rmdir d0/d27/d81/df/d11/d3b/d51/d54 0 2024-03-23T08:23:23.341 INFO:tasks.workunit.client.1.smithi193.stdout:4/524: dread d11/d1a/d33/d44/f54 [0,4194304] 0 2024-03-23T08:23:23.345 INFO:tasks.workunit.client.1.smithi193.stdout:7/434: creat d6/d9/d1f/f94 x:0 0 0 2024-03-23T08:23:23.348 INFO:tasks.workunit.client.1.smithi193.stdout:3/432: rmdir d1/de/d13/d21 39 2024-03-23T08:23:23.349 INFO:tasks.workunit.client.1.smithi193.stdout:8/469: truncate f3 1992349 0 2024-03-23T08:23:23.350 INFO:tasks.workunit.client.1.smithi193.stdout:8/470: read df/f21 [323216,107065] 0 2024-03-23T08:23:23.350 INFO:tasks.workunit.client.1.smithi193.stdout:5/540: rmdir d0/d1c/d7a/db1/d72/d73/d57 39 2024-03-23T08:23:23.351 INFO:tasks.workunit.client.1.smithi193.stdout:5/541: readlink d0/d5a/la6 0 2024-03-23T08:23:23.351 INFO:tasks.workunit.client.1.smithi193.stdout:1/484: sync 2024-03-23T08:23:23.351 INFO:tasks.workunit.client.1.smithi193.stdout:8/471: fsync df/d10/d3e/f5f 0 2024-03-23T08:23:23.351 INFO:tasks.workunit.client.1.smithi193.stdout:1/485: dread - da/d13/f90 zero size 2024-03-23T08:23:23.354 INFO:tasks.workunit.client.1.smithi193.stdout:4/525: rename d11/d19/d22 to d11/d1a/d33/d8f/db8 0 2024-03-23T08:23:23.355 INFO:tasks.workunit.client.1.smithi193.stdout:2/557: truncate d0/d1b/d19/f5c 735709 0 2024-03-23T08:23:23.358 INFO:tasks.workunit.client.1.smithi193.stdout:9/536: symlink d2/d7/db/d17/d4b/d59/d97/d9d/lb3 0 2024-03-23T08:23:23.359 INFO:tasks.workunit.client.1.smithi193.stdout:5/542: creat d0/d1f/fc4 x:0 0 0 2024-03-23T08:23:23.359 INFO:tasks.workunit.client.1.smithi193.stdout:8/472: mknod df/d10/d67/ca0 0 2024-03-23T08:23:23.361 INFO:tasks.workunit.client.1.smithi193.stdout:6/419: dwrite d0/d27/d81/df/d11/d3b/f76 [0,4194304] 0 2024-03-23T08:23:23.364 INFO:tasks.workunit.client.1.smithi193.stdout:9/537: dread - d2/d7/f8e zero size 2024-03-23T08:23:23.364 INFO:tasks.workunit.client.1.smithi193.stdout:1/486: dwrite da/d13/f6f [0,4194304] 0 2024-03-23T08:23:23.364 INFO:tasks.workunit.client.1.smithi193.stdout:2/558: readlink d0/d1b/l51 0 2024-03-23T08:23:23.365 INFO:tasks.workunit.client.1.smithi193.stdout:8/473: chown df/d10/d3e/f6b 182 1 2024-03-23T08:23:23.369 INFO:tasks.workunit.client.1.smithi193.stdout:9/538: chown d2/d7/d31/f40 1 1 2024-03-23T08:23:23.371 INFO:tasks.workunit.client.1.smithi193.stdout:5/543: readlink d0/d1c/d7a/db1/d72/d73/l33 0 2024-03-23T08:23:23.376 INFO:tasks.workunit.client.1.smithi193.stdout:2/559: chown d0/d1b/d14/d66/fa0 0 1 2024-03-23T08:23:23.378 INFO:tasks.workunit.client.1.smithi193.stdout:0/478: dwrite db/d11/d14/d24/f8d [0,4194304] 0 2024-03-23T08:23:23.378 INFO:tasks.workunit.client.1.smithi193.stdout:3/433: creat d1/de/d13/d21/d27/d7c/f9a x:0 0 0 2024-03-23T08:23:23.379 INFO:tasks.workunit.client.1.smithi193.stdout:6/420: write d0/d27/d81/df/d11/d36/d61/d12/f1e [3780674,121829] 0 2024-03-23T08:23:23.379 INFO:tasks.workunit.client.1.smithi193.stdout:8/474: rmdir df/d10/d3e 39 2024-03-23T08:23:23.379 INFO:tasks.workunit.client.1.smithi193.stdout:2/560: fdatasync d0/d1b/d19/f7f 0 2024-03-23T08:23:23.380 INFO:tasks.workunit.client.1.smithi193.stdout:7/435: write d6/d9/d1f/d23/d33/d3c/d41/f79 [1174093,124011] 0 2024-03-23T08:23:23.382 INFO:tasks.workunit.client.1.smithi193.stdout:5/544: creat d0/d1c/d7a/db1/d72/d99/fc5 x:0 0 0 2024-03-23T08:23:23.384 INFO:tasks.workunit.client.1.smithi193.stdout:7/436: chown d6/d9/c37 410911228 1 2024-03-23T08:23:23.385 INFO:tasks.workunit.client.1.smithi193.stdout:9/539: rename d2/d7/d31/d5f/d73 to d2/d7/db/d1b/d43/d71/db4 0 2024-03-23T08:23:23.386 INFO:tasks.workunit.client.1.smithi193.stdout:8/475: mknod df/d10/d19/ca1 0 2024-03-23T08:23:23.387 INFO:tasks.workunit.client.1.smithi193.stdout:7/437: write d6/d9/d1f/d23/d33/d38/d3b/f5a [475353,124153] 0 2024-03-23T08:23:23.388 INFO:tasks.workunit.client.1.smithi193.stdout:9/540: write d2/d7/d31/d5f/f68 [678696,93886] 0 2024-03-23T08:23:23.390 INFO:tasks.workunit.client.1.smithi193.stdout:5/545: mkdir d0/d1f/dc6 0 2024-03-23T08:23:23.390 INFO:tasks.workunit.client.1.smithi193.stdout:2/561: fsync d0/d1b/d14/d25/f5a 0 2024-03-23T08:23:23.391 INFO:tasks.workunit.client.1.smithi193.stdout:8/476: chown df/f21 0 1 2024-03-23T08:23:23.392 INFO:tasks.workunit.client.1.smithi193.stdout:6/421: symlink d0/d27/d81/df/d11/d36/d61/d7c/l82 0 2024-03-23T08:23:23.392 INFO:tasks.workunit.client.1.smithi193.stdout:3/434: mkdir d1/de/d13/d21/d50/d9b 0 2024-03-23T08:23:23.393 INFO:tasks.workunit.client.1.smithi193.stdout:1/487: write da/d13/f49 [223512,69914] 0 2024-03-23T08:23:23.393 INFO:tasks.workunit.client.1.smithi193.stdout:7/438: mknod d6/d9/d1f/d4d/c95 0 2024-03-23T08:23:23.394 INFO:tasks.workunit.client.1.smithi193.stdout:4/526: dread d11/d1a/d33/d42/d52/f6c [0,4194304] 0 2024-03-23T08:23:23.396 INFO:tasks.workunit.client.1.smithi193.stdout:8/477: chown df/d62/l65 5 1 2024-03-23T08:23:23.396 INFO:tasks.workunit.client.1.smithi193.stdout:9/541: symlink d2/d7/db/d1b/d5c/lb5 0 2024-03-23T08:23:23.397 INFO:tasks.workunit.client.1.smithi193.stdout:4/527: chown d11/d1a/d33/d44/f82 487575 1 2024-03-23T08:23:23.398 INFO:tasks.workunit.client.1.smithi193.stdout:9/542: dread - d2/d7/db/f2e zero size 2024-03-23T08:23:23.399 INFO:tasks.workunit.client.1.smithi193.stdout:4/528: write d11/d1a/d33/d44/f82 [835615,17184] 0 2024-03-23T08:23:23.399 INFO:tasks.workunit.client.1.smithi193.stdout:6/422: rmdir d0/d27/d81/df/d11/d36 39 2024-03-23T08:23:23.401 INFO:tasks.workunit.client.1.smithi193.stdout:3/435: mknod d1/de/d13/d96/d99/c9c 0 2024-03-23T08:23:23.401 INFO:tasks.workunit.client.1.smithi193.stdout:1/488: mknod da/dd/df/d7e/ca3 0 2024-03-23T08:23:23.401 INFO:tasks.workunit.client.1.smithi193.stdout:4/529: chown d11/d1a/d33/d42/f28 116884312 1 2024-03-23T08:23:23.402 INFO:tasks.workunit.client.1.smithi193.stdout:7/439: symlink d6/d9/d1f/d3e/d42/l96 0 2024-03-23T08:23:23.402 INFO:tasks.workunit.client.1.smithi193.stdout:4/530: chown d11/d95 5146 1 2024-03-23T08:23:23.403 INFO:tasks.workunit.client.1.smithi193.stdout:8/478: creat df/d62/fa2 x:0 0 0 2024-03-23T08:23:23.403 INFO:tasks.workunit.client.1.smithi193.stdout:1/489: read da/f12 [1599250,79634] 0 2024-03-23T08:23:23.403 INFO:tasks.workunit.client.1.smithi193.stdout:5/546: mknod d0/d1c/d7a/db1/d72/d73/d57/d6d/cc7 0 2024-03-23T08:23:23.407 INFO:tasks.workunit.client.1.smithi193.stdout:8/479: readlink df/d10/d3e/l7e 0 2024-03-23T08:23:23.407 INFO:tasks.workunit.client.1.smithi193.stdout:1/490: rmdir da/dd/df/d75 39 2024-03-23T08:23:23.410 INFO:tasks.workunit.client.1.smithi193.stdout:9/543: dwrite d2/fc [4194304,4194304] 0 2024-03-23T08:23:23.410 INFO:tasks.workunit.client.1.smithi193.stdout:5/547: creat d0/d1c/d7a/d86/d9f/fc8 x:0 0 0 2024-03-23T08:23:23.411 INFO:tasks.workunit.client.1.smithi193.stdout:7/440: rename d6/d9/d1f/d4d/d6b/l90 to d6/d9/d1f/d23/l97 0 2024-03-23T08:23:23.411 INFO:tasks.workunit.client.1.smithi193.stdout:8/480: write df/f48 [1368526,36254] 0 2024-03-23T08:23:23.415 INFO:tasks.workunit.client.1.smithi193.stdout:2/562: truncate d0/f71 1900220 0 2024-03-23T08:23:23.416 INFO:tasks.workunit.client.1.smithi193.stdout:8/481: readlink df/d4b/d4d/l28 0 2024-03-23T08:23:23.417 INFO:tasks.workunit.client.1.smithi193.stdout:3/436: truncate d1/de/d13/f24 2833899 0 2024-03-23T08:23:23.417 INFO:tasks.workunit.client.1.smithi193.stdout:1/491: write da/dd/df/d7e/d86/f89 [264179,86567] 0 2024-03-23T08:23:23.420 INFO:tasks.workunit.client.1.smithi193.stdout:5/548: truncate d0/fa 778634 0 2024-03-23T08:23:23.420 INFO:tasks.workunit.client.1.smithi193.stdout:4/531: dwrite d11/d48/d50/f5b [4194304,4194304] 0 2024-03-23T08:23:23.421 INFO:tasks.workunit.client.1.smithi193.stdout:7/441: mknod d6/d9/d1f/d23/d33/d3c/d41/c98 0 2024-03-23T08:23:23.421 INFO:tasks.workunit.client.1.smithi193.stdout:6/423: dread d0/d27/d81/df/d11/d36/d61/d12/d21/f29 [0,4194304] 0 2024-03-23T08:23:23.424 INFO:tasks.workunit.client.1.smithi193.stdout:1/492: unlink da/dd/df/f17 0 2024-03-23T08:23:23.425 INFO:tasks.workunit.client.1.smithi193.stdout:5/549: write d0/d1f/fbb [656821,120531] 0 2024-03-23T08:23:23.426 INFO:tasks.workunit.client.1.smithi193.stdout:4/532: dread d11/d1a/d33/d8f/db8/f45 [0,4194304] 0 2024-03-23T08:23:23.433 INFO:tasks.workunit.client.1.smithi193.stdout:4/533: write d11/d95/f9a [790463,25854] 0 2024-03-23T08:23:23.433 INFO:tasks.workunit.client.1.smithi193.stdout:1/493: rmdir da/dd/d8d 0 2024-03-23T08:23:23.436 INFO:tasks.workunit.client.1.smithi193.stdout:8/482: dwrite df/d4b/d4d/d2a/f3c [0,4194304] 0 2024-03-23T08:23:23.440 INFO:tasks.workunit.client.1.smithi193.stdout:2/563: dread d0/d1b/d14/d66/d50/f52 [0,4194304] 0 2024-03-23T08:23:23.440 INFO:tasks.workunit.client.1.smithi193.stdout:1/494: readlink da/dd/l69 0 2024-03-23T08:23:23.444 INFO:tasks.workunit.client.1.smithi193.stdout:4/534: truncate d11/d19/f20 3896842 0 2024-03-23T08:23:23.444 INFO:tasks.workunit.client.1.smithi193.stdout:8/483: stat df/d10/d3e/d54/f89 0 2024-03-23T08:23:23.449 INFO:tasks.workunit.client.1.smithi193.stdout:8/484: mknod df/d10/d3e/d4e/ca3 0 2024-03-23T08:23:23.449 INFO:tasks.workunit.client.1.smithi193.stdout:9/544: dread d2/d7/db/d32/f5a [0,4194304] 0 2024-03-23T08:23:23.450 INFO:tasks.workunit.client.1.smithi193.stdout:2/564: getdents d0/d1b/d77 0 2024-03-23T08:23:23.450 INFO:tasks.workunit.client.1.smithi193.stdout:8/485: write df/d10/d19/f4f [243085,90132] 0 2024-03-23T08:23:23.452 INFO:tasks.workunit.client.1.smithi193.stdout:2/565: stat d0/d1b/d14/d56/da3 0 2024-03-23T08:23:23.453 INFO:tasks.workunit.client.1.smithi193.stdout:4/535: truncate d11/d1a/d33/d42/f29 2348176 0 2024-03-23T08:23:23.455 INFO:tasks.workunit.client.1.smithi193.stdout:7/442: dread d6/d9/d1f/d3e/d42/f60 [0,4194304] 0 2024-03-23T08:23:23.457 INFO:tasks.workunit.client.1.smithi193.stdout:2/566: mkdir d0/d1b/d14/d66/d50/d92/da9 0 2024-03-23T08:23:23.461 INFO:tasks.workunit.client.1.smithi193.stdout:7/443: creat d6/d9/d1f/d4d/d7c/f99 x:0 0 0 2024-03-23T08:23:23.463 INFO:tasks.workunit.client.1.smithi193.stdout:4/536: truncate d11/d1a/d33/d42/d52/d70/f74 99113 0 2024-03-23T08:23:23.464 INFO:tasks.workunit.client.1.smithi193.stdout:7/444: fdatasync d6/f39 0 2024-03-23T08:23:23.466 INFO:tasks.workunit.client.1.smithi193.stdout:2/567: getdents d0 0 2024-03-23T08:23:23.467 INFO:tasks.workunit.client.1.smithi193.stdout:9/545: dwrite d2/d7/db/d7a/d95/fa7 [0,4194304] 0 2024-03-23T08:23:23.467 INFO:tasks.workunit.client.1.smithi193.stdout:7/445: readlink d6/l6e 0 2024-03-23T08:23:23.472 INFO:tasks.workunit.client.1.smithi193.stdout:8/486: dwrite df/d4b/f71 [0,4194304] 0 2024-03-23T08:23:23.474 INFO:tasks.workunit.client.1.smithi193.stdout:4/537: mkdir d11/d2d/d58/d80/db9 0 2024-03-23T08:23:23.482 INFO:tasks.workunit.client.1.smithi193.stdout:8/487: creat df/d63/d84/fa4 x:0 0 0 2024-03-23T08:23:23.483 INFO:tasks.workunit.client.1.smithi193.stdout:5/550: dwrite d0/d1c/d7a/db1/d72/d73/d45/f96 [0,4194304] 0 2024-03-23T08:23:23.483 INFO:tasks.workunit.client.1.smithi193.stdout:4/538: creat d11/d1a/d33/d42/d52/d70/fba x:0 0 0 2024-03-23T08:23:23.486 INFO:tasks.workunit.client.1.smithi193.stdout:1/495: dread da/dd/df/f1d [0,4194304] 0 2024-03-23T08:23:23.488 INFO:tasks.workunit.client.1.smithi193.stdout:5/551: chown d0/d1c/d3b/d81/cbe 1063657320 1 2024-03-23T08:23:23.489 INFO:tasks.workunit.client.1.smithi193.stdout:2/568: dwrite d0/d1b/f9e [0,4194304] 0 2024-03-23T08:23:23.490 INFO:tasks.workunit.client.1.smithi193.stdout:7/446: dread d6/d9/d1f/d23/f45 [0,4194304] 0 2024-03-23T08:23:23.491 INFO:tasks.workunit.client.1.smithi193.stdout:5/552: symlink d0/d1f/lc9 0 2024-03-23T08:23:23.493 INFO:tasks.workunit.client.1.smithi193.stdout:1/496: write da/dd/df/d52/f6e [554653,45574] 0 2024-03-23T08:23:23.494 INFO:tasks.workunit.client.1.smithi193.stdout:7/447: write d6/d9/d1f/d23/d33/d38/d46/d82/f8c [965464,119311] 0 2024-03-23T08:23:23.495 INFO:tasks.workunit.client.1.smithi193.stdout:1/497: write da/d13/d3f/d5a/f8c [646647,11967] 0 2024-03-23T08:23:23.496 INFO:tasks.workunit.client.1.smithi193.stdout:5/553: creat d0/d1f/fca x:0 0 0 2024-03-23T08:23:23.497 INFO:tasks.workunit.client.1.smithi193.stdout:9/546: write d2/d7/f2b [61698,69713] 0 2024-03-23T08:23:23.498 INFO:tasks.workunit.client.1.smithi193.stdout:1/498: dread - da/d13/f90 zero size 2024-03-23T08:23:23.499 INFO:tasks.workunit.client.1.smithi193.stdout:4/539: write d11/d19/f20 [3894961,69281] 0 2024-03-23T08:23:23.499 INFO:tasks.workunit.client.1.smithi193.stdout:5/554: chown d0/d1c/d7a/f7d 56461027 1 2024-03-23T08:23:23.500 INFO:tasks.workunit.client.1.smithi193.stdout:7/448: mknod d6/d9/c9a 0 2024-03-23T08:23:23.501 INFO:tasks.workunit.client.1.smithi193.stdout:1/499: fdatasync da/dd/df/d7e/d7f/f8a 0 2024-03-23T08:23:23.501 INFO:tasks.workunit.client.1.smithi193.stdout:5/555: chown d0/f7 29535214 1 2024-03-23T08:23:23.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/547: write d2/d7/db/d17/f54 [5007743,109039] 0 2024-03-23T08:23:23.505 INFO:tasks.workunit.client.1.smithi193.stdout:4/540: write d11/d95/f9a [1795186,127454] 0 2024-03-23T08:23:23.505 INFO:tasks.workunit.client.1.smithi193.stdout:9/548: mknod d2/d7/db/d7a/cb6 0 2024-03-23T08:23:23.508 INFO:tasks.workunit.client.1.smithi193.stdout:7/449: write d6/d9/d1f/d23/d33/d38/d3b/f5a [416619,23756] 0 2024-03-23T08:23:23.508 INFO:tasks.workunit.client.1.smithi193.stdout:0/479: sync 2024-03-23T08:23:23.509 INFO:tasks.workunit.client.1.smithi193.stdout:1/500: symlink da/d13/la4 0 2024-03-23T08:23:23.509 INFO:tasks.workunit.client.1.smithi193.stdout:4/541: dread - d11/d1a/d33/d42/f71 zero size 2024-03-23T08:23:23.512 INFO:tasks.workunit.client.1.smithi193.stdout:9/549: unlink d2/d7/db/d32/c86 0 2024-03-23T08:23:23.512 INFO:tasks.workunit.client.1.smithi193.stdout:2/569: dwrite d0/d1b/d14/f41 [0,4194304] 0 2024-03-23T08:23:23.512 INFO:tasks.workunit.client.1.smithi193.stdout:5/556: mknod d0/d1f/d93/dc0/ccb 0 2024-03-23T08:23:23.513 INFO:tasks.workunit.client.1.smithi193.stdout:0/480: read - db/d11/d14/d21/d33/d48/f5d zero size 2024-03-23T08:23:23.514 INFO:tasks.workunit.client.1.smithi193.stdout:1/501: write da/f12 [7233514,39973] 0 2024-03-23T08:23:23.515 INFO:tasks.workunit.client.1.smithi193.stdout:4/542: mkdir d11/dbb 0 2024-03-23T08:23:23.515 INFO:tasks.workunit.client.1.smithi193.stdout:2/570: write d0/d1b/d14/f40 [2120247,14407] 0 2024-03-23T08:23:23.519 INFO:tasks.workunit.client.1.smithi193.stdout:2/571: write d0/d1b/f9e [66840,59258] 0 2024-03-23T08:23:23.519 INFO:tasks.workunit.client.1.smithi193.stdout:0/481: write db/d11/d14/d24/d47/d59/f79 [2904510,96894] 0 2024-03-23T08:23:23.522 INFO:tasks.workunit.client.1.smithi193.stdout:1/502: dread da/f12 [0,4194304] 0 2024-03-23T08:23:23.524 INFO:tasks.workunit.client.1.smithi193.stdout:7/450: rename d6/d9/d1f/c58 to d6/d9/d1f/d23/d33/c9b 0 2024-03-23T08:23:23.525 INFO:tasks.workunit.client.1.smithi193.stdout:7/451: read d6/f14 [1715536,36104] 0 2024-03-23T08:23:23.528 INFO:tasks.workunit.client.1.smithi193.stdout:0/482: creat db/d11/d14/d24/d2a/fa9 x:0 0 0 2024-03-23T08:23:23.528 INFO:tasks.workunit.client.1.smithi193.stdout:5/557: getdents d0/d1c/d3b/d85/d97 0 2024-03-23T08:23:23.530 INFO:tasks.workunit.client.1.smithi193.stdout:4/543: rename d11/d19 to d11/d2d/d58/d8b/dbc 0 2024-03-23T08:23:23.533 INFO:tasks.workunit.client.1.smithi193.stdout:0/483: readlink db/d11/d14/d51/l58 0 2024-03-23T08:23:23.537 INFO:tasks.workunit.client.1.smithi193.stdout:2/572: dwrite d0/d1b/d3b/f46 [0,4194304] 0 2024-03-23T08:23:23.537 INFO:tasks.workunit.client.1.smithi193.stdout:9/550: dread d2/d7/f2b [0,4194304] 0 2024-03-23T08:23:23.537 INFO:tasks.workunit.client.1.smithi193.stdout:0/484: stat db/c30 0 2024-03-23T08:23:23.537 INFO:tasks.workunit.client.1.smithi193.stdout:5/558: rename d0/f11 to d0/d1c/d3b/d85/d97/fcc 0 2024-03-23T08:23:23.537 INFO:tasks.workunit.client.1.smithi193.stdout:7/452: truncate d6/d9/d1f/d23/d33/d38/d3b/f5a 1432546 0 2024-03-23T08:23:23.540 INFO:tasks.workunit.client.1.smithi193.stdout:1/503: dwrite da/d13/d3f/f54 [0,4194304] 0 2024-03-23T08:23:23.550 INFO:tasks.workunit.client.1.smithi193.stdout:9/551: creat d2/d7/db/d17/d4b/d8b/fb7 x:0 0 0 2024-03-23T08:23:23.550 INFO:tasks.workunit.client.1.smithi193.stdout:4/544: dwrite d11/d1a/d33/f39 [4194304,4194304] 0 2024-03-23T08:23:23.550 INFO:tasks.workunit.client.1.smithi193.stdout:1/504: mknod da/dd/df/d7e/d86/d95/ca5 0 2024-03-23T08:23:23.552 INFO:tasks.workunit.client.1.smithi193.stdout:5/559: write d0/d1c/d3b/d4b/f95 [234551,66021] 0 2024-03-23T08:23:23.560 INFO:tasks.workunit.client.1.smithi193.stdout:4/545: write d11/d1a/d33/d44/f4a [2675815,9502] 0 2024-03-23T08:23:23.560 INFO:tasks.workunit.client.1.smithi193.stdout:7/453: dwrite d6/d9/d1f/d23/d33/d3c/d41/f51 [0,4194304] 0 2024-03-23T08:23:23.562 INFO:tasks.workunit.client.1.smithi193.stdout:5/560: mkdir d0/d1f/d93/dcd 0 2024-03-23T08:23:23.563 INFO:tasks.workunit.client.1.smithi193.stdout:1/505: creat da/dd/fa6 x:0 0 0 2024-03-23T08:23:23.565 INFO:tasks.workunit.client.1.smithi193.stdout:7/454: unlink d6/f8 0 2024-03-23T08:23:23.565 INFO:tasks.workunit.client.1.smithi193.stdout:0/485: dwrite db/d11/d14/d3b/f63 [0,4194304] 0 2024-03-23T08:23:23.566 INFO:tasks.workunit.client.1.smithi193.stdout:4/546: creat d11/d8e/dac/fbd x:0 0 0 2024-03-23T08:23:23.573 INFO:tasks.workunit.client.1.smithi193.stdout:9/552: dwrite d2/d7/db/d7a/d95/da9/fac [0,4194304] 0 2024-03-23T08:23:23.573 INFO:tasks.workunit.client.1.smithi193.stdout:0/486: write db/d11/d14/f95 [331532,101702] 0 2024-03-23T08:23:23.574 INFO:tasks.workunit.client.1.smithi193.stdout:4/547: write d11/d1a/d33/f39 [4497518,112021] 0 2024-03-23T08:23:23.575 INFO:tasks.workunit.client.1.smithi193.stdout:1/506: truncate da/dd/df/d7e/d86/f89 1646648 0 2024-03-23T08:23:23.575 INFO:tasks.workunit.client.1.smithi193.stdout:9/553: chown d2/d7/d31 25 1 2024-03-23T08:23:23.576 INFO:tasks.workunit.client.1.smithi193.stdout:5/561: rmdir d0/d1f/dc6 0 2024-03-23T08:23:23.577 INFO:tasks.workunit.client.1.smithi193.stdout:9/554: read - d2/f8f zero size 2024-03-23T08:23:23.580 INFO:tasks.workunit.client.1.smithi193.stdout:4/548: write f9 [3220162,76463] 0 2024-03-23T08:23:23.582 INFO:tasks.workunit.client.1.smithi193.stdout:0/487: creat db/d11/d14/d24/d47/d59/faa x:0 0 0 2024-03-23T08:23:23.583 INFO:tasks.workunit.client.1.smithi193.stdout:1/507: write da/dd/f14 [1063335,89992] 0 2024-03-23T08:23:23.584 INFO:tasks.workunit.client.1.smithi193.stdout:9/555: creat d2/d7/db/d7a/d95/da9/fb8 x:0 0 0 2024-03-23T08:23:23.586 INFO:tasks.workunit.client.1.smithi193.stdout:7/455: dread d6/d9/d1f/d3e/d65/d70/f73 [0,4194304] 0 2024-03-23T08:23:23.588 INFO:tasks.workunit.client.1.smithi193.stdout:7/456: chown d6/d9/d1f/d23/d33/d3c/d41/f79 817334273 1 2024-03-23T08:23:23.588 INFO:tasks.workunit.client.1.smithi193.stdout:4/549: rename d11/d48 to d11/d1a/d33/d42/db2/dbe 0 2024-03-23T08:23:23.593 INFO:tasks.workunit.client.1.smithi193.stdout:7/457: rename d6/d9/d1f/d23/d33/d38/d3b/f52 to d6/d9/d1f/d23/d33/d38/d66/f9c 0 2024-03-23T08:23:23.595 INFO:tasks.workunit.client.1.smithi193.stdout:7/458: chown d6/f14 4651085 1 2024-03-23T08:23:23.597 INFO:tasks.workunit.client.1.smithi193.stdout:0/488: getdents db/d11/d14/d24/d47/d59 0 2024-03-23T08:23:23.597 INFO:tasks.workunit.client.1.smithi193.stdout:5/562: truncate d0/d1c/d7a/db1/d72/d73/f44 3565304 0 2024-03-23T08:23:23.599 INFO:tasks.workunit.client.1.smithi193.stdout:4/550: write d11/d1a/d33/d8f/db8/f45 [943004,40707] 0 2024-03-23T08:23:23.602 INFO:tasks.workunit.client.1.smithi193.stdout:5/563: write d0/f1a [1128937,7304] 0 2024-03-23T08:23:23.602 INFO:tasks.workunit.client.1.smithi193.stdout:9/556: dwrite d2/d7/f2b [0,4194304] 0 2024-03-23T08:23:23.602 INFO:tasks.workunit.client.1.smithi193.stdout:1/508: dwrite da/dd/f62 [0,4194304] 0 2024-03-23T08:23:23.605 INFO:tasks.workunit.client.1.smithi193.stdout:0/489: creat db/d11/d14/d21/d33/d69/fab x:0 0 0 2024-03-23T08:23:23.624 INFO:tasks.workunit.client.1.smithi193.stdout:3/437: sync 2024-03-23T08:23:23.640 INFO:tasks.workunit.client.1.smithi193.stdout:7/459: dwrite d6/d9/d1f/d3e/d42/f59 [0,4194304] 0 2024-03-23T08:23:23.641 INFO:tasks.workunit.client.1.smithi193.stdout:0/490: write db/d11/d14/d24/d2a/d4b/f84 [1024771,56352] 0 2024-03-23T08:23:23.642 INFO:tasks.workunit.client.1.smithi193.stdout:7/460: stat d6/l74 0 2024-03-23T08:23:23.642 INFO:tasks.workunit.client.1.smithi193.stdout:4/551: dwrite d11/d27/d85/f8a [0,4194304] 0 2024-03-23T08:23:23.643 INFO:tasks.workunit.client.1.smithi193.stdout:0/491: truncate db/f87 1190009 0 2024-03-23T08:23:23.645 INFO:tasks.workunit.client.1.smithi193.stdout:4/552: mknod d11/d2d/d58/d80/d9f/cbf 0 2024-03-23T08:23:23.645 INFO:tasks.workunit.client.1.smithi193.stdout:7/461: read - d6/d9/d1f/d23/d33/d38/d46/f61 zero size 2024-03-23T08:23:23.648 INFO:tasks.workunit.client.1.smithi193.stdout:7/462: rmdir d6/d9 39 2024-03-23T08:23:23.648 INFO:tasks.workunit.client.1.smithi193.stdout:3/438: dread d1/de/d13/d21/d27/f1c [0,4194304] 0 2024-03-23T08:23:23.650 INFO:tasks.workunit.client.1.smithi193.stdout:0/492: unlink db/d11/d14/d24/d2a/c44 0 2024-03-23T08:23:23.654 INFO:tasks.workunit.client.1.smithi193.stdout:7/463: mkdir d6/d9/d1f/d9d 0 2024-03-23T08:23:23.656 INFO:tasks.workunit.client.1.smithi193.stdout:4/553: dwrite d11/d2d/d58/d8b/dbc/f94 [0,4194304] 0 2024-03-23T08:23:23.663 INFO:tasks.workunit.client.1.smithi193.stdout:7/464: mknod d6/d9/d1f/d23/d33/d38/d3b/d6c/c9e 0 2024-03-23T08:23:23.663 INFO:tasks.workunit.client.1.smithi193.stdout:4/554: mknod d11/d1a/d33/d42/d52/d70/cc0 0 2024-03-23T08:23:23.663 INFO:tasks.workunit.client.1.smithi193.stdout:4/555: chown f9 0 1 2024-03-23T08:23:23.664 INFO:tasks.workunit.client.1.smithi193.stdout:7/465: write d6/d9/d1f/d23/d33/d38/d3b/f5a [389013,82438] 0 2024-03-23T08:23:23.666 INFO:tasks.workunit.client.1.smithi193.stdout:4/556: creat d11/d2d/d3a/fc1 x:0 0 0 2024-03-23T08:23:23.669 INFO:tasks.workunit.client.1.smithi193.stdout:7/466: link d6/d9/d1f/d23/d33/f6d d6/d9/d1f/d3e/d65/d7d/f9f 0 2024-03-23T08:23:23.669 INFO:tasks.workunit.client.1.smithi193.stdout:5/564: dread d0/d5a/f75 [0,4194304] 0 2024-03-23T08:23:23.673 INFO:tasks.workunit.client.1.smithi193.stdout:5/565: creat d0/d1c/d3b/fce x:0 0 0 2024-03-23T08:23:23.673 INFO:tasks.workunit.client.1.smithi193.stdout:7/467: fsync d6/d9/d1f/d23/d33/d38/d46/f5b 0 2024-03-23T08:23:23.673 INFO:tasks.workunit.client.1.smithi193.stdout:4/557: dwrite d11/d2d/d3a/f76 [0,4194304] 0 2024-03-23T08:23:23.674 INFO:tasks.workunit.client.1.smithi193.stdout:3/439: truncate d1/de/d13/d21/d25/d34/f47 201638 0 2024-03-23T08:23:23.674 INFO:tasks.workunit.client.1.smithi193.stdout:0/493: truncate db/d11/d14/d3b/f63 47445 0 2024-03-23T08:23:23.676 INFO:tasks.workunit.client.1.smithi193.stdout:7/468: write d6/d9/d1f/f30 [5910687,26615] 0 2024-03-23T08:23:23.676 INFO:tasks.workunit.client.1.smithi193.stdout:5/566: fdatasync d0/d1c/d7a/db1/d72/d73/d57/d6d/d8e/fa5 0 2024-03-23T08:23:23.677 INFO:tasks.workunit.client.1.smithi193.stdout:0/494: chown db/d11/d14/d24/d2a/c70 0 1 2024-03-23T08:23:23.677 INFO:tasks.workunit.client.1.smithi193.stdout:7/469: readlink d6/d9/l25 0 2024-03-23T08:23:23.679 INFO:tasks.workunit.client.1.smithi193.stdout:5/567: rmdir d0/d1c/d7a 39 2024-03-23T08:23:23.680 INFO:tasks.workunit.client.1.smithi193.stdout:3/440: getdents d1/de/d13/d21/d25 0 2024-03-23T08:23:23.684 INFO:tasks.workunit.client.1.smithi193.stdout:0/495: rename db/d11/d14/d24/d2a/d4d/f66 to db/d11/d14/d24/d47/fac 0 2024-03-23T08:23:23.685 INFO:tasks.workunit.client.1.smithi193.stdout:5/568: creat d0/d5a/fcf x:0 0 0 2024-03-23T08:23:23.692 INFO:tasks.workunit.client.1.smithi193.stdout:5/569: chown d0/d1c/d7a/db1/d72/d73/d57/d6d/d8e/fab 3 1 2024-03-23T08:23:23.693 INFO:tasks.workunit.client.1.smithi193.stdout:5/570: chown d0/d1f/d93/dcd 61278244 1 2024-03-23T08:23:23.694 INFO:tasks.workunit.client.1.smithi193.stdout:5/571: fsync d0/d1c/d3b/d4b/f84 0 2024-03-23T08:23:23.695 INFO:tasks.workunit.client.1.smithi193.stdout:0/496: truncate db/d11/d14/d51/f99 203094 0 2024-03-23T08:23:23.697 INFO:tasks.workunit.client.1.smithi193.stdout:7/470: dwrite d6/d9/d1f/d23/f2e [0,4194304] 0 2024-03-23T08:23:23.701 INFO:tasks.workunit.client.1.smithi193.stdout:0/497: creat db/d11/d14/fad x:0 0 0 2024-03-23T08:23:23.702 INFO:tasks.workunit.client.1.smithi193.stdout:8/488: sync 2024-03-23T08:23:23.702 INFO:tasks.workunit.client.1.smithi193.stdout:6/424: sync 2024-03-23T08:23:23.702 INFO:tasks.workunit.client.1.smithi193.stdout:2/573: sync 2024-03-23T08:23:23.705 INFO:tasks.workunit.client.1.smithi193.stdout:7/471: rename d6/d9/d1f/d23/d33/d38/d3b/d6c/f77 to d6/d9/d1f/d9d/fa0 0 2024-03-23T08:23:23.706 INFO:tasks.workunit.client.1.smithi193.stdout:7/472: chown d6/d9/c13 2154 1 2024-03-23T08:23:23.707 INFO:tasks.workunit.client.1.smithi193.stdout:5/572: link d0/d1f/d93/cba d0/d1c/cd0 0 2024-03-23T08:23:23.710 INFO:tasks.workunit.client.1.smithi193.stdout:0/498: dwrite db/d11/d14/d24/f8d [0,4194304] 0 2024-03-23T08:23:23.711 INFO:tasks.workunit.client.1.smithi193.stdout:6/425: mknod d0/d27/d81/df/c83 0 2024-03-23T08:23:23.713 INFO:tasks.workunit.client.1.smithi193.stdout:7/473: rmdir d6/d9/d1f/d23/d33/d38/d3b 39 2024-03-23T08:23:23.713 INFO:tasks.workunit.client.1.smithi193.stdout:7/474: fdatasync d6/f32 0 2024-03-23T08:23:23.714 INFO:tasks.workunit.client.1.smithi193.stdout:2/574: mknod d0/d1b/d14/d66/caa 0 2024-03-23T08:23:23.714 INFO:tasks.workunit.client.1.smithi193.stdout:2/575: dread - d0/d28/fa8 zero size 2024-03-23T08:23:23.715 INFO:tasks.workunit.client.1.smithi193.stdout:5/573: creat d0/d1f/d93/dc0/fd1 x:0 0 0 2024-03-23T08:23:23.715 INFO:tasks.workunit.client.1.smithi193.stdout:2/576: fsync d0/d28/f47 0 2024-03-23T08:23:23.717 INFO:tasks.workunit.client.1.smithi193.stdout:8/489: link df/d10/d67/f7d df/d10/d19/fa5 0 2024-03-23T08:23:23.719 INFO:tasks.workunit.client.1.smithi193.stdout:0/499: write db/d11/d14/f1f [28391,22111] 0 2024-03-23T08:23:23.720 INFO:tasks.workunit.client.1.smithi193.stdout:6/426: creat d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f84 x:0 0 0 2024-03-23T08:23:23.763 INFO:tasks.workunit.client.1.smithi193.stdout:7/475: mkdir d6/d9/d1f/d9d/da1 0 2024-03-23T08:23:23.764 INFO:tasks.workunit.client.1.smithi193.stdout:8/490: creat df/d4b/d4d/d9c/fa6 x:0 0 0 2024-03-23T08:23:23.764 INFO:tasks.workunit.client.1.smithi193.stdout:2/577: write d0/d1b/d14/f45 [506496,36329] 0 2024-03-23T08:23:23.764 INFO:tasks.workunit.client.1.smithi193.stdout:6/427: write d0/d27/d81/df/d11/d36/f6e [73647,1927] 0 2024-03-23T08:23:23.766 INFO:tasks.workunit.client.1.smithi193.stdout:7/476: write d6/d9/d1f/d3e/f5e [823976,68468] 0 2024-03-23T08:23:23.767 INFO:tasks.workunit.client.1.smithi193.stdout:5/574: mkdir d0/d1c/d3b/dbc/dd2 0 2024-03-23T08:23:23.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/500: mknod db/d11/d14/d24/cae 0 2024-03-23T08:23:23.771 INFO:tasks.workunit.client.1.smithi193.stdout:7/477: creat d6/d9/d1f/d4d/fa2 x:0 0 0 2024-03-23T08:23:23.772 INFO:tasks.workunit.client.1.smithi193.stdout:6/428: mknod d0/d27/d81/df/d11/d36/d61/d75/c85 0 2024-03-23T08:23:23.773 INFO:tasks.workunit.client.1.smithi193.stdout:5/575: link d0/d1f/la9 d0/d1c/d7a/db1/d72/d73/d57/d6d/ld3 0 2024-03-23T08:23:23.774 INFO:tasks.workunit.client.1.smithi193.stdout:0/501: write db/d11/d14/d21/d33/d48/d85/fa7 [503780,96375] 0 2024-03-23T08:23:23.778 INFO:tasks.workunit.client.1.smithi193.stdout:2/578: dwrite d0/d28/d33/f70 [0,4194304] 0 2024-03-23T08:23:23.778 INFO:tasks.workunit.client.1.smithi193.stdout:0/502: symlink db/d52/laf 0 2024-03-23T08:23:23.779 INFO:tasks.workunit.client.1.smithi193.stdout:7/478: write d6/d9/d1f/d23/d33/d38/d3b/d6c/f8d [582417,104591] 0 2024-03-23T08:23:23.781 INFO:tasks.workunit.client.1.smithi193.stdout:8/491: dwrite df/d10/d3e/d4e/f5b [0,4194304] 0 2024-03-23T08:23:23.784 INFO:tasks.workunit.client.1.smithi193.stdout:5/576: write d0/d1c/d7a/db1/d72/d73/d57/d6d/fb8 [938653,31776] 0 2024-03-23T08:23:23.787 INFO:tasks.workunit.client.1.smithi193.stdout:0/503: stat db/d11/d14/d24/d47/d59/faa 0 2024-03-23T08:23:23.787 INFO:tasks.workunit.client.1.smithi193.stdout:8/492: unlink df/d4b/d4d/l7c 0 2024-03-23T08:23:23.789 INFO:tasks.workunit.client.1.smithi193.stdout:0/504: mknod db/d52/cb0 0 2024-03-23T08:23:23.793 INFO:tasks.workunit.client.1.smithi193.stdout:8/493: truncate df/d10/d19/fa5 593375 0 2024-03-23T08:23:23.797 INFO:tasks.workunit.client.1.smithi193.stdout:0/505: mknod db/d11/d14/d21/d33/d69/cb1 0 2024-03-23T08:23:23.797 INFO:tasks.workunit.client.1.smithi193.stdout:8/494: symlink df/d10/la7 0 2024-03-23T08:23:23.798 INFO:tasks.workunit.client.1.smithi193.stdout:5/577: dread d0/d1c/d3b/d4b/f84 [0,4194304] 0 2024-03-23T08:23:23.799 INFO:tasks.workunit.client.1.smithi193.stdout:7/479: dwrite d6/d9/f78 [0,4194304] 0 2024-03-23T08:23:23.800 INFO:tasks.workunit.client.1.smithi193.stdout:6/429: dwrite d0/d27/d81/df/d11/d36/d61/d12/d21/f73 [4194304,4194304] 0 2024-03-23T08:23:23.801 INFO:tasks.workunit.client.1.smithi193.stdout:9/557: sync 2024-03-23T08:23:23.801 INFO:tasks.workunit.client.1.smithi193.stdout:8/495: write df/d63/d84/fa4 [1011271,24451] 0 2024-03-23T08:23:23.804 INFO:tasks.workunit.client.1.smithi193.stdout:7/480: read - d6/d9/d1f/d3e/d65/d70/f7f zero size 2024-03-23T08:23:23.804 INFO:tasks.workunit.client.1.smithi193.stdout:9/558: fdatasync d2/d7/db/f3b 0 2024-03-23T08:23:23.804 INFO:tasks.workunit.client.1.smithi193.stdout:0/506: creat db/d11/d14/d21/d33/d69/d88/fb2 x:0 0 0 2024-03-23T08:23:23.807 INFO:tasks.workunit.client.1.smithi193.stdout:6/430: chown d0/d27/d81/df/d11/d3b/d6c/f7d 85650792 1 2024-03-23T08:23:23.807 INFO:tasks.workunit.client.1.smithi193.stdout:0/507: chown db/d11/d14/d21/d33/d69/d88 27966009 1 2024-03-23T08:23:23.807 INFO:tasks.workunit.client.1.smithi193.stdout:8/496: unlink df/d62/d8b/f96 0 2024-03-23T08:23:23.808 INFO:tasks.workunit.client.1.smithi193.stdout:7/481: chown d6/d9/d1f/d23/l97 506 1 2024-03-23T08:23:23.808 INFO:tasks.workunit.client.1.smithi193.stdout:9/559: symlink d2/d7/db/d17/d33/d70/lb9 0 2024-03-23T08:23:23.809 INFO:tasks.workunit.client.1.smithi193.stdout:0/508: readlink db/d52/laf 0 2024-03-23T08:23:23.810 INFO:tasks.workunit.client.1.smithi193.stdout:8/497: readlink df/d10/d67/l98 0 2024-03-23T08:23:23.812 INFO:tasks.workunit.client.1.smithi193.stdout:7/482: write d6/d9/d1f/d23/d33/d38/d3b/f55 [4657342,41498] 0 2024-03-23T08:23:23.814 INFO:tasks.workunit.client.1.smithi193.stdout:6/431: rename d0/d27/d81/df/d11/d36/d61/d12/f60 to d0/d27/d81/f86 0 2024-03-23T08:23:23.817 INFO:tasks.workunit.client.1.smithi193.stdout:8/498: unlink df/d63/d84/fa4 0 2024-03-23T08:23:23.817 INFO:tasks.workunit.client.1.smithi193.stdout:6/432: chown d0/d27/c78 1629010 1 2024-03-23T08:23:23.821 INFO:tasks.workunit.client.1.smithi193.stdout:9/560: getdents d2/d7 0 2024-03-23T08:23:23.822 INFO:tasks.workunit.client.1.smithi193.stdout:8/499: mkdir df/d63/d84/da8 0 2024-03-23T08:23:23.822 INFO:tasks.workunit.client.1.smithi193.stdout:0/509: link db/l2f db/d11/d14/lb3 0 2024-03-23T08:23:23.824 INFO:tasks.workunit.client.1.smithi193.stdout:7/483: dwrite d6/d9/d1f/d4d/d7c/f99 [0,4194304] 0 2024-03-23T08:23:23.825 INFO:tasks.workunit.client.1.smithi193.stdout:9/561: symlink d2/d7/db/d17/d4b/lba 0 2024-03-23T08:23:23.826 INFO:tasks.workunit.client.1.smithi193.stdout:6/433: mkdir d0/d27/d81/d87 0 2024-03-23T08:23:23.828 INFO:tasks.workunit.client.1.smithi193.stdout:8/500: truncate df/f21 180331 0 2024-03-23T08:23:23.829 INFO:tasks.workunit.client.1.smithi193.stdout:8/501: read df/d4b/d4d/d2a/f2b [220886,49306] 0 2024-03-23T08:23:23.830 INFO:tasks.workunit.client.1.smithi193.stdout:0/510: truncate db/d11/d14/d24/f25 341647 0 2024-03-23T08:23:23.831 INFO:tasks.workunit.client.1.smithi193.stdout:2/579: truncate d0/d1b/d3b/f46 2810735 0 2024-03-23T08:23:23.832 INFO:tasks.workunit.client.1.smithi193.stdout:9/562: creat d2/d7/db/d1b/fbb x:0 0 0 2024-03-23T08:23:23.834 INFO:tasks.workunit.client.1.smithi193.stdout:0/511: dread db/d11/d14/d51/f56 [0,4194304] 0 2024-03-23T08:23:23.836 INFO:tasks.workunit.client.1.smithi193.stdout:8/502: rename df/d10/d19/f26 to df/d10/d67/fa9 0 2024-03-23T08:23:23.838 INFO:tasks.workunit.client.1.smithi193.stdout:8/503: write df/d10/d3e/d54/f89 [1053378,73422] 0 2024-03-23T08:23:23.838 INFO:tasks.workunit.client.1.smithi193.stdout:2/580: write d0/d1b/f8 [7943761,36934] 0 2024-03-23T08:23:23.847 INFO:tasks.workunit.client.1.smithi193.stdout:6/434: creat d0/d27/d81/df/d11/d3b/d6c/f88 x:0 0 0 2024-03-23T08:23:23.849 INFO:tasks.workunit.client.1.smithi193.stdout:2/581: chown d0/d1b/l38 605 1 2024-03-23T08:23:23.849 INFO:tasks.workunit.client.1.smithi193.stdout:8/504: creat df/d10/d3e/d54/faa x:0 0 0 2024-03-23T08:23:23.852 INFO:tasks.workunit.client.1.smithi193.stdout:8/505: chown df/d46/f47 1551744 1 2024-03-23T08:23:23.852 INFO:tasks.workunit.client.1.smithi193.stdout:7/484: getdents d6/d9 0 2024-03-23T08:23:23.854 INFO:tasks.workunit.client.1.smithi193.stdout:2/582: dread - d0/d1b/d14/d66/d50/d8f/f90 zero size 2024-03-23T08:23:23.855 INFO:tasks.workunit.client.1.smithi193.stdout:6/435: rename d0/d27/d81/df/d11/d36/d61/d3a to d0/d27/d81/df/d11/d36/d61/d3a/d7a/d89 22 2024-03-23T08:23:23.857 INFO:tasks.workunit.client.1.smithi193.stdout:9/563: dread d2/f98 [0,4194304] 0 2024-03-23T08:23:23.857 INFO:tasks.workunit.client.1.smithi193.stdout:7/485: dread - d6/d9/d1f/d23/d33/d38/d66/f9c zero size 2024-03-23T08:23:23.857 INFO:tasks.workunit.client.1.smithi193.stdout:8/506: dread df/d63/f68 [0,4194304] 0 2024-03-23T08:23:23.858 INFO:tasks.workunit.client.1.smithi193.stdout:0/512: dwrite db/d11/d14/d24/d2a/f2e [0,4194304] 0 2024-03-23T08:23:23.860 INFO:tasks.workunit.client.1.smithi193.stdout:8/507: fsync df/d10/d19/f4f 0 2024-03-23T08:23:23.861 INFO:tasks.workunit.client.1.smithi193.stdout:6/436: chown d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f84 351626794 1 2024-03-23T08:23:23.864 INFO:tasks.workunit.client.1.smithi193.stdout:9/564: creat d2/d7/db/d1b/d43/d71/db4/d78/fbc x:0 0 0 2024-03-23T08:23:23.864 INFO:tasks.workunit.client.1.smithi193.stdout:0/513: mknod db/d11/d14/d21/d33/d69/d9b/cb4 0 2024-03-23T08:23:23.865 INFO:tasks.workunit.client.1.smithi193.stdout:8/508: symlink df/d10/d67/d7f/lab 0 2024-03-23T08:23:23.868 INFO:tasks.workunit.client.1.smithi193.stdout:9/565: mknod d2/d7/db/d17/cbd 0 2024-03-23T08:23:23.870 INFO:tasks.workunit.client.1.smithi193.stdout:2/583: dwrite d0/d1b/d14/d66/d50/d92/f9a [0,4194304] 0 2024-03-23T08:23:23.870 INFO:tasks.workunit.client.1.smithi193.stdout:6/437: rename d0/d27/d81/df/d11/d36/f68 to d0/d27/d81/df/d11/d36/d61/d3a/d7a/f8a 0 2024-03-23T08:23:23.872 INFO:tasks.workunit.client.1.smithi193.stdout:8/509: write df/d10/d67/fa9 [886608,9774] 0 2024-03-23T08:23:23.872 INFO:tasks.workunit.client.1.smithi193.stdout:7/486: dread d6/d9/d1f/d23/d33/d38/d3b/f48 [0,4194304] 0 2024-03-23T08:23:23.873 INFO:tasks.workunit.client.1.smithi193.stdout:0/514: creat db/d11/d14/d21/d33/d69/d9b/fb5 x:0 0 0 2024-03-23T08:23:23.880 INFO:tasks.workunit.client.1.smithi193.stdout:0/515: symlink db/lb6 0 2024-03-23T08:23:23.880 INFO:tasks.workunit.client.1.smithi193.stdout:7/487: chown d6/d9/d1f/d23/d33/d38/d3b/d6c 13756424 1 2024-03-23T08:23:23.880 INFO:tasks.workunit.client.1.smithi193.stdout:0/516: chown db/d11/d14/d24/d2a/fa9 0 1 2024-03-23T08:23:23.882 INFO:tasks.workunit.client.1.smithi193.stdout:6/438: creat d0/f8b x:0 0 0 2024-03-23T08:23:23.883 INFO:tasks.workunit.client.1.smithi193.stdout:7/488: creat d6/d9/d1f/d23/d33/d38/d66/fa3 x:0 0 0 2024-03-23T08:23:23.884 INFO:tasks.workunit.client.1.smithi193.stdout:0/517: fdatasync db/d11/d14/d24/f25 0 2024-03-23T08:23:23.886 INFO:tasks.workunit.client.1.smithi193.stdout:6/439: symlink d0/d27/d81/df/d11/d36/d61/d7c/d69/l8c 0 2024-03-23T08:23:23.886 INFO:tasks.workunit.client.1.smithi193.stdout:0/518: readlink db/l2f 0 2024-03-23T08:23:23.887 INFO:tasks.workunit.client.1.smithi193.stdout:7/489: creat d6/d9/d1f/fa4 x:0 0 0 2024-03-23T08:23:23.888 INFO:tasks.workunit.client.1.smithi193.stdout:6/440: creat d0/d27/d81/df/d11/d56/f8d x:0 0 0 2024-03-23T08:23:23.889 INFO:tasks.workunit.client.1.smithi193.stdout:0/519: write db/d11/d14/d24/d2a/f83 [1108030,72616] 0 2024-03-23T08:23:23.889 INFO:tasks.workunit.client.1.smithi193.stdout:1/509: sync 2024-03-23T08:23:23.889 INFO:tasks.workunit.client.1.smithi193.stdout:4/558: sync 2024-03-23T08:23:23.891 INFO:tasks.workunit.client.1.smithi193.stdout:9/566: dread d2/d7/d31/d3c/f6e [0,4194304] 0 2024-03-23T08:23:23.894 INFO:tasks.workunit.client.1.smithi193.stdout:4/559: write d11/d1a/d33/d8f/db8/f25 [4032230,714] 0 2024-03-23T08:23:23.895 INFO:tasks.workunit.client.1.smithi193.stdout:7/490: link d6/d9/d1f/d23/d33/d3c/f4c d6/d9/fa5 0 2024-03-23T08:23:23.895 INFO:tasks.workunit.client.1.smithi193.stdout:4/560: chown d11/d1a/d33/d8f 59803703 1 2024-03-23T08:23:23.896 INFO:tasks.workunit.client.1.smithi193.stdout:6/441: dread d0/d27/d81/df/d11/d36/f6e [0,4194304] 0 2024-03-23T08:23:23.898 INFO:tasks.workunit.client.1.smithi193.stdout:1/510: link da/dd/fa6 da/d13/d3f/fa7 0 2024-03-23T08:23:23.899 INFO:tasks.workunit.client.1.smithi193.stdout:2/584: dread d0/d1b/d14/d25/f29 [0,4194304] 0 2024-03-23T08:23:23.899 INFO:tasks.workunit.client.1.smithi193.stdout:1/511: dread - da/d13/f90 zero size 2024-03-23T08:23:23.901 INFO:tasks.workunit.client.1.smithi193.stdout:0/520: chown db/d11/d14/d24/d2a/c46 4451109 1 2024-03-23T08:23:23.905 INFO:tasks.workunit.client.1.smithi193.stdout:9/567: creat d2/d7/d31/d3c/fbe x:0 0 0 2024-03-23T08:23:23.906 INFO:tasks.workunit.client.1.smithi193.stdout:8/510: dwrite df/d4b/d4d/f69 [0,4194304] 0 2024-03-23T08:23:23.906 INFO:tasks.workunit.client.1.smithi193.stdout:1/512: chown da/dd/df/d7e/d7f/f8a 2 1 2024-03-23T08:23:23.909 INFO:tasks.workunit.client.1.smithi193.stdout:6/442: dread d0/d27/d81/f86 [0,4194304] 0 2024-03-23T08:23:23.910 INFO:tasks.workunit.client.1.smithi193.stdout:3/441: sync 2024-03-23T08:23:23.912 INFO:tasks.workunit.client.1.smithi193.stdout:0/521: unlink db/d11/d14/d24/d2a/f2e 0 2024-03-23T08:23:23.914 INFO:tasks.workunit.client.1.smithi193.stdout:9/568: symlink d2/d7/d31/d5f/lbf 0 2024-03-23T08:23:23.914 INFO:tasks.workunit.client.1.smithi193.stdout:1/513: write da/d13/d3f/d5a/d76/d77/f6c [1033338,114990] 0 2024-03-23T08:23:23.917 INFO:tasks.workunit.client.1.smithi193.stdout:0/522: write db/d11/d14/f1f [1564572,29927] 0 2024-03-23T08:23:23.921 INFO:tasks.workunit.client.1.smithi193.stdout:9/569: creat d2/d7/db/d7a/d95/fc0 x:0 0 0 2024-03-23T08:23:23.972 INFO:tasks.workunit.client.1.smithi193.stdout:7/491: rename d6/d9/d1f/d4d/c95 to d6/d9/d1f/d23/d69/ca6 0 2024-03-23T08:23:23.974 INFO:tasks.workunit.client.1.smithi193.stdout:8/511: dwrite df/f2c [0,4194304] 0 2024-03-23T08:23:23.974 INFO:tasks.workunit.client.1.smithi193.stdout:4/561: dwrite d11/d1a/d33/d42/d52/f3f [0,4194304] 0 2024-03-23T08:23:23.976 INFO:tasks.workunit.client.1.smithi193.stdout:9/570: write d2/d7/db/f2e [155696,109926] 0 2024-03-23T08:23:23.976 INFO:tasks.workunit.client.1.smithi193.stdout:2/585: dread d0/d1b/d3b/f7a [0,4194304] 0 2024-03-23T08:23:23.979 INFO:tasks.workunit.client.1.smithi193.stdout:3/442: rename d1/de/d13/d21/d59/d60/c73 to d1/de/d13/d21/d27/d7c/c9d 0 2024-03-23T08:23:23.979 INFO:tasks.workunit.client.1.smithi193.stdout:0/523: creat db/d11/d14/d24/d47/d59/d8c/fb7 x:0 0 0 2024-03-23T08:23:23.980 INFO:tasks.workunit.client.1.smithi193.stdout:4/562: chown d11/d1a/d33/d42/d52/f87 54492794 1 2024-03-23T08:23:23.982 INFO:tasks.workunit.client.1.smithi193.stdout:2/586: chown d0/d1b/d14/c85 904954 1 2024-03-23T08:23:23.985 INFO:tasks.workunit.client.1.smithi193.stdout:4/563: truncate d11/d1a/d33/d42/db2/dbe/d50/f5b 9025745 0 2024-03-23T08:23:23.985 INFO:tasks.workunit.client.1.smithi193.stdout:7/492: mknod d6/d9/d1f/d4d/d7c/d92/ca7 0 2024-03-23T08:23:23.986 INFO:tasks.workunit.client.1.smithi193.stdout:8/512: dread df/d4b/f71 [0,4194304] 0 2024-03-23T08:23:23.987 INFO:tasks.workunit.client.1.smithi193.stdout:6/443: getdents d0/d27/d81/df/d11/d36/d61/d3a/d7a 0 2024-03-23T08:23:23.988 INFO:tasks.workunit.client.1.smithi193.stdout:0/524: mkdir db/d52/db8 0 2024-03-23T08:23:23.989 INFO:tasks.workunit.client.1.smithi193.stdout:4/564: chown d11/d1a/d33/d44/c46 113340 1 2024-03-23T08:23:23.990 INFO:tasks.workunit.client.1.smithi193.stdout:9/571: creat d2/d7/db/d1b/d43/d9a/fc1 x:0 0 0 2024-03-23T08:23:23.991 INFO:tasks.workunit.client.1.smithi193.stdout:3/443: rename d1/c29 to d1/de/d13/d96/c9e 0 2024-03-23T08:23:23.992 INFO:tasks.workunit.client.1.smithi193.stdout:8/513: truncate df/d10/d3e/d54/faa 125820 0 2024-03-23T08:23:23.992 INFO:tasks.workunit.client.1.smithi193.stdout:2/587: dwrite d0/d1b/f8 [4194304,4194304] 0 2024-03-23T08:23:23.992 INFO:tasks.workunit.client.1.smithi193.stdout:7/493: mknod d6/d9/d1f/d23/d69/ca8 0 2024-03-23T08:23:23.993 INFO:tasks.workunit.client.1.smithi193.stdout:6/444: fdatasync d0/d27/d81/f86 0 2024-03-23T08:23:23.998 INFO:tasks.workunit.client.1.smithi193.stdout:9/572: symlink d2/lc2 0 2024-03-23T08:23:23.998 INFO:tasks.workunit.client.1.smithi193.stdout:7/494: creat d6/d9/d1f/d4d/fa9 x:0 0 0 2024-03-23T08:23:24.000 INFO:tasks.workunit.client.1.smithi193.stdout:3/444: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f 0 2024-03-23T08:23:24.002 INFO:tasks.workunit.client.1.smithi193.stdout:9/573: chown d2/d7/db/d1b/d5c/c61 1178 1 2024-03-23T08:23:24.002 INFO:tasks.workunit.client.1.smithi193.stdout:8/514: rename df/c42 to df/cac 0 2024-03-23T08:23:24.003 INFO:tasks.workunit.client.1.smithi193.stdout:0/525: write db/d11/d14/d24/f39 [2986051,116095] 0 2024-03-23T08:23:24.005 INFO:tasks.workunit.client.1.smithi193.stdout:2/588: dread d0/d1b/d3b/f46 [0,4194304] 0 2024-03-23T08:23:24.006 INFO:tasks.workunit.client.1.smithi193.stdout:3/445: rmdir d1/de/d13/d21/d50/d9b 0 2024-03-23T08:23:24.007 INFO:tasks.workunit.client.1.smithi193.stdout:8/515: creat df/d62/fad x:0 0 0 2024-03-23T08:23:24.008 INFO:tasks.workunit.client.1.smithi193.stdout:0/526: stat db/d11/d14/d21/l3d 0 2024-03-23T08:23:24.008 INFO:tasks.workunit.client.1.smithi193.stdout:9/574: symlink d2/d7/db/d32/lc3 0 2024-03-23T08:23:24.009 INFO:tasks.workunit.client.1.smithi193.stdout:5/578: sync 2024-03-23T08:23:24.011 INFO:tasks.workunit.client.1.smithi193.stdout:7/495: rename d6/d9/d1f/d23/d33/d38/d46/f61 to d6/d9/d1f/d3e/d65/d70/faa 0 2024-03-23T08:23:24.011 INFO:tasks.workunit.client.1.smithi193.stdout:9/575: chown d2/d7/d31/d5f/d9c 0 1 2024-03-23T08:23:24.011 INFO:tasks.workunit.client.1.smithi193.stdout:4/565: dread d11/d1a/f1c [4194304,4194304] 0 2024-03-23T08:23:24.012 INFO:tasks.workunit.client.1.smithi193.stdout:8/516: write df/f33 [1385164,81739] 0 2024-03-23T08:23:24.013 INFO:tasks.workunit.client.1.smithi193.stdout:3/446: write d1/de/d13/d21/d27/f63 [792740,65017] 0 2024-03-23T08:23:24.016 INFO:tasks.workunit.client.1.smithi193.stdout:2/589: creat d0/d1b/d14/d66/d50/d92/da9/fab x:0 0 0 2024-03-23T08:23:24.017 INFO:tasks.workunit.client.1.smithi193.stdout:6/445: dwrite d0/d27/d81/df/d11/d36/d61/d7c/d69/f7b [0,4194304] 0 2024-03-23T08:23:24.017 INFO:tasks.workunit.client.1.smithi193.stdout:9/576: unlink d2/l88 0 2024-03-23T08:23:24.020 INFO:tasks.workunit.client.1.smithi193.stdout:2/590: stat d0/d1b/d19/l3c 0 2024-03-23T08:23:24.020 INFO:tasks.workunit.client.1.smithi193.stdout:0/527: truncate db/d11/d14/f19 4433844 0 2024-03-23T08:23:24.020 INFO:tasks.workunit.client.1.smithi193.stdout:5/579: write d0/d1c/d7a/db1/f76 [4787853,50105] 0 2024-03-23T08:23:24.020 INFO:tasks.workunit.client.1.smithi193.stdout:3/447: write d1/f1e [78890,43715] 0 2024-03-23T08:23:24.022 INFO:tasks.workunit.client.1.smithi193.stdout:5/580: chown d0/d1c/d3b/d4b/f95 2122745410 1 2024-03-23T08:23:24.025 INFO:tasks.workunit.client.1.smithi193.stdout:0/528: fdatasync db/d11/f40 0 2024-03-23T08:23:24.025 INFO:tasks.workunit.client.1.smithi193.stdout:8/517: rename df/d46 to df/d62/d8b/dae 0 2024-03-23T08:23:24.026 INFO:tasks.workunit.client.1.smithi193.stdout:9/577: creat d2/d7/db/d17/fc4 x:0 0 0 2024-03-23T08:23:24.027 INFO:tasks.workunit.client.1.smithi193.stdout:5/581: mknod d0/d1c/d3b/cd4 0 2024-03-23T08:23:24.029 INFO:tasks.workunit.client.1.smithi193.stdout:4/566: symlink d11/d2d/d58/d8b/dbc/lc2 0 2024-03-23T08:23:24.029 INFO:tasks.workunit.client.1.smithi193.stdout:5/582: stat d0/d1c 0 2024-03-23T08:23:24.031 INFO:tasks.workunit.client.1.smithi193.stdout:2/591: mknod d0/d28/cac 0 2024-03-23T08:23:24.031 INFO:tasks.workunit.client.1.smithi193.stdout:7/496: dwrite d6/d9/f6f [0,4194304] 0 2024-03-23T08:23:24.032 INFO:tasks.workunit.client.1.smithi193.stdout:8/518: symlink df/d63/d76/laf 0 2024-03-23T08:23:24.033 INFO:tasks.workunit.client.1.smithi193.stdout:3/448: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/fa0 x:0 0 0 2024-03-23T08:23:24.035 INFO:tasks.workunit.client.1.smithi193.stdout:2/592: read d0/d1b/d14/d25/f29 [117467,2168] 0 2024-03-23T08:23:24.036 INFO:tasks.workunit.client.1.smithi193.stdout:5/583: mknod d0/d1f/d93/cd5 0 2024-03-23T08:23:24.037 INFO:tasks.workunit.client.1.smithi193.stdout:4/567: creat d11/d2d/d58/d80/fc3 x:0 0 0 2024-03-23T08:23:24.037 INFO:tasks.workunit.client.1.smithi193.stdout:6/446: creat d0/d27/d81/df/d11/d36/d61/f8e x:0 0 0 2024-03-23T08:23:24.042 INFO:tasks.workunit.client.1.smithi193.stdout:7/497: creat d6/d9/d1f/d3e/fab x:0 0 0 2024-03-23T08:23:24.043 INFO:tasks.workunit.client.1.smithi193.stdout:2/593: symlink d0/d28/d33/lad 0 2024-03-23T08:23:24.045 INFO:tasks.workunit.client.1.smithi193.stdout:5/584: mknod d0/d1c/d7a/db1/d72/d73/d57/cd6 0 2024-03-23T08:23:24.045 INFO:tasks.workunit.client.1.smithi193.stdout:9/578: dwrite d2/d7/db/d7a/d95/da9/fb8 [0,4194304] 0 2024-03-23T08:23:24.047 INFO:tasks.workunit.client.1.smithi193.stdout:0/529: dwrite db/d11/d14/d24/d2a/d4d/f7a [0,4194304] 0 2024-03-23T08:23:24.048 INFO:tasks.workunit.client.1.smithi193.stdout:4/568: unlink d11/d1a/d33/d42/d52/d70/l9c 0 2024-03-23T08:23:24.050 INFO:tasks.workunit.client.1.smithi193.stdout:6/447: mkdir d0/d27/d81/df/d11/d36/d61/d3a/d8f 0 2024-03-23T08:23:24.051 INFO:tasks.workunit.client.1.smithi193.stdout:7/498: mkdir d6/d9/d1f/d23/d33/d38/d66/dac 0 2024-03-23T08:23:24.054 INFO:tasks.workunit.client.1.smithi193.stdout:2/594: mknod d0/d1b/d14/cae 0 2024-03-23T08:23:24.055 INFO:tasks.workunit.client.1.smithi193.stdout:6/448: fsync d0/d27/d81/df/d11/d3b/f66 0 2024-03-23T08:23:24.057 INFO:tasks.workunit.client.1.smithi193.stdout:4/569: unlink d11/d2d/d58/d8b/dbc/c69 0 2024-03-23T08:23:24.058 INFO:tasks.workunit.client.1.smithi193.stdout:0/530: creat db/d11/d14/d24/d2a/d96/fb9 x:0 0 0 2024-03-23T08:23:24.058 INFO:tasks.workunit.client.1.smithi193.stdout:2/595: chown d0/l11 23187274 1 2024-03-23T08:23:24.059 INFO:tasks.workunit.client.1.smithi193.stdout:2/596: readlink d0/d1b/l38 0 2024-03-23T08:23:24.060 INFO:tasks.workunit.client.1.smithi193.stdout:0/531: readlink db/d11/d14/d24/d47/l6d 0 2024-03-23T08:23:24.060 INFO:tasks.workunit.client.1.smithi193.stdout:4/570: write d11/d1a/d33/d42/d52/d70/f74 [714451,74677] 0 2024-03-23T08:23:24.061 INFO:tasks.workunit.client.1.smithi193.stdout:5/585: rename d0/d1c/d3b/d81/c90 to d0/d1c/cd7 0 2024-03-23T08:23:24.064 INFO:tasks.workunit.client.1.smithi193.stdout:5/586: chown d0/d5a/f75 493721 1 2024-03-23T08:23:24.067 INFO:tasks.workunit.client.1.smithi193.stdout:6/449: getdents d0/d27/d81/df/d11/d36/d61/d7c 0 2024-03-23T08:23:24.069 INFO:tasks.workunit.client.1.smithi193.stdout:3/449: dread d1/de/d13/f24 [0,4194304] 0 2024-03-23T08:23:24.069 INFO:tasks.workunit.client.1.smithi193.stdout:7/499: dwrite d6/d9/d1f/d23/d33/d38/d3b/f48 [0,4194304] 0 2024-03-23T08:23:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:23 smithi193 ceph-mon[48370]: pgmap v21: 65 pgs: 65 active+clean; 912 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 105 MiB/s wr, 376 op/s 2024-03-23T08:23:24.076 INFO:tasks.workunit.client.1.smithi193.stdout:3/450: dwrite d1/f1e [0,4194304] 0 2024-03-23T08:23:24.081 INFO:tasks.workunit.client.1.smithi193.stdout:2/597: dread d0/f2d [0,4194304] 0 2024-03-23T08:23:24.088 INFO:tasks.workunit.client.1.smithi193.stdout:2/598: dwrite d0/d28/d33/f70 [0,4194304] 0 2024-03-23T08:23:24.089 INFO:tasks.workunit.client.1.smithi193.stdout:2/599: chown d0/d28/d33/c81 10791278 1 2024-03-23T08:23:24.094 INFO:tasks.workunit.client.1.smithi193.stdout:1/514: sync 2024-03-23T08:23:24.191 INFO:tasks.workunit.client.1.smithi193.stdout:2/600: dread d0/d1b/d14/d66/d4e/f5b [0,4194304] 0 2024-03-23T08:23:24.193 INFO:tasks.workunit.client.1.smithi193.stdout:5/587: rename d0/d1f/d93/dc0 to d0/d1c/d7a/db1/d72/d99/dd8 0 2024-03-23T08:23:24.194 INFO:tasks.workunit.client.1.smithi193.stdout:5/588: readlink d0/d5a/l71 0 2024-03-23T08:23:24.195 INFO:tasks.workunit.client.1.smithi193.stdout:6/450: creat d0/d27/d81/df/d11/d3b/d6c/f90 x:0 0 0 2024-03-23T08:23:24.195 INFO:tasks.workunit.client.1.smithi193.stdout:5/589: chown d0/d1c/d3b/f7f 3996202 1 2024-03-23T08:23:24.196 INFO:tasks.workunit.client.1.smithi193.stdout:2/601: dwrite d0/d1b/d3b/f58 [0,4194304] 0 2024-03-23T08:23:24.198 INFO:tasks.workunit.client.1.smithi193.stdout:6/451: chown d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f84 11073642 1 2024-03-23T08:23:24.203 INFO:tasks.workunit.client.1.smithi193.stdout:2/602: dwrite d0/d1b/d3b/f7a [0,4194304] 0 2024-03-23T08:23:24.205 INFO:tasks.workunit.client.1.smithi193.stdout:8/519: sync 2024-03-23T08:23:24.209 INFO:tasks.workunit.client.1.smithi193.stdout:3/451: mknod d1/de/d13/d21/d25/d2b/d3c/d40/ca1 0 2024-03-23T08:23:24.214 INFO:tasks.workunit.client.1.smithi193.stdout:0/532: rmdir db/d11/d14 39 2024-03-23T08:23:24.215 INFO:tasks.workunit.client.1.smithi193.stdout:1/515: write da/f12 [8656960,69093] 0 2024-03-23T08:23:24.216 INFO:tasks.workunit.client.1.smithi193.stdout:9/579: truncate d2/d7/db/d17/f54 583061 0 2024-03-23T08:23:24.216 INFO:tasks.workunit.client.1.smithi193.stdout:1/516: chown da/d13/d24/f97 13266283 1 2024-03-23T08:23:24.219 INFO:tasks.workunit.client.1.smithi193.stdout:6/452: rmdir d0/d27/d81/df/d11/d36/d61 39 2024-03-23T08:23:24.220 INFO:tasks.workunit.client.1.smithi193.stdout:2/603: fdatasync d0/d1b/d14/d66/d50/f52 0 2024-03-23T08:23:24.222 INFO:tasks.workunit.client.1.smithi193.stdout:4/571: truncate d11/f92 131605 0 2024-03-23T08:23:24.225 INFO:tasks.workunit.client.1.smithi193.stdout:9/580: symlink d2/d7/d31/d3c/d79/lc5 0 2024-03-23T08:23:24.229 INFO:tasks.workunit.client.1.smithi193.stdout:8/520: truncate df/f34 818801 0 2024-03-23T08:23:24.229 INFO:tasks.workunit.client.1.smithi193.stdout:4/572: rmdir d11 39 2024-03-23T08:23:24.232 INFO:tasks.workunit.client.1.smithi193.stdout:0/533: dread db/d11/d14/d51/f56 [0,4194304] 0 2024-03-23T08:23:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:23 smithi070 ceph-mon[39699]: pgmap v21: 65 pgs: 65 active+clean; 912 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 105 MiB/s wr, 376 op/s 2024-03-23T08:23:24.240 INFO:tasks.workunit.client.1.smithi193.stdout:6/453: write d0/d27/d81/df/d11/d36/d61/d3a/f6b [1896457,123417] 0 2024-03-23T08:23:24.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/517: truncate da/dd/df/f1d 3280349 0 2024-03-23T08:23:24.246 INFO:tasks.workunit.client.1.smithi193.stdout:9/581: dwrite d2/d7/db/d17/f53 [0,4194304] 0 2024-03-23T08:23:24.246 INFO:tasks.workunit.client.1.smithi193.stdout:1/518: rmdir da 39 2024-03-23T08:23:24.247 INFO:tasks.workunit.client.1.smithi193.stdout:8/521: dread df/d4b/d4d/f5c [0,4194304] 0 2024-03-23T08:23:24.249 INFO:tasks.workunit.client.1.smithi193.stdout:8/522: dread - df/d10/d3e/d54/f85 zero size 2024-03-23T08:23:24.249 INFO:tasks.workunit.client.1.smithi193.stdout:9/582: chown d2/d7/db/d1b/d43/f84 13 1 2024-03-23T08:23:24.250 INFO:tasks.workunit.client.1.smithi193.stdout:6/454: rename d0/d27/d81/df/d11/d3b/d6c/f7d to d0/d27/f91 0 2024-03-23T08:23:24.251 INFO:tasks.workunit.client.1.smithi193.stdout:4/573: dread d11/d1a/d33/d42/d52/f3f [0,4194304] 0 2024-03-23T08:23:24.253 INFO:tasks.workunit.client.1.smithi193.stdout:1/519: creat da/fa8 x:0 0 0 2024-03-23T08:23:24.256 INFO:tasks.workunit.client.1.smithi193.stdout:1/520: mkdir da/dd/da9 0 2024-03-23T08:23:24.257 INFO:tasks.workunit.client.1.smithi193.stdout:6/455: symlink d0/d27/d81/df/d11/d36/d61/d12/l92 0 2024-03-23T08:23:24.259 INFO:tasks.workunit.client.1.smithi193.stdout:1/521: chown da/dd/df/d7e/d7f/f8a 9776634 1 2024-03-23T08:23:24.260 INFO:tasks.workunit.client.1.smithi193.stdout:4/574: dwrite fb [0,4194304] 0 2024-03-23T08:23:24.264 INFO:tasks.workunit.client.1.smithi193.stdout:1/522: rename da/d13/d3f/d5a/c5d to da/dd/df/d7e/caa 0 2024-03-23T08:23:24.267 INFO:tasks.workunit.client.1.smithi193.stdout:4/575: symlink d11/d2d/d58/d80/db9/lc4 0 2024-03-23T08:23:24.269 INFO:tasks.workunit.client.1.smithi193.stdout:1/523: creat da/d13/d3f/d5a/fab x:0 0 0 2024-03-23T08:23:24.272 INFO:tasks.workunit.client.1.smithi193.stdout:1/524: dread - da/dd/df/d52/d82/f83 zero size 2024-03-23T08:23:24.273 INFO:tasks.workunit.client.1.smithi193.stdout:8/523: truncate df/d10/d67/f7d 419693 0 2024-03-23T08:23:24.274 INFO:tasks.workunit.client.1.smithi193.stdout:1/525: stat da/dd 0 2024-03-23T08:23:24.276 INFO:tasks.workunit.client.1.smithi193.stdout:8/524: stat df/d10/f7b 0 2024-03-23T08:23:24.277 INFO:tasks.workunit.client.1.smithi193.stdout:9/583: write d2/d7/db/d17/f54 [1138682,119708] 0 2024-03-23T08:23:24.280 INFO:tasks.workunit.client.1.smithi193.stdout:1/526: read da/dd/df/f29 [1368763,78816] 0 2024-03-23T08:23:24.280 INFO:tasks.workunit.client.1.smithi193.stdout:4/576: dwrite f9 [4194304,4194304] 0 2024-03-23T08:23:24.284 INFO:tasks.workunit.client.1.smithi193.stdout:8/525: creat df/d10/d3e/fb0 x:0 0 0 2024-03-23T08:23:24.284 INFO:tasks.workunit.client.1.smithi193.stdout:1/527: mkdir da/d13/d24/dac 0 2024-03-23T08:23:24.285 INFO:tasks.workunit.client.1.smithi193.stdout:6/456: dwrite d0/d27/d81/df/d11/d36/d61/d12/d21/f29 [0,4194304] 0 2024-03-23T08:23:24.287 INFO:tasks.workunit.client.1.smithi193.stdout:4/577: fdatasync f4 0 2024-03-23T08:23:24.288 INFO:tasks.workunit.client.1.smithi193.stdout:8/526: mknod df/d10/d19/cb1 0 2024-03-23T08:23:24.288 INFO:tasks.workunit.client.1.smithi193.stdout:4/578: chown d11/d8e/dac 105 1 2024-03-23T08:23:24.293 INFO:tasks.workunit.client.1.smithi193.stdout:8/527: mkdir df/d4b/d4d/d2a/db2 0 2024-03-23T08:23:24.293 INFO:tasks.workunit.client.1.smithi193.stdout:4/579: symlink d11/d2d/d58/d80/d9f/lc5 0 2024-03-23T08:23:24.297 INFO:tasks.workunit.client.1.smithi193.stdout:4/580: mkdir d11/d8e/dac/dc6 0 2024-03-23T08:23:24.298 INFO:tasks.workunit.client.1.smithi193.stdout:7/500: sync 2024-03-23T08:23:24.298 INFO:tasks.workunit.client.1.smithi193.stdout:1/528: dwrite da/d13/d3f/d5a/d76/d77/f6c [0,4194304] 0 2024-03-23T08:23:24.299 INFO:tasks.workunit.client.1.smithi193.stdout:7/501: chown d6/d9/d1f/d3e 218 1 2024-03-23T08:23:24.299 INFO:tasks.workunit.client.1.smithi193.stdout:4/581: fsync d11/d1a/d33/f39 0 2024-03-23T08:23:24.301 INFO:tasks.workunit.client.1.smithi193.stdout:4/582: chown d11/d1a/d33/d42/d52/d70/cc0 31630189 1 2024-03-23T08:23:24.301 INFO:tasks.workunit.client.1.smithi193.stdout:1/529: mknod da/dd/df/d7e/cad 0 2024-03-23T08:23:24.302 INFO:tasks.workunit.client.1.smithi193.stdout:7/502: mknod d6/d9/d1f/d4d/d6b/cad 0 2024-03-23T08:23:24.304 INFO:tasks.workunit.client.1.smithi193.stdout:4/583: mknod d11/d1a/d33/d44/cc7 0 2024-03-23T08:23:24.306 INFO:tasks.workunit.client.1.smithi193.stdout:4/584: truncate d11/d8e/dac/fbd 815771 0 2024-03-23T08:23:24.308 INFO:tasks.workunit.client.1.smithi193.stdout:8/528: write df/d10/d19/f8c [178125,128578] 0 2024-03-23T08:23:24.309 INFO:tasks.workunit.client.1.smithi193.stdout:8/529: rmdir df/d10/d19 39 2024-03-23T08:23:24.310 INFO:tasks.workunit.client.1.smithi193.stdout:4/585: mknod d11/d1a/d33/d42/d52/d70/cc8 0 2024-03-23T08:23:24.311 INFO:tasks.workunit.client.1.smithi193.stdout:4/586: dread - d11/d2d/d58/d80/d9f/fa2 zero size 2024-03-23T08:23:24.313 INFO:tasks.workunit.client.1.smithi193.stdout:4/587: symlink d11/d8e/dac/lc9 0 2024-03-23T08:23:24.314 INFO:tasks.workunit.client.1.smithi193.stdout:7/503: write d6/d9/d1f/d23/d33/d3c/f4b [555428,30054] 0 2024-03-23T08:23:24.315 INFO:tasks.workunit.client.1.smithi193.stdout:4/588: creat d11/d2d/d58/d80/fca x:0 0 0 2024-03-23T08:23:24.316 INFO:tasks.workunit.client.1.smithi193.stdout:7/504: fsync d6/d9/d1f/d3e/d42/f71 0 2024-03-23T08:23:24.316 INFO:tasks.workunit.client.1.smithi193.stdout:4/589: write d11/d8d/f91 [666468,55585] 0 2024-03-23T08:23:24.318 INFO:tasks.workunit.client.1.smithi193.stdout:4/590: chown d11/d1a/d33/d7f/caf 18621753 1 2024-03-23T08:23:24.319 INFO:tasks.workunit.client.1.smithi193.stdout:7/505: creat d6/d9/d4e/fae x:0 0 0 2024-03-23T08:23:24.321 INFO:tasks.workunit.client.1.smithi193.stdout:6/457: dread d0/f40 [0,4194304] 0 2024-03-23T08:23:24.323 INFO:tasks.workunit.client.1.smithi193.stdout:6/458: write d0/d27/d81/df/d11/f30 [52771,60874] 0 2024-03-23T08:23:24.326 INFO:tasks.workunit.client.1.smithi193.stdout:4/591: dread d11/d27/d85/f8a [0,4194304] 0 2024-03-23T08:23:24.328 INFO:tasks.workunit.client.1.smithi193.stdout:6/459: symlink d0/d27/d81/df/d11/d36/d61/d3a/l93 0 2024-03-23T08:23:24.330 INFO:tasks.workunit.client.1.smithi193.stdout:7/506: dwrite d6/d9/d1f/d23/d33/d38/d3b/f48 [0,4194304] 0 2024-03-23T08:23:24.334 INFO:tasks.workunit.client.1.smithi193.stdout:6/460: getdents d0/d27/d81/df/d11 0 2024-03-23T08:23:24.336 INFO:tasks.workunit.client.1.smithi193.stdout:7/507: dread d6/d9/d1f/d23/d33/d38/d3b/d6c/f8d [0,4194304] 0 2024-03-23T08:23:24.337 INFO:tasks.workunit.client.1.smithi193.stdout:4/592: dwrite d11/d1a/d33/d42/f29 [0,4194304] 0 2024-03-23T08:23:24.337 INFO:tasks.workunit.client.1.smithi193.stdout:6/461: write d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f7e [60261,60444] 0 2024-03-23T08:23:24.339 INFO:tasks.workunit.client.1.smithi193.stdout:7/508: mkdir d6/d9/d1f/daf 0 2024-03-23T08:23:24.340 INFO:tasks.workunit.client.1.smithi193.stdout:4/593: mknod d11/d1a/ccb 0 2024-03-23T08:23:24.343 INFO:tasks.workunit.client.1.smithi193.stdout:7/509: mknod d6/d9/d1f/d23/d33/d38/d66/dac/cb0 0 2024-03-23T08:23:24.346 INFO:tasks.workunit.client.1.smithi193.stdout:7/510: mknod d6/d9/d1f/daf/cb1 0 2024-03-23T08:23:24.346 INFO:tasks.workunit.client.1.smithi193.stdout:7/511: chown c2 4552953 1 2024-03-23T08:23:24.349 INFO:tasks.workunit.client.1.smithi193.stdout:7/512: rename d6/d9/d1f/d4d/d7c/f99 to d6/d9/d1f/d23/d33/d3c/d41/d47/fb2 0 2024-03-23T08:23:24.351 INFO:tasks.workunit.client.1.smithi193.stdout:7/513: fsync d6/d9/d1f/d3e/d65/d70/faa 0 2024-03-23T08:23:24.355 INFO:tasks.workunit.client.1.smithi193.stdout:7/514: rename d6/d9/d1f/d4d/d6b/l87 to d6/d9/d1f/d23/d33/d38/d46/d82/lb3 0 2024-03-23T08:23:24.356 INFO:tasks.workunit.client.1.smithi193.stdout:7/515: readlink d6/d9/d1f/d23/d33/l54 0 2024-03-23T08:23:24.357 INFO:tasks.workunit.client.1.smithi193.stdout:7/516: write d6/d9/d1f/d23/d33/d38/d3b/f5a [451118,95749] 0 2024-03-23T08:23:24.357 INFO:tasks.workunit.client.1.smithi193.stdout:7/517: chown d6/f14 25440 1 2024-03-23T08:23:24.360 INFO:tasks.workunit.client.1.smithi193.stdout:7/518: creat d6/d9/d1f/d3e/d65/d7d/fb4 x:0 0 0 2024-03-23T08:23:24.369 INFO:tasks.workunit.client.1.smithi193.stdout:7/519: write d6/d9/f19 [11032,19719] 0 2024-03-23T08:23:24.372 INFO:tasks.workunit.client.1.smithi193.stdout:7/520: mknod d6/d9/d1f/d3e/d65/d7d/cb5 0 2024-03-23T08:23:24.373 INFO:tasks.workunit.client.1.smithi193.stdout:7/521: read d6/d9/d1f/d23/f45 [3838516,14700] 0 2024-03-23T08:23:24.373 INFO:tasks.workunit.client.1.smithi193.stdout:7/522: chown d6/d9/d1f/d4d/d6b/cad 664663 1 2024-03-23T08:23:24.374 INFO:tasks.workunit.client.1.smithi193.stdout:7/523: fsync d6/d9/d1f/d23/d33/d3c/f4b 0 2024-03-23T08:23:24.377 INFO:tasks.workunit.client.1.smithi193.stdout:7/524: symlink d6/d9/d1f/d23/d33/d38/lb6 0 2024-03-23T08:23:24.382 INFO:tasks.workunit.client.1.smithi193.stdout:7/525: dwrite d6/f39 [0,4194304] 0 2024-03-23T08:23:24.385 INFO:tasks.workunit.client.1.smithi193.stdout:7/526: write d6/d9/d1f/d23/d33/d38/d66/f9c [889538,59382] 0 2024-03-23T08:23:24.387 INFO:tasks.workunit.client.1.smithi193.stdout:7/527: read d6/d9/d1f/f50 [1042792,77312] 0 2024-03-23T08:23:24.393 INFO:tasks.workunit.client.1.smithi193.stdout:7/528: dwrite d6/d9/d1f/d23/f2e [0,4194304] 0 2024-03-23T08:23:24.398 INFO:tasks.workunit.client.1.smithi193.stdout:7/529: dread d6/d9/f6f [0,4194304] 0 2024-03-23T08:23:24.399 INFO:tasks.workunit.client.1.smithi193.stdout:7/530: rmdir d6/d9/d1f/daf 39 2024-03-23T08:23:24.460 INFO:tasks.workunit.client.1.smithi193.stdout:5/590: sync 2024-03-23T08:23:24.462 INFO:tasks.workunit.client.1.smithi193.stdout:5/591: unlink d0/l40 0 2024-03-23T08:23:24.465 INFO:tasks.workunit.client.1.smithi193.stdout:5/592: mkdir d0/d1c/d7a/db1/d72/d73/d57/d6d/dd9 0 2024-03-23T08:23:24.466 INFO:tasks.workunit.client.1.smithi193.stdout:5/593: read - d0/d1c/d3b/d81/f9e zero size 2024-03-23T08:23:24.468 INFO:tasks.workunit.client.1.smithi193.stdout:5/594: unlink d0/d1c/d3b/f7f 0 2024-03-23T08:23:24.469 INFO:tasks.workunit.client.1.smithi193.stdout:5/595: symlink d0/d1c/d3b/d4b/lda 0 2024-03-23T08:23:24.474 INFO:tasks.workunit.client.1.smithi193.stdout:5/596: dwrite d0/d1c/d7a/db1/f8b [0,4194304] 0 2024-03-23T08:23:24.477 INFO:tasks.workunit.client.1.smithi193.stdout:5/597: write d0/d1c/d7a/db1/d72/d99/f9b [4519594,60322] 0 2024-03-23T08:23:24.481 INFO:tasks.workunit.client.1.smithi193.stdout:5/598: fdatasync d0/d1c/d7a/db1/f38 0 2024-03-23T08:23:24.484 INFO:tasks.workunit.client.1.smithi193.stdout:5/599: creat d0/d1f/d93/fdb x:0 0 0 2024-03-23T08:23:24.484 INFO:tasks.workunit.client.1.smithi193.stdout:5/600: readlink d0/d1f/lc9 0 2024-03-23T08:23:24.486 INFO:tasks.workunit.client.1.smithi193.stdout:5/601: fdatasync d0/f4d 0 2024-03-23T08:23:24.487 INFO:tasks.workunit.client.1.smithi193.stdout:5/602: write d0/d5a/fcf [1006236,15157] 0 2024-03-23T08:23:24.491 INFO:tasks.workunit.client.1.smithi193.stdout:5/603: creat d0/d1c/d7a/db1/d72/d73/d45/fdc x:0 0 0 2024-03-23T08:23:24.535 INFO:tasks.workunit.client.1.smithi193.stdout:0/534: sync 2024-03-23T08:23:24.535 INFO:tasks.workunit.client.1.smithi193.stdout:2/604: sync 2024-03-23T08:23:24.535 INFO:tasks.workunit.client.1.smithi193.stdout:9/584: sync 2024-03-23T08:23:24.535 INFO:tasks.workunit.client.1.smithi193.stdout:3/452: sync 2024-03-23T08:23:24.537 INFO:tasks.workunit.client.1.smithi193.stdout:9/585: read d2/d7/f1e [3666130,108985] 0 2024-03-23T08:23:24.538 INFO:tasks.workunit.client.1.smithi193.stdout:3/453: truncate d1/de/d13/d21/d25/d2b/d3c/f6a 681667 0 2024-03-23T08:23:24.539 INFO:tasks.workunit.client.1.smithi193.stdout:2/605: unlink d0/d1b/d14/d66/d4e/f5b 0 2024-03-23T08:23:24.539 INFO:tasks.workunit.client.1.smithi193.stdout:9/586: creat d2/d7/db/d7a/d95/fc6 x:0 0 0 2024-03-23T08:23:24.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/606: mkdir d0/d28/daf 0 2024-03-23T08:23:24.541 INFO:tasks.workunit.client.1.smithi193.stdout:3/454: mkdir d1/de/d13/d21/d50/da2 0 2024-03-23T08:23:24.542 INFO:tasks.workunit.client.1.smithi193.stdout:9/587: creat d2/d7/db/d1b/d43/d71/fc7 x:0 0 0 2024-03-23T08:23:24.544 INFO:tasks.workunit.client.1.smithi193.stdout:3/455: write d1/de/d13/d21/d25/d2b/d3c/d4a/f93 [4751106,68228] 0 2024-03-23T08:23:24.546 INFO:tasks.workunit.client.1.smithi193.stdout:2/607: dread d0/d1b/d3b/f46 [0,4194304] 0 2024-03-23T08:23:24.546 INFO:tasks.workunit.client.1.smithi193.stdout:9/588: dread - d2/d7/db/d1b/d43/d71/db4/d78/fbc zero size 2024-03-23T08:23:24.548 INFO:tasks.workunit.client.1.smithi193.stdout:2/608: write d0/d1b/d14/d25/f89 [736780,49986] 0 2024-03-23T08:23:24.549 INFO:tasks.workunit.client.1.smithi193.stdout:9/589: write d2/d7/db/f2e [626717,128227] 0 2024-03-23T08:23:24.552 INFO:tasks.workunit.client.1.smithi193.stdout:0/535: dread db/d11/d14/d21/d33/f43 [4194304,4194304] 0 2024-03-23T08:23:24.552 INFO:tasks.workunit.client.1.smithi193.stdout:2/609: mkdir d0/d1b/d14/d56/da3/db0 0 2024-03-23T08:23:24.553 INFO:tasks.workunit.client.1.smithi193.stdout:2/610: dread - d0/d28/fa8 zero size 2024-03-23T08:23:24.554 INFO:tasks.workunit.client.1.smithi193.stdout:0/536: read - db/d11/d14/d24/d2a/d4d/fa1 zero size 2024-03-23T08:23:24.554 INFO:tasks.workunit.client.1.smithi193.stdout:3/456: write d1/de/d13/d21/f31 [1406762,3861] 0 2024-03-23T08:23:24.556 INFO:tasks.workunit.client.1.smithi193.stdout:2/611: symlink d0/d1b/d77/lb1 0 2024-03-23T08:23:24.556 INFO:tasks.workunit.client.1.smithi193.stdout:3/457: rmdir d1 39 2024-03-23T08:23:24.557 INFO:tasks.workunit.client.1.smithi193.stdout:0/537: chown db/d11/d14/d21/l55 3437 1 2024-03-23T08:23:24.561 INFO:tasks.workunit.client.1.smithi193.stdout:3/458: getdents d1 0 2024-03-23T08:23:24.561 INFO:tasks.workunit.client.1.smithi193.stdout:0/538: creat db/d11/d14/d51/fba x:0 0 0 2024-03-23T08:23:24.563 INFO:tasks.workunit.client.1.smithi193.stdout:9/590: dwrite d2/d7/db/d1b/f4d [0,4194304] 0 2024-03-23T08:23:24.563 INFO:tasks.workunit.client.1.smithi193.stdout:7/531: sync 2024-03-23T08:23:24.563 INFO:tasks.workunit.client.1.smithi193.stdout:8/530: sync 2024-03-23T08:23:24.563 INFO:tasks.workunit.client.1.smithi193.stdout:1/530: sync 2024-03-23T08:23:24.563 INFO:tasks.workunit.client.1.smithi193.stdout:6/462: sync 2024-03-23T08:23:24.563 INFO:tasks.workunit.client.1.smithi193.stdout:4/594: sync 2024-03-23T08:23:24.564 INFO:tasks.workunit.client.1.smithi193.stdout:3/459: readlink d1/de/d13/d21/d27/l38 0 2024-03-23T08:23:24.565 INFO:tasks.workunit.client.1.smithi193.stdout:7/532: fsync d6/f32 0 2024-03-23T08:23:24.566 INFO:tasks.workunit.client.1.smithi193.stdout:2/612: dread d0/f7 [4194304,4194304] 0 2024-03-23T08:23:24.569 INFO:tasks.workunit.client.1.smithi193.stdout:1/531: mknod da/dd/df/d75/cae 0 2024-03-23T08:23:24.569 INFO:tasks.workunit.client.1.smithi193.stdout:3/460: write d1/fc [3841026,81973] 0 2024-03-23T08:23:24.570 INFO:tasks.workunit.client.1.smithi193.stdout:9/591: read d2/d7/db/d17/d33/f63 [318036,130937] 0 2024-03-23T08:23:24.571 INFO:tasks.workunit.client.1.smithi193.stdout:8/531: creat df/fb3 x:0 0 0 2024-03-23T08:23:24.573 INFO:tasks.workunit.client.1.smithi193.stdout:2/613: symlink d0/d1b/d77/d9b/lb2 0 2024-03-23T08:23:24.574 INFO:tasks.workunit.client.1.smithi193.stdout:7/533: mkdir d6/d9/d1f/d23/d33/d38/d3b/db7 0 2024-03-23T08:23:24.576 INFO:tasks.workunit.client.1.smithi193.stdout:1/532: stat da/dd/df/d2e/l72 0 2024-03-23T08:23:24.582 INFO:tasks.workunit.client.1.smithi193.stdout:1/533: read - da/d13/d3f/fa7 zero size 2024-03-23T08:23:24.582 INFO:tasks.workunit.client.1.smithi193.stdout:6/463: rename d0/d27/d81/df/d11/d36/d61/d12/d21/c5b to d0/d27/d81/df/d11/d36/d61/d7c/d69/c94 0 2024-03-23T08:23:24.582 INFO:tasks.workunit.client.1.smithi193.stdout:8/532: link df/c3a df/d10/d67/cb4 0 2024-03-23T08:23:24.585 INFO:tasks.workunit.client.1.smithi193.stdout:4/595: dwrite d11/d2d/d58/d80/fca [0,4194304] 0 2024-03-23T08:23:24.588 INFO:tasks.workunit.client.1.smithi193.stdout:7/534: truncate d6/d9/d1f/d23/d33/d3c/f8b 914194 0 2024-03-23T08:23:24.594 INFO:tasks.workunit.client.1.smithi193.stdout:2/614: dwrite d0/d37/d48/f55 [0,4194304] 0 2024-03-23T08:23:24.595 INFO:tasks.workunit.client.1.smithi193.stdout:0/539: dread db/f1e [0,4194304] 0 2024-03-23T08:23:24.595 INFO:tasks.workunit.client.1.smithi193.stdout:8/533: truncate df/d10/d3e/d54/f85 114950 0 2024-03-23T08:23:24.596 INFO:tasks.workunit.client.1.smithi193.stdout:4/596: readlink d11/d1a/d33/d42/db2/dbe/d50/lad 0 2024-03-23T08:23:24.597 INFO:tasks.workunit.client.1.smithi193.stdout:1/534: creat da/dd/faf x:0 0 0 2024-03-23T08:23:24.597 INFO:tasks.workunit.client.1.smithi193.stdout:7/535: chown d6/d9/d1f/f2a 34260 1 2024-03-23T08:23:24.598 INFO:tasks.workunit.client.1.smithi193.stdout:3/461: dread d1/de/f16 [0,4194304] 0 2024-03-23T08:23:24.599 INFO:tasks.workunit.client.1.smithi193.stdout:4/597: chown d11/d1a/f72 332684039 1 2024-03-23T08:23:24.601 INFO:tasks.workunit.client.1.smithi193.stdout:2/615: truncate d0/d28/f43 311185 0 2024-03-23T08:23:24.601 INFO:tasks.workunit.client.1.smithi193.stdout:0/540: symlink db/d11/d14/d3b/d71/lbb 0 2024-03-23T08:23:24.603 INFO:tasks.workunit.client.1.smithi193.stdout:1/535: mkdir da/dd/df/d52/d82/db0 0 2024-03-23T08:23:24.604 INFO:tasks.workunit.client.1.smithi193.stdout:0/541: fsync db/d11/d14/d24/d47/d59/f79 0 2024-03-23T08:23:24.605 INFO:tasks.workunit.client.1.smithi193.stdout:7/536: mknod d6/d9/d1f/d23/d33/d38/d3b/d6c/cb8 0 2024-03-23T08:23:24.605 INFO:tasks.workunit.client.1.smithi193.stdout:3/462: fsync d1/de/d13/d21/d27/f1f 0 2024-03-23T08:23:24.607 INFO:tasks.workunit.client.1.smithi193.stdout:2/616: dread - d0/d1b/d14/d66/d50/f91 zero size 2024-03-23T08:23:24.608 INFO:tasks.workunit.client.1.smithi193.stdout:9/592: dread d2/d7/f37 [0,4194304] 0 2024-03-23T08:23:24.609 INFO:tasks.workunit.client.1.smithi193.stdout:1/536: fsync da/d13/d24/f7b 0 2024-03-23T08:23:24.609 INFO:tasks.workunit.client.1.smithi193.stdout:3/463: fdatasync d1/de/d13/d21/f81 0 2024-03-23T08:23:24.611 INFO:tasks.workunit.client.1.smithi193.stdout:4/598: dread d11/d2d/d3a/f76 [0,4194304] 0 2024-03-23T08:23:24.613 INFO:tasks.workunit.client.1.smithi193.stdout:0/542: unlink db/d11/d14/d21/d33/d48/f5d 0 2024-03-23T08:23:24.613 INFO:tasks.workunit.client.1.smithi193.stdout:9/593: readlink d2/d7/db/d17/d4b/d59/d97/d9d/lb3 0 2024-03-23T08:23:24.614 INFO:tasks.workunit.client.1.smithi193.stdout:0/543: readlink db/l94 0 2024-03-23T08:23:24.614 INFO:tasks.workunit.client.1.smithi193.stdout:3/464: write d1/de/f70 [344941,102549] 0 2024-03-23T08:23:24.614 INFO:tasks.workunit.client.1.smithi193.stdout:1/537: creat da/dd/df/d7e/d86/d95/fb1 x:0 0 0 2024-03-23T08:23:24.616 INFO:tasks.workunit.client.1.smithi193.stdout:2/617: creat d0/d1b/da7/fb3 x:0 0 0 2024-03-23T08:23:24.617 INFO:tasks.workunit.client.1.smithi193.stdout:4/599: mknod d11/ccc 0 2024-03-23T08:23:24.619 INFO:tasks.workunit.client.1.smithi193.stdout:9/594: mknod d2/d7/d31/cc8 0 2024-03-23T08:23:24.620 INFO:tasks.workunit.client.1.smithi193.stdout:0/544: mkdir db/d11/d14/d24/d2a/d4d/dbc 0 2024-03-23T08:23:24.622 INFO:tasks.workunit.client.1.smithi193.stdout:2/618: creat d0/d1b/d14/d56/da3/fb4 x:0 0 0 2024-03-23T08:23:24.623 INFO:tasks.workunit.client.1.smithi193.stdout:3/465: dread d1/de/d13/d21/d25/d34/f47 [0,4194304] 0 2024-03-23T08:23:24.624 INFO:tasks.workunit.client.1.smithi193.stdout:7/537: write d6/d9/f1c [415604,77608] 0 2024-03-23T08:23:24.626 INFO:tasks.workunit.client.1.smithi193.stdout:6/464: dread d0/d27/d81/df/d11/f5a [0,4194304] 0 2024-03-23T08:23:24.629 INFO:tasks.workunit.client.1.smithi193.stdout:1/538: dwrite da/dd/df/d7e/d86/d95/fb1 [0,4194304] 0 2024-03-23T08:23:24.629 INFO:tasks.workunit.client.1.smithi193.stdout:3/466: truncate d1/de/d13/f88 274641 0 2024-03-23T08:23:24.631 INFO:tasks.workunit.client.1.smithi193.stdout:1/539: chown da/dd/da9 54193733 1 2024-03-23T08:23:24.632 INFO:tasks.workunit.client.1.smithi193.stdout:4/600: dwrite d11/f83 [4194304,4194304] 0 2024-03-23T08:23:24.638 INFO:tasks.workunit.client.1.smithi193.stdout:6/465: truncate d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f7e 979407 0 2024-03-23T08:23:24.639 INFO:tasks.workunit.client.1.smithi193.stdout:4/601: write d11/d1a/d33/d42/db2/dbe/d50/d49/f4f [805882,51942] 0 2024-03-23T08:23:24.642 INFO:tasks.workunit.client.1.smithi193.stdout:2/619: creat d0/d28/d9f/fb5 x:0 0 0 2024-03-23T08:23:24.645 INFO:tasks.workunit.client.1.smithi193.stdout:3/467: dwrite d1/de/f70 [0,4194304] 0 2024-03-23T08:23:24.645 INFO:tasks.workunit.client.1.smithi193.stdout:7/538: symlink d6/d9/d1f/d3e/lb9 0 2024-03-23T08:23:24.648 INFO:tasks.workunit.client.1.smithi193.stdout:0/545: rmdir db/d11/d14/d24/d2a/d4d/dbc 0 2024-03-23T08:23:24.650 INFO:tasks.workunit.client.1.smithi193.stdout:4/602: dread d11/d1a/d33/d42/db2/dbe/d50/f61 [0,4194304] 0 2024-03-23T08:23:24.651 INFO:tasks.workunit.client.1.smithi193.stdout:9/595: truncate d2/d7/db/d32/f3f 570966 0 2024-03-23T08:23:24.651 INFO:tasks.workunit.client.1.smithi193.stdout:2/620: write d0/d37/f8a [365150,51396] 0 2024-03-23T08:23:24.652 INFO:tasks.workunit.client.1.smithi193.stdout:7/539: write d6/d9/d1f/d3e/d65/d70/faa [95666,69974] 0 2024-03-23T08:23:24.653 INFO:tasks.workunit.client.1.smithi193.stdout:1/540: dwrite da/d13/d24/f58 [0,4194304] 0 2024-03-23T08:23:24.653 INFO:tasks.workunit.client.1.smithi193.stdout:0/546: write db/d11/d14/d21/f41 [2063219,44811] 0 2024-03-23T08:23:24.655 INFO:tasks.workunit.client.1.smithi193.stdout:9/596: fsync d2/d7/db/d1b/f4d 0 2024-03-23T08:23:24.661 INFO:tasks.workunit.client.1.smithi193.stdout:4/603: symlink d11/d2d/da8/lcd 0 2024-03-23T08:23:24.661 INFO:tasks.workunit.client.1.smithi193.stdout:9/597: fdatasync d2/d7/db/d1b/f4c 0 2024-03-23T08:23:24.663 INFO:tasks.workunit.client.1.smithi193.stdout:5/604: sync 2024-03-23T08:23:24.664 INFO:tasks.workunit.client.1.smithi193.stdout:1/541: read da/dd/df/f2d [1071536,27743] 0 2024-03-23T08:23:24.665 INFO:tasks.workunit.client.1.smithi193.stdout:1/542: readlink da/d13/d3f/d5a/l70 0 2024-03-23T08:23:24.666 INFO:tasks.workunit.client.1.smithi193.stdout:3/468: rename d1/de/d13/d96/d99/c9c to d1/de/d13/d21/d27/ca3 0 2024-03-23T08:23:24.666 INFO:tasks.workunit.client.1.smithi193.stdout:1/543: readlink da/l57 0 2024-03-23T08:23:24.666 INFO:tasks.workunit.client.1.smithi193.stdout:7/540: rmdir d6/d9/d1f/d23/d33/d38/d66/dac 39 2024-03-23T08:23:24.672 INFO:tasks.workunit.client.1.smithi193.stdout:2/621: creat d0/d1b/d14/d56/da3/db0/fb6 x:0 0 0 2024-03-23T08:23:24.672 INFO:tasks.workunit.client.1.smithi193.stdout:6/466: dread d0/d27/d81/df/d11/d36/f79 [4194304,4194304] 0 2024-03-23T08:23:24.675 INFO:tasks.workunit.client.1.smithi193.stdout:5/605: creat d0/d1c/d7a/db1/d72/d73/d45/da2/fdd x:0 0 0 2024-03-23T08:23:24.676 INFO:tasks.workunit.client.1.smithi193.stdout:1/544: mkdir da/dd/df/d7e/d7f/db2 0 2024-03-23T08:23:24.676 INFO:tasks.workunit.client.1.smithi193.stdout:3/469: mknod d1/de/d13/d21/d50/d44/ca4 0 2024-03-23T08:23:24.682 INFO:tasks.workunit.client.1.smithi193.stdout:9/598: rename d2/d7/f2b to d2/d7/db/d17/d33/db2/fc9 0 2024-03-23T08:23:24.682 INFO:tasks.workunit.client.1.smithi193.stdout:6/467: dread d0/d27/d81/df/f4f [0,4194304] 0 2024-03-23T08:23:24.685 INFO:tasks.workunit.client.1.smithi193.stdout:2/622: dwrite d0/d28/d33/d72/fa5 [0,4194304] 0 2024-03-23T08:23:24.687 INFO:tasks.workunit.client.1.smithi193.stdout:7/541: rename d6/d9/d1f/d23/d33/d3c/l67 to d6/d9/d4e/lba 0 2024-03-23T08:23:24.692 INFO:tasks.workunit.client.1.smithi193.stdout:2/623: fdatasync d0/d1b/da7/fb3 0 2024-03-23T08:23:24.694 INFO:tasks.workunit.client.1.smithi193.stdout:9/599: getdents d2/d7/db/d17 0 2024-03-23T08:23:24.697 INFO:tasks.workunit.client.1.smithi193.stdout:6/468: creat d0/d27/d81/df/d11/d36/d61/d12/f95 x:0 0 0 2024-03-23T08:23:24.697 INFO:tasks.workunit.client.1.smithi193.stdout:2/624: dread - d0/d1b/da7/fb3 zero size 2024-03-23T08:23:24.699 INFO:tasks.workunit.client.1.smithi193.stdout:1/545: dwrite da/dd/df/d7e/d7f/f8a [4194304,4194304] 0 2024-03-23T08:23:24.699 INFO:tasks.workunit.client.1.smithi193.stdout:3/470: link d1/de/d13/d21/d50/d44/ca4 d1/de/d13/d21/d25/d2b/d3c/d40/ca5 0 2024-03-23T08:23:24.702 INFO:tasks.workunit.client.1.smithi193.stdout:4/604: dwrite d11/d1a/d33/d42/f59 [0,4194304] 0 2024-03-23T08:23:24.707 INFO:tasks.workunit.client.1.smithi193.stdout:5/606: dread d0/d1c/d7a/db1/d72/d73/d45/f96 [0,4194304] 0 2024-03-23T08:23:24.708 INFO:tasks.workunit.client.1.smithi193.stdout:1/546: readlink da/dd/df/d2e/l85 0 2024-03-23T08:23:24.708 INFO:tasks.workunit.client.1.smithi193.stdout:9/600: link d2/d7/db/d17/cbd d2/d7/cca 0 2024-03-23T08:23:24.710 INFO:tasks.workunit.client.1.smithi193.stdout:4/605: mkdir d11/d1a/d33/dce 0 2024-03-23T08:23:24.710 INFO:tasks.workunit.client.1.smithi193.stdout:2/625: write d0/d1b/f8 [1170272,116018] 0 2024-03-23T08:23:24.712 INFO:tasks.workunit.client.1.smithi193.stdout:3/471: unlink d1/de/d13/d21/d25/d2b/d3c/d40/ca1 0 2024-03-23T08:23:24.714 INFO:tasks.workunit.client.1.smithi193.stdout:9/601: truncate d2/f8f 717675 0 2024-03-23T08:23:24.717 INFO:tasks.workunit.client.1.smithi193.stdout:7/542: dwrite d6/d9/d1f/d23/d33/d38/f91 [0,4194304] 0 2024-03-23T08:23:24.723 INFO:tasks.workunit.client.1.smithi193.stdout:1/547: creat da/dd/df/d7e/d86/fb3 x:0 0 0 2024-03-23T08:23:24.723 INFO:tasks.workunit.client.1.smithi193.stdout:5/607: write d0/d1c/d7a/db1/d72/d73/d45/fa0 [91198,12943] 0 2024-03-23T08:23:24.726 INFO:tasks.workunit.client.1.smithi193.stdout:6/469: dwrite d0/d27/d81/df/d11/d36/f44 [0,4194304] 0 2024-03-23T08:23:24.728 INFO:tasks.workunit.client.1.smithi193.stdout:9/602: write d2/d7/db/d1b/d43/d71/db4/d78/f9b [1072503,22345] 0 2024-03-23T08:23:24.730 INFO:tasks.workunit.client.1.smithi193.stdout:2/626: getdents d0/d1b/d19 0 2024-03-23T08:23:24.733 INFO:tasks.workunit.client.1.smithi193.stdout:4/606: dwrite fb [0,4194304] 0 2024-03-23T08:23:24.734 INFO:tasks.workunit.client.1.smithi193.stdout:1/548: mkdir da/dd/df/d2e/db4 0 2024-03-23T08:23:24.735 INFO:tasks.workunit.client.1.smithi193.stdout:8/534: sync 2024-03-23T08:23:24.737 INFO:tasks.workunit.client.1.smithi193.stdout:2/627: write d0/d1b/d14/d56/da3/db0/fb6 [431946,19508] 0 2024-03-23T08:23:24.737 INFO:tasks.workunit.client.1.smithi193.stdout:5/608: mknod d0/d1c/d7a/db1/d72/d73/d57/d6d/dd9/cde 0 2024-03-23T08:23:24.738 INFO:tasks.workunit.client.1.smithi193.stdout:4/607: creat d11/d8d/fcf x:0 0 0 2024-03-23T08:23:24.739 INFO:tasks.workunit.client.1.smithi193.stdout:2/628: chown d0/d28/d9f 14612373 1 2024-03-23T08:23:24.739 INFO:tasks.workunit.client.1.smithi193.stdout:1/549: symlink da/dd/df/d7e/d86/d95/lb5 0 2024-03-23T08:23:24.740 INFO:tasks.workunit.client.1.smithi193.stdout:8/535: truncate df/d4b/d4d/d9c/f9f 207399 0 2024-03-23T08:23:24.741 INFO:tasks.workunit.client.1.smithi193.stdout:6/470: rename d0/d27/d81/df/d11/d36/d61/d12/f95 to d0/d27/d81/df/d11/d36/d61/d3a/d7a/f96 0 2024-03-23T08:23:24.742 INFO:tasks.workunit.client.1.smithi193.stdout:7/543: dwrite d6/d9/d1f/d23/d33/d38/d3b/f55 [0,4194304] 0 2024-03-23T08:23:24.743 INFO:tasks.workunit.client.1.smithi193.stdout:4/608: mknod d11/d8d/cd0 0 2024-03-23T08:23:24.745 INFO:tasks.workunit.client.1.smithi193.stdout:2/629: write d0/d1b/f9e [2632801,54241] 0 2024-03-23T08:23:24.750 INFO:tasks.workunit.client.1.smithi193.stdout:1/550: truncate da/d13/d24/f45 286147 0 2024-03-23T08:23:24.751 INFO:tasks.workunit.client.1.smithi193.stdout:7/544: mknod d6/d9/d4e/d81/cbb 0 2024-03-23T08:23:24.752 INFO:tasks.workunit.client.1.smithi193.stdout:6/471: write d0/d27/d81/df/d11/d56/f8d [800329,107980] 0 2024-03-23T08:23:24.753 INFO:tasks.workunit.client.1.smithi193.stdout:2/630: fdatasync d0/d1b/d3b/f46 0 2024-03-23T08:23:24.758 INFO:tasks.workunit.client.1.smithi193.stdout:5/609: dwrite d0/d1c/d3b/f94 [4194304,4194304] 0 2024-03-23T08:23:24.761 INFO:tasks.workunit.client.1.smithi193.stdout:2/631: mkdir d0/d28/d33/db7 0 2024-03-23T08:23:24.766 INFO:tasks.workunit.client.1.smithi193.stdout:3/472: dwrite d1/de/f10 [0,4194304] 0 2024-03-23T08:23:24.768 INFO:tasks.workunit.client.1.smithi193.stdout:4/609: write d11/f92 [418001,46668] 0 2024-03-23T08:23:24.768 INFO:tasks.workunit.client.1.smithi193.stdout:2/632: creat d0/d28/d9f/fb8 x:0 0 0 2024-03-23T08:23:24.771 INFO:tasks.workunit.client.1.smithi193.stdout:7/545: truncate d6/d9/d1f/d23/d33/d38/d46/f5b 4575215 0 2024-03-23T08:23:24.774 INFO:tasks.workunit.client.1.smithi193.stdout:6/472: dwrite d0/d27/d81/f20 [0,4194304] 0 2024-03-23T08:23:24.775 INFO:tasks.workunit.client.1.smithi193.stdout:4/610: chown d11/f92 0 1 2024-03-23T08:23:24.782 INFO:tasks.workunit.client.1.smithi193.stdout:6/473: write d0/d27/d81/f86 [2897853,6272] 0 2024-03-23T08:23:24.782 INFO:tasks.workunit.client.1.smithi193.stdout:8/536: dread df/d10/f3f [0,4194304] 0 2024-03-23T08:23:24.782 INFO:tasks.workunit.client.1.smithi193.stdout:3/473: fdatasync d1/de/d13/d21/d25/d2b/f39 0 2024-03-23T08:23:24.783 INFO:tasks.workunit.client.1.smithi193.stdout:7/546: mknod d6/d9/cbc 0 2024-03-23T08:23:24.784 INFO:tasks.workunit.client.1.smithi193.stdout:4/611: creat d11/d1a/d33/d8f/db8/fd1 x:0 0 0 2024-03-23T08:23:24.785 INFO:tasks.workunit.client.1.smithi193.stdout:4/612: write d11/d27/d77/fa0 [804694,125372] 0 2024-03-23T08:23:24.786 INFO:tasks.workunit.client.1.smithi193.stdout:7/547: mkdir d6/dbd 0 2024-03-23T08:23:24.788 INFO:tasks.workunit.client.1.smithi193.stdout:1/551: dread da/dd/df/f29 [0,4194304] 0 2024-03-23T08:23:24.789 INFO:tasks.workunit.client.1.smithi193.stdout:3/474: truncate d1/de/d13/d21/d25/d34/f8f 627421 0 2024-03-23T08:23:24.790 INFO:tasks.workunit.client.1.smithi193.stdout:4/613: symlink d11/d1a/d33/d42/d52/ld2 0 2024-03-23T08:23:24.791 INFO:tasks.workunit.client.1.smithi193.stdout:7/548: fdatasync d6/d9/d1f/d23/d33/d38/f91 0 2024-03-23T08:23:24.791 INFO:tasks.workunit.client.1.smithi193.stdout:6/474: creat d0/d27/d81/df/d11/d36/f97 x:0 0 0 2024-03-23T08:23:24.793 INFO:tasks.workunit.client.1.smithi193.stdout:1/552: mknod da/dd/cb6 0 2024-03-23T08:23:24.794 INFO:tasks.workunit.client.1.smithi193.stdout:3/475: symlink d1/de/d13/d21/d59/la6 0 2024-03-23T08:23:24.794 INFO:tasks.workunit.client.1.smithi193.stdout:8/537: link df/d10/c75 df/d4b/cb5 0 2024-03-23T08:23:24.795 INFO:tasks.workunit.client.1.smithi193.stdout:4/614: rmdir d11/d2d/d58/d80 39 2024-03-23T08:23:24.796 INFO:tasks.workunit.client.1.smithi193.stdout:8/538: dread - df/d63/d84/f8a zero size 2024-03-23T08:23:24.798 INFO:tasks.workunit.client.1.smithi193.stdout:6/475: link d0/c67 d0/d27/d81/df/d11/d3b/d51/c98 0 2024-03-23T08:23:24.798 INFO:tasks.workunit.client.1.smithi193.stdout:4/615: write d11/d1a/d33/d42/db2/dbe/d50/d49/f4f [1940271,63899] 0 2024-03-23T08:23:24.800 INFO:tasks.workunit.client.1.smithi193.stdout:0/547: sync 2024-03-23T08:23:24.800 INFO:tasks.workunit.client.1.smithi193.stdout:3/476: creat d1/de/d13/d21/d50/d6f/fa7 x:0 0 0 2024-03-23T08:23:24.802 INFO:tasks.workunit.client.1.smithi193.stdout:6/476: symlink d0/d27/d81/df/d11/d36/d61/d7c/l99 0 2024-03-23T08:23:24.803 INFO:tasks.workunit.client.1.smithi193.stdout:3/477: chown d1/de/d13/d21/d50 1176737 1 2024-03-23T08:23:24.804 INFO:tasks.workunit.client.1.smithi193.stdout:6/477: symlink d0/d27/l9a 0 2024-03-23T08:23:24.805 INFO:tasks.workunit.client.1.smithi193.stdout:0/548: write db/d11/d14/d24/d47/d59/faa [658671,103989] 0 2024-03-23T08:23:24.807 INFO:tasks.workunit.client.1.smithi193.stdout:5/610: dread d0/d1c/d7a/db1/f76 [0,4194304] 0 2024-03-23T08:23:24.807 INFO:tasks.workunit.client.1.smithi193.stdout:0/549: chown db/d52 0 1 2024-03-23T08:23:24.807 INFO:tasks.workunit.client.1.smithi193.stdout:4/616: getdents d11/d2d/d58/d8b/dbc/d88/d99 0 2024-03-23T08:23:24.808 INFO:tasks.workunit.client.1.smithi193.stdout:0/550: fdatasync db/d52/f5c 0 2024-03-23T08:23:24.809 INFO:tasks.workunit.client.1.smithi193.stdout:5/611: mkdir d0/d5a/ddf 0 2024-03-23T08:23:24.810 INFO:tasks.workunit.client.1.smithi193.stdout:4/617: unlink d11/d2d/d3a/f6f 0 2024-03-23T08:23:24.813 INFO:tasks.workunit.client.1.smithi193.stdout:7/549: truncate d6/d9/d1f/d3e/d65/d70/f83 740771 0 2024-03-23T08:23:24.813 INFO:tasks.workunit.client.1.smithi193.stdout:0/551: creat db/d11/d14/d24/d2a/d4b/fbd x:0 0 0 2024-03-23T08:23:24.816 INFO:tasks.workunit.client.1.smithi193.stdout:5/612: fsync d0/d1c/d7a/db1/d72/d99/fb5 0 2024-03-23T08:23:24.816 INFO:tasks.workunit.client.1.smithi193.stdout:6/478: dwrite d0/d27/f91 [0,4194304] 0 2024-03-23T08:23:24.818 INFO:tasks.workunit.client.1.smithi193.stdout:8/539: truncate df/d10/f17 668710 0 2024-03-23T08:23:24.820 INFO:tasks.workunit.client.1.smithi193.stdout:8/540: readlink df/d4b/d4d/l28 0 2024-03-23T08:23:24.824 INFO:tasks.workunit.client.1.smithi193.stdout:3/478: dread d1/de/d13/d21/d25/d2b/f35 [0,4194304] 0 2024-03-23T08:23:24.824 INFO:tasks.workunit.client.1.smithi193.stdout:1/553: dwrite da/dd/df/d2e/f93 [0,4194304] 0 2024-03-23T08:23:24.826 INFO:tasks.workunit.client.1.smithi193.stdout:3/479: write d1/de/d13/f33 [5096573,26610] 0 2024-03-23T08:23:24.828 INFO:tasks.workunit.client.1.smithi193.stdout:4/618: creat d11/d1a/d33/d42/db2/dbe/fd3 x:0 0 0 2024-03-23T08:23:24.829 INFO:tasks.workunit.client.1.smithi193.stdout:5/613: symlink d0/le0 0 2024-03-23T08:23:24.830 INFO:tasks.workunit.client.1.smithi193.stdout:4/619: chown d11/d1a/d33/d42/d52/ld2 5254 1 2024-03-23T08:23:24.832 INFO:tasks.workunit.client.1.smithi193.stdout:4/620: fsync d11/d1a/d33/d42/d52/d70/fba 0 2024-03-23T08:23:24.832 INFO:tasks.workunit.client.1.smithi193.stdout:5/614: truncate d0/d1c/d7a/f7d 29992 0 2024-03-23T08:23:24.838 INFO:tasks.workunit.client.1.smithi193.stdout:3/480: rename d1/de/d13/d21/d50/d6f/d6e/d94 to d1/de/d13/d21/d25/d34/da8 0 2024-03-23T08:23:24.839 INFO:tasks.workunit.client.1.smithi193.stdout:8/541: creat df/d10/d19/fb6 x:0 0 0 2024-03-23T08:23:24.841 INFO:tasks.workunit.client.1.smithi193.stdout:6/479: creat d0/d27/d81/df/f9b x:0 0 0 2024-03-23T08:23:24.842 INFO:tasks.workunit.client.1.smithi193.stdout:1/554: symlink da/d13/lb7 0 2024-03-23T08:23:24.843 INFO:tasks.workunit.client.1.smithi193.stdout:8/542: mknod df/d4b/cb7 0 2024-03-23T08:23:24.844 INFO:tasks.workunit.client.1.smithi193.stdout:0/552: getdents db/d11/d14/d24/d2a/d96 0 2024-03-23T08:23:24.854 INFO:tasks.workunit.client.1.smithi193.stdout:4/621: dread d11/d1a/f1c [8388608,4194304] 0 2024-03-23T08:23:24.855 INFO:tasks.workunit.client.1.smithi193.stdout:6/480: fdatasync d0/d27/d81/df/d11/d36/d61/d12/f1b 0 2024-03-23T08:23:24.856 INFO:tasks.workunit.client.1.smithi193.stdout:3/481: dwrite d1/de/d13/d21/d27/f1f [0,4194304] 0 2024-03-23T08:23:24.857 INFO:tasks.workunit.client.1.smithi193.stdout:8/543: rename df/f12 to df/d4b/d4d/d9c/fb8 0 2024-03-23T08:23:24.857 INFO:tasks.workunit.client.1.smithi193.stdout:4/622: write fb [4225874,107358] 0 2024-03-23T08:23:24.861 INFO:tasks.workunit.client.1.smithi193.stdout:4/623: readlink d11/d1a/d33/l56 0 2024-03-23T08:23:24.866 INFO:tasks.workunit.client.1.smithi193.stdout:3/482: chown d1/de/d13/d21/d50/d6f/fa7 4284857 1 2024-03-23T08:23:24.868 INFO:tasks.workunit.client.1.smithi193.stdout:0/553: dwrite db/d11/d14/d21/f78 [0,4194304] 0 2024-03-23T08:23:24.870 INFO:tasks.workunit.client.1.smithi193.stdout:3/483: write d1/f1e [5506931,117067] 0 2024-03-23T08:23:24.871 INFO:tasks.workunit.client.1.smithi193.stdout:5/615: dread d0/f12 [0,4194304] 0 2024-03-23T08:23:24.874 INFO:tasks.workunit.client.1.smithi193.stdout:6/481: dwrite d0/d27/d81/f86 [0,4194304] 0 2024-03-23T08:23:24.876 INFO:tasks.workunit.client.1.smithi193.stdout:4/624: truncate d11/d1a/d33/d42/db2/dbe/d50/f61 1114077 0 2024-03-23T08:23:24.876 INFO:tasks.workunit.client.1.smithi193.stdout:9/603: sync 2024-03-23T08:23:24.879 INFO:tasks.workunit.client.1.smithi193.stdout:8/544: dwrite df/d10/d3e/fb0 [0,4194304] 0 2024-03-23T08:23:24.880 INFO:tasks.workunit.client.1.smithi193.stdout:6/482: fdatasync d0/d27/d81/df/f9b 0 2024-03-23T08:23:24.882 INFO:tasks.workunit.client.1.smithi193.stdout:5/616: mkdir d0/d1f/d93/de1 0 2024-03-23T08:23:24.883 INFO:tasks.workunit.client.1.smithi193.stdout:1/555: dread da/f59 [0,4194304] 0 2024-03-23T08:23:24.886 INFO:tasks.workunit.client.1.smithi193.stdout:3/484: creat d1/de/d78/fa9 x:0 0 0 2024-03-23T08:23:24.889 INFO:tasks.workunit.client.1.smithi193.stdout:1/556: read da/dd/f14 [2242409,70755] 0 2024-03-23T08:23:24.892 INFO:tasks.workunit.client.1.smithi193.stdout:4/625: fsync d11/d1a/d33/d8f/db8/f75 0 2024-03-23T08:23:24.893 INFO:tasks.workunit.client.1.smithi193.stdout:6/483: unlink d0/d27/d81/df/d11/d36/d61/d12/d21/f29 0 2024-03-23T08:23:24.893 INFO:tasks.workunit.client.1.smithi193.stdout:8/545: fsync df/f34 0 2024-03-23T08:23:24.899 INFO:tasks.workunit.client.1.smithi193.stdout:8/546: mknod df/d10/d3e/d54/cb9 0 2024-03-23T08:23:24.903 INFO:tasks.workunit.client.1.smithi193.stdout:1/557: getdents da/d13/d24 0 2024-03-23T08:23:24.906 INFO:tasks.workunit.client.1.smithi193.stdout:5/617: dwrite d0/d1c/d7a/db1/d72/d73/f6c [0,4194304] 0 2024-03-23T08:23:24.906 INFO:tasks.workunit.client.1.smithi193.stdout:9/604: dwrite d2/d7/d31/f4f [4194304,4194304] 0 2024-03-23T08:23:24.911 INFO:tasks.workunit.client.1.smithi193.stdout:0/554: dread db/d11/d14/d21/f49 [0,4194304] 0 2024-03-23T08:23:24.911 INFO:tasks.workunit.client.1.smithi193.stdout:5/618: mkdir d0/d1c/d3b/d4b/de2 0 2024-03-23T08:23:24.912 INFO:tasks.workunit.client.1.smithi193.stdout:6/484: rename d0/d27/d81/df/d11/d36/d61/d3a/d7a to d0/d27/d81/df/d11/d3b/d51/d9c 0 2024-03-23T08:23:24.916 INFO:tasks.workunit.client.1.smithi193.stdout:9/605: truncate d2/d7/db/d1b/f81 855713 0 2024-03-23T08:23:24.916 INFO:tasks.workunit.client.1.smithi193.stdout:0/555: readlink db/d11/d14/d24/d47/l5b 0 2024-03-23T08:23:24.917 INFO:tasks.workunit.client.1.smithi193.stdout:5/619: dread - d0/d1c/d7a/db1/d72/d73/d57/fb4 zero size 2024-03-23T08:23:24.918 INFO:tasks.workunit.client.1.smithi193.stdout:4/626: dread d11/d1a/d33/d42/db2/dbe/d50/d49/f4f [0,4194304] 0 2024-03-23T08:23:24.919 INFO:tasks.workunit.client.1.smithi193.stdout:1/558: rename da/dd/df/d2e/f50 to da/dd/df/d52/d82/fb8 0 2024-03-23T08:23:24.920 INFO:tasks.workunit.client.1.smithi193.stdout:3/485: dread d1/de/d13/d21/d25/d2b/f39 [4194304,4194304] 0 2024-03-23T08:23:24.921 INFO:tasks.workunit.client.1.smithi193.stdout:9/606: creat d2/d7/fcb x:0 0 0 2024-03-23T08:23:24.921 INFO:tasks.workunit.client.1.smithi193.stdout:6/485: mkdir d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/d9d 0 2024-03-23T08:23:24.922 INFO:tasks.workunit.client.1.smithi193.stdout:5/620: mknod d0/d1f/d93/de1/ce3 0 2024-03-23T08:23:24.924 INFO:tasks.workunit.client.1.smithi193.stdout:2/633: sync 2024-03-23T08:23:24.924 INFO:tasks.workunit.client.1.smithi193.stdout:1/559: chown da/dd/df/d2e/c3e 492 1 2024-03-23T08:23:24.925 INFO:tasks.workunit.client.1.smithi193.stdout:9/607: chown d2/d7/d31/d3c/f6e 10714103 1 2024-03-23T08:23:24.925 INFO:tasks.workunit.client.1.smithi193.stdout:5/621: dread - d0/d1f/d93/fdb zero size 2024-03-23T08:23:24.926 INFO:tasks.workunit.client.1.smithi193.stdout:3/486: readlink d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/l92 0 2024-03-23T08:23:24.928 INFO:tasks.workunit.client.1.smithi193.stdout:8/547: dwrite df/d10/f3b [0,4194304] 0 2024-03-23T08:23:24.934 INFO:tasks.workunit.client.1.smithi193.stdout:0/556: write db/d11/f12 [3438310,29889] 0 2024-03-23T08:23:24.934 INFO:tasks.workunit.client.1.smithi193.stdout:9/608: chown d2/d7/db/d17/f54 21 1 2024-03-23T08:23:24.946 INFO:tasks.workunit.client.1.smithi193.stdout:5/622: rename d0/d1c/d3b/d85/d97/lb2 to d0/d1f/d93/de1/le4 0 2024-03-23T08:23:24.946 INFO:tasks.workunit.client.1.smithi193.stdout:9/609: chown d2/d7/db/d32/d6b/c72 4057800 1 2024-03-23T08:23:24.948 INFO:tasks.workunit.client.1.smithi193.stdout:0/557: creat db/d11/d14/d24/d2a/fbe x:0 0 0 2024-03-23T08:23:24.953 INFO:tasks.workunit.client.1.smithi193.stdout:4/627: dwrite d11/d1a/d33/d42/db2/dbe/d50/fab [0,4194304] 0 2024-03-23T08:23:24.955 INFO:tasks.workunit.client.1.smithi193.stdout:6/486: dwrite d0/d27/d81/df/d11/d36/d61/f8e [0,4194304] 0 2024-03-23T08:23:24.958 INFO:tasks.workunit.client.1.smithi193.stdout:1/560: dwrite da/d13/d3f/d5a/f67 [0,4194304] 0 2024-03-23T08:23:24.961 INFO:tasks.workunit.client.1.smithi193.stdout:5/623: chown d0/d1f/la9 26270 1 2024-03-23T08:23:24.965 INFO:tasks.workunit.client.1.smithi193.stdout:3/487: dwrite d1/de/d13/d96/d99/f2d [4194304,4194304] 0 2024-03-23T08:23:24.966 INFO:tasks.workunit.client.1.smithi193.stdout:2/634: dwrite d0/d1b/f21 [0,4194304] 0 2024-03-23T08:23:24.967 INFO:tasks.workunit.client.1.smithi193.stdout:8/548: dwrite df/d4b/d4d/f45 [0,4194304] 0 2024-03-23T08:23:24.969 INFO:tasks.workunit.client.1.smithi193.stdout:1/561: mknod da/dd/df/d75/cb9 0 2024-03-23T08:23:24.973 INFO:tasks.workunit.client.1.smithi193.stdout:9/610: getdents d2/d7/db/d17/d4b/d8d 0 2024-03-23T08:23:24.973 INFO:tasks.workunit.client.1.smithi193.stdout:0/558: creat db/d11/d14/d24/fbf x:0 0 0 2024-03-23T08:23:24.974 INFO:tasks.workunit.client.1.smithi193.stdout:4/628: rename d11/d2d/d58/d80/c9b to d11/d2d/da8/cd4 0 2024-03-23T08:23:24.975 INFO:tasks.workunit.client.1.smithi193.stdout:5/624: read d0/d1c/d7a/db1/d72/d99/fb5 [785154,73922] 0 2024-03-23T08:23:24.976 INFO:tasks.workunit.client.1.smithi193.stdout:1/562: write da/dd/f78 [365277,12872] 0 2024-03-23T08:23:24.977 INFO:tasks.workunit.client.1.smithi193.stdout:8/549: chown df/d4b/d4d/c3d 20253 1 2024-03-23T08:23:24.980 INFO:tasks.workunit.client.1.smithi193.stdout:3/488: creat d1/de/d13/d21/d25/d2b/d3c/faa x:0 0 0 2024-03-23T08:23:24.980 INFO:tasks.workunit.client.1.smithi193.stdout:8/550: readlink df/d62/d8b/dae/l5e 0 2024-03-23T08:23:24.980 INFO:tasks.workunit.client.1.smithi193.stdout:0/559: symlink db/d11/d14/d21/d33/d48/d85/lc0 0 2024-03-23T08:23:24.984 INFO:tasks.workunit.client.1.smithi193.stdout:4/629: unlink d11/d2d/d58/d8b/dbc/f20 0 2024-03-23T08:23:24.986 INFO:tasks.workunit.client.1.smithi193.stdout:6/487: truncate d0/d27/d81/df/d11/d36/d61/d12/f35 3071087 0 2024-03-23T08:23:24.987 INFO:tasks.workunit.client.1.smithi193.stdout:1/563: symlink da/lba 0 2024-03-23T08:23:24.988 INFO:tasks.workunit.client.1.smithi193.stdout:5/625: dwrite d0/d1c/d7a/db1/f76 [0,4194304] 0 2024-03-23T08:23:24.988 INFO:tasks.workunit.client.1.smithi193.stdout:6/488: stat d0/d27/d81/df/d11/d36/d61/d12/d21/c64 0 2024-03-23T08:23:24.991 INFO:tasks.workunit.client.1.smithi193.stdout:4/630: rmdir d11/d27 39 2024-03-23T08:23:24.991 INFO:tasks.workunit.client.1.smithi193.stdout:7/550: sync 2024-03-23T08:23:24.991 INFO:tasks.workunit.client.1.smithi193.stdout:8/551: symlink df/lba 0 2024-03-23T08:23:24.994 INFO:tasks.workunit.client.1.smithi193.stdout:3/489: mkdir d1/de/d13/d21/dab 0 2024-03-23T08:23:24.996 INFO:tasks.workunit.client.1.smithi193.stdout:7/551: chown d6/f14 15301 1 2024-03-23T08:23:24.997 INFO:tasks.workunit.client.1.smithi193.stdout:0/560: getdents db/d11/d14/d24/d2a/d4b 0 2024-03-23T08:23:24.999 INFO:tasks.workunit.client.1.smithi193.stdout:4/631: write d11/d2d/d3a/fc1 [612263,77003] 0 2024-03-23T08:23:24.999 INFO:tasks.workunit.client.1.smithi193.stdout:3/490: symlink d1/de/d13/d96/lac 0 2024-03-23T08:23:25.000 INFO:tasks.workunit.client.1.smithi193.stdout:6/489: unlink d0/d27/d81/df/d11/d36/d61/d12/f52 0 2024-03-23T08:23:25.001 INFO:tasks.workunit.client.1.smithi193.stdout:7/552: chown d6/d9/fa5 3906303 1 2024-03-23T08:23:25.002 INFO:tasks.workunit.client.1.smithi193.stdout:5/626: link d0/d1c/f24 d0/d1c/d7a/db1/d72/d73/d45/daa/fe5 0 2024-03-23T08:23:25.003 INFO:tasks.workunit.client.1.smithi193.stdout:8/552: creat df/d10/fbb x:0 0 0 2024-03-23T08:23:25.005 INFO:tasks.workunit.client.1.smithi193.stdout:1/564: dwrite da/dd/df/d52/f7a [0,4194304] 0 2024-03-23T08:23:25.009 INFO:tasks.workunit.client.1.smithi193.stdout:2/635: dread d0/d1b/d14/d56/d7c/d7e/f88 [0,4194304] 0 2024-03-23T08:23:25.009 INFO:tasks.workunit.client.1.smithi193.stdout:5/627: dread - d0/d1c/d7a/db1/d72/d99/dd8/fd1 zero size 2024-03-23T08:23:25.010 INFO:tasks.workunit.client.1.smithi193.stdout:4/632: unlink d11/f92 0 2024-03-23T08:23:25.013 INFO:tasks.workunit.client.1.smithi193.stdout:6/490: getdents d0/d27/d72 0 2024-03-23T08:23:25.014 INFO:tasks.workunit.client.1.smithi193.stdout:2/636: chown d0/d1b/d14/cae 56 1 2024-03-23T08:23:25.015 INFO:tasks.workunit.client.1.smithi193.stdout:3/491: link d1/de/d13/d21/f81 d1/de/d28/fad 0 2024-03-23T08:23:25.015 INFO:tasks.workunit.client.1.smithi193.stdout:1/565: rename da/d13/ca1 to da/dd/df/d7e/d7f/cbb 0 2024-03-23T08:23:25.017 INFO:tasks.workunit.client.1.smithi193.stdout:0/561: dwrite db/d11/d14/d24/d47/d59/f72 [0,4194304] 0 2024-03-23T08:23:25.022 INFO:tasks.workunit.client.1.smithi193.stdout:1/566: read - da/dd/f64 zero size 2024-03-23T08:23:25.023 INFO:tasks.workunit.client.1.smithi193.stdout:8/553: dwrite df/d10/d19/f72 [0,4194304] 0 2024-03-23T08:23:25.024 INFO:tasks.workunit.client.1.smithi193.stdout:2/637: creat d0/d28/daf/fb9 x:0 0 0 2024-03-23T08:23:25.025 INFO:tasks.workunit.client.1.smithi193.stdout:3/492: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/dae 0 2024-03-23T08:23:25.031 INFO:tasks.workunit.client.1.smithi193.stdout:8/554: creat df/d63/fbc x:0 0 0 2024-03-23T08:23:25.031 INFO:tasks.workunit.client.1.smithi193.stdout:0/562: truncate db/d11/d14/d24/d47/f61 326390 0 2024-03-23T08:23:25.031 INFO:tasks.workunit.client.1.smithi193.stdout:0/563: chown db/d11/d14/d24/d2a/d96 2 1 2024-03-23T08:23:25.031 INFO:tasks.workunit.client.1.smithi193.stdout:1/567: read da/d13/d3f/f54 [943144,88751] 0 2024-03-23T08:23:25.031 INFO:tasks.workunit.client.1.smithi193.stdout:8/555: truncate df/fb3 1034843 0 2024-03-23T08:23:25.034 INFO:tasks.workunit.client.1.smithi193.stdout:0/564: fdatasync db/d11/d14/d24/d47/d59/f79 0 2024-03-23T08:23:25.034 INFO:tasks.workunit.client.1.smithi193.stdout:5/628: write d0/d1c/d7a/db1/d72/d73/d45/daa/fe5 [4370734,6336] 0 2024-03-23T08:23:25.037 INFO:tasks.workunit.client.1.smithi193.stdout:3/493: dread d1/de/f10 [4194304,4194304] 0 2024-03-23T08:23:25.038 INFO:tasks.workunit.client.1.smithi193.stdout:2/638: truncate d0/d1b/d14/f40 433904 0 2024-03-23T08:23:25.038 INFO:tasks.workunit.client.1.smithi193.stdout:8/556: write df/d10/d3e/d4e/f9d [792773,79978] 0 2024-03-23T08:23:25.038 INFO:tasks.workunit.client.1.smithi193.stdout:1/568: dread da/d13/d3f/f87 [0,4194304] 0 2024-03-23T08:23:25.040 INFO:tasks.workunit.client.1.smithi193.stdout:0/565: mkdir db/d11/d14/d21/d33/d69/d9b/dc1 0 2024-03-23T08:23:25.041 INFO:tasks.workunit.client.1.smithi193.stdout:5/629: write d0/d1c/d7a/db1/d72/d73/d45/fa0 [908425,68818] 0 2024-03-23T08:23:25.042 INFO:tasks.workunit.client.1.smithi193.stdout:2/639: write d0/d1b/d19/f80 [625404,99464] 0 2024-03-23T08:23:25.042 INFO:tasks.workunit.client.1.smithi193.stdout:1/569: write da/dd/f39 [295156,15028] 0 2024-03-23T08:23:25.043 INFO:tasks.workunit.client.1.smithi193.stdout:8/557: mknod df/d10/d19/cbd 0 2024-03-23T08:23:25.044 INFO:tasks.workunit.client.1.smithi193.stdout:5/630: chown d0/d1c/d7a/db1/d72/d73/f6c 1532 1 2024-03-23T08:23:25.045 INFO:tasks.workunit.client.1.smithi193.stdout:0/566: creat db/d11/d14/d3b/d71/fc2 x:0 0 0 2024-03-23T08:23:25.045 INFO:tasks.workunit.client.1.smithi193.stdout:1/570: chown da/d13/c6d 3322 1 2024-03-23T08:23:25.045 INFO:tasks.workunit.client.1.smithi193.stdout:3/494: mkdir d1/de/d13/d21/d27/daf 0 2024-03-23T08:23:25.045 INFO:tasks.workunit.client.1.smithi193.stdout:8/558: mknod df/d10/d3e/d4e/cbe 0 2024-03-23T08:23:25.046 INFO:tasks.workunit.client.1.smithi193.stdout:2/640: stat d0/d1b/d14/d66/d50/d92/f9a 0 2024-03-23T08:23:25.049 INFO:tasks.workunit.client.1.smithi193.stdout:1/571: fdatasync da/dd/df/d7e/d7f/f8a 0 2024-03-23T08:23:25.051 INFO:tasks.workunit.client.1.smithi193.stdout:5/631: creat d0/d1c/d7a/db1/d72/d73/d57/d6d/fe6 x:0 0 0 2024-03-23T08:23:25.052 INFO:tasks.workunit.client.1.smithi193.stdout:1/572: truncate f8 1185152 0 2024-03-23T08:23:25.056 INFO:tasks.workunit.client.1.smithi193.stdout:1/573: chown da/dd/df/d75/d98 21580 1 2024-03-23T08:23:25.056 INFO:tasks.workunit.client.1.smithi193.stdout:3/495: truncate d1/de/d13/d21/d25/d2b/f39 840931 0 2024-03-23T08:23:25.062 INFO:tasks.workunit.client.1.smithi193.stdout:1/574: fsync da/dd/df/f29 0 2024-03-23T08:23:25.064 INFO:tasks.workunit.client.1.smithi193.stdout:5/632: chown d0/d1c/d7a/db1/d72/d73/d45/f56 8493187 1 2024-03-23T08:23:25.064 INFO:tasks.workunit.client.1.smithi193.stdout:0/567: dwrite db/d11/d14/d24/f7d [0,4194304] 0 2024-03-23T08:23:25.065 INFO:tasks.workunit.client.1.smithi193.stdout:4/633: dread d11/d1a/d33/d8f/db8/f25 [0,4194304] 0 2024-03-23T08:23:25.068 INFO:tasks.workunit.client.1.smithi193.stdout:8/559: dwrite df/d10/d67/f6c [0,4194304] 0 2024-03-23T08:23:25.069 INFO:tasks.workunit.client.1.smithi193.stdout:3/496: symlink d1/de/d13/d21/d27/daf/lb0 0 2024-03-23T08:23:25.071 INFO:tasks.workunit.client.1.smithi193.stdout:2/641: dwrite d0/d1b/d14/d66/d50/d8f/f90 [0,4194304] 0 2024-03-23T08:23:25.071 INFO:tasks.workunit.client.1.smithi193.stdout:4/634: creat d11/fd5 x:0 0 0 2024-03-23T08:23:25.071 INFO:tasks.workunit.client.1.smithi193.stdout:5/633: readlink d0/d1c/d7a/db1/d72/d73/d45/d6e/lac 0 2024-03-23T08:23:25.073 INFO:tasks.workunit.client.1.smithi193.stdout:4/635: dread - d11/d8d/f97 zero size 2024-03-23T08:23:25.075 INFO:tasks.workunit.client.1.smithi193.stdout:1/575: mknod da/dd/df/d7c/cbc 0 2024-03-23T08:23:25.076 INFO:tasks.workunit.client.1.smithi193.stdout:2/642: write d0/d37/d48/f55 [4838618,129993] 0 2024-03-23T08:23:25.078 INFO:tasks.workunit.client.1.smithi193.stdout:1/576: chown da/c2c 75399071 1 2024-03-23T08:23:25.078 INFO:tasks.workunit.client.1.smithi193.stdout:4/636: write d11/d1a/d33/d44/f82 [1882212,73379] 0 2024-03-23T08:23:25.078 INFO:tasks.workunit.client.1.smithi193.stdout:5/634: fdatasync d0/d1c/d7a/db1/d72/d73/d45/da2/fdd 0 2024-03-23T08:23:25.078 INFO:tasks.workunit.client.1.smithi193.stdout:3/497: creat d1/de/d13/d21/d25/d2b/fb1 x:0 0 0 2024-03-23T08:23:25.079 INFO:tasks.workunit.client.1.smithi193.stdout:1/577: readlink da/d13/d24/l31 0 2024-03-23T08:23:25.080 INFO:tasks.workunit.client.1.smithi193.stdout:4/637: readlink d11/d2d/l89 0 2024-03-23T08:23:25.082 INFO:tasks.workunit.client.1.smithi193.stdout:8/560: creat df/d10/d3e/fbf x:0 0 0 2024-03-23T08:23:25.085 INFO:tasks.workunit.client.1.smithi193.stdout:2/643: getdents d0/d28/daf 0 2024-03-23T08:23:25.086 INFO:tasks.workunit.client.1.smithi193.stdout:1/578: write da/d13/d3f/d5a/d76/d77/f6c [1144305,46237] 0 2024-03-23T08:23:25.086 INFO:tasks.workunit.client.1.smithi193.stdout:4/638: mknod d11/d1a/d33/d42/db2/dbe/cd6 0 2024-03-23T08:23:25.090 INFO:tasks.workunit.client.1.smithi193.stdout:0/568: truncate db/d11/d14/d24/f8d 1183436 0 2024-03-23T08:23:25.097 INFO:tasks.workunit.client.1.smithi193.stdout:9/611: sync 2024-03-23T08:23:25.097 INFO:tasks.workunit.client.1.smithi193.stdout:7/553: sync 2024-03-23T08:23:25.098 INFO:tasks.workunit.client.1.smithi193.stdout:3/498: write d1/de/d28/fad [1854870,36956] 0 2024-03-23T08:23:25.100 INFO:tasks.workunit.client.1.smithi193.stdout:5/635: dwrite d0/d1c/d3b/fce [0,4194304] 0 2024-03-23T08:23:25.102 INFO:tasks.workunit.client.1.smithi193.stdout:8/561: dwrite df/d10/fbb [0,4194304] 0 2024-03-23T08:23:25.103 INFO:tasks.workunit.client.1.smithi193.stdout:2/644: truncate d0/d1b/d14/f95 3816296 0 2024-03-23T08:23:25.111 INFO:tasks.workunit.client.1.smithi193.stdout:9/612: write d2/d7/f11 [5330871,92979] 0 2024-03-23T08:23:25.114 INFO:tasks.workunit.client.1.smithi193.stdout:1/579: dwrite da/d13/d24/f79 [0,4194304] 0 2024-03-23T08:23:25.115 INFO:tasks.workunit.client.1.smithi193.stdout:7/554: write d6/d9/d1f/d23/d33/d3c/f4b [1728428,130753] 0 2024-03-23T08:23:25.115 INFO:tasks.workunit.client.1.smithi193.stdout:5/636: truncate d0/d1f/fb3 5010888 0 2024-03-23T08:23:25.116 INFO:tasks.workunit.client.1.smithi193.stdout:2/645: mkdir d0/d1b/d14/d66/d50/d92/da9/dba 0 2024-03-23T08:23:25.118 INFO:tasks.workunit.client.1.smithi193.stdout:4/639: dwrite d11/d1a/d33/d42/d52/d70/f74 [0,4194304] 0 2024-03-23T08:23:25.122 INFO:tasks.workunit.client.1.smithi193.stdout:0/569: link db/d11/d14/d24/f25 db/d11/d14/d21/d33/d48/fc3 0 2024-03-23T08:23:25.125 INFO:tasks.workunit.client.1.smithi193.stdout:8/562: truncate df/d10/d19/fa5 33857 0 2024-03-23T08:23:25.126 INFO:tasks.workunit.client.1.smithi193.stdout:2/646: creat d0/d1b/d14/d56/da3/fbb x:0 0 0 2024-03-23T08:23:25.126 INFO:tasks.workunit.client.1.smithi193.stdout:5/637: mknod d0/d1c/d7a/db1/d72/d73/d57/ce7 0 2024-03-23T08:23:25.126 INFO:tasks.workunit.client.1.smithi193.stdout:4/640: write d11/d2d/d58/d80/d9f/fa2 [499974,61508] 0 2024-03-23T08:23:25.129 INFO:tasks.workunit.client.1.smithi193.stdout:8/563: write df/f1f [4040997,28596] 0 2024-03-23T08:23:25.130 INFO:tasks.workunit.client.1.smithi193.stdout:3/499: rename d1/de/d13/d21/d25/d2b/d3c/d40/c90 to d1/de/d13/d21/d50/cb2 0 2024-03-23T08:23:25.130 INFO:tasks.workunit.client.1.smithi193.stdout:8/564: dread - df/d62/fa2 zero size 2024-03-23T08:23:25.131 INFO:tasks.workunit.client.1.smithi193.stdout:4/641: write d11/d1a/d33/d42/db2/dbe/d50/f5b [1625623,2576] 0 2024-03-23T08:23:25.131 INFO:tasks.workunit.client.1.smithi193.stdout:5/638: chown d0/d1c/d7a/db1/d72/d73/d57/fb4 748674 1 2024-03-23T08:23:25.134 INFO:tasks.workunit.client.1.smithi193.stdout:1/580: rename da/d13/d3f/c99 to da/dd/df/d7e/d86/cbd 0 2024-03-23T08:23:25.134 INFO:tasks.workunit.client.1.smithi193.stdout:3/500: fdatasync d1/de/d13/f88 0 2024-03-23T08:23:25.139 INFO:tasks.workunit.client.1.smithi193.stdout:5/639: write d0/fe [4057342,114512] 0 2024-03-23T08:23:25.139 INFO:tasks.workunit.client.1.smithi193.stdout:9/613: write d2/d7/d31/d5f/f83 [411025,1602] 0 2024-03-23T08:23:25.141 INFO:tasks.workunit.client.1.smithi193.stdout:4/642: rmdir d11/d2d/d58/d8b 39 2024-03-23T08:23:25.144 INFO:tasks.workunit.client.1.smithi193.stdout:2/647: dwrite d0/d1b/d14/d66/d50/d92/da9/fab [0,4194304] 0 2024-03-23T08:23:25.145 INFO:tasks.workunit.client.1.smithi193.stdout:6/491: sync 2024-03-23T08:23:25.146 INFO:tasks.workunit.client.1.smithi193.stdout:3/501: dread - d1/de/d13/d21/d50/d44/f54 zero size 2024-03-23T08:23:25.147 INFO:tasks.workunit.client.1.smithi193.stdout:0/570: link db/d11/d14/d51/f99 db/d11/d14/d21/d33/d69/d9b/dc1/fc4 0 2024-03-23T08:23:25.147 INFO:tasks.workunit.client.1.smithi193.stdout:2/648: stat d0/d1b 0 2024-03-23T08:23:25.148 INFO:tasks.workunit.client.1.smithi193.stdout:1/581: dread da/d13/d3f/d5a/f8c [0,4194304] 0 2024-03-23T08:23:25.148 INFO:tasks.workunit.client.1.smithi193.stdout:6/492: fsync d0/d27/f91 0 2024-03-23T08:23:25.149 INFO:tasks.workunit.client.1.smithi193.stdout:4/643: unlink d11/d2d/d58/d80/fca 0 2024-03-23T08:23:25.152 INFO:tasks.workunit.client.1.smithi193.stdout:9/614: creat d2/d7/db/d17/d33/db2/fcc x:0 0 0 2024-03-23T08:23:25.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/644: chown d11/d2d/d58/d80/d9d 30336 1 2024-03-23T08:23:25.157 INFO:tasks.workunit.client.1.smithi193.stdout:3/502: rmdir d1/de/d13/d96 39 2024-03-23T08:23:25.157 INFO:tasks.workunit.client.1.smithi193.stdout:8/565: dread df/d63/f68 [0,4194304] 0 2024-03-23T08:23:25.159 INFO:tasks.workunit.client.1.smithi193.stdout:1/582: write da/d13/f6f [2943766,85358] 0 2024-03-23T08:23:25.161 INFO:tasks.workunit.client.1.smithi193.stdout:8/566: chown df/d10/la7 15 1 2024-03-23T08:23:25.162 INFO:tasks.workunit.client.1.smithi193.stdout:9/615: chown d2/d7/db/d1b/d5c/f94 371 1 2024-03-23T08:23:25.163 INFO:tasks.workunit.client.1.smithi193.stdout:7/555: dread d6/f2d [0,4194304] 0 2024-03-23T08:23:25.165 INFO:tasks.workunit.client.1.smithi193.stdout:3/503: write d1/de/d28/fad [2002144,126833] 0 2024-03-23T08:23:25.167 INFO:tasks.workunit.client.1.smithi193.stdout:0/571: dread db/d11/d14/d24/d2a/f83 [0,4194304] 0 2024-03-23T08:23:25.170 INFO:tasks.workunit.client.1.smithi193.stdout:9/616: mknod d2/d7/db/d17/d4b/ccd 0 2024-03-23T08:23:25.173 INFO:tasks.workunit.client.1.smithi193.stdout:4/645: truncate d11/d8e/dac/fbd 404977 0 2024-03-23T08:23:25.174 INFO:tasks.workunit.client.1.smithi193.stdout:2/649: dwrite d0/d1b/d14/d66/d50/d8f/f90 [0,4194304] 0 2024-03-23T08:23:25.176 INFO:tasks.workunit.client.1.smithi193.stdout:6/493: rename d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/l3c to d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/l9e 0 2024-03-23T08:23:25.177 INFO:tasks.workunit.client.1.smithi193.stdout:0/572: chown db/l20 1839650096 1 2024-03-23T08:23:25.179 INFO:tasks.workunit.client.1.smithi193.stdout:5/640: dwrite d0/fa [0,4194304] 0 2024-03-23T08:23:25.179 INFO:tasks.workunit.client.1.smithi193.stdout:3/504: chown d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f 81 1 2024-03-23T08:23:25.181 INFO:tasks.workunit.client.1.smithi193.stdout:2/650: write d0/d1b/d14/d66/d50/f93 [4162090,124399] 0 2024-03-23T08:23:25.182 INFO:tasks.workunit.client.1.smithi193.stdout:9/617: chown d2/d7/f36 2570405 1 2024-03-23T08:23:25.185 INFO:tasks.workunit.client.1.smithi193.stdout:7/556: getdents d6/d9/d1f/daf 0 2024-03-23T08:23:25.187 INFO:tasks.workunit.client.1.smithi193.stdout:8/567: dread df/d10/d19/f1b [0,4194304] 0 2024-03-23T08:23:25.187 INFO:tasks.workunit.client.1.smithi193.stdout:0/573: readlink db/d11/d14/d21/d33/l3c 0 2024-03-23T08:23:25.187 INFO:tasks.workunit.client.1.smithi193.stdout:5/641: chown d0/d1c/d7a/db1/d72/d73 39704 1 2024-03-23T08:23:25.189 INFO:tasks.workunit.client.1.smithi193.stdout:1/583: dwrite da/dd/df/f3b [0,4194304] 0 2024-03-23T08:23:25.192 INFO:tasks.workunit.client.1.smithi193.stdout:6/494: symlink d0/d27/d81/df/d11/d36/d61/d7c/l9f 0 2024-03-23T08:23:25.193 INFO:tasks.workunit.client.1.smithi193.stdout:2/651: dread - d0/d1b/d14/d66/fa0 zero size 2024-03-23T08:23:25.194 INFO:tasks.workunit.client.1.smithi193.stdout:3/505: rmdir d1/de/d13/d21/d27/daf 39 2024-03-23T08:23:25.196 INFO:tasks.workunit.client.1.smithi193.stdout:8/568: creat df/d62/d8b/fc0 x:0 0 0 2024-03-23T08:23:25.198 INFO:tasks.workunit.client.1.smithi193.stdout:2/652: readlink d0/d1b/d19/l3c 0 2024-03-23T08:23:25.199 INFO:tasks.workunit.client.1.smithi193.stdout:7/557: link d6/f32 d6/d9/d1f/d4d/d7c/d92/fbe 0 2024-03-23T08:23:25.200 INFO:tasks.workunit.client.1.smithi193.stdout:9/618: dwrite d2/d7/d31/f4f [4194304,4194304] 0 2024-03-23T08:23:25.200 INFO:tasks.workunit.client.1.smithi193.stdout:0/574: creat db/d11/d14/d24/d2a/d4d/d93/fc5 x:0 0 0 2024-03-23T08:23:25.200 INFO:tasks.workunit.client.1.smithi193.stdout:1/584: read da/d13/d24/f58 [634813,15002] 0 2024-03-23T08:23:25.201 INFO:tasks.workunit.client.1.smithi193.stdout:5/642: symlink d0/d1c/d7a/db1/d72/d73/d57/le8 0 2024-03-23T08:23:25.204 INFO:tasks.workunit.client.1.smithi193.stdout:2/653: rmdir d0/d28 39 2024-03-23T08:23:25.205 INFO:tasks.workunit.client.1.smithi193.stdout:3/506: stat d1/de/d13/d21/d50/d44/f54 0 2024-03-23T08:23:25.205 INFO:tasks.workunit.client.1.smithi193.stdout:6/495: symlink d0/d27/d81/df/d11/d36/d61/d3a/la0 0 2024-03-23T08:23:25.206 INFO:tasks.workunit.client.1.smithi193.stdout:5/643: mkdir d0/d5a/de9 0 2024-03-23T08:23:25.207 INFO:tasks.workunit.client.1.smithi193.stdout:0/575: mkdir db/dc6 0 2024-03-23T08:23:25.208 INFO:tasks.workunit.client.1.smithi193.stdout:0/576: readlink db/l94 0 2024-03-23T08:23:25.210 INFO:tasks.workunit.client.1.smithi193.stdout:9/619: write d2/d7/db/d17/d33/f64 [3099182,45924] 0 2024-03-23T08:23:25.210 INFO:tasks.workunit.client.1.smithi193.stdout:4/646: truncate d11/d1a/d33/d8f/db8/f6b 2489455 0 2024-03-23T08:23:25.211 INFO:tasks.workunit.client.1.smithi193.stdout:6/496: write d0/d27/d81/df/d11/f30 [532574,80667] 0 2024-03-23T08:23:25.212 INFO:tasks.workunit.client.1.smithi193.stdout:1/585: mknod da/d13/d24/dac/cbe 0 2024-03-23T08:23:25.215 INFO:tasks.workunit.client.1.smithi193.stdout:9/620: chown d2/d7/fcb 1870 1 2024-03-23T08:23:25.215 INFO:tasks.workunit.client.1.smithi193.stdout:5/644: rename d0/le0 to d0/d5a/lea 0 2024-03-23T08:23:25.216 INFO:tasks.workunit.client.1.smithi193.stdout:8/569: dwrite df/f33 [0,4194304] 0 2024-03-23T08:23:25.220 INFO:tasks.workunit.client.1.smithi193.stdout:1/586: readlink da/dd/l60 0 2024-03-23T08:23:25.224 INFO:tasks.workunit.client.1.smithi193.stdout:7/558: dwrite d6/d9/d1f/d23/d33/d38/d3b/f5a [0,4194304] 0 2024-03-23T08:23:25.228 INFO:tasks.workunit.client.1.smithi193.stdout:3/507: rename d1/c5 to d1/de/d13/d21/d59/cb3 0 2024-03-23T08:23:25.228 INFO:tasks.workunit.client.1.smithi193.stdout:4/647: creat d11/d1a/d33/dce/fd7 x:0 0 0 2024-03-23T08:23:25.228 INFO:tasks.workunit.client.1.smithi193.stdout:6/497: truncate d0/d27/d81/df/d11/d3b/d51/d9c/f8a 499943 0 2024-03-23T08:23:25.229 INFO:tasks.workunit.client.1.smithi193.stdout:2/654: dread d0/d1b/d14/f40 [0,4194304] 0 2024-03-23T08:23:25.230 INFO:tasks.workunit.client.1.smithi193.stdout:8/570: creat df/d10/d83/fc1 x:0 0 0 2024-03-23T08:23:25.232 INFO:tasks.workunit.client.1.smithi193.stdout:1/587: read da/dd/f78 [277327,64275] 0 2024-03-23T08:23:25.234 INFO:tasks.workunit.client.1.smithi193.stdout:6/498: chown d0/d27/d81/df/d11/d36 32086 1 2024-03-23T08:23:25.235 INFO:tasks.workunit.client.1.smithi193.stdout:7/559: truncate d6/d9/d1f/d3e/d65/d70/f7f 818483 0 2024-03-23T08:23:25.235 INFO:tasks.workunit.client.1.smithi193.stdout:2/655: chown d0/d28/d33/c87 20 1 2024-03-23T08:23:25.238 INFO:tasks.workunit.client.1.smithi193.stdout:8/571: rename df/d10/f3f to df/d10/d67/d7f/fc2 0 2024-03-23T08:23:25.240 INFO:tasks.workunit.client.1.smithi193.stdout:2/656: read - d0/d1b/da7/fb3 zero size 2024-03-23T08:23:25.242 INFO:tasks.workunit.client.1.smithi193.stdout:6/499: write d0/d27/d81/df/d11/d36/f44 [1269033,96230] 0 2024-03-23T08:23:25.243 INFO:tasks.workunit.client.1.smithi193.stdout:4/648: rename d11/d27/f73 to d11/d2d/da8/fd8 0 2024-03-23T08:23:25.243 INFO:tasks.workunit.client.1.smithi193.stdout:0/577: dwrite db/d11/d14/d21/f22 [0,4194304] 0 2024-03-23T08:23:25.245 INFO:tasks.workunit.client.1.smithi193.stdout:2/657: fsync d0/f1d 0 2024-03-23T08:23:25.245 INFO:tasks.workunit.client.1.smithi193.stdout:8/572: truncate df/d10/d19/f4f 3717507 0 2024-03-23T08:23:25.251 INFO:tasks.workunit.client.1.smithi193.stdout:7/560: creat d6/d9/d1f/d4d/d7c/d8e/fbf x:0 0 0 2024-03-23T08:23:25.252 INFO:tasks.workunit.client.1.smithi193.stdout:3/508: write d1/de/d13/d21/d27/f63 [1654947,61945] 0 2024-03-23T08:23:25.253 INFO:tasks.workunit.client.1.smithi193.stdout:6/500: write d0/d27/d81/df/d11/f5a [2811352,18777] 0 2024-03-23T08:23:25.253 INFO:tasks.workunit.client.1.smithi193.stdout:0/578: rename db/d11/d14/d21/d33/d69/d88 to db/d11/d14/d24/d47/dc7 0 2024-03-23T08:23:25.254 INFO:tasks.workunit.client.1.smithi193.stdout:2/658: creat d0/d1b/d77/d9b/fbc x:0 0 0 2024-03-23T08:23:25.258 INFO:tasks.workunit.client.1.smithi193.stdout:0/579: unlink db/d11/d14/f95 0 2024-03-23T08:23:25.258 INFO:tasks.workunit.client.1.smithi193.stdout:7/561: dread - d6/d9/d4e/d81/f93 zero size 2024-03-23T08:23:25.261 INFO:tasks.workunit.client.1.smithi193.stdout:2/659: write d0/d1b/d14/d66/d50/f93 [2034404,93122] 0 2024-03-23T08:23:25.261 INFO:tasks.workunit.client.1.smithi193.stdout:4/649: dwrite d11/f83 [8388608,4194304] 0 2024-03-23T08:23:25.263 INFO:tasks.workunit.client.1.smithi193.stdout:0/580: read db/f87 [13059,26481] 0 2024-03-23T08:23:25.264 INFO:tasks.workunit.client.1.smithi193.stdout:3/509: mknod d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/cb4 0 2024-03-23T08:23:25.267 INFO:tasks.workunit.client.1.smithi193.stdout:4/650: read d11/d2d/f68 [49023,90734] 0 2024-03-23T08:23:25.287 INFO:tasks.workunit.client.1.smithi193.stdout:7/562: mkdir d6/d9/d1f/d23/d33/d38/dc0 0 2024-03-23T08:23:25.288 INFO:tasks.workunit.client.1.smithi193.stdout:3/510: write d1/de/d13/d21/d25/d2b/fb1 [349552,8745] 0 2024-03-23T08:23:25.289 INFO:tasks.workunit.client.1.smithi193.stdout:8/573: truncate df/d10/d3e/f6b 3837222 0 2024-03-23T08:23:25.289 INFO:tasks.workunit.client.1.smithi193.stdout:6/501: dwrite d0/d27/d81/df/d11/d3b/f76 [0,4194304] 0 2024-03-23T08:23:25.290 INFO:tasks.workunit.client.1.smithi193.stdout:1/588: dwrite da/d13/d3f/f44 [0,4194304] 0 2024-03-23T08:23:25.291 INFO:tasks.workunit.client.1.smithi193.stdout:8/574: stat df/d62/fad 0 2024-03-23T08:23:25.294 INFO:tasks.workunit.client.1.smithi193.stdout:4/651: mknod d11/d1a/d65/cd9 0 2024-03-23T08:23:25.297 INFO:tasks.workunit.client.1.smithi193.stdout:3/511: write d1/de/d13/d21/d25/d2b/d3c/f6a [1157555,85527] 0 2024-03-23T08:23:25.300 INFO:tasks.workunit.client.1.smithi193.stdout:0/581: rename db/d11/d14/d21/d33/l3f to db/d11/d14/d21/d33/d69/d9b/dc1/lc8 0 2024-03-23T08:23:25.302 INFO:tasks.workunit.client.1.smithi193.stdout:1/589: chown da/dd/df/d52/d82/fb8 2002325 1 2024-03-23T08:23:25.303 INFO:tasks.workunit.client.1.smithi193.stdout:2/660: dwrite d0/d1b/d14/f40 [0,4194304] 0 2024-03-23T08:23:25.304 INFO:tasks.workunit.client.1.smithi193.stdout:6/502: mkdir d0/d27/d81/da1 0 2024-03-23T08:23:25.306 INFO:tasks.workunit.client.1.smithi193.stdout:4/652: mknod d11/d1a/d33/d42/db2/dbe/cda 0 2024-03-23T08:23:25.307 INFO:tasks.workunit.client.1.smithi193.stdout:1/590: write da/d13/f7d [431211,100532] 0 2024-03-23T08:23:25.308 INFO:tasks.workunit.client.1.smithi193.stdout:8/575: dwrite df/f18 [0,4194304] 0 2024-03-23T08:23:25.308 INFO:tasks.workunit.client.1.smithi193.stdout:0/582: mkdir db/d11/d14/d24/dc9 0 2024-03-23T08:23:25.311 INFO:tasks.workunit.client.1.smithi193.stdout:3/512: truncate d1/de/d13/d96/d99/f55 696424 0 2024-03-23T08:23:25.313 INFO:tasks.workunit.client.1.smithi193.stdout:4/653: symlink d11/d8d/ldb 0 2024-03-23T08:23:25.313 INFO:tasks.workunit.client.1.smithi193.stdout:3/513: fsync d1/de/d13/d21/d25/d2b/d3c/f6a 0 2024-03-23T08:23:25.314 INFO:tasks.workunit.client.1.smithi193.stdout:8/576: mknod df/d10/d3e/d54/cc3 0 2024-03-23T08:23:25.315 INFO:tasks.workunit.client.1.smithi193.stdout:6/503: symlink d0/d27/d81/df/d11/d36/la2 0 2024-03-23T08:23:25.316 INFO:tasks.workunit.client.1.smithi193.stdout:2/661: symlink d0/d28/d33/db7/lbd 0 2024-03-23T08:23:25.320 INFO:tasks.workunit.client.1.smithi193.stdout:0/583: rename db/d11/d14/d24/d2a/d4d/f7a to db/d11/d14/d21/d33/d69/d9b/fca 0 2024-03-23T08:23:25.321 INFO:tasks.workunit.client.1.smithi193.stdout:0/584: chown db/d11/d14/d24/d47/d59/l6e 107599871 1 2024-03-23T08:23:25.322 INFO:tasks.workunit.client.1.smithi193.stdout:8/577: unlink df/d10/d67/c79 0 2024-03-23T08:23:25.323 INFO:tasks.workunit.client.1.smithi193.stdout:8/578: write df/d4b/d4d/f69 [3977574,10892] 0 2024-03-23T08:23:25.328 INFO:tasks.workunit.client.1.smithi193.stdout:1/591: link da/dd/df/d7e/la2 da/d13/d24/dac/lbf 0 2024-03-23T08:23:25.329 INFO:tasks.workunit.client.1.smithi193.stdout:3/514: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/fb5 x:0 0 0 2024-03-23T08:23:25.330 INFO:tasks.workunit.client.1.smithi193.stdout:2/662: creat d0/d37/d48/fbe x:0 0 0 2024-03-23T08:23:25.331 INFO:tasks.workunit.client.1.smithi193.stdout:3/515: chown d1/de/d13/d96/d99/c61 109017 1 2024-03-23T08:23:25.332 INFO:tasks.workunit.client.1.smithi193.stdout:0/585: link db/d11/d14/d21/f49 db/d11/d14/d3b/d71/fcb 0 2024-03-23T08:23:25.333 INFO:tasks.workunit.client.1.smithi193.stdout:1/592: symlink da/d13/d3f/d5a/d76/d77/lc0 0 2024-03-23T08:23:25.333 INFO:tasks.workunit.client.1.smithi193.stdout:4/654: link d11/d1a/d33/d42/db2/dbe/la7 d11/d2d/d58/d80/db9/ldc 0 2024-03-23T08:23:25.337 INFO:tasks.workunit.client.1.smithi193.stdout:3/516: mknod d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/cb6 0 2024-03-23T08:23:25.337 INFO:tasks.workunit.client.1.smithi193.stdout:0/586: creat db/d11/d14/d24/d2a/d4d/d93/fcc x:0 0 0 2024-03-23T08:23:25.340 INFO:tasks.workunit.client.1.smithi193.stdout:8/579: rename df/d4b/d4d/c3d to df/d10/cc4 0 2024-03-23T08:23:25.341 INFO:tasks.workunit.client.1.smithi193.stdout:1/593: creat da/dd/df/d7e/d7f/db2/fc1 x:0 0 0 2024-03-23T08:23:25.343 INFO:tasks.workunit.client.1.smithi193.stdout:0/587: mknod db/d11/d14/d51/ccd 0 2024-03-23T08:23:25.344 INFO:tasks.workunit.client.1.smithi193.stdout:2/663: link d0/d1b/d14/d66/d4e/c94 d0/d37/cbf 0 2024-03-23T08:23:25.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/517: dwrite d1/de/d13/d21/d25/f7e [0,4194304] 0 2024-03-23T08:23:25.346 INFO:tasks.workunit.client.1.smithi193.stdout:2/664: chown d0/d1b/d19/f7f 1789046042 1 2024-03-23T08:23:25.347 INFO:tasks.workunit.client.1.smithi193.stdout:8/580: mkdir df/d63/d84/dc5 0 2024-03-23T08:23:25.347 INFO:tasks.workunit.client.1.smithi193.stdout:2/665: chown d0/d28/d33/d72 28066788 1 2024-03-23T08:23:25.347 INFO:tasks.workunit.client.1.smithi193.stdout:4/655: rmdir d11/d1a/db0 0 2024-03-23T08:23:25.349 INFO:tasks.workunit.client.1.smithi193.stdout:3/518: creat d1/de/d13/d21/d59/d60/fb7 x:0 0 0 2024-03-23T08:23:25.350 INFO:tasks.workunit.client.1.smithi193.stdout:0/588: creat db/d11/d14/d21/d33/d69/d9b/fce x:0 0 0 2024-03-23T08:23:25.350 INFO:tasks.workunit.client.1.smithi193.stdout:1/594: symlink da/dd/df/d52/d82/db0/lc2 0 2024-03-23T08:23:25.352 INFO:tasks.workunit.client.1.smithi193.stdout:2/666: mkdir d0/d28/d33/db7/dc0 0 2024-03-23T08:23:25.352 INFO:tasks.workunit.client.1.smithi193.stdout:1/595: readlink da/dd/df/d2e/l5b 0 2024-03-23T08:23:25.353 INFO:tasks.workunit.client.1.smithi193.stdout:2/667: dread - d0/d28/daf/fb9 zero size 2024-03-23T08:23:25.354 INFO:tasks.workunit.client.1.smithi193.stdout:3/519: rename d1/de/d13/d96/d99/c52 to d1/de/d13/d96/cb8 0 2024-03-23T08:23:25.354 INFO:tasks.workunit.client.1.smithi193.stdout:2/668: write d0/d1b/d3b/f58 [3295570,95117] 0 2024-03-23T08:23:25.355 INFO:tasks.workunit.client.1.smithi193.stdout:8/581: getdents df/d63/d84/da8 0 2024-03-23T08:23:25.356 INFO:tasks.workunit.client.1.smithi193.stdout:0/589: mkdir db/d52/db8/dcf 0 2024-03-23T08:23:25.357 INFO:tasks.workunit.client.1.smithi193.stdout:0/590: fsync db/d11/d14/f1f 0 2024-03-23T08:23:25.359 INFO:tasks.workunit.client.1.smithi193.stdout:3/520: chown d1/fc 1 1 2024-03-23T08:23:25.361 INFO:tasks.workunit.client.1.smithi193.stdout:1/596: creat da/d13/fc3 x:0 0 0 2024-03-23T08:23:25.362 INFO:tasks.workunit.client.1.smithi193.stdout:4/656: getdents d11/d1a/d33/d8f 0 2024-03-23T08:23:25.363 INFO:tasks.workunit.client.1.smithi193.stdout:2/669: rename d0/l11 to d0/lc1 0 2024-03-23T08:23:25.366 INFO:tasks.workunit.client.1.smithi193.stdout:8/582: dwrite f3 [4194304,4194304] 0 2024-03-23T08:23:25.366 INFO:tasks.workunit.client.1.smithi193.stdout:2/670: dread - d0/d1b/d3b/f79 zero size 2024-03-23T08:23:25.370 INFO:tasks.workunit.client.1.smithi193.stdout:0/591: truncate db/d11/d14/d21/f49 290911 0 2024-03-23T08:23:25.376 INFO:tasks.workunit.client.1.smithi193.stdout:3/521: dwrite d1/de/d13/d21/f23 [0,4194304] 0 2024-03-23T08:23:25.377 INFO:tasks.workunit.client.1.smithi193.stdout:2/671: mkdir d0/d1b/d14/d25/dc2 0 2024-03-23T08:23:25.381 INFO:tasks.workunit.client.1.smithi193.stdout:1/597: dwrite da/d13/f90 [0,4194304] 0 2024-03-23T08:23:25.387 INFO:tasks.workunit.client.1.smithi193.stdout:8/583: rename df/d62 to df/dc6 0 2024-03-23T08:23:25.388 INFO:tasks.workunit.client.1.smithi193.stdout:0/592: write db/d11/d14/d24/d2a/d4d/d93/fcc [30362,3342] 0 2024-03-23T08:23:25.393 INFO:tasks.workunit.client.1.smithi193.stdout:3/522: chown d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/c67 1731356 1 2024-03-23T08:23:25.394 INFO:tasks.workunit.client.1.smithi193.stdout:0/593: dread db/d11/d14/d24/d47/fac [0,4194304] 0 2024-03-23T08:23:25.396 INFO:tasks.workunit.client.1.smithi193.stdout:4/657: dread d11/d1a/d33/d8f/db8/f2b [0,4194304] 0 2024-03-23T08:23:25.399 INFO:tasks.workunit.client.1.smithi193.stdout:0/594: rename db/d11/d14/d24/d2a/f83 to db/d11/d14/d21/fd0 0 2024-03-23T08:23:25.402 INFO:tasks.workunit.client.1.smithi193.stdout:2/672: dread d0/d1b/d14/d25/f6c [0,4194304] 0 2024-03-23T08:23:25.404 INFO:tasks.workunit.client.1.smithi193.stdout:4/658: dwrite d11/d1a/d33/d44/f4a [0,4194304] 0 2024-03-23T08:23:25.405 INFO:tasks.workunit.client.1.smithi193.stdout:2/673: mknod d0/d28/d33/db7/cc3 0 2024-03-23T08:23:25.406 INFO:tasks.workunit.client.1.smithi193.stdout:2/674: dread - d0/d37/d48/fbe zero size 2024-03-23T08:23:25.410 INFO:tasks.workunit.client.1.smithi193.stdout:2/675: creat d0/d1b/d14/d66/d4e/fc4 x:0 0 0 2024-03-23T08:23:25.411 INFO:tasks.workunit.client.1.smithi193.stdout:3/523: dread d1/de/d13/d21/d59/d60/f6b [4194304,4194304] 0 2024-03-23T08:23:25.413 INFO:tasks.workunit.client.1.smithi193.stdout:5/645: sync 2024-03-23T08:23:25.413 INFO:tasks.workunit.client.1.smithi193.stdout:9/621: sync 2024-03-23T08:23:25.414 INFO:tasks.workunit.client.1.smithi193.stdout:4/659: link d11/d1a/d33/d8f/db8/f26 d11/d95/fdd 0 2024-03-23T08:23:25.416 INFO:tasks.workunit.client.1.smithi193.stdout:2/676: getdents d0/d1b/d14/d66/d50/d8f 0 2024-03-23T08:23:25.417 INFO:tasks.workunit.client.1.smithi193.stdout:5/646: creat d0/d1f/d93/feb x:0 0 0 2024-03-23T08:23:25.421 INFO:tasks.workunit.client.1.smithi193.stdout:4/660: dread - d11/d1a/d33/d7f/f81 zero size 2024-03-23T08:23:25.428 INFO:tasks.workunit.client.1.smithi193.stdout:4/661: chown d11/cb4 3606217 1 2024-03-23T08:23:25.430 INFO:tasks.workunit.client.1.smithi193.stdout:1/598: dwrite da/dd/df/f4b [0,4194304] 0 2024-03-23T08:23:25.434 INFO:tasks.workunit.client.1.smithi193.stdout:0/595: dread db/d11/d14/d3b/d71/f7f [0,4194304] 0 2024-03-23T08:23:25.435 INFO:tasks.workunit.client.1.smithi193.stdout:3/524: dwrite d1/de/d13/d21/d59/d60/fb7 [0,4194304] 0 2024-03-23T08:23:25.438 INFO:tasks.workunit.client.1.smithi193.stdout:8/584: dwrite df/d10/d19/fa5 [0,4194304] 0 2024-03-23T08:23:25.438 INFO:tasks.workunit.client.1.smithi193.stdout:4/662: creat d11/d2d/fde x:0 0 0 2024-03-23T08:23:25.439 INFO:tasks.workunit.client.1.smithi193.stdout:5/647: rename d0/d5a/l71 to d0/d1c/d7a/db1/d72/d73/d57/lec 0 2024-03-23T08:23:25.440 INFO:tasks.workunit.client.1.smithi193.stdout:9/622: dwrite d2/d7/db/d1b/d43/d9a/fc1 [0,4194304] 0 2024-03-23T08:23:25.441 INFO:tasks.workunit.client.1.smithi193.stdout:3/525: dread - d1/de/d78/fa9 zero size 2024-03-23T08:23:25.443 INFO:tasks.workunit.client.1.smithi193.stdout:0/596: creat db/d11/d14/d24/d2a/d4d/d93/fd1 x:0 0 0 2024-03-23T08:23:25.444 INFO:tasks.workunit.client.1.smithi193.stdout:5/648: chown d0/d1c/d7a/db1/d72/d73/d45 124 1 2024-03-23T08:23:25.451 INFO:tasks.workunit.client.1.smithi193.stdout:9/623: rename d2/d7/db/f3b to d2/fce 0 2024-03-23T08:23:25.456 INFO:tasks.workunit.client.1.smithi193.stdout:1/599: dwrite da/d13/d3f/d5a/f61 [0,4194304] 0 2024-03-23T08:23:25.459 INFO:tasks.workunit.client.1.smithi193.stdout:8/585: dwrite df/d10/d19/fb6 [0,4194304] 0 2024-03-23T08:23:25.463 INFO:tasks.workunit.client.1.smithi193.stdout:9/624: symlink d2/d7/db/d17/d4b/d59/d75/lcf 0 2024-03-23T08:23:25.466 INFO:tasks.workunit.client.1.smithi193.stdout:3/526: rename d1/de/d13/d21/dab to d1/de/d13/d21/d25/d2b/d3c/d4a/d85/db9 0 2024-03-23T08:23:25.466 INFO:tasks.workunit.client.1.smithi193.stdout:1/600: mkdir da/dd/df/d7e/d86/d95/dc4 0 2024-03-23T08:23:25.468 INFO:tasks.workunit.client.1.smithi193.stdout:3/527: chown d1/de/d78 25 1 2024-03-23T08:23:25.470 INFO:tasks.workunit.client.1.smithi193.stdout:9/625: link d2/fc d2/d7/db/d17/d33/d70/fd0 0 2024-03-23T08:23:25.470 INFO:tasks.workunit.client.1.smithi193.stdout:1/601: mknod da/d13/d3f/d5a/d76/cc5 0 2024-03-23T08:23:25.471 INFO:tasks.workunit.client.1.smithi193.stdout:4/663: dread d11/d1a/d33/d42/d52/f3f [0,4194304] 0 2024-03-23T08:23:25.471 INFO:tasks.workunit.client.1.smithi193.stdout:3/528: creat d1/de/d13/d21/d25/d34/da8/fba x:0 0 0 2024-03-23T08:23:25.474 INFO:tasks.workunit.client.1.smithi193.stdout:8/586: dread df/d10/d67/d7f/fc2 [0,4194304] 0 2024-03-23T08:23:25.474 INFO:tasks.workunit.client.1.smithi193.stdout:0/597: truncate db/d11/d14/d21/fd0 992807 0 2024-03-23T08:23:25.474 INFO:tasks.workunit.client.1.smithi193.stdout:5/649: dread d0/d1c/f24 [0,4194304] 0 2024-03-23T08:23:25.476 INFO:tasks.workunit.client.1.smithi193.stdout:5/650: stat d0/fe 0 2024-03-23T08:23:25.480 INFO:tasks.workunit.client.1.smithi193.stdout:0/598: mkdir db/d11/d14/d21/dd2 0 2024-03-23T08:23:25.482 INFO:tasks.workunit.client.1.smithi193.stdout:4/664: link d11/d27/d77/f7e d11/d2d/d58/d80/db9/fdf 0 2024-03-23T08:23:25.486 INFO:tasks.workunit.client.1.smithi193.stdout:9/626: dwrite d2/d7/fcb [0,4194304] 0 2024-03-23T08:23:25.489 INFO:tasks.workunit.client.1.smithi193.stdout:5/651: rename d0/d1c/d7a/db1/d72/d99/fc3 to d0/d1c/d7a/fed 0 2024-03-23T08:23:25.491 INFO:tasks.workunit.client.1.smithi193.stdout:5/652: read - d0/d1c/d3b/d81/f9e zero size 2024-03-23T08:23:25.492 INFO:tasks.workunit.client.1.smithi193.stdout:3/529: dwrite d1/de/d13/d21/d27/d7c/f9a [0,4194304] 0 2024-03-23T08:23:25.492 INFO:tasks.workunit.client.1.smithi193.stdout:0/599: symlink db/d11/d14/d9c/ld3 0 2024-03-23T08:23:25.494 INFO:tasks.workunit.client.1.smithi193.stdout:9/627: fsync d2/d7/db/d7a/d95/da9/fac 0 2024-03-23T08:23:25.494 INFO:tasks.workunit.client.1.smithi193.stdout:1/602: dread da/d13/f34 [0,4194304] 0 2024-03-23T08:23:25.495 INFO:tasks.workunit.client.1.smithi193.stdout:4/665: rename d11/d27/d77 to d11/d1a/d33/d7f/de0 0 2024-03-23T08:23:25.498 INFO:tasks.workunit.client.1.smithi193.stdout:9/628: chown d2/d7/d31/d5f 52 1 2024-03-23T08:23:25.498 INFO:tasks.workunit.client.1.smithi193.stdout:1/603: read - da/d13/d3f/d5a/f9d zero size 2024-03-23T08:23:25.500 INFO:tasks.workunit.client.1.smithi193.stdout:0/600: mkdir db/d11/d14/d24/d47/d59/dd4 0 2024-03-23T08:23:25.501 INFO:tasks.workunit.client.1.smithi193.stdout:9/629: fsync d2/d7/f8e 0 2024-03-23T08:23:25.501 INFO:tasks.workunit.client.1.smithi193.stdout:1/604: fsync da/d13/f49 0 2024-03-23T08:23:25.501 INFO:tasks.workunit.client.1.smithi193.stdout:8/587: dread df/d4b/d4d/f1e [0,4194304] 0 2024-03-23T08:23:25.502 INFO:tasks.workunit.client.1.smithi193.stdout:5/653: rename d0/f1d to d0/d1c/d3b/d85/d97/fee 0 2024-03-23T08:23:25.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/630: stat d2/d7/d31 0 2024-03-23T08:23:25.505 INFO:tasks.workunit.client.1.smithi193.stdout:1/605: truncate da/d13/d3f/d5a/f67 4548270 0 2024-03-23T08:23:25.506 INFO:tasks.workunit.client.1.smithi193.stdout:3/530: creat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fbb x:0 0 0 2024-03-23T08:23:25.508 INFO:tasks.workunit.client.1.smithi193.stdout:4/666: rename d11/d2d/d58/d8b/dbc/d88 to d11/d27/d85/de1 0 2024-03-23T08:23:25.509 INFO:tasks.workunit.client.1.smithi193.stdout:0/601: mknod db/d11/d14/d21/d33/d69/d9b/cd5 0 2024-03-23T08:23:25.510 INFO:tasks.workunit.client.1.smithi193.stdout:4/667: chown d11/d1a/d33/d7f 4114008 1 2024-03-23T08:23:25.510 INFO:tasks.workunit.client.1.smithi193.stdout:1/606: mknod da/d13/d3f/d5a/d8e/cc6 0 2024-03-23T08:23:25.511 INFO:tasks.workunit.client.1.smithi193.stdout:8/588: symlink df/d10/lc7 0 2024-03-23T08:23:25.512 INFO:tasks.workunit.client.1.smithi193.stdout:9/631: dwrite d2/d7/db/d1b/fbb [0,4194304] 0 2024-03-23T08:23:25.515 INFO:tasks.workunit.client.1.smithi193.stdout:5/654: creat d0/d1c/d7a/db1/d72/fef x:0 0 0 2024-03-23T08:23:25.516 INFO:tasks.workunit.client.1.smithi193.stdout:0/602: creat db/d52/db8/dcf/fd6 x:0 0 0 2024-03-23T08:23:25.516 INFO:tasks.workunit.client.1.smithi193.stdout:9/632: write d2/d7/f6f [136285,49847] 0 2024-03-23T08:23:25.517 INFO:tasks.workunit.client.1.smithi193.stdout:3/531: dread d1/de/d13/d21/d25/f7e [0,4194304] 0 2024-03-23T08:23:25.517 INFO:tasks.workunit.client.1.smithi193.stdout:4/668: mknod d11/d2d/d58/d80/db9/ce2 0 2024-03-23T08:23:25.518 INFO:tasks.workunit.client.1.smithi193.stdout:1/607: creat da/dd/df/d52/d82/fc7 x:0 0 0 2024-03-23T08:23:25.523 INFO:tasks.workunit.client.1.smithi193.stdout:5/655: mknod d0/d5a/ddf/cf0 0 2024-03-23T08:23:25.530 INFO:tasks.workunit.client.1.smithi193.stdout:5/656: readlink d0/d1c/d7a/db1/d72/d73/l33 0 2024-03-23T08:23:25.530 INFO:tasks.workunit.client.1.smithi193.stdout:0/603: link db/d11/f12 db/d11/d14/d24/d47/d59/fd7 0 2024-03-23T08:23:25.530 INFO:tasks.workunit.client.1.smithi193.stdout:9/633: dwrite d2/d7/f11 [0,4194304] 0 2024-03-23T08:23:25.530 INFO:tasks.workunit.client.1.smithi193.stdout:7/563: sync 2024-03-23T08:23:25.532 INFO:tasks.workunit.client.1.smithi193.stdout:9/634: truncate d2/d7/db/d7a/d95/fc0 473667 0 2024-03-23T08:23:25.532 INFO:tasks.workunit.client.1.smithi193.stdout:3/532: rename d1/de/d13/d21/d25/l4b to d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/lbc 0 2024-03-23T08:23:25.533 INFO:tasks.workunit.client.1.smithi193.stdout:5/657: mknod d0/d1c/d3b/cf1 0 2024-03-23T08:23:25.539 INFO:tasks.workunit.client.1.smithi193.stdout:4/669: dread d11/d1a/d33/d42/f29 [0,4194304] 0 2024-03-23T08:23:25.539 INFO:tasks.workunit.client.1.smithi193.stdout:9/635: dwrite d2/d7/db/d7a/f9f [0,4194304] 0 2024-03-23T08:23:25.540 INFO:tasks.workunit.client.1.smithi193.stdout:8/589: truncate df/d4b/d4d/d2a/f3c 4184557 0 2024-03-23T08:23:25.541 INFO:tasks.workunit.client.1.smithi193.stdout:9/636: chown d2/d7/db/d32/d6b/f6c 7 1 2024-03-23T08:23:25.542 INFO:tasks.workunit.client.1.smithi193.stdout:8/590: write df/d10/d3e/d54/f89 [736880,11970] 0 2024-03-23T08:23:25.542 INFO:tasks.workunit.client.1.smithi193.stdout:9/637: chown d2/d7/db/d1b/d43/d71 1158945654 1 2024-03-23T08:23:25.543 INFO:tasks.workunit.client.1.smithi193.stdout:8/591: stat df/d4b/d4d/c41 0 2024-03-23T08:23:25.547 INFO:tasks.workunit.client.1.smithi193.stdout:1/608: truncate da/d13/d24/f58 3163343 0 2024-03-23T08:23:25.547 INFO:tasks.workunit.client.1.smithi193.stdout:6/504: sync 2024-03-23T08:23:25.547 INFO:tasks.workunit.client.1.smithi193.stdout:2/677: sync 2024-03-23T08:23:25.548 INFO:tasks.workunit.client.1.smithi193.stdout:0/604: creat db/d11/fd8 x:0 0 0 2024-03-23T08:23:25.548 INFO:tasks.workunit.client.1.smithi193.stdout:9/638: write d2/d7/db/d1b/f4d [3467251,70433] 0 2024-03-23T08:23:25.548 INFO:tasks.workunit.client.1.smithi193.stdout:7/564: creat d6/d9/d1f/d23/d33/d38/d66/fc1 x:0 0 0 2024-03-23T08:23:25.549 INFO:tasks.workunit.client.1.smithi193.stdout:4/670: mknod d11/d2d/d58/d80/db9/ce3 0 2024-03-23T08:23:25.550 INFO:tasks.workunit.client.1.smithi193.stdout:3/533: rename d1/f1e to d1/fbd 0 2024-03-23T08:23:25.551 INFO:tasks.workunit.client.1.smithi193.stdout:8/592: rmdir df/dc6/d8b/dae 39 2024-03-23T08:23:25.551 INFO:tasks.workunit.client.1.smithi193.stdout:4/671: chown d11/d1a/f1c 1323873 1 2024-03-23T08:23:25.552 INFO:tasks.workunit.client.1.smithi193.stdout:2/678: symlink d0/d1b/d19/lc5 0 2024-03-23T08:23:25.552 INFO:tasks.workunit.client.1.smithi193.stdout:6/505: write d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f7e [457082,13680] 0 2024-03-23T08:23:25.555 INFO:tasks.workunit.client.1.smithi193.stdout:0/605: creat db/d11/d14/d24/d47/d59/fd9 x:0 0 0 2024-03-23T08:23:25.555 INFO:tasks.workunit.client.1.smithi193.stdout:6/506: fdatasync d0/d27/d81/df/d11/d3b/d6c/f90 0 2024-03-23T08:23:25.557 INFO:tasks.workunit.client.1.smithi193.stdout:4/672: mkdir d11/d95/de4 0 2024-03-23T08:23:25.560 INFO:tasks.workunit.client.1.smithi193.stdout:9/639: truncate d2/d7/db/d32/f3f 1269500 0 2024-03-23T08:23:25.561 INFO:tasks.workunit.client.1.smithi193.stdout:5/658: getdents d0/d1c/d7a/db1/d72/d73/d57/d6d 0 2024-03-23T08:23:25.566 INFO:tasks.workunit.client.1.smithi193.stdout:0/606: rename db/d11/d14/d9c to db/d11/d14/d24/d47/d59/d8c/dda 0 2024-03-23T08:23:25.568 INFO:tasks.workunit.client.1.smithi193.stdout:4/673: creat d11/d2d/d58/d8b/dbc/d84/fe5 x:0 0 0 2024-03-23T08:23:25.569 INFO:tasks.workunit.client.1.smithi193.stdout:9/640: creat d2/d7/d31/d5f/fd1 x:0 0 0 2024-03-23T08:23:25.569 INFO:tasks.workunit.client.1.smithi193.stdout:2/679: dwrite d0/d1b/d14/d66/f8e [0,4194304] 0 2024-03-23T08:23:25.570 INFO:tasks.workunit.client.1.smithi193.stdout:0/607: readlink db/d11/d14/d24/d47/d59/d8c/dda/ld3 0 2024-03-23T08:23:25.571 INFO:tasks.workunit.client.1.smithi193.stdout:8/593: dread df/d63/f81 [0,4194304] 0 2024-03-23T08:23:25.571 INFO:tasks.workunit.client.1.smithi193.stdout:6/507: fdatasync d0/d27/d81/df/d11/d36/d61/d12/f35 0 2024-03-23T08:23:25.572 INFO:tasks.workunit.client.1.smithi193.stdout:7/565: link d6/d9/d1f/d23/d33/d38/d3b/d6c/c9e d6/d9/d1f/d23/d33/d38/d46/cc2 0 2024-03-23T08:23:25.574 INFO:tasks.workunit.client.1.smithi193.stdout:5/659: rename d0/d1c/d3b/d85 to d0/d1c/d3b/d81/df2 0 2024-03-23T08:23:25.575 INFO:tasks.workunit.client.1.smithi193.stdout:9/641: write d2/d7/db/d1b/f4d [2316169,77054] 0 2024-03-23T08:23:25.576 INFO:tasks.workunit.client.1.smithi193.stdout:4/674: mkdir d11/d27/d85/de1/de6 0 2024-03-23T08:23:25.578 INFO:tasks.workunit.client.1.smithi193.stdout:3/534: chown d1/de/d13/d21/d50/d6f/d6e 115 1 2024-03-23T08:23:25.580 INFO:tasks.workunit.client.1.smithi193.stdout:4/675: write d11/d1a/d33/d42/db2/dbe/d50/d49/f5c [5801114,53256] 0 2024-03-23T08:23:25.583 INFO:tasks.workunit.client.1.smithi193.stdout:4/676: readlink d11/d8d/ldb 0 2024-03-23T08:23:25.584 INFO:tasks.workunit.client.1.smithi193.stdout:7/566: symlink d6/d9/d1f/d4d/d6b/lc3 0 2024-03-23T08:23:25.588 INFO:tasks.workunit.client.1.smithi193.stdout:9/642: chown d2/d7/db/d1b/d43/d71/c7f 1 1 2024-03-23T08:23:25.590 INFO:tasks.workunit.client.1.smithi193.stdout:0/608: dread db/d11/d14/d24/d47/d59/f79 [4194304,4194304] 0 2024-03-23T08:23:25.590 INFO:tasks.workunit.client.1.smithi193.stdout:5/660: dwrite d0/d1c/d7a/db1/d72/d73/d45/da2/fdd [0,4194304] 0 2024-03-23T08:23:25.590 INFO:tasks.workunit.client.1.smithi193.stdout:3/535: dread d1/f84 [0,4194304] 0 2024-03-23T08:23:25.593 INFO:tasks.workunit.client.1.smithi193.stdout:4/677: symlink d11/d8e/le7 0 2024-03-23T08:23:25.594 INFO:tasks.workunit.client.1.smithi193.stdout:2/680: write d0/d1b/d14/d25/f5a [126712,63643] 0 2024-03-23T08:23:25.594 INFO:tasks.workunit.client.1.smithi193.stdout:3/536: stat d1/de/d13/d21/d27/d7c/f87 0 2024-03-23T08:23:25.596 INFO:tasks.workunit.client.1.smithi193.stdout:4/678: dread - d11/d2d/d58/d80/fc3 zero size 2024-03-23T08:23:25.596 INFO:tasks.workunit.client.1.smithi193.stdout:6/508: dread d0/d27/d81/df/d11/d36/d61/f8e [0,4194304] 0 2024-03-23T08:23:25.598 INFO:tasks.workunit.client.1.smithi193.stdout:9/643: creat d2/d7/db/d32/fd2 x:0 0 0 2024-03-23T08:23:25.599 INFO:tasks.workunit.client.1.smithi193.stdout:5/661: mknod d0/d1c/d7a/db1/d72/d99/cf3 0 2024-03-23T08:23:25.600 INFO:tasks.workunit.client.1.smithi193.stdout:8/594: getdents df/d63/d76 0 2024-03-23T08:23:25.602 INFO:tasks.workunit.client.1.smithi193.stdout:3/537: rename d1/de/d78/l7b to d1/de/d78/lbe 0 2024-03-23T08:23:25.602 INFO:tasks.workunit.client.1.smithi193.stdout:7/567: truncate d6/d9/d1f/d3e/d65/d70/f83 1588017 0 2024-03-23T08:23:25.603 INFO:tasks.workunit.client.1.smithi193.stdout:4/679: creat d11/d1a/d65/fe8 x:0 0 0 2024-03-23T08:23:25.605 INFO:tasks.workunit.client.1.smithi193.stdout:9/644: creat d2/d7/d31/d3c/d79/fd3 x:0 0 0 2024-03-23T08:23:25.607 INFO:tasks.workunit.client.1.smithi193.stdout:5/662: creat d0/d1c/d3b/ff4 x:0 0 0 2024-03-23T08:23:25.608 INFO:tasks.workunit.client.1.smithi193.stdout:2/681: dread d0/d1b/d14/d66/d50/d92/da9/fab [0,4194304] 0 2024-03-23T08:23:25.608 INFO:tasks.workunit.client.1.smithi193.stdout:6/509: getdents d0/d27/d72 0 2024-03-23T08:23:25.610 INFO:tasks.workunit.client.1.smithi193.stdout:7/568: mknod d6/d9/d1f/d23/d33/d38/d3b/cc4 0 2024-03-23T08:23:25.611 INFO:tasks.workunit.client.1.smithi193.stdout:0/609: dwrite db/d11/d14/d24/d2a/d4d/f81 [0,4194304] 0 2024-03-23T08:23:25.611 INFO:tasks.workunit.client.1.smithi193.stdout:8/595: creat df/fc8 x:0 0 0 2024-03-23T08:23:25.614 INFO:tasks.workunit.client.1.smithi193.stdout:6/510: chown d0/d27/d81/df/f9b 5785067 1 2024-03-23T08:23:25.615 INFO:tasks.workunit.client.1.smithi193.stdout:5/663: chown d0/d1c/d7a/db1/d72/d73/d45/da2/lc1 231282 1 2024-03-23T08:23:25.617 INFO:tasks.workunit.client.1.smithi193.stdout:8/596: chown df/d10/d19/f8c 9 1 2024-03-23T08:23:25.619 INFO:tasks.workunit.client.1.smithi193.stdout:3/538: link d1/de/d13/d21/d25/d34/c8b d1/de/d13/d21/d25/d2b/cbf 0 2024-03-23T08:23:25.621 INFO:tasks.workunit.client.1.smithi193.stdout:2/682: fdatasync d0/d1b/d14/d66/d50/d92/da9/fab 0 2024-03-23T08:23:25.622 INFO:tasks.workunit.client.1.smithi193.stdout:9/645: rename d2/d7/db/d7a/d95/da9/fac to d2/d7/db/d7a/d95/fd4 0 2024-03-23T08:23:25.626 INFO:tasks.workunit.client.1.smithi193.stdout:8/597: chown df/d4b/d4d/f5c 6964982 1 2024-03-23T08:23:25.627 INFO:tasks.workunit.client.1.smithi193.stdout:5/664: creat d0/ff5 x:0 0 0 2024-03-23T08:23:25.629 INFO:tasks.workunit.client.1.smithi193.stdout:4/680: dwrite d11/d1a/d33/d42/d52/d70/fba [0,4194304] 0 2024-03-23T08:23:25.634 INFO:tasks.workunit.client.1.smithi193.stdout:2/683: mkdir d0/d1b/d14/d56/da3/db0/dc6 0 2024-03-23T08:23:25.636 INFO:tasks.workunit.client.1.smithi193.stdout:7/569: dwrite d6/d9/f1c [0,4194304] 0 2024-03-23T08:23:25.637 INFO:tasks.workunit.client.1.smithi193.stdout:0/610: dwrite db/d11/d14/fad [0,4194304] 0 2024-03-23T08:23:25.641 INFO:tasks.workunit.client.1.smithi193.stdout:8/598: mknod df/d10/d3e/d54/cc9 0 2024-03-23T08:23:25.641 INFO:tasks.workunit.client.1.smithi193.stdout:9/646: fdatasync d2/d7/db/d17/d33/db2/fc9 0 2024-03-23T08:23:25.645 INFO:tasks.workunit.client.1.smithi193.stdout:2/684: unlink d0/d28/cac 0 2024-03-23T08:23:25.645 INFO:tasks.workunit.client.1.smithi193.stdout:5/665: dread - d0/d1c/d7a/d86/d9f/fa8 zero size 2024-03-23T08:23:25.648 INFO:tasks.workunit.client.1.smithi193.stdout:0/611: unlink db/d11/d14/d3b/c9d 0 2024-03-23T08:23:25.648 INFO:tasks.workunit.client.1.smithi193.stdout:6/511: dwrite d0/d27/d81/df/f25 [0,4194304] 0 2024-03-23T08:23:25.648 INFO:tasks.workunit.client.1.smithi193.stdout:3/539: dwrite d1/de/d28/fad [0,4194304] 0 2024-03-23T08:23:25.659 INFO:tasks.workunit.client.1.smithi193.stdout:7/570: rename d6/d9/d1f/d23/d33/d38/d46/l64 to d6/d9/d1f/d3e/d65/lc5 0 2024-03-23T08:23:25.660 INFO:tasks.workunit.client.1.smithi193.stdout:3/540: write d1/de/d13/d21/d50/d6f/d6e/f72 [1273685,103143] 0 2024-03-23T08:23:25.661 INFO:tasks.workunit.client.1.smithi193.stdout:8/599: dread df/d10/d67/d7f/fc2 [0,4194304] 0 2024-03-23T08:23:25.664 INFO:tasks.workunit.client.1.smithi193.stdout:5/666: dwrite d0/d1c/d3b/fce [0,4194304] 0 2024-03-23T08:23:25.664 INFO:tasks.workunit.client.1.smithi193.stdout:2/685: dwrite d0/d1b/d14/d66/f8e [0,4194304] 0 2024-03-23T08:23:25.666 INFO:tasks.workunit.client.1.smithi193.stdout:7/571: creat d6/d9/d1f/d4d/fc6 x:0 0 0 2024-03-23T08:23:25.672 INFO:tasks.workunit.client.1.smithi193.stdout:2/686: write d0/d28/d9f/fb8 [37991,92577] 0 2024-03-23T08:23:25.674 INFO:tasks.workunit.client.1.smithi193.stdout:6/512: getdents d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/d9d 0 2024-03-23T08:23:25.675 INFO:tasks.workunit.client.1.smithi193.stdout:3/541: write d1/de/d13/f62 [863047,129264] 0 2024-03-23T08:23:25.679 INFO:tasks.workunit.client.1.smithi193.stdout:6/513: stat d0/d27/d81/df/d11/d3b/l58 0 2024-03-23T08:23:25.681 INFO:tasks.workunit.client.1.smithi193.stdout:8/600: dwrite df/d10/d3e/d4e/f88 [0,4194304] 0 2024-03-23T08:23:25.681 INFO:tasks.workunit.client.1.smithi193.stdout:7/572: mkdir d6/d9/d1f/d23/d33/d3c/d41/d47/dc7 0 2024-03-23T08:23:25.684 INFO:tasks.workunit.client.1.smithi193.stdout:8/601: chown df/d4b/d4d/d2a/db2 54370965 1 2024-03-23T08:23:25.686 INFO:tasks.workunit.client.1.smithi193.stdout:5/667: dread d0/d1c/d7a/db1/f38 [0,4194304] 0 2024-03-23T08:23:25.686 INFO:tasks.workunit.client.1.smithi193.stdout:6/514: unlink d0/d27/c78 0 2024-03-23T08:23:25.689 INFO:tasks.workunit.client.1.smithi193.stdout:8/602: mkdir df/d10/d19/dca 0 2024-03-23T08:23:25.693 INFO:tasks.workunit.client.1.smithi193.stdout:8/603: fsync df/d10/f17 0 2024-03-23T08:23:25.694 INFO:tasks.workunit.client.1.smithi193.stdout:1/609: sync 2024-03-23T08:23:25.699 INFO:tasks.workunit.client.1.smithi193.stdout:2/687: truncate d0/d28/d33/d72/fa5 3435422 0 2024-03-23T08:23:25.699 INFO:tasks.workunit.client.1.smithi193.stdout:7/573: dwrite d6/d9/d1f/d9d/fa0 [0,4194304] 0 2024-03-23T08:23:25.700 INFO:tasks.workunit.client.1.smithi193.stdout:8/604: mkdir df/d10/d83/dcb 0 2024-03-23T08:23:25.702 INFO:tasks.workunit.client.1.smithi193.stdout:5/668: dread d0/d1c/d7a/db1/d72/d73/d45/fa0 [0,4194304] 0 2024-03-23T08:23:25.703 INFO:tasks.workunit.client.1.smithi193.stdout:9/647: dread d2/d7/f36 [4194304,4194304] 0 2024-03-23T08:23:25.704 INFO:tasks.workunit.client.1.smithi193.stdout:6/515: dwrite d0/d27/d81/df/d11/d3b/d6c/f90 [0,4194304] 0 2024-03-23T08:23:25.706 INFO:tasks.workunit.client.1.smithi193.stdout:9/648: stat d2/d7/db/d17/d33/d70/lb9 0 2024-03-23T08:23:25.708 INFO:tasks.workunit.client.1.smithi193.stdout:5/669: write d0/f12 [6883240,8298] 0 2024-03-23T08:23:25.710 INFO:tasks.workunit.client.1.smithi193.stdout:8/605: read df/d10/f17 [605373,13493] 0 2024-03-23T08:23:25.711 INFO:tasks.workunit.client.1.smithi193.stdout:1/610: mknod da/dd/df/d7e/d86/d95/dc4/cc8 0 2024-03-23T08:23:25.712 INFO:tasks.workunit.client.1.smithi193.stdout:7/574: truncate d6/d9/f15 1749713 0 2024-03-23T08:23:25.715 INFO:tasks.workunit.client.1.smithi193.stdout:9/649: mkdir d2/d7/db/d17/d1f/dd5 0 2024-03-23T08:23:25.715 INFO:tasks.workunit.client.1.smithi193.stdout:2/688: creat d0/d1b/d14/d66/d50/d92/da9/dba/fc7 x:0 0 0 2024-03-23T08:23:25.717 INFO:tasks.workunit.client.1.smithi193.stdout:1/611: mknod da/dd/df/d7e/d7f/cc9 0 2024-03-23T08:23:25.718 INFO:tasks.workunit.client.1.smithi193.stdout:8/606: chown df/f21 18643883 1 2024-03-23T08:23:25.718 INFO:tasks.workunit.client.1.smithi193.stdout:6/516: creat d0/d27/d81/df/d11/d36/fa3 x:0 0 0 2024-03-23T08:23:25.719 INFO:tasks.workunit.client.1.smithi193.stdout:2/689: chown d0/d1b/l38 2997 1 2024-03-23T08:23:25.722 INFO:tasks.workunit.client.1.smithi193.stdout:5/670: dwrite d0/d1f/f49 [4194304,4194304] 0 2024-03-23T08:23:25.723 INFO:tasks.workunit.client.1.smithi193.stdout:1/612: creat da/dd/df/d7c/fca x:0 0 0 2024-03-23T08:23:25.725 INFO:tasks.workunit.client.1.smithi193.stdout:8/607: mkdir df/dcc 0 2024-03-23T08:23:25.727 INFO:tasks.workunit.client.1.smithi193.stdout:1/613: read da/d13/d3f/d5a/f61 [3433983,43250] 0 2024-03-23T08:23:25.728 INFO:tasks.workunit.client.1.smithi193.stdout:3/542: dread d1/de/d13/d21/d50/d44/f79 [0,4194304] 0 2024-03-23T08:23:25.729 INFO:tasks.workunit.client.1.smithi193.stdout:6/517: rmdir d0/d27/d81/df/d11/d3b/d51 39 2024-03-23T08:23:25.733 INFO:tasks.workunit.client.1.smithi193.stdout:7/575: dwrite d6/d9/d1f/f30 [4194304,4194304] 0 2024-03-23T08:23:25.734 INFO:tasks.workunit.client.1.smithi193.stdout:1/614: read da/d13/f7d [412895,68668] 0 2024-03-23T08:23:25.734 INFO:tasks.workunit.client.1.smithi193.stdout:9/650: write d2/d7/db/d1b/faf [6056359,52456] 0 2024-03-23T08:23:25.734 INFO:tasks.workunit.client.1.smithi193.stdout:8/608: symlink df/d10/d3e/d54/lcd 0 2024-03-23T08:23:25.739 INFO:tasks.workunit.client.1.smithi193.stdout:3/543: creat d1/de/d13/d21/d50/d6f/d6e/fc0 x:0 0 0 2024-03-23T08:23:25.741 INFO:tasks.workunit.client.1.smithi193.stdout:6/518: symlink d0/d27/d72/la4 0 2024-03-23T08:23:25.741 INFO:tasks.workunit.client.1.smithi193.stdout:7/576: read d6/d9/d1f/d23/d33/d3c/f4b [1573297,82592] 0 2024-03-23T08:23:25.743 INFO:tasks.workunit.client.1.smithi193.stdout:1/615: read da/d13/d24/f7b [3333255,112151] 0 2024-03-23T08:23:25.744 INFO:tasks.workunit.client.1.smithi193.stdout:8/609: mknod df/cce 0 2024-03-23T08:23:25.744 INFO:tasks.workunit.client.1.smithi193.stdout:3/544: stat d1/de/d78/lbe 0 2024-03-23T08:23:25.745 INFO:tasks.workunit.client.1.smithi193.stdout:9/651: creat d2/d7/db/d32/fd6 x:0 0 0 2024-03-23T08:23:25.745 INFO:tasks.workunit.client.1.smithi193.stdout:2/690: dwrite d0/d1b/d14/d66/d50/d92/da9/dba/fc7 [0,4194304] 0 2024-03-23T08:23:25.747 INFO:tasks.workunit.client.1.smithi193.stdout:5/671: dwrite d0/d1c/d7a/db1/f8b [0,4194304] 0 2024-03-23T08:23:25.747 INFO:tasks.workunit.client.1.smithi193.stdout:4/681: sync 2024-03-23T08:23:25.748 INFO:tasks.workunit.client.1.smithi193.stdout:6/519: chown d0/d27/d81/df/d11/d36/d61/d3a/f45 264328147 1 2024-03-23T08:23:25.752 INFO:tasks.workunit.client.1.smithi193.stdout:7/577: truncate d6/d9/d1f/d5f/f8a 1004751 0 2024-03-23T08:23:25.756 INFO:tasks.workunit.client.1.smithi193.stdout:1/616: creat da/dd/df/d7e/fcb x:0 0 0 2024-03-23T08:23:25.758 INFO:tasks.workunit.client.1.smithi193.stdout:7/578: chown d6/d9/d1f/d23/d33/d3c/d41/d47/fb2 25 1 2024-03-23T08:23:25.759 INFO:tasks.workunit.client.1.smithi193.stdout:6/520: read - d0/d27/d81/df/d11/d36/f97 zero size 2024-03-23T08:23:25.759 INFO:tasks.workunit.client.1.smithi193.stdout:2/691: creat d0/d1b/d14/d66/d50/d92/fc8 x:0 0 0 2024-03-23T08:23:25.761 INFO:tasks.workunit.client.1.smithi193.stdout:1/617: unlink da/c2c 0 2024-03-23T08:23:25.762 INFO:tasks.workunit.client.1.smithi193.stdout:4/682: mkdir d11/d2d/d58/d8b/dbc/de9 0 2024-03-23T08:23:25.763 INFO:tasks.workunit.client.1.smithi193.stdout:3/545: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/dae/lc1 0 2024-03-23T08:23:25.763 INFO:tasks.workunit.client.1.smithi193.stdout:7/579: chown d6/d9/d1f/d3e/d42/f71 366 1 2024-03-23T08:23:25.765 INFO:tasks.workunit.client.1.smithi193.stdout:6/521: dread - d0/d27/d81/df/d11/d36/f97 zero size 2024-03-23T08:23:25.766 INFO:tasks.workunit.client.1.smithi193.stdout:5/672: dwrite d0/d1c/fb0 [0,4194304] 0 2024-03-23T08:23:25.768 INFO:tasks.workunit.client.1.smithi193.stdout:3/546: mknod d1/de/d13/d21/d25/d34/da8/cc2 0 2024-03-23T08:23:25.768 INFO:tasks.workunit.client.1.smithi193.stdout:4/683: chown d11/d1a/d33/d44/f6e 2049086 1 2024-03-23T08:23:25.769 INFO:tasks.workunit.client.1.smithi193.stdout:1/618: truncate da/d13/d24/f43 4261085 0 2024-03-23T08:23:25.769 INFO:tasks.workunit.client.1.smithi193.stdout:2/692: rename d0/d1b/d14/d56/da3/fbb to d0/d28/d33/d72/fc9 0 2024-03-23T08:23:25.774 INFO:tasks.workunit.client.1.smithi193.stdout:4/684: write d11/d95/f9a [2228106,36257] 0 2024-03-23T08:23:25.775 INFO:tasks.workunit.client.1.smithi193.stdout:7/580: symlink d6/d9/d1f/d23/d33/d38/d3b/lc8 0 2024-03-23T08:23:25.776 INFO:tasks.workunit.client.1.smithi193.stdout:6/522: dread - d0/d27/d81/df/d11/d3b/d6c/f88 zero size 2024-03-23T08:23:25.778 INFO:tasks.workunit.client.1.smithi193.stdout:1/619: creat da/d13/d3f/d5a/fcc x:0 0 0 2024-03-23T08:23:25.778 INFO:tasks.workunit.client.1.smithi193.stdout:8/610: dwrite df/f34 [0,4194304] 0 2024-03-23T08:23:25.781 INFO:tasks.workunit.client.1.smithi193.stdout:2/693: creat d0/d1b/d14/d56/d7c/d7e/fca x:0 0 0 2024-03-23T08:23:25.781 INFO:tasks.workunit.client.1.smithi193.stdout:4/685: rmdir d11/d8e/dac 39 2024-03-23T08:23:25.781 INFO:tasks.workunit.client.1.smithi193.stdout:0/612: sync 2024-03-23T08:23:25.781 INFO:tasks.workunit.client.1.smithi193.stdout:6/523: fdatasync d0/f8b 0 2024-03-23T08:23:25.784 INFO:tasks.workunit.client.1.smithi193.stdout:8/611: mknod df/dcc/ccf 0 2024-03-23T08:23:25.786 INFO:tasks.workunit.client.1.smithi193.stdout:1/620: unlink da/d13/f34 0 2024-03-23T08:23:25.787 INFO:tasks.workunit.client.1.smithi193.stdout:2/694: dread - d0/d1b/d14/d66/d50/f91 zero size 2024-03-23T08:23:25.790 INFO:tasks.workunit.client.1.smithi193.stdout:6/524: write d0/d27/d81/f86 [2511803,116936] 0 2024-03-23T08:23:25.792 INFO:tasks.workunit.client.1.smithi193.stdout:3/547: dwrite d1/de/d13/d21/f31 [0,4194304] 0 2024-03-23T08:23:25.793 INFO:tasks.workunit.client.1.smithi193.stdout:8/612: symlink df/d4b/d4d/d2a/db2/ld0 0 2024-03-23T08:23:25.797 INFO:tasks.workunit.client.1.smithi193.stdout:2/695: unlink d0/d28/f62 0 2024-03-23T08:23:25.798 INFO:tasks.workunit.client.1.smithi193.stdout:9/652: dread d2/d7/f6f [0,4194304] 0 2024-03-23T08:23:25.801 INFO:tasks.workunit.client.1.smithi193.stdout:3/548: write d1/de/d28/fad [3763199,85550] 0 2024-03-23T08:23:25.802 INFO:tasks.workunit.client.1.smithi193.stdout:0/613: mkdir db/d11/d14/d24/d47/d59/d8c/dda/ddb 0 2024-03-23T08:23:25.803 INFO:tasks.workunit.client.1.smithi193.stdout:2/696: write d0/d28/f47 [4791515,65634] 0 2024-03-23T08:23:25.807 INFO:tasks.workunit.client.1.smithi193.stdout:1/621: getdents da/d13/d3f/d5a/d76 0 2024-03-23T08:23:25.807 INFO:tasks.workunit.client.1.smithi193.stdout:7/581: dwrite d6/d9/d1f/d3e/d42/f60 [0,4194304] 0 2024-03-23T08:23:25.807 INFO:tasks.workunit.client.1.smithi193.stdout:5/673: dread d0/d1c/d3b/f94 [0,4194304] 0 2024-03-23T08:23:25.809 INFO:tasks.workunit.client.1.smithi193.stdout:4/686: dread d11/d2d/d58/d80/db9/fdf [0,4194304] 0 2024-03-23T08:23:25.816 INFO:tasks.workunit.client.1.smithi193.stdout:6/525: chown d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/l9e 261540 1 2024-03-23T08:23:25.816 INFO:tasks.workunit.client.1.smithi193.stdout:9/653: write d2/d7/db/d7a/f87 [1028671,77679] 0 2024-03-23T08:23:25.816 INFO:tasks.workunit.client.1.smithi193.stdout:0/614: mknod db/d11/d14/d21/d33/d48/d85/cdc 0 2024-03-23T08:23:25.816 INFO:tasks.workunit.client.1.smithi193.stdout:1/622: fdatasync da/dd/df/d7e/f84 0 2024-03-23T08:23:25.816 INFO:tasks.workunit.client.1.smithi193.stdout:3/549: rmdir d1/de/d13/d21/d27/daf 39 2024-03-23T08:23:25.818 INFO:tasks.workunit.client.1.smithi193.stdout:7/582: write d6/d9/d1f/d23/f45 [4503247,89075] 0 2024-03-23T08:23:25.819 INFO:tasks.workunit.client.1.smithi193.stdout:8/613: dread df/d10/d19/f1b [0,4194304] 0 2024-03-23T08:23:25.824 INFO:tasks.workunit.client.1.smithi193.stdout:7/583: unlink d6/l74 0 2024-03-23T08:23:25.826 INFO:tasks.workunit.client.1.smithi193.stdout:3/550: mkdir d1/de/d13/d21/d59/dc3 0 2024-03-23T08:23:25.826 INFO:tasks.workunit.client.1.smithi193.stdout:2/697: dwrite d0/d1b/f13 [0,4194304] 0 2024-03-23T08:23:25.829 INFO:tasks.workunit.client.1.smithi193.stdout:9/654: rmdir d2/d7/db/d17/d1f/dd5 0 2024-03-23T08:23:25.829 INFO:tasks.workunit.client.1.smithi193.stdout:8/614: mknod df/d4b/cd1 0 2024-03-23T08:23:25.832 INFO:tasks.workunit.client.1.smithi193.stdout:0/615: truncate db/d11/d14/d24/d47/d59/fd9 395156 0 2024-03-23T08:23:25.833 INFO:tasks.workunit.client.1.smithi193.stdout:7/584: rename d6/d9/d1f to d6/d9/d1f/d5f/dc9 22 2024-03-23T08:23:25.835 INFO:tasks.workunit.client.1.smithi193.stdout:8/615: read - df/d10/d3e/fbf zero size 2024-03-23T08:23:25.836 INFO:tasks.workunit.client.1.smithi193.stdout:6/526: dwrite d0/d27/d81/df/f9b [0,4194304] 0 2024-03-23T08:23:25.837 INFO:tasks.workunit.client.1.smithi193.stdout:7/585: chown d6/d9/d1f/d3e 14104 1 2024-03-23T08:23:25.837 INFO:tasks.workunit.client.1.smithi193.stdout:8/616: readlink df/d10/lc7 0 2024-03-23T08:23:25.840 INFO:tasks.workunit.client.1.smithi193.stdout:1/623: dread da/d13/d3f/d5a/f8b [0,4194304] 0 2024-03-23T08:23:25.845 INFO:tasks.workunit.client.1.smithi193.stdout:6/527: write d0/d27/d81/f4e [1122865,102571] 0 2024-03-23T08:23:25.845 INFO:tasks.workunit.client.1.smithi193.stdout:0/616: dread db/d11/d14/d3b/f63 [0,4194304] 0 2024-03-23T08:23:25.846 INFO:tasks.workunit.client.1.smithi193.stdout:7/586: mkdir d6/d9/d4e/d81/dca 0 2024-03-23T08:23:25.846 INFO:tasks.workunit.client.1.smithi193.stdout:1/624: chown da/dd/df/d2e/f4f 126707495 1 2024-03-23T08:23:25.847 INFO:tasks.workunit.client.1.smithi193.stdout:4/687: dread d11/d1a/d33/f39 [0,4194304] 0 2024-03-23T08:23:25.850 INFO:tasks.workunit.client.1.smithi193.stdout:5/674: truncate d0/d1c/d7a/db1/d72/d73/d45/f66 1627381 0 2024-03-23T08:23:25.850 INFO:tasks.workunit.client.1.smithi193.stdout:8/617: rename df/d10/d19 to df/d10/d3e/d54/dd2 0 2024-03-23T08:23:25.851 INFO:tasks.workunit.client.1.smithi193.stdout:7/587: fsync d6/f2d 0 2024-03-23T08:23:25.852 INFO:tasks.workunit.client.1.smithi193.stdout:9/655: dwrite d2/d7/db/d1b/faf [8388608,4194304] 0 2024-03-23T08:23:25.863 INFO:tasks.workunit.client.1.smithi193.stdout:2/698: dwrite d0/d1b/d14/d56/d7c/d7e/f88 [0,4194304] 0 2024-03-23T08:23:25.864 INFO:tasks.workunit.client.1.smithi193.stdout:5/675: readlink d0/d1c/d7a/db1/d72/d73/d57/l88 0 2024-03-23T08:23:25.864 INFO:tasks.workunit.client.1.smithi193.stdout:0/617: write db/d11/d14/d3b/d71/f7f [98364,11661] 0 2024-03-23T08:23:25.869 INFO:tasks.workunit.client.1.smithi193.stdout:4/688: mknod d11/d2d/d58/cea 0 2024-03-23T08:23:25.869 INFO:tasks.workunit.client.1.smithi193.stdout:8/618: write df/dc6/d8b/dae/f90 [45399,37810] 0 2024-03-23T08:23:25.869 INFO:tasks.workunit.client.1.smithi193.stdout:6/528: symlink d0/d27/d81/df/d11/d36/d61/d12/la5 0 2024-03-23T08:23:25.871 INFO:tasks.workunit.client.1.smithi193.stdout:7/588: unlink d6/d9/d1f/d4d/d7c/d92/fbe 0 2024-03-23T08:23:25.873 INFO:tasks.workunit.client.1.smithi193.stdout:8/619: chown ca 1797813 1 2024-03-23T08:23:25.874 INFO:tasks.workunit.client.1.smithi193.stdout:4/689: read d11/d2d/f4b [6687029,123817] 0 2024-03-23T08:23:25.874 INFO:tasks.workunit.client.1.smithi193.stdout:2/699: chown d0/d1b/d14/d25/c8d 209144 1 2024-03-23T08:23:25.876 INFO:tasks.workunit.client.1.smithi193.stdout:9/656: creat d2/d7/db/d1b/d43/d71/db4/d78/fd7 x:0 0 0 2024-03-23T08:23:25.877 INFO:tasks.workunit.client.1.smithi193.stdout:5/676: rmdir d0/d1f/d93/de1 39 2024-03-23T08:23:25.877 INFO:tasks.workunit.client.1.smithi193.stdout:3/551: dread d1/de/d13/d21/d27/f1c [0,4194304] 0 2024-03-23T08:23:25.878 INFO:tasks.workunit.client.1.smithi193.stdout:0/618: mkdir db/d11/d14/d24/ddd 0 2024-03-23T08:23:25.878 INFO:tasks.workunit.client.1.smithi193.stdout:4/690: mknod d11/d27/ceb 0 2024-03-23T08:23:25.879 INFO:tasks.workunit.client.1.smithi193.stdout:7/589: rename d6/d9/d4e/f72 to d6/d9/d1f/daf/fcb 0 2024-03-23T08:23:25.880 INFO:tasks.workunit.client.1.smithi193.stdout:6/529: creat d0/d27/d81/fa6 x:0 0 0 2024-03-23T08:23:25.881 INFO:tasks.workunit.client.1.smithi193.stdout:8/620: creat df/d63/d84/dc5/fd3 x:0 0 0 2024-03-23T08:23:25.882 INFO:tasks.workunit.client.1.smithi193.stdout:9/657: unlink d2/d7/d31/d3c/d79/lc5 0 2024-03-23T08:23:25.882 INFO:tasks.workunit.client.1.smithi193.stdout:2/700: write d0/d1b/d14/d66/d50/d92/da9/fab [1559443,34602] 0 2024-03-23T08:23:25.884 INFO:tasks.workunit.client.1.smithi193.stdout:9/658: stat d2/d7/f1e 0 2024-03-23T08:23:25.885 INFO:tasks.workunit.client.1.smithi193.stdout:2/701: read - d0/d28/fa8 zero size 2024-03-23T08:23:25.886 INFO:tasks.workunit.client.1.smithi193.stdout:4/691: truncate d11/d1a/d65/fe8 708244 0 2024-03-23T08:23:25.888 INFO:tasks.workunit.client.1.smithi193.stdout:7/590: creat d6/d9/d1f/d23/d33/d3c/d41/d47/fcc x:0 0 0 2024-03-23T08:23:25.888 INFO:tasks.workunit.client.1.smithi193.stdout:4/692: stat f4 0 2024-03-23T08:23:25.889 INFO:tasks.workunit.client.1.smithi193.stdout:6/530: rmdir d0/d27/d81/df/d11/d36/d61/d7c 39 2024-03-23T08:23:25.891 INFO:tasks.workunit.client.1.smithi193.stdout:1/625: dread da/dd/df/d7e/d86/d95/fb1 [0,4194304] 0 2024-03-23T08:23:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:9/659: creat d2/d7/db/d17/d4b/fd8 x:0 0 0 2024-03-23T08:23:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:3/552: dwrite d1/de/d13/f24 [0,4194304] 0 2024-03-23T08:23:25.894 INFO:tasks.workunit.client.1.smithi193.stdout:7/591: chown d6/d9/d1f/d3e/f5e 1 1 2024-03-23T08:23:25.894 INFO:tasks.workunit.client.1.smithi193.stdout:4/693: fsync d11/d1a/d33/dce/fd7 0 2024-03-23T08:23:25.900 INFO:tasks.workunit.client.1.smithi193.stdout:6/531: write d0/d27/d81/fa6 [252044,33652] 0 2024-03-23T08:23:25.904 INFO:tasks.workunit.client.1.smithi193.stdout:1/626: symlink da/dd/df/d7e/d86/d95/dc4/lcd 0 2024-03-23T08:23:25.905 INFO:tasks.workunit.client.1.smithi193.stdout:9/660: write d2/d7/f1e [3343123,95531] 0 2024-03-23T08:23:25.907 INFO:tasks.workunit.client.1.smithi193.stdout:7/592: creat d6/d9/d1f/d9d/da1/fcd x:0 0 0 2024-03-23T08:23:25.909 INFO:tasks.workunit.client.1.smithi193.stdout:0/619: dread db/d11/d14/d24/d47/d59/f79 [0,4194304] 0 2024-03-23T08:23:25.911 INFO:tasks.workunit.client.1.smithi193.stdout:5/677: dread d0/d1f/f41 [0,4194304] 0 2024-03-23T08:23:25.911 INFO:tasks.workunit.client.1.smithi193.stdout:2/702: dwrite d0/d1b/d14/d66/d4e/fc4 [0,4194304] 0 2024-03-23T08:23:25.912 INFO:tasks.workunit.client.1.smithi193.stdout:9/661: chown d2/d7/db/d32/fd6 73 1 2024-03-23T08:23:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:6/532: dread d0/d27/d81/df/f4f [0,4194304] 0 2024-03-23T08:23:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:7/593: chown d6/d9/d1f/d3e/d65/d7d/f9f 289812 1 2024-03-23T08:23:25.917 INFO:tasks.workunit.client.1.smithi193.stdout:2/703: fsync d0/d1b/d19/f7f 0 2024-03-23T08:23:25.918 INFO:tasks.workunit.client.1.smithi193.stdout:5/678: mknod d0/d1c/d7a/cf6 0 2024-03-23T08:23:25.921 INFO:tasks.workunit.client.1.smithi193.stdout:8/621: dread df/d10/d3e/fb0 [0,4194304] 0 2024-03-23T08:23:25.922 INFO:tasks.workunit.client.1.smithi193.stdout:0/620: getdents db/d52/db8/dcf 0 2024-03-23T08:23:25.924 INFO:tasks.workunit.client.1.smithi193.stdout:6/533: mknod d0/d27/d81/df/d11/ca7 0 2024-03-23T08:23:25.925 INFO:tasks.workunit.client.1.smithi193.stdout:7/594: symlink d6/d9/d1f/daf/lce 0 2024-03-23T08:23:25.926 INFO:tasks.workunit.client.1.smithi193.stdout:1/627: dwrite da/dd/df/d52/f7a [0,4194304] 0 2024-03-23T08:23:25.926 INFO:tasks.workunit.client.1.smithi193.stdout:5/679: mkdir d0/d1c/df7 0 2024-03-23T08:23:25.930 INFO:tasks.workunit.client.1.smithi193.stdout:6/534: write d0/d27/f91 [248355,80809] 0 2024-03-23T08:23:25.931 INFO:tasks.workunit.client.1.smithi193.stdout:8/622: read df/d10/d3e/d4e/f5b [895532,116356] 0 2024-03-23T08:23:25.931 INFO:tasks.workunit.client.1.smithi193.stdout:3/553: dwrite d1/de/d13/d21/d25/d34/f91 [0,4194304] 0 2024-03-23T08:23:25.933 INFO:tasks.workunit.client.1.smithi193.stdout:9/662: link d2/d7/db/f34 d2/d7/db/d17/d33/fd9 0 2024-03-23T08:23:25.933 INFO:tasks.workunit.client.1.smithi193.stdout:4/694: dread d11/d2d/f4b [0,4194304] 0 2024-03-23T08:23:25.934 INFO:tasks.workunit.client.1.smithi193.stdout:0/621: mkdir db/d11/d14/d21/d33/d69/dde 0 2024-03-23T08:23:25.935 INFO:tasks.workunit.client.1.smithi193.stdout:8/623: chown df/f21 3842185 1 2024-03-23T08:23:25.937 INFO:tasks.workunit.client.1.smithi193.stdout:7/595: fdatasync d6/d9/f1b 0 2024-03-23T08:23:25.940 INFO:tasks.workunit.client.1.smithi193.stdout:1/628: write da/dd/df/d7e/d86/d95/fb1 [4311619,61192] 0 2024-03-23T08:23:25.941 INFO:tasks.workunit.client.1.smithi193.stdout:2/704: dwrite d0/d1b/d14/d56/da3/db0/fb6 [0,4194304] 0 2024-03-23T08:23:25.943 INFO:tasks.workunit.client.1.smithi193.stdout:2/705: dread - d0/d1b/da7/fb3 zero size 2024-03-23T08:23:25.943 INFO:tasks.workunit.client.1.smithi193.stdout:3/554: readlink d1/de/d13/d21/d25/d34/l75 0 2024-03-23T08:23:25.943 INFO:tasks.workunit.client.1.smithi193.stdout:6/535: creat d0/d27/d81/d87/fa8 x:0 0 0 2024-03-23T08:23:25.947 INFO:tasks.workunit.client.1.smithi193.stdout:2/706: chown d0/d1b/d19/c34 0 1 2024-03-23T08:23:25.947 INFO:tasks.workunit.client.1.smithi193.stdout:1/629: truncate da/dd/f78 1025640 0 2024-03-23T08:23:25.948 INFO:tasks.workunit.client.1.smithi193.stdout:4/695: mknod d11/d2d/d58/d8b/dbc/cec 0 2024-03-23T08:23:25.950 INFO:tasks.workunit.client.1.smithi193.stdout:8/624: mkdir df/d63/d84/da8/dd4 0 2024-03-23T08:23:25.952 INFO:tasks.workunit.client.1.smithi193.stdout:8/625: stat df/d63/d84/da8/dd4 0 2024-03-23T08:23:25.954 INFO:tasks.workunit.client.1.smithi193.stdout:8/626: readlink df/d4b/l97 0 2024-03-23T08:23:25.954 INFO:tasks.workunit.client.1.smithi193.stdout:5/680: truncate d0/d1c/d7a/db1/d72/d73/d45/f56 52301 0 2024-03-23T08:23:25.955 INFO:tasks.workunit.client.1.smithi193.stdout:9/663: write d2/d7/db/d17/d33/fd9 [332905,22131] 0 2024-03-23T08:23:25.958 INFO:tasks.workunit.client.1.smithi193.stdout:6/536: mkdir d0/d27/d81/da1/da9 0 2024-03-23T08:23:25.961 INFO:tasks.workunit.client.1.smithi193.stdout:4/696: mkdir d11/d27/d85/de1/d99/ded 0 2024-03-23T08:23:25.962 INFO:tasks.workunit.client.1.smithi193.stdout:5/681: fdatasync d0/d1c/d3b/f94 0 2024-03-23T08:23:25.963 INFO:tasks.workunit.client.1.smithi193.stdout:8/627: symlink df/dc6/d8b/dae/ld5 0 2024-03-23T08:23:25.966 INFO:tasks.workunit.client.1.smithi193.stdout:2/707: dwrite d0/d28/fa8 [0,4194304] 0 2024-03-23T08:23:25.969 INFO:tasks.workunit.client.1.smithi193.stdout:7/596: truncate d6/d9/fa 20651 0 2024-03-23T08:23:25.969 INFO:tasks.workunit.client.1.smithi193.stdout:1/630: dwrite da/dd/df/d7e/d7f/db2/fc1 [0,4194304] 0 2024-03-23T08:23:25.975 INFO:tasks.workunit.client.1.smithi193.stdout:0/622: dwrite db/d11/d14/d24/f4a [0,4194304] 0 2024-03-23T08:23:25.976 INFO:tasks.workunit.client.1.smithi193.stdout:8/628: write df/d10/d3e/d54/faa [400627,87247] 0 2024-03-23T08:23:25.977 INFO:tasks.workunit.client.1.smithi193.stdout:3/555: dwrite d1/de/d13/d21/d25/d34/f47 [0,4194304] 0 2024-03-23T08:23:25.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:25 smithi070 ceph-mon[39699]: pgmap v22: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 40 MiB/s rd, 160 MiB/s wr, 408 op/s 2024-03-23T08:23:25.983 INFO:tasks.workunit.client.1.smithi193.stdout:5/682: creat d0/d1c/d3b/d81/df2/ff8 x:0 0 0 2024-03-23T08:23:25.983 INFO:tasks.workunit.client.1.smithi193.stdout:1/631: dread - da/d13/d3f/d5a/f9d zero size 2024-03-23T08:23:25.987 INFO:tasks.workunit.client.1.smithi193.stdout:9/664: getdents d2/d7/db/d1b/d43/d71 0 2024-03-23T08:23:25.987 INFO:tasks.workunit.client.1.smithi193.stdout:6/537: read - d0/d27/d81/df/d11/d36/f97 zero size 2024-03-23T08:23:25.988 INFO:tasks.workunit.client.1.smithi193.stdout:5/683: write d0/d1c/d3b/d81/df2/ff8 [996295,50822] 0 2024-03-23T08:23:25.991 INFO:tasks.workunit.client.1.smithi193.stdout:3/556: getdents d1/de/d13/d21/d27/d7c 0 2024-03-23T08:23:25.993 INFO:tasks.workunit.client.1.smithi193.stdout:1/632: truncate da/dd/df/d7e/f84 951534 0 2024-03-23T08:23:25.995 INFO:tasks.workunit.client.1.smithi193.stdout:7/597: rename d6/d9/d1f/d5f/l8f to d6/d9/d1f/d23/d33/d38/dc0/lcf 0 2024-03-23T08:23:25.996 INFO:tasks.workunit.client.1.smithi193.stdout:0/623: dwrite db/d11/d14/f19 [0,4194304] 0 2024-03-23T08:23:25.996 INFO:tasks.workunit.client.1.smithi193.stdout:5/684: truncate d0/f1a 1892283 0 2024-03-23T08:23:25.996 INFO:tasks.workunit.client.1.smithi193.stdout:3/557: read d1/de/d13/f62 [516537,72525] 0 2024-03-23T08:23:25.997 INFO:tasks.workunit.client.1.smithi193.stdout:9/665: creat d2/d7/db/d17/d4b/d8b/fda x:0 0 0 2024-03-23T08:23:25.998 INFO:tasks.workunit.client.1.smithi193.stdout:6/538: unlink d0/d27/d81/df/d11/c65 0 2024-03-23T08:23:26.053 INFO:tasks.workunit.client.1.smithi193.stdout:9/666: write d2/d7/db/d17/d33/d70/f7b [155232,49660] 0 2024-03-23T08:23:26.054 INFO:tasks.workunit.client.1.smithi193.stdout:4/697: truncate f4 3596876 0 2024-03-23T08:23:26.056 INFO:tasks.workunit.client.1.smithi193.stdout:8/629: dread df/d4b/d4d/f69 [4194304,4194304] 0 2024-03-23T08:23:26.072 INFO:tasks.workunit.client.1.smithi193.stdout:8/630: stat df/f21 0 2024-03-23T08:23:26.073 INFO:tasks.workunit.client.1.smithi193.stdout:0/624: write db/d11/d14/d24/f5f [2021528,8625] 0 2024-03-23T08:23:26.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:25 smithi193 ceph-mon[48370]: pgmap v22: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 40 MiB/s rd, 160 MiB/s wr, 408 op/s 2024-03-23T08:23:26.078 INFO:tasks.workunit.client.1.smithi193.stdout:4/698: mknod d11/d1a/d33/d42/db2/cee 0 2024-03-23T08:23:26.081 INFO:tasks.workunit.client.1.smithi193.stdout:1/633: dwrite da/d13/d24/f79 [4194304,4194304] 0 2024-03-23T08:23:26.081 INFO:tasks.workunit.client.1.smithi193.stdout:6/539: mkdir d0/d27/d81/df/d11/d3b/d51/daa 0 2024-03-23T08:23:26.082 INFO:tasks.workunit.client.1.smithi193.stdout:2/708: dread d0/d1b/f8 [4194304,4194304] 0 2024-03-23T08:23:26.082 INFO:tasks.workunit.client.1.smithi193.stdout:5/685: dwrite d0/d1c/d3b/fce [4194304,4194304] 0 2024-03-23T08:23:26.082 INFO:tasks.workunit.client.1.smithi193.stdout:3/558: dwrite d1/de/d13/d96/d99/f64 [0,4194304] 0 2024-03-23T08:23:26.088 INFO:tasks.workunit.client.1.smithi193.stdout:8/631: dread df/d10/d3e/d54/faa [0,4194304] 0 2024-03-23T08:23:26.090 INFO:tasks.workunit.client.1.smithi193.stdout:3/559: chown d1/de/d13/d21/d27/d7c/c9d 1 1 2024-03-23T08:23:26.090 INFO:tasks.workunit.client.1.smithi193.stdout:5/686: mkdir d0/d1c/d3b/d4b/df9 0 2024-03-23T08:23:26.092 INFO:tasks.workunit.client.1.smithi193.stdout:1/634: mkdir da/dd/df/d52/dce 0 2024-03-23T08:23:26.092 INFO:tasks.workunit.client.1.smithi193.stdout:9/667: dwrite d2/d7/db/d17/d33/f5d [0,4194304] 0 2024-03-23T08:23:26.100 INFO:tasks.workunit.client.1.smithi193.stdout:2/709: rename d0/d28/f43 to d0/d1b/d14/d66/d4e/fcb 0 2024-03-23T08:23:26.101 INFO:tasks.workunit.client.1.smithi193.stdout:3/560: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/fc4 x:0 0 0 2024-03-23T08:23:26.102 INFO:tasks.workunit.client.1.smithi193.stdout:0/625: dwrite db/d11/d14/fad [0,4194304] 0 2024-03-23T08:23:26.102 INFO:tasks.workunit.client.1.smithi193.stdout:9/668: chown d2/d7/db/d7a/f87 82955280 1 2024-03-23T08:23:26.104 INFO:tasks.workunit.client.1.smithi193.stdout:8/632: link df/d10/d67/c80 df/d63/d76/cd6 0 2024-03-23T08:23:26.106 INFO:tasks.workunit.client.1.smithi193.stdout:2/710: stat d0/d1b/d14/d56/da3 0 2024-03-23T08:23:26.106 INFO:tasks.workunit.client.1.smithi193.stdout:4/699: dwrite d11/d2d/d58/d8b/dbc/f2f [0,4194304] 0 2024-03-23T08:23:26.116 INFO:tasks.workunit.client.1.smithi193.stdout:6/540: dread d0/d27/d81/df/d11/d56/f5c [0,4194304] 0 2024-03-23T08:23:26.119 INFO:tasks.workunit.client.1.smithi193.stdout:5/687: dwrite d0/d1f/fb3 [0,4194304] 0 2024-03-23T08:23:26.119 INFO:tasks.workunit.client.1.smithi193.stdout:3/561: truncate d1/de/d78/fa9 257834 0 2024-03-23T08:23:26.122 INFO:tasks.workunit.client.1.smithi193.stdout:8/633: rename df/d10/d3e/l7e to df/d4b/d4d/d2a/db2/ld7 0 2024-03-23T08:23:26.123 INFO:tasks.workunit.client.1.smithi193.stdout:2/711: truncate d0/d1b/d14/d66/d50/f91 769649 0 2024-03-23T08:23:26.126 INFO:tasks.workunit.client.1.smithi193.stdout:1/635: dwrite da/dd/df/d2e/f48 [0,4194304] 0 2024-03-23T08:23:26.130 INFO:tasks.workunit.client.1.smithi193.stdout:2/712: chown d0/d28/f8c 67 1 2024-03-23T08:23:26.131 INFO:tasks.workunit.client.1.smithi193.stdout:9/669: symlink d2/d7/db/d17/d4b/d8d/ldb 0 2024-03-23T08:23:26.131 INFO:tasks.workunit.client.1.smithi193.stdout:4/700: dread d11/d1a/d33/d7f/de0/fa0 [0,4194304] 0 2024-03-23T08:23:26.134 INFO:tasks.workunit.client.1.smithi193.stdout:0/626: dwrite db/d11/d14/d3b/d71/fc2 [0,4194304] 0 2024-03-23T08:23:26.135 INFO:tasks.workunit.client.1.smithi193.stdout:1/636: mkdir da/d13/d24/dac/dcf 0 2024-03-23T08:23:26.136 INFO:tasks.workunit.client.1.smithi193.stdout:9/670: write d2/d7/db/d17/d33/d70/f7b [1158254,747] 0 2024-03-23T08:23:26.140 INFO:tasks.workunit.client.1.smithi193.stdout:8/634: link df/d10/d3e/fb0 df/d63/d84/fd8 0 2024-03-23T08:23:26.141 INFO:tasks.workunit.client.1.smithi193.stdout:5/688: link d0/d1c/d3b/cd4 d0/d1c/d3b/dbc/dd2/cfa 0 2024-03-23T08:23:26.146 INFO:tasks.workunit.client.1.smithi193.stdout:1/637: write da/dd/df/d7e/fcb [571183,32371] 0 2024-03-23T08:23:26.146 INFO:tasks.workunit.client.1.smithi193.stdout:9/671: mkdir d2/d7/db/d1b/d43/d71/db4/d78/ddc 0 2024-03-23T08:23:26.147 INFO:tasks.workunit.client.1.smithi193.stdout:1/638: chown da/c21 29 1 2024-03-23T08:23:26.148 INFO:tasks.workunit.client.1.smithi193.stdout:4/701: dwrite d11/d2d/d58/d80/d9f/fa2 [0,4194304] 0 2024-03-23T08:23:26.150 INFO:tasks.workunit.client.1.smithi193.stdout:3/562: dread d1/de/d13/d21/d27/f63 [0,4194304] 0 2024-03-23T08:23:26.150 INFO:tasks.workunit.client.1.smithi193.stdout:9/672: read d2/d7/db/d7a/d95/fd4 [2217888,121496] 0 2024-03-23T08:23:26.151 INFO:tasks.workunit.client.1.smithi193.stdout:1/639: symlink da/d13/d3f/ld0 0 2024-03-23T08:23:26.151 INFO:tasks.workunit.client.1.smithi193.stdout:3/563: chown d1/de/d13/d21 46 1 2024-03-23T08:23:26.153 INFO:tasks.workunit.client.1.smithi193.stdout:1/640: chown da/d13/d24/f43 31401 1 2024-03-23T08:23:26.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/702: creat d11/d27/d85/de1/d99/ded/fef x:0 0 0 2024-03-23T08:23:26.154 INFO:tasks.workunit.client.1.smithi193.stdout:9/673: mknod d2/d7/db/d17/d4b/d8b/cdd 0 2024-03-23T08:23:26.157 INFO:tasks.workunit.client.1.smithi193.stdout:1/641: fdatasync f5 0 2024-03-23T08:23:26.157 INFO:tasks.workunit.client.1.smithi193.stdout:9/674: write d2/d7/db/d1b/f81 [1078374,29286] 0 2024-03-23T08:23:26.158 INFO:tasks.workunit.client.1.smithi193.stdout:1/642: chown da/dd/df/d52/d82 15663 1 2024-03-23T08:23:26.160 INFO:tasks.workunit.client.1.smithi193.stdout:1/643: fdatasync da/dd/f62 0 2024-03-23T08:23:26.161 INFO:tasks.workunit.client.1.smithi193.stdout:4/703: getdents d11/d2d/d3a 0 2024-03-23T08:23:26.162 INFO:tasks.workunit.client.1.smithi193.stdout:9/675: mkdir d2/d7/db/d32/d6b/dde 0 2024-03-23T08:23:26.166 INFO:tasks.workunit.client.1.smithi193.stdout:6/541: dread d0/d27/d81/f20 [0,4194304] 0 2024-03-23T08:23:26.167 INFO:tasks.workunit.client.1.smithi193.stdout:9/676: truncate d2/d7/db/d32/f4e 1218312 0 2024-03-23T08:23:26.169 INFO:tasks.workunit.client.1.smithi193.stdout:1/644: dwrite da/dd/df/d7e/d7f/db2/fc1 [0,4194304] 0 2024-03-23T08:23:26.171 INFO:tasks.workunit.client.1.smithi193.stdout:9/677: chown d2/d7/db/d32/d6b/c72 1753759 1 2024-03-23T08:23:26.174 INFO:tasks.workunit.client.1.smithi193.stdout:7/598: sync 2024-03-23T08:23:26.175 INFO:tasks.workunit.client.1.smithi193.stdout:9/678: creat d2/d7/db/d32/d6b/dde/fdf x:0 0 0 2024-03-23T08:23:26.176 INFO:tasks.workunit.client.1.smithi193.stdout:4/704: fdatasync d11/d2d/d58/d80/d9f/fa2 0 2024-03-23T08:23:26.178 INFO:tasks.workunit.client.1.smithi193.stdout:6/542: link d0/d27/d81/df/d11/d36/d61/d7c/d69/l8c d0/d27/d81/d87/lab 0 2024-03-23T08:23:26.179 INFO:tasks.workunit.client.1.smithi193.stdout:9/679: stat d2/d7/db/d17/d4b/d8d 0 2024-03-23T08:23:26.183 INFO:tasks.workunit.client.1.smithi193.stdout:2/713: dread d0/d1b/d14/d25/f29 [0,4194304] 0 2024-03-23T08:23:26.183 INFO:tasks.workunit.client.1.smithi193.stdout:7/599: rename d6/d9/d1f/d23/d33/d38/d66/f9c to d6/d9/d1f/d23/d69/fd0 0 2024-03-23T08:23:26.184 INFO:tasks.workunit.client.1.smithi193.stdout:1/645: dwrite da/dd/df/d7e/d86/d95/fb1 [0,4194304] 0 2024-03-23T08:23:26.186 INFO:tasks.workunit.client.1.smithi193.stdout:8/635: truncate df/f34 3662524 0 2024-03-23T08:23:26.189 INFO:tasks.workunit.client.1.smithi193.stdout:5/689: dread d0/d1c/d7a/db1/d72/d73/d57/d6d/d8e/fa7 [0,4194304] 0 2024-03-23T08:23:26.189 INFO:tasks.workunit.client.1.smithi193.stdout:9/680: symlink d2/d7/db/d17/d4b/d59/d97/d9d/le0 0 2024-03-23T08:23:26.190 INFO:tasks.workunit.client.1.smithi193.stdout:1/646: chown da/dd/l60 87458721 1 2024-03-23T08:23:26.192 INFO:tasks.workunit.client.1.smithi193.stdout:8/636: chown df/d4b/d4d/d2a/l4c 799131861 1 2024-03-23T08:23:26.192 INFO:tasks.workunit.client.1.smithi193.stdout:4/705: truncate d11/d1a/d33/d8f/db8/f45 628431 0 2024-03-23T08:23:26.194 INFO:tasks.workunit.client.1.smithi193.stdout:3/564: dread d1/de/d13/d21/d25/f7e [0,4194304] 0 2024-03-23T08:23:26.194 INFO:tasks.workunit.client.1.smithi193.stdout:7/600: write d6/d9/f6f [4453165,126647] 0 2024-03-23T08:23:26.196 INFO:tasks.workunit.client.1.smithi193.stdout:6/543: getdents d0/d27/d81/df/d11 0 2024-03-23T08:23:26.196 INFO:tasks.workunit.client.1.smithi193.stdout:0/627: dwrite db/d11/d14/d24/d2a/d4b/f84 [0,4194304] 0 2024-03-23T08:23:26.197 INFO:tasks.workunit.client.1.smithi193.stdout:8/637: write df/dc6/fad [52522,116456] 0 2024-03-23T08:23:26.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/714: creat d0/d28/d33/db7/dc0/fcc x:0 0 0 2024-03-23T08:23:26.205 INFO:tasks.workunit.client.1.smithi193.stdout:4/706: write d11/d1a/d33/d42/d52/d70/fba [3950321,98725] 0 2024-03-23T08:23:26.207 INFO:tasks.workunit.client.1.smithi193.stdout:3/565: readlink d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/l92 0 2024-03-23T08:23:26.210 INFO:tasks.workunit.client.1.smithi193.stdout:7/601: dread d6/d9/d1f/d3e/d65/d70/f7f [0,4194304] 0 2024-03-23T08:23:26.210 INFO:tasks.workunit.client.1.smithi193.stdout:6/544: truncate d0/d27/d81/fa6 709373 0 2024-03-23T08:23:26.212 INFO:tasks.workunit.client.1.smithi193.stdout:5/690: truncate d0/d1c/d7a/db1/d72/d73/d57/d6d/fb8 2000131 0 2024-03-23T08:23:26.212 INFO:tasks.workunit.client.1.smithi193.stdout:1/647: dwrite da/dd/df/f25 [0,4194304] 0 2024-03-23T08:23:26.213 INFO:tasks.workunit.client.1.smithi193.stdout:3/566: chown d1/de/d13/d96/d99/l48 1625 1 2024-03-23T08:23:26.214 INFO:tasks.workunit.client.1.smithi193.stdout:4/707: rmdir d11/d2d/d58/d80 39 2024-03-23T08:23:26.215 INFO:tasks.workunit.client.1.smithi193.stdout:2/715: fsync d0/d28/d33/d72/fc9 0 2024-03-23T08:23:26.215 INFO:tasks.workunit.client.1.smithi193.stdout:0/628: write db/d11/d14/d24/f7d [3584838,21231] 0 2024-03-23T08:23:26.218 INFO:tasks.workunit.client.1.smithi193.stdout:6/545: creat d0/d27/d72/fac x:0 0 0 2024-03-23T08:23:26.222 INFO:tasks.workunit.client.1.smithi193.stdout:3/567: creat d1/de/d28/fc5 x:0 0 0 2024-03-23T08:23:26.224 INFO:tasks.workunit.client.1.smithi193.stdout:8/638: dwrite df/d10/d3e/d54/dd2/f4f [0,4194304] 0 2024-03-23T08:23:26.224 INFO:tasks.workunit.client.1.smithi193.stdout:5/691: read - d0/d1c/d7a/db1/d72/d73/d45/f5f zero size 2024-03-23T08:23:26.226 INFO:tasks.workunit.client.1.smithi193.stdout:6/546: rmdir d0/d27/d81/df/d11/d3b/d51 39 2024-03-23T08:23:26.230 INFO:tasks.workunit.client.1.smithi193.stdout:3/568: fsync d1/de/d13/d96/d99/f55 0 2024-03-23T08:23:26.231 INFO:tasks.workunit.client.1.smithi193.stdout:4/708: creat d11/d2d/d58/d80/ff0 x:0 0 0 2024-03-23T08:23:26.231 INFO:tasks.workunit.client.1.smithi193.stdout:7/602: write d6/d9/fd [1415339,101413] 0 2024-03-23T08:23:26.234 INFO:tasks.workunit.client.1.smithi193.stdout:5/692: mknod d0/d1c/d7a/d86/d9f/cfb 0 2024-03-23T08:23:26.234 INFO:tasks.workunit.client.1.smithi193.stdout:0/629: mknod db/d11/d14/d24/d47/d59/d8c/dda/ddb/cdf 0 2024-03-23T08:23:26.236 INFO:tasks.workunit.client.1.smithi193.stdout:1/648: symlink da/d13/d24/dac/dcf/ld1 0 2024-03-23T08:23:26.238 INFO:tasks.workunit.client.1.smithi193.stdout:3/569: mkdir d1/de/d13/d21/d59/dc6 0 2024-03-23T08:23:26.241 INFO:tasks.workunit.client.1.smithi193.stdout:4/709: truncate d11/d1a/d33/d7f/f81 213003 0 2024-03-23T08:23:26.242 INFO:tasks.workunit.client.1.smithi193.stdout:9/681: dwrite d2/d7/db/d32/f4e [0,4194304] 0 2024-03-23T08:23:26.242 INFO:tasks.workunit.client.1.smithi193.stdout:1/649: chown da/dd/df/d2e/l72 8404488 1 2024-03-23T08:23:26.244 INFO:tasks.workunit.client.1.smithi193.stdout:2/716: dwrite d0/d1b/d14/d66/f8e [4194304,4194304] 0 2024-03-23T08:23:26.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/650: stat da/f4e 0 2024-03-23T08:23:26.306 INFO:tasks.workunit.client.1.smithi193.stdout:5/693: mkdir d0/d1c/d7a/db1/d72/d73/d57/dfc 0 2024-03-23T08:23:26.307 INFO:tasks.workunit.client.1.smithi193.stdout:7/603: rename d6/d9/d1f/f30 to d6/d9/d1f/d23/d33/d38/d46/d82/fd1 0 2024-03-23T08:23:26.318 INFO:tasks.workunit.client.1.smithi193.stdout:5/694: dread d0/d1c/d3b/d81/df2/d97/fcc [0,4194304] 0 2024-03-23T08:23:26.319 INFO:tasks.workunit.client.1.smithi193.stdout:7/604: dread d6/d9/d1f/d23/d33/d38/f91 [0,4194304] 0 2024-03-23T08:23:26.320 INFO:tasks.workunit.client.1.smithi193.stdout:7/605: chown d6/d9/d1f/d23/d33/d38/d3b/cc4 5 1 2024-03-23T08:23:26.321 INFO:tasks.workunit.client.1.smithi193.stdout:7/606: fdatasync d6/d9/d1f/d23/d33/d38/f68 0 2024-03-23T08:23:26.332 INFO:tasks.workunit.client.1.smithi193.stdout:3/570: unlink d1/lb 0 2024-03-23T08:23:26.333 INFO:tasks.workunit.client.1.smithi193.stdout:4/710: creat d11/d27/d85/de1/d99/ff1 x:0 0 0 2024-03-23T08:23:26.334 INFO:tasks.workunit.client.1.smithi193.stdout:1/651: mkdir da/dd/df/d7c/dd2 0 2024-03-23T08:23:26.335 INFO:tasks.workunit.client.1.smithi193.stdout:9/682: mkdir d2/d7/db/d17/d33/d70/de1 0 2024-03-23T08:23:26.338 INFO:tasks.workunit.client.1.smithi193.stdout:6/547: rename d0/d27/d81/df/d11/d36/d61/d7c/l9f to d0/d27/d81/da1/lad 0 2024-03-23T08:23:26.339 INFO:tasks.workunit.client.1.smithi193.stdout:8/639: dwrite df/d4b/d4d/d2a/f3c [0,4194304] 0 2024-03-23T08:23:26.343 INFO:tasks.workunit.client.1.smithi193.stdout:5/695: read d0/f15 [1013242,74801] 0 2024-03-23T08:23:26.344 INFO:tasks.workunit.client.1.smithi193.stdout:7/607: mkdir d6/d9/d1f/daf/dd2 0 2024-03-23T08:23:26.345 INFO:tasks.workunit.client.1.smithi193.stdout:4/711: fsync d11/d1a/d33/d42/d52/f3f 0 2024-03-23T08:23:26.346 INFO:tasks.workunit.client.1.smithi193.stdout:5/696: fsync d0/d1f/fca 0 2024-03-23T08:23:26.348 INFO:tasks.workunit.client.1.smithi193.stdout:9/683: write d2/d7/db/d1b/d43/f84 [707409,46697] 0 2024-03-23T08:23:26.349 INFO:tasks.workunit.client.1.smithi193.stdout:2/717: rename d0/d28/d9f/fb8 to d0/d28/d33/d72/fcd 0 2024-03-23T08:23:26.351 INFO:tasks.workunit.client.1.smithi193.stdout:6/548: truncate d0/d27/d81/df/d11/d36/d61/f48 773343 0 2024-03-23T08:23:26.351 INFO:tasks.workunit.client.1.smithi193.stdout:0/630: dwrite db/d11/d14/d24/d47/f61 [0,4194304] 0 2024-03-23T08:23:26.353 INFO:tasks.workunit.client.1.smithi193.stdout:0/631: chown db/d11/d14/d24/l86 206365 1 2024-03-23T08:23:26.355 INFO:tasks.workunit.client.1.smithi193.stdout:1/652: write da/f4e [4541919,118050] 0 2024-03-23T08:23:26.355 INFO:tasks.workunit.client.1.smithi193.stdout:4/712: fdatasync d11/d2d/f4b 0 2024-03-23T08:23:26.361 INFO:tasks.workunit.client.1.smithi193.stdout:9/684: symlink d2/d7/db/d17/d33/le2 0 2024-03-23T08:23:26.361 INFO:tasks.workunit.client.1.smithi193.stdout:2/718: dwrite d0/d1b/d3b/f58 [0,4194304] 0 2024-03-23T08:23:26.364 INFO:tasks.workunit.client.1.smithi193.stdout:3/571: dwrite d1/de/d13/d21/d59/d60/f6b [4194304,4194304] 0 2024-03-23T08:23:26.368 INFO:tasks.workunit.client.1.smithi193.stdout:4/713: mkdir d11/d27/df2 0 2024-03-23T08:23:26.368 INFO:tasks.workunit.client.1.smithi193.stdout:3/572: write d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/fc4 [438197,116858] 0 2024-03-23T08:23:26.369 INFO:tasks.workunit.client.1.smithi193.stdout:7/608: creat d6/d9/d1f/fd3 x:0 0 0 2024-03-23T08:23:26.370 INFO:tasks.workunit.client.1.smithi193.stdout:9/685: mkdir d2/d7/d31/d5f/de3 0 2024-03-23T08:23:26.372 INFO:tasks.workunit.client.1.smithi193.stdout:9/686: read - d2/d7/db/d32/fd2 zero size 2024-03-23T08:23:26.372 INFO:tasks.workunit.client.1.smithi193.stdout:4/714: dread d11/d1a/d33/d7f/f81 [0,4194304] 0 2024-03-23T08:23:26.374 INFO:tasks.workunit.client.1.smithi193.stdout:2/719: unlink d0/d1b/d14/d66/d4e/l57 0 2024-03-23T08:23:26.374 INFO:tasks.workunit.client.1.smithi193.stdout:7/609: dread d6/d9/d1f/d3e/d65/d7d/f9f [0,4194304] 0 2024-03-23T08:23:26.378 INFO:tasks.workunit.client.1.smithi193.stdout:8/640: getdents df/d4b/d4d 0 2024-03-23T08:23:26.379 INFO:tasks.workunit.client.1.smithi193.stdout:3/573: unlink d1/de/d13/d21/d50/d44/f54 0 2024-03-23T08:23:26.380 INFO:tasks.workunit.client.1.smithi193.stdout:2/720: write d0/d1b/f13 [136899,61346] 0 2024-03-23T08:23:26.384 INFO:tasks.workunit.client.1.smithi193.stdout:7/610: dread - d6/d9/d1f/d3e/d65/d7d/fb4 zero size 2024-03-23T08:23:26.389 INFO:tasks.workunit.client.1.smithi193.stdout:8/641: symlink df/d63/d84/dc5/ld9 0 2024-03-23T08:23:26.389 INFO:tasks.workunit.client.1.smithi193.stdout:4/715: readlink d11/d1a/d33/d42/d52/d70/lb5 0 2024-03-23T08:23:26.390 INFO:tasks.workunit.client.1.smithi193.stdout:0/632: dread db/d11/d14/d21/f22 [0,4194304] 0 2024-03-23T08:23:26.391 INFO:tasks.workunit.client.1.smithi193.stdout:5/697: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:26.393 INFO:tasks.workunit.client.1.smithi193.stdout:2/721: mkdir d0/d1b/d14/d66/d4e/dce 0 2024-03-23T08:23:26.396 INFO:tasks.workunit.client.1.smithi193.stdout:9/687: dwrite d2/d7/db/d17/d33/fd9 [0,4194304] 0 2024-03-23T08:23:26.396 INFO:tasks.workunit.client.1.smithi193.stdout:8/642: write df/f2c [1746263,115678] 0 2024-03-23T08:23:26.401 INFO:tasks.workunit.client.1.smithi193.stdout:7/611: fsync d6/d9/f21 0 2024-03-23T08:23:26.402 INFO:tasks.workunit.client.1.smithi193.stdout:6/549: dwrite d0/d27/d81/df/d11/d3b/d51/d9c/f96 [0,4194304] 0 2024-03-23T08:23:26.405 INFO:tasks.workunit.client.1.smithi193.stdout:4/716: readlink d11/d8e/dac/lc9 0 2024-03-23T08:23:26.405 INFO:tasks.workunit.client.1.smithi193.stdout:0/633: fsync db/d11/d14/d24/d2a/d96/fb9 0 2024-03-23T08:23:26.407 INFO:tasks.workunit.client.1.smithi193.stdout:7/612: chown d6/d9/d1f/f2a 1581645047 1 2024-03-23T08:23:26.408 INFO:tasks.workunit.client.1.smithi193.stdout:9/688: write d2/f8f [1546927,358] 0 2024-03-23T08:23:26.413 INFO:tasks.workunit.client.1.smithi193.stdout:3/574: dwrite d1/de/f71 [0,4194304] 0 2024-03-23T08:23:26.414 INFO:tasks.workunit.client.1.smithi193.stdout:5/698: mkdir d0/d1c/d7a/db1/d72/d73/d45/d6e/d91/dfd 0 2024-03-23T08:23:26.418 INFO:tasks.workunit.client.1.smithi193.stdout:0/634: write db/d11/d14/d24/d47/dc7/fb2 [437768,81189] 0 2024-03-23T08:23:26.421 INFO:tasks.workunit.client.1.smithi193.stdout:9/689: write d2/d7/db/d17/f53 [356795,117954] 0 2024-03-23T08:23:26.424 INFO:tasks.workunit.client.1.smithi193.stdout:8/643: link df/d63/d84/c91 df/d10/cda 0 2024-03-23T08:23:26.426 INFO:tasks.workunit.client.1.smithi193.stdout:7/613: mknod d6/d9/d4e/d81/dca/cd4 0 2024-03-23T08:23:26.429 INFO:tasks.workunit.client.1.smithi193.stdout:9/690: chown d2/d7/db/d17 59 1 2024-03-23T08:23:26.431 INFO:tasks.workunit.client.1.smithi193.stdout:2/722: dwrite d0/d1b/d3b/f7a [0,4194304] 0 2024-03-23T08:23:26.434 INFO:tasks.workunit.client.1.smithi193.stdout:4/717: dwrite f5 [0,4194304] 0 2024-03-23T08:23:26.434 INFO:tasks.workunit.client.1.smithi193.stdout:7/614: creat d6/d9/d1f/d4d/d6b/fd5 x:0 0 0 2024-03-23T08:23:26.436 INFO:tasks.workunit.client.1.smithi193.stdout:2/723: readlink d0/d1b/d19/lc5 0 2024-03-23T08:23:26.437 INFO:tasks.workunit.client.1.smithi193.stdout:3/575: write d1/de/d13/f62 [27853,58793] 0 2024-03-23T08:23:26.440 INFO:tasks.workunit.client.1.smithi193.stdout:9/691: creat d2/d7/db/d7a/d95/da9/fe4 x:0 0 0 2024-03-23T08:23:26.444 INFO:tasks.workunit.client.1.smithi193.stdout:5/699: dwrite d0/d1c/d7a/db1/d72/d99/fc5 [0,4194304] 0 2024-03-23T08:23:26.451 INFO:tasks.workunit.client.1.smithi193.stdout:0/635: dwrite db/d11/d14/d24/d47/d59/faa [0,4194304] 0 2024-03-23T08:23:26.453 INFO:tasks.workunit.client.1.smithi193.stdout:9/692: readlink d2/l52 0 2024-03-23T08:23:26.455 INFO:tasks.workunit.client.1.smithi193.stdout:4/718: getdents d11/d1a/d33/d7f/de0 0 2024-03-23T08:23:26.455 INFO:tasks.workunit.client.1.smithi193.stdout:6/550: dwrite d0/d27/d81/df/d11/d36/d61/f48 [0,4194304] 0 2024-03-23T08:23:26.457 INFO:tasks.workunit.client.1.smithi193.stdout:5/700: stat d0/d1f/d93/de1 0 2024-03-23T08:23:26.459 INFO:tasks.workunit.client.1.smithi193.stdout:2/724: dread d0/d1b/d3b/f58 [0,4194304] 0 2024-03-23T08:23:26.462 INFO:tasks.workunit.client.1.smithi193.stdout:5/701: dread - d0/d1c/d7a/d86/d9f/fc8 zero size 2024-03-23T08:23:26.462 INFO:tasks.workunit.client.1.smithi193.stdout:4/719: dread - d11/d1a/d33/d8f/db8/f75 zero size 2024-03-23T08:23:26.463 INFO:tasks.workunit.client.1.smithi193.stdout:0/636: truncate db/d11/f28 347943 0 2024-03-23T08:23:26.465 INFO:tasks.workunit.client.1.smithi193.stdout:6/551: mkdir d0/d27/d81/df/dae 0 2024-03-23T08:23:26.466 INFO:tasks.workunit.client.1.smithi193.stdout:2/725: mkdir d0/dcf 0 2024-03-23T08:23:26.466 INFO:tasks.workunit.client.1.smithi193.stdout:0/637: write db/d11/d14/d21/d33/d69/d9b/fce [414560,961] 0 2024-03-23T08:23:26.468 INFO:tasks.workunit.client.1.smithi193.stdout:4/720: rename d11/d8e/dac/dc6 to d11/d2d/d58/d80/df3 0 2024-03-23T08:23:26.471 INFO:tasks.workunit.client.1.smithi193.stdout:5/702: link d0/d1c/d7a/d86/d9f/fc8 d0/d1f/d93/dcd/ffe 0 2024-03-23T08:23:26.472 INFO:tasks.workunit.client.1.smithi193.stdout:6/552: dwrite d0/f8b [0,4194304] 0 2024-03-23T08:23:26.473 INFO:tasks.workunit.client.1.smithi193.stdout:2/726: creat d0/d28/fd0 x:0 0 0 2024-03-23T08:23:26.474 INFO:tasks.workunit.client.1.smithi193.stdout:5/703: fdatasync d0/d1c/d7a/db1/d72/d73/d57/d6d/fb8 0 2024-03-23T08:23:26.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/727: symlink d0/d1b/d14/d66/d50/d8f/ld1 0 2024-03-23T08:23:26.476 INFO:tasks.workunit.client.1.smithi193.stdout:5/704: mkdir d0/d1c/d3b/d4b/dff 0 2024-03-23T08:23:26.478 INFO:tasks.workunit.client.1.smithi193.stdout:5/705: chown d0/d1c/d7a/db1/d72/d73/d45/da2/lc1 54126 1 2024-03-23T08:23:26.478 INFO:tasks.workunit.client.1.smithi193.stdout:6/553: rmdir d0/d27/d81/df/d11/d36 39 2024-03-23T08:23:26.484 INFO:tasks.workunit.client.1.smithi193.stdout:9/693: write d2/d7/db/d17/d33/d70/fd0 [2780098,68826] 0 2024-03-23T08:23:26.484 INFO:tasks.workunit.client.1.smithi193.stdout:3/576: truncate d1/de/d78/fa9 38993 0 2024-03-23T08:23:26.487 INFO:tasks.workunit.client.1.smithi193.stdout:6/554: creat d0/d27/d81/df/faf x:0 0 0 2024-03-23T08:23:26.487 INFO:tasks.workunit.client.1.smithi193.stdout:4/721: dread d11/d2d/d58/d80/db9/fdf [4194304,4194304] 0 2024-03-23T08:23:26.489 INFO:tasks.workunit.client.1.smithi193.stdout:9/694: write d2/d7/db/d32/f4e [5229137,90060] 0 2024-03-23T08:23:26.491 INFO:tasks.workunit.client.1.smithi193.stdout:0/638: write db/f87 [1804738,18327] 0 2024-03-23T08:23:26.493 INFO:tasks.workunit.client.1.smithi193.stdout:7/615: dwrite d6/d9/d1f/d23/d33/d3c/d41/f4a [0,4194304] 0 2024-03-23T08:23:26.494 INFO:tasks.workunit.client.1.smithi193.stdout:5/706: getdents d0/d1c/d7a/db1/d72/d73/d57/d6d/d8e 0 2024-03-23T08:23:26.495 INFO:tasks.workunit.client.1.smithi193.stdout:9/695: fsync d2/d7/db/d1b/d43/d71/db4/d78/f9b 0 2024-03-23T08:23:26.496 INFO:tasks.workunit.client.1.smithi193.stdout:9/696: readlink d2/d7/db/d7a/laa 0 2024-03-23T08:23:26.499 INFO:tasks.workunit.client.1.smithi193.stdout:7/616: mknod d6/d9/d1f/d23/d33/d38/d3b/cd6 0 2024-03-23T08:23:26.499 INFO:tasks.workunit.client.1.smithi193.stdout:0/639: fsync db/d11/d14/d21/f2c 0 2024-03-23T08:23:26.500 INFO:tasks.workunit.client.1.smithi193.stdout:2/728: truncate d0/d1b/d14/d56/d7c/d7e/f88 3700267 0 2024-03-23T08:23:26.501 INFO:tasks.workunit.client.1.smithi193.stdout:9/697: read d2/d7/fcb [3054142,128248] 0 2024-03-23T08:23:26.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/698: chown d2/d7/db/d17/d1f 91826 1 2024-03-23T08:23:26.503 INFO:tasks.workunit.client.1.smithi193.stdout:4/722: mknod d11/d8e/cf4 0 2024-03-23T08:23:26.506 INFO:tasks.workunit.client.1.smithi193.stdout:0/640: creat db/d11/d14/d24/d47/d59/d8c/dda/fe0 x:0 0 0 2024-03-23T08:23:26.507 INFO:tasks.workunit.client.1.smithi193.stdout:2/729: rmdir d0/d28/d33/db7/dc0 39 2024-03-23T08:23:26.508 INFO:tasks.workunit.client.1.smithi193.stdout:3/577: write d1/f84 [631251,70969] 0 2024-03-23T08:23:26.509 INFO:tasks.workunit.client.1.smithi193.stdout:2/730: fsync d0/d1b/d14/d66/d50/d92/fc8 0 2024-03-23T08:23:26.510 INFO:tasks.workunit.client.1.smithi193.stdout:9/699: mkdir d2/d7/de5 0 2024-03-23T08:23:26.512 INFO:tasks.workunit.client.1.smithi193.stdout:5/707: mknod d0/d5a/dbf/c100 0 2024-03-23T08:23:26.515 INFO:tasks.workunit.client.1.smithi193.stdout:5/708: dread - d0/d1c/d7a/db1/d72/fef zero size 2024-03-23T08:23:26.518 INFO:tasks.workunit.client.1.smithi193.stdout:2/731: creat d0/d1b/d14/d56/d7c/fd2 x:0 0 0 2024-03-23T08:23:26.519 INFO:tasks.workunit.client.1.smithi193.stdout:4/723: dread d11/d1a/d33/d42/f4d [0,4194304] 0 2024-03-23T08:23:26.520 INFO:tasks.workunit.client.1.smithi193.stdout:3/578: dwrite d1/de/d13/d21/d59/d60/fb7 [4194304,4194304] 0 2024-03-23T08:23:26.521 INFO:tasks.workunit.client.1.smithi193.stdout:0/641: link db/d11/d14/d21/d33/d69/d9b/fca db/d11/d14/d21/dd2/fe1 0 2024-03-23T08:23:26.522 INFO:tasks.workunit.client.1.smithi193.stdout:6/555: dread d0/d27/d81/df/d11/d36/d61/d3a/f45 [0,4194304] 0 2024-03-23T08:23:26.527 INFO:tasks.workunit.client.1.smithi193.stdout:6/556: fdatasync d0/d27/d81/f33 0 2024-03-23T08:23:26.527 INFO:tasks.workunit.client.1.smithi193.stdout:9/700: dread d2/fc [0,4194304] 0 2024-03-23T08:23:26.527 INFO:tasks.workunit.client.1.smithi193.stdout:3/579: truncate d1/de/d13/d21/d27/f1c 4194637 0 2024-03-23T08:23:26.528 INFO:tasks.workunit.client.1.smithi193.stdout:2/732: rename d0/d1b/d14/d66/d50/f93 to d0/d1b/fd3 0 2024-03-23T08:23:26.529 INFO:tasks.workunit.client.1.smithi193.stdout:5/709: getdents d0/d1c/d3b/d81/df2/d97 0 2024-03-23T08:23:26.530 INFO:tasks.workunit.client.1.smithi193.stdout:5/710: dread - d0/d1c/d3b/d81/df2/d97/fee zero size 2024-03-23T08:23:26.531 INFO:tasks.workunit.client.1.smithi193.stdout:3/580: creat d1/de/d78/fc7 x:0 0 0 2024-03-23T08:23:26.535 INFO:tasks.workunit.client.1.smithi193.stdout:5/711: creat d0/d5a/dbf/f101 x:0 0 0 2024-03-23T08:23:26.536 INFO:tasks.workunit.client.1.smithi193.stdout:6/557: dwrite d0/d27/d81/f86 [0,4194304] 0 2024-03-23T08:23:26.539 INFO:tasks.workunit.client.1.smithi193.stdout:5/712: mkdir d0/d1c/d7a/d86/d9f/d102 0 2024-03-23T08:23:26.542 INFO:tasks.workunit.client.1.smithi193.stdout:5/713: chown d0/d1c/d7a/fed 277 1 2024-03-23T08:23:26.544 INFO:tasks.workunit.client.1.smithi193.stdout:0/642: dwrite db/d11/f12 [0,4194304] 0 2024-03-23T08:23:26.545 INFO:tasks.workunit.client.1.smithi193.stdout:0/643: chown db/d11/f12 8 1 2024-03-23T08:23:26.547 INFO:tasks.workunit.client.1.smithi193.stdout:0/644: dread - db/d11/d14/d24/d47/d59/f7c zero size 2024-03-23T08:23:26.548 INFO:tasks.workunit.client.1.smithi193.stdout:3/581: dread d1/de/f70 [0,4194304] 0 2024-03-23T08:23:26.588 INFO:tasks.workunit.client.1.smithi193.stdout:1/653: sync 2024-03-23T08:23:26.598 INFO:tasks.workunit.client.1.smithi193.stdout:0/645: dread - db/d11/d14/d24/d2a/fa9 zero size 2024-03-23T08:23:26.598 INFO:tasks.workunit.client.1.smithi193.stdout:3/582: fsync d1/de/f16 0 2024-03-23T08:23:26.599 INFO:tasks.workunit.client.1.smithi193.stdout:6/558: truncate d0/d27/d81/df/d11/d36/d61/d12/d21/f73 5126587 0 2024-03-23T08:23:26.601 INFO:tasks.workunit.client.1.smithi193.stdout:3/583: readlink d1/de/d13/d21/d50/d6f/l5f 0 2024-03-23T08:23:26.602 INFO:tasks.workunit.client.1.smithi193.stdout:6/559: dread - d0/d27/d81/df/d11/d3b/d6c/f88 zero size 2024-03-23T08:23:26.602 INFO:tasks.workunit.client.1.smithi193.stdout:0/646: mkdir db/d11/d14/d21/d33/d48/de2 0 2024-03-23T08:23:26.604 INFO:tasks.workunit.client.1.smithi193.stdout:6/560: write d0/f8b [4796233,91204] 0 2024-03-23T08:23:26.609 INFO:tasks.workunit.client.1.smithi193.stdout:6/561: write d0/d27/d81/df/d11/d36/f44 [2557552,71743] 0 2024-03-23T08:23:26.610 INFO:tasks.workunit.client.1.smithi193.stdout:0/647: dread db/d11/d14/d21/d33/d69/d9b/fce [0,4194304] 0 2024-03-23T08:23:26.610 INFO:tasks.workunit.client.1.smithi193.stdout:1/654: dwrite da/d13/d24/f7b [0,4194304] 0 2024-03-23T08:23:26.611 INFO:tasks.workunit.client.1.smithi193.stdout:9/701: dread d2/f4 [0,4194304] 0 2024-03-23T08:23:26.614 INFO:tasks.workunit.client.1.smithi193.stdout:9/702: write d2/d7/db/d17/fc4 [350325,60659] 0 2024-03-23T08:23:26.616 INFO:tasks.workunit.client.1.smithi193.stdout:6/562: dread d0/d27/d81/df/d11/d3b/d51/d9c/f8a [0,4194304] 0 2024-03-23T08:23:26.618 INFO:tasks.workunit.client.1.smithi193.stdout:6/563: read d0/d27/d81/f4e [2332444,75101] 0 2024-03-23T08:23:26.619 INFO:tasks.workunit.client.1.smithi193.stdout:3/584: dread d1/de/d13/f33 [0,4194304] 0 2024-03-23T08:23:26.619 INFO:tasks.workunit.client.1.smithi193.stdout:6/564: chown d0/d27/d81/df/d11/d36/d61/d12 988 1 2024-03-23T08:23:26.620 INFO:tasks.workunit.client.1.smithi193.stdout:9/703: mknod d2/d7/db/d7a/ce6 0 2024-03-23T08:23:26.622 INFO:tasks.workunit.client.1.smithi193.stdout:0/648: rmdir db/d11/d14/d21/d33/d48/de2 0 2024-03-23T08:23:26.625 INFO:tasks.workunit.client.1.smithi193.stdout:6/565: mknod d0/d27/d81/df/d11/d3b/cb0 0 2024-03-23T08:23:26.626 INFO:tasks.workunit.client.1.smithi193.stdout:1/655: dwrite da/dd/fa6 [0,4194304] 0 2024-03-23T08:23:26.628 INFO:tasks.workunit.client.1.smithi193.stdout:1/656: dread - da/dd/df/d7e/d86/fb3 zero size 2024-03-23T08:23:26.630 INFO:tasks.workunit.client.1.smithi193.stdout:0/649: rename db/d11/d14/d24/d47/d59/d8c/dda to db/d11/d14/d24/d2a/de3 0 2024-03-23T08:23:26.631 INFO:tasks.workunit.client.1.smithi193.stdout:9/704: link d2/d7/db/c10 d2/d7/d31/d5f/d9c/ce7 0 2024-03-23T08:23:26.632 INFO:tasks.workunit.client.1.smithi193.stdout:6/566: mkdir d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/db1 0 2024-03-23T08:23:26.633 INFO:tasks.workunit.client.1.smithi193.stdout:9/705: symlink d2/d7/db/d17/le8 0 2024-03-23T08:23:26.634 INFO:tasks.workunit.client.1.smithi193.stdout:9/706: chown d2/d7/db/d17/d33/f63 2 1 2024-03-23T08:23:26.635 INFO:tasks.workunit.client.1.smithi193.stdout:6/567: mkdir d0/d27/d81/df/d11/d3b/d51/db2 0 2024-03-23T08:23:26.636 INFO:tasks.workunit.client.1.smithi193.stdout:9/707: write d2/d7/db/d7a/d95/da9/fe4 [627958,93341] 0 2024-03-23T08:23:26.636 INFO:tasks.workunit.client.1.smithi193.stdout:3/585: truncate d1/de/d13/d21/d59/d60/fb7 5002947 0 2024-03-23T08:23:26.638 INFO:tasks.workunit.client.1.smithi193.stdout:3/586: dread - d1/de/d13/d21/d25/d2b/d3c/faa zero size 2024-03-23T08:23:26.639 INFO:tasks.workunit.client.1.smithi193.stdout:0/650: link db/d11/d14/d24/d2a/d4d/d93/fd1 db/d11/d14/d21/d33/d48/fe4 0 2024-03-23T08:23:26.640 INFO:tasks.workunit.client.1.smithi193.stdout:1/657: getdents da/dd/df/d2e 0 2024-03-23T08:23:26.641 INFO:tasks.workunit.client.1.smithi193.stdout:9/708: creat d2/d7/db/d1b/d5c/fe9 x:0 0 0 2024-03-23T08:23:26.642 INFO:tasks.workunit.client.1.smithi193.stdout:9/709: stat d2/d7/db/c41 0 2024-03-23T08:23:26.642 INFO:tasks.workunit.client.1.smithi193.stdout:6/568: mknod d0/d27/d81/df/d11/d36/d61/d3a/cb3 0 2024-03-23T08:23:26.643 INFO:tasks.workunit.client.1.smithi193.stdout:6/569: write d0/d27/f91 [5229624,43171] 0 2024-03-23T08:23:26.647 INFO:tasks.workunit.client.1.smithi193.stdout:9/710: mknod d2/d7/db/d32/d6b/cea 0 2024-03-23T08:23:26.650 INFO:tasks.workunit.client.1.smithi193.stdout:1/658: mknod da/d13/cd3 0 2024-03-23T08:23:26.651 INFO:tasks.workunit.client.1.smithi193.stdout:9/711: creat d2/d7/db/d17/d33/feb x:0 0 0 2024-03-23T08:23:26.652 INFO:tasks.workunit.client.1.smithi193.stdout:9/712: write d2/f7d [1373450,67359] 0 2024-03-23T08:23:26.653 INFO:tasks.workunit.client.1.smithi193.stdout:3/587: creat d1/de/d13/d21/d50/fc8 x:0 0 0 2024-03-23T08:23:26.653 INFO:tasks.workunit.client.1.smithi193.stdout:0/651: write db/f6f [13907,82105] 0 2024-03-23T08:23:26.654 INFO:tasks.workunit.client.1.smithi193.stdout:9/713: write d2/d7/db/d17/d4b/d8b/fb7 [875110,74864] 0 2024-03-23T08:23:26.656 INFO:tasks.workunit.client.1.smithi193.stdout:9/714: chown d2/d7/db/d17/d33/cab 7854 1 2024-03-23T08:23:26.656 INFO:tasks.workunit.client.1.smithi193.stdout:1/659: rmdir da/dd/df 39 2024-03-23T08:23:26.659 INFO:tasks.workunit.client.1.smithi193.stdout:0/652: rename db/d11/d14/d51/ccd to db/dc6/ce5 0 2024-03-23T08:23:26.660 INFO:tasks.workunit.client.1.smithi193.stdout:1/660: symlink da/d13/d3f/d5a/d76/ld4 0 2024-03-23T08:23:26.660 INFO:tasks.workunit.client.1.smithi193.stdout:0/653: fsync db/d11/d14/d24/d2a/de3/fe0 0 2024-03-23T08:23:26.660 INFO:tasks.workunit.client.1.smithi193.stdout:0/654: chown db/c30 16977 1 2024-03-23T08:23:26.662 INFO:tasks.workunit.client.1.smithi193.stdout:3/588: rename d1/de/d13/c66 to d1/de/d13/d21/d25/d34/da8/cc9 0 2024-03-23T08:23:26.666 INFO:tasks.workunit.client.1.smithi193.stdout:9/715: rename d2/d7/d31 to d2/d7/db/d1b/d43/d71/db4/dec 0 2024-03-23T08:23:26.668 INFO:tasks.workunit.client.1.smithi193.stdout:3/589: mkdir d1/de/d13/d21/d50/d6f/dca 0 2024-03-23T08:23:26.668 INFO:tasks.workunit.client.1.smithi193.stdout:0/655: link db/c30 db/d11/d14/d24/d47/d59/ce6 0 2024-03-23T08:23:26.670 INFO:tasks.workunit.client.1.smithi193.stdout:9/716: creat d2/d7/db/d1b/d5c/fed x:0 0 0 2024-03-23T08:23:26.671 INFO:tasks.workunit.client.1.smithi193.stdout:3/590: read d1/de/d13/d21/d50/d6f/d6e/f72 [56951,120000] 0 2024-03-23T08:23:26.672 INFO:tasks.workunit.client.1.smithi193.stdout:7/617: sync 2024-03-23T08:23:26.672 INFO:tasks.workunit.client.1.smithi193.stdout:8/644: sync 2024-03-23T08:23:26.672 INFO:tasks.workunit.client.1.smithi193.stdout:1/661: truncate da/dd/df/d2e/f48 517605 0 2024-03-23T08:23:26.674 INFO:tasks.workunit.client.1.smithi193.stdout:7/618: truncate d6/d9/d1f/f94 901246 0 2024-03-23T08:23:26.676 INFO:tasks.workunit.client.1.smithi193.stdout:9/717: rename d2/d7/db/d17/d4b/d8b/cdd to d2/d7/db/d17/d4b/d59/d97/d9d/cee 0 2024-03-23T08:23:26.676 INFO:tasks.workunit.client.1.smithi193.stdout:8/645: mknod df/d10/d83/cdb 0 2024-03-23T08:23:26.677 INFO:tasks.workunit.client.1.smithi193.stdout:1/662: mkdir da/dd/df/d2e/dd5 0 2024-03-23T08:23:26.682 INFO:tasks.workunit.client.1.smithi193.stdout:3/591: dwrite d1/de/d13/d21/f23 [0,4194304] 0 2024-03-23T08:23:26.683 INFO:tasks.workunit.client.1.smithi193.stdout:0/656: dwrite db/d11/d14/d51/f56 [0,4194304] 0 2024-03-23T08:23:26.685 INFO:tasks.workunit.client.1.smithi193.stdout:7/619: getdents d6/d9/d1f/d4d/d7c/d92 0 2024-03-23T08:23:26.687 INFO:tasks.workunit.client.1.smithi193.stdout:3/592: chown d1/de/d13/d21/d25/l42 2860897 1 2024-03-23T08:23:26.687 INFO:tasks.workunit.client.1.smithi193.stdout:1/663: fdatasync da/dd/df/d2e/f48 0 2024-03-23T08:23:26.687 INFO:tasks.workunit.client.1.smithi193.stdout:7/620: chown d6/d9/d1f/d5f 13349228 1 2024-03-23T08:23:26.689 INFO:tasks.workunit.client.1.smithi193.stdout:3/593: stat d1/de/d13/d21/f23 0 2024-03-23T08:23:26.690 INFO:tasks.workunit.client.1.smithi193.stdout:9/718: write d2/d7/db/d1b/d43/d71/db4/dec/f40 [1586384,5424] 0 2024-03-23T08:23:26.691 INFO:tasks.workunit.client.1.smithi193.stdout:0/657: fdatasync db/d11/d14/d24/d2a/d4b/fbd 0 2024-03-23T08:23:26.694 INFO:tasks.workunit.client.1.smithi193.stdout:3/594: chown d1/de/d13/d21/d25/d2b/d3c/d40/ca5 4085173 1 2024-03-23T08:23:26.696 INFO:tasks.workunit.client.1.smithi193.stdout:9/719: read d2/d7/db/d1b/d43/d71/db4/dec/d3c/f6e [3394267,110485] 0 2024-03-23T08:23:26.700 INFO:tasks.workunit.client.1.smithi193.stdout:1/664: dwrite f8 [0,4194304] 0 2024-03-23T08:23:26.703 INFO:tasks.workunit.client.1.smithi193.stdout:7/621: write d6/d9/fa5 [904027,31817] 0 2024-03-23T08:23:26.707 INFO:tasks.workunit.client.1.smithi193.stdout:8/646: dread df/f56 [0,4194304] 0 2024-03-23T08:23:26.712 INFO:tasks.workunit.client.1.smithi193.stdout:8/647: symlink df/dcc/ldc 0 2024-03-23T08:23:26.714 INFO:tasks.workunit.client.1.smithi193.stdout:7/622: rename d6/d9/d1f/d23/d33/d38/dc0/lcf to d6/d9/d1f/d23/ld7 0 2024-03-23T08:23:26.714 INFO:tasks.workunit.client.1.smithi193.stdout:8/648: fsync df/dc6/fad 0 2024-03-23T08:23:26.716 INFO:tasks.workunit.client.1.smithi193.stdout:3/595: dwrite d1/de/d78/fa9 [0,4194304] 0 2024-03-23T08:23:26.717 INFO:tasks.workunit.client.1.smithi193.stdout:1/665: dwrite da/d13/d3f/d5a/f8c [0,4194304] 0 2024-03-23T08:23:26.718 INFO:tasks.workunit.client.1.smithi193.stdout:8/649: creat df/d63/d84/fdd x:0 0 0 2024-03-23T08:23:26.718 INFO:tasks.workunit.client.1.smithi193.stdout:7/623: dread - d6/d9/d4e/fae zero size 2024-03-23T08:23:26.721 INFO:tasks.workunit.client.1.smithi193.stdout:1/666: symlink da/dd/df/d7c/ld6 0 2024-03-23T08:23:26.724 INFO:tasks.workunit.client.1.smithi193.stdout:3/596: symlink d1/de/d13/d21/d59/dc6/lcb 0 2024-03-23T08:23:26.724 INFO:tasks.workunit.client.1.smithi193.stdout:0/658: dread db/d11/d14/d21/f27 [0,4194304] 0 2024-03-23T08:23:26.724 INFO:tasks.workunit.client.1.smithi193.stdout:8/650: truncate df/d4b/d4d/f22 35757 0 2024-03-23T08:23:26.728 INFO:tasks.workunit.client.1.smithi193.stdout:9/720: dread d2/d7/f37 [0,4194304] 0 2024-03-23T08:23:26.731 INFO:tasks.workunit.client.1.smithi193.stdout:8/651: rmdir df/dc6/d8b 39 2024-03-23T08:23:26.732 INFO:tasks.workunit.client.1.smithi193.stdout:9/721: creat d2/d7/db/d1b/fef x:0 0 0 2024-03-23T08:23:26.733 INFO:tasks.workunit.client.1.smithi193.stdout:3/597: read d1/de/f10 [3328098,29923] 0 2024-03-23T08:23:26.734 INFO:tasks.workunit.client.1.smithi193.stdout:8/652: write f3 [5964897,122815] 0 2024-03-23T08:23:26.737 INFO:tasks.workunit.client.1.smithi193.stdout:9/722: symlink d2/d7/db/d32/lf0 0 2024-03-23T08:23:26.738 INFO:tasks.workunit.client.1.smithi193.stdout:0/659: dwrite db/d11/d14/d24/f8d [0,4194304] 0 2024-03-23T08:23:26.738 INFO:tasks.workunit.client.1.smithi193.stdout:3/598: symlink d1/de/d13/d21/d50/d6f/lcc 0 2024-03-23T08:23:26.741 INFO:tasks.workunit.client.1.smithi193.stdout:0/660: write db/d11/fa6 [22532,69691] 0 2024-03-23T08:23:26.744 INFO:tasks.workunit.client.1.smithi193.stdout:8/653: rename df/dc6/d8b/dae/f47 to df/d63/d84/fde 0 2024-03-23T08:23:26.744 INFO:tasks.workunit.client.1.smithi193.stdout:7/624: dread d6/d9/d1f/d3e/d65/f75 [0,4194304] 0 2024-03-23T08:23:26.748 INFO:tasks.workunit.client.1.smithi193.stdout:8/654: creat df/dcc/fdf x:0 0 0 2024-03-23T08:23:26.751 INFO:tasks.workunit.client.1.smithi193.stdout:9/723: dwrite d2/d7/db/d1b/f4c [0,4194304] 0 2024-03-23T08:23:26.751 INFO:tasks.workunit.client.1.smithi193.stdout:1/667: truncate da/d13/d24/f79 1829301 0 2024-03-23T08:23:26.751 INFO:tasks.workunit.client.1.smithi193.stdout:0/661: truncate db/d11/d14/d24/d2a/d4d/d93/fcc 478999 0 2024-03-23T08:23:26.752 INFO:tasks.workunit.client.1.smithi193.stdout:3/599: creat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fcd x:0 0 0 2024-03-23T08:23:26.758 INFO:tasks.workunit.client.1.smithi193.stdout:0/662: dread db/d11/d14/d21/f27 [4194304,4194304] 0 2024-03-23T08:23:26.761 INFO:tasks.workunit.client.1.smithi193.stdout:0/663: mknod db/d11/d14/d51/ce7 0 2024-03-23T08:23:26.761 INFO:tasks.workunit.client.1.smithi193.stdout:7/625: link d6/d9/d1f/d23/d33/c9b d6/d9/d1f/d23/d33/d38/d46/cd8 0 2024-03-23T08:23:26.770 INFO:tasks.workunit.client.1.smithi193.stdout:4/724: sync 2024-03-23T08:23:26.772 INFO:tasks.workunit.client.1.smithi193.stdout:0/664: dwrite db/d11/d14/d24/d2a/fa9 [0,4194304] 0 2024-03-23T08:23:26.774 INFO:tasks.workunit.client.1.smithi193.stdout:4/725: mkdir d11/d2d/d58/d80/d9d/df5 0 2024-03-23T08:23:26.775 INFO:tasks.workunit.client.1.smithi193.stdout:7/626: truncate d6/d9/d1f/d23/d33/d38/d46/d82/fd1 3021708 0 2024-03-23T08:23:26.778 INFO:tasks.workunit.client.1.smithi193.stdout:7/627: write d6/d9/d1f/d9d/da1/fcd [1040755,49308] 0 2024-03-23T08:23:26.779 INFO:tasks.workunit.client.1.smithi193.stdout:8/655: dread df/d4b/d4d/f5c [0,4194304] 0 2024-03-23T08:23:26.782 INFO:tasks.workunit.client.1.smithi193.stdout:1/668: dread da/f4e [0,4194304] 0 2024-03-23T08:23:26.782 INFO:tasks.workunit.client.1.smithi193.stdout:4/726: creat d11/d2d/d58/d80/df3/ff6 x:0 0 0 2024-03-23T08:23:26.785 INFO:tasks.workunit.client.1.smithi193.stdout:0/665: dread db/d11/d14/d24/d2a/fa9 [0,4194304] 0 2024-03-23T08:23:26.785 INFO:tasks.workunit.client.1.smithi193.stdout:8/656: unlink df/d10/d3e/f6b 0 2024-03-23T08:23:26.789 INFO:tasks.workunit.client.1.smithi193.stdout:1/669: getdents da/dd/df/d7e/d7f/db2 0 2024-03-23T08:23:26.789 INFO:tasks.workunit.client.1.smithi193.stdout:8/657: fsync df/d4b/f71 0 2024-03-23T08:23:26.791 INFO:tasks.workunit.client.1.smithi193.stdout:0/666: rename db/l20 to db/d11/d14/d24/d47/le8 0 2024-03-23T08:23:26.792 INFO:tasks.workunit.client.1.smithi193.stdout:1/670: rmdir da/dd/df 39 2024-03-23T08:23:26.794 INFO:tasks.workunit.client.1.smithi193.stdout:7/628: dwrite d6/d9/d1f/d23/d33/d3c/d41/f51 [4194304,4194304] 0 2024-03-23T08:23:26.796 INFO:tasks.workunit.client.1.smithi193.stdout:4/727: dwrite d11/d2d/fde [0,4194304] 0 2024-03-23T08:23:26.797 INFO:tasks.workunit.client.1.smithi193.stdout:7/629: dread - d6/d9/d1f/d3e/d65/d7d/fb4 zero size 2024-03-23T08:23:26.799 INFO:tasks.workunit.client.1.smithi193.stdout:7/630: stat d6/d9/f6f 0 2024-03-23T08:23:26.800 INFO:tasks.workunit.client.1.smithi193.stdout:4/728: stat d11/d1a/d33/f39 0 2024-03-23T08:23:26.804 INFO:tasks.workunit.client.1.smithi193.stdout:4/729: write d11/d1a/d33/d8f/db8/f31 [7493263,114744] 0 2024-03-23T08:23:26.804 INFO:tasks.workunit.client.1.smithi193.stdout:7/631: creat d6/d9/d1f/d23/d33/d38/dc0/fd9 x:0 0 0 2024-03-23T08:23:26.805 INFO:tasks.workunit.client.1.smithi193.stdout:4/730: chown d11/d2d/d58/d80/d9f/lc5 15434 1 2024-03-23T08:23:26.806 INFO:tasks.workunit.client.1.smithi193.stdout:4/731: chown d11/d2d/d58/d80/db9/ce3 31316196 1 2024-03-23T08:23:26.808 INFO:tasks.workunit.client.1.smithi193.stdout:4/732: write d11/d2d/d58/d80/d9f/fa2 [4797431,107433] 0 2024-03-23T08:23:26.812 INFO:tasks.workunit.client.1.smithi193.stdout:4/733: rmdir d11/d1a/d33/d8f/db8 39 2024-03-23T08:23:26.814 INFO:tasks.workunit.client.1.smithi193.stdout:7/632: dwrite d6/d9/f78 [0,4194304] 0 2024-03-23T08:23:26.816 INFO:tasks.workunit.client.1.smithi193.stdout:4/734: creat d11/d2d/d58/d8b/ff7 x:0 0 0 2024-03-23T08:23:26.818 INFO:tasks.workunit.client.1.smithi193.stdout:4/735: symlink d11/d8d/lf8 0 2024-03-23T08:23:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:7/633: rename d6/d9/cbc to d6/d9/d1f/d23/d33/d38/cda 0 2024-03-23T08:23:26.822 INFO:tasks.workunit.client.1.smithi193.stdout:4/736: write d11/d1a/d33/d44/f54 [1798205,77859] 0 2024-03-23T08:23:26.823 INFO:tasks.workunit.client.1.smithi193.stdout:0/667: dread db/d11/d14/d24/f4a [0,4194304] 0 2024-03-23T08:23:26.823 INFO:tasks.workunit.client.1.smithi193.stdout:7/634: creat d6/d9/d1f/d3e/d42/fdb x:0 0 0 2024-03-23T08:23:26.824 INFO:tasks.workunit.client.1.smithi193.stdout:8/658: dread df/d63/f81 [0,4194304] 0 2024-03-23T08:23:26.827 INFO:tasks.workunit.client.1.smithi193.stdout:0/668: chown db/d11/d14/d24/d47/d59/ce6 10323461 1 2024-03-23T08:23:26.828 INFO:tasks.workunit.client.1.smithi193.stdout:8/659: mknod df/dc6/ce0 0 2024-03-23T08:23:26.829 INFO:tasks.workunit.client.1.smithi193.stdout:1/671: truncate da/dd/df/d7e/d7f/db2/fc1 1327717 0 2024-03-23T08:23:26.830 INFO:tasks.workunit.client.1.smithi193.stdout:8/660: truncate df/d63/f81 4612293 0 2024-03-23T08:23:26.831 INFO:tasks.workunit.client.1.smithi193.stdout:7/635: dwrite d6/d9/f6f [0,4194304] 0 2024-03-23T08:23:26.834 INFO:tasks.workunit.client.1.smithi193.stdout:0/669: dread db/f1e [0,4194304] 0 2024-03-23T08:23:26.834 INFO:tasks.workunit.client.1.smithi193.stdout:1/672: unlink da/dd/df/d52/f53 0 2024-03-23T08:23:26.834 INFO:tasks.workunit.client.1.smithi193.stdout:7/636: truncate d6/d9/d1f/d23/d33/d38/d66/fc1 949351 0 2024-03-23T08:23:26.839 INFO:tasks.workunit.client.1.smithi193.stdout:7/637: symlink d6/d9/d1f/d23/d33/d38/d3b/ldc 0 2024-03-23T08:23:26.841 INFO:tasks.workunit.client.1.smithi193.stdout:1/673: mknod da/dd/df/d7c/dd2/cd7 0 2024-03-23T08:23:26.844 INFO:tasks.workunit.client.1.smithi193.stdout:1/674: unlink da/d13/c74 0 2024-03-23T08:23:26.844 INFO:tasks.workunit.client.1.smithi193.stdout:7/638: dread d6/d9/d1f/f50 [0,4194304] 0 2024-03-23T08:23:26.845 INFO:tasks.workunit.client.1.smithi193.stdout:1/675: dread - da/d13/d3f/d5a/fab zero size 2024-03-23T08:23:26.847 INFO:tasks.workunit.client.1.smithi193.stdout:7/639: mknod d6/d9/d1f/d23/cdd 0 2024-03-23T08:23:26.848 INFO:tasks.workunit.client.1.smithi193.stdout:0/670: write db/d11/d14/d24/d47/d59/f7c [858559,77631] 0 2024-03-23T08:23:26.849 INFO:tasks.workunit.client.1.smithi193.stdout:7/640: write d6/d9/d1f/d4d/fc6 [462945,70316] 0 2024-03-23T08:23:26.851 INFO:tasks.workunit.client.1.smithi193.stdout:8/661: dwrite df/d4b/d4d/f5c [0,4194304] 0 2024-03-23T08:23:26.853 INFO:tasks.workunit.client.1.smithi193.stdout:8/662: truncate df/dcc/fdf 254653 0 2024-03-23T08:23:26.854 INFO:tasks.workunit.client.1.smithi193.stdout:8/663: write df/d63/fbc [828270,81429] 0 2024-03-23T08:23:26.855 INFO:tasks.workunit.client.1.smithi193.stdout:0/671: symlink db/d11/d14/d24/d2a/d4d/le9 0 2024-03-23T08:23:26.856 INFO:tasks.workunit.client.1.smithi193.stdout:8/664: readlink df/d63/d76/laf 0 2024-03-23T08:23:26.857 INFO:tasks.workunit.client.1.smithi193.stdout:7/641: creat d6/d9/d1f/d4d/d7c/d92/fde x:0 0 0 2024-03-23T08:23:26.858 INFO:tasks.workunit.client.1.smithi193.stdout:7/642: stat d6/d9/d1f/d3e/d65/d7d 0 2024-03-23T08:23:26.858 INFO:tasks.workunit.client.1.smithi193.stdout:7/643: fdatasync d6/f39 0 2024-03-23T08:23:26.859 INFO:tasks.workunit.client.1.smithi193.stdout:0/672: chown db/d11/d14/d21/c9e 206804 1 2024-03-23T08:23:26.860 INFO:tasks.workunit.client.1.smithi193.stdout:0/673: stat db/d11/fd8 0 2024-03-23T08:23:26.861 INFO:tasks.workunit.client.1.smithi193.stdout:8/665: symlink df/d4b/d4d/d9c/le1 0 2024-03-23T08:23:26.861 INFO:tasks.workunit.client.1.smithi193.stdout:1/676: write da/d13/d3f/d5a/f9d [235445,100897] 0 2024-03-23T08:23:26.870 INFO:tasks.workunit.client.1.smithi193.stdout:1/677: write da/dd/df/d7e/d86/d95/fb1 [3950315,112175] 0 2024-03-23T08:23:26.872 INFO:tasks.workunit.client.1.smithi193.stdout:7/644: dwrite d6/d9/d1f/d4d/fc6 [0,4194304] 0 2024-03-23T08:23:26.876 INFO:tasks.workunit.client.1.smithi193.stdout:0/674: symlink db/d11/d14/d21/lea 0 2024-03-23T08:23:26.877 INFO:tasks.workunit.client.1.smithi193.stdout:8/666: creat df/d4b/d4d/d9c/fe2 x:0 0 0 2024-03-23T08:23:26.878 INFO:tasks.workunit.client.1.smithi193.stdout:8/667: stat df/f1f 0 2024-03-23T08:23:26.879 INFO:tasks.workunit.client.1.smithi193.stdout:7/645: dwrite d6/d9/d1f/f94 [0,4194304] 0 2024-03-23T08:23:26.879 INFO:tasks.workunit.client.1.smithi193.stdout:1/678: creat da/dd/df/d7c/fd8 x:0 0 0 2024-03-23T08:23:26.884 INFO:tasks.workunit.client.1.smithi193.stdout:7/646: mknod d6/d9/d4e/d81/cdf 0 2024-03-23T08:23:26.886 INFO:tasks.workunit.client.1.smithi193.stdout:0/675: rename db/d11/d14/l18 to db/d11/d14/d21/dd2/leb 0 2024-03-23T08:23:26.887 INFO:tasks.workunit.client.1.smithi193.stdout:8/668: creat df/fe3 x:0 0 0 2024-03-23T08:23:26.889 INFO:tasks.workunit.client.1.smithi193.stdout:7/647: mkdir d6/d9/d1f/d9d/da1/de0 0 2024-03-23T08:23:26.890 INFO:tasks.workunit.client.1.smithi193.stdout:0/676: creat db/d11/d14/d24/d47/fec x:0 0 0 2024-03-23T08:23:26.891 INFO:tasks.workunit.client.1.smithi193.stdout:0/677: dread - db/f8f zero size 2024-03-23T08:23:26.891 INFO:tasks.workunit.client.1.smithi193.stdout:8/669: creat df/d4b/fe4 x:0 0 0 2024-03-23T08:23:26.894 INFO:tasks.workunit.client.1.smithi193.stdout:8/670: creat df/d63/d76/fe5 x:0 0 0 2024-03-23T08:23:26.897 INFO:tasks.workunit.client.1.smithi193.stdout:8/671: creat df/d10/d83/fe6 x:0 0 0 2024-03-23T08:23:26.898 INFO:tasks.workunit.client.1.smithi193.stdout:8/672: readlink df/d10/lc7 0 2024-03-23T08:23:26.899 INFO:tasks.workunit.client.1.smithi193.stdout:0/678: dwrite db/d11/d14/d24/d2a/d4d/d93/fc5 [0,4194304] 0 2024-03-23T08:23:26.899 INFO:tasks.workunit.client.1.smithi193.stdout:8/673: chown df/d10/d67/f6c 51587310 1 2024-03-23T08:23:26.900 INFO:tasks.workunit.client.1.smithi193.stdout:8/674: stat df/d4b/d4d/d2a/c6f 0 2024-03-23T08:23:26.902 INFO:tasks.workunit.client.1.smithi193.stdout:8/675: read df/d10/d3e/d4e/f88 [1852495,54818] 0 2024-03-23T08:23:26.903 INFO:tasks.workunit.client.1.smithi193.stdout:8/676: truncate df/d4b/d4d/d9c/fe2 407959 0 2024-03-23T08:23:26.904 INFO:tasks.workunit.client.1.smithi193.stdout:8/677: readlink df/d4b/d4d/l51 0 2024-03-23T08:23:26.908 INFO:tasks.workunit.client.1.smithi193.stdout:8/678: fsync df/d63/f68 0 2024-03-23T08:23:26.910 INFO:tasks.workunit.client.1.smithi193.stdout:1/679: dwrite da/d13/d24/f79 [0,4194304] 0 2024-03-23T08:23:26.911 INFO:tasks.workunit.client.1.smithi193.stdout:7/648: dwrite d6/d9/d1f/d3e/d65/d70/f83 [0,4194304] 0 2024-03-23T08:23:26.913 INFO:tasks.workunit.client.1.smithi193.stdout:8/679: fdatasync df/f21 0 2024-03-23T08:23:26.915 INFO:tasks.workunit.client.1.smithi193.stdout:7/649: creat d6/d9/d1f/daf/fe1 x:0 0 0 2024-03-23T08:23:26.916 INFO:tasks.workunit.client.1.smithi193.stdout:8/680: write df/d63/d84/f8a [221606,109399] 0 2024-03-23T08:23:26.916 INFO:tasks.workunit.client.1.smithi193.stdout:1/680: truncate da/d13/fc3 750543 0 2024-03-23T08:23:26.920 INFO:tasks.workunit.client.1.smithi193.stdout:7/650: getdents d6/d9/d1f/daf/dd2 0 2024-03-23T08:23:26.926 INFO:tasks.workunit.client.1.smithi193.stdout:8/681: dread df/d10/f17 [0,4194304] 0 2024-03-23T08:23:26.926 INFO:tasks.workunit.client.1.smithi193.stdout:1/681: dwrite da/dd/f64 [0,4194304] 0 2024-03-23T08:23:26.932 INFO:tasks.workunit.client.1.smithi193.stdout:1/682: dread da/d13/d3f/fa7 [0,4194304] 0 2024-03-23T08:23:26.933 INFO:tasks.workunit.client.1.smithi193.stdout:8/682: link f0 df/d63/d84/dc5/fe7 0 2024-03-23T08:23:26.936 INFO:tasks.workunit.client.1.smithi193.stdout:8/683: symlink df/dc6/le8 0 2024-03-23T08:23:26.937 INFO:tasks.workunit.client.1.smithi193.stdout:8/684: fdatasync df/d10/d83/fc1 0 2024-03-23T08:23:26.937 INFO:tasks.workunit.client.1.smithi193.stdout:7/651: truncate d6/d9/d1f/d23/d33/d3c/f8b 101431 0 2024-03-23T08:23:26.939 INFO:tasks.workunit.client.1.smithi193.stdout:8/685: symlink df/d10/d3e/d54/le9 0 2024-03-23T08:23:26.940 INFO:tasks.workunit.client.1.smithi193.stdout:1/683: dread da/dd/df/d7e/d86/f89 [0,4194304] 0 2024-03-23T08:23:26.941 INFO:tasks.workunit.client.1.smithi193.stdout:8/686: mknod df/d63/d76/cea 0 2024-03-23T08:23:26.942 INFO:tasks.workunit.client.1.smithi193.stdout:8/687: dread - df/d63/d76/fe5 zero size 2024-03-23T08:23:26.942 INFO:tasks.workunit.client.1.smithi193.stdout:1/684: readlink da/dd/df/d7e/d86/d95/dc4/lcd 0 2024-03-23T08:23:26.942 INFO:tasks.workunit.client.1.smithi193.stdout:1/685: dread - da/dd/faf zero size 2024-03-23T08:23:26.943 INFO:tasks.workunit.client.1.smithi193.stdout:1/686: chown da/dd/df/d75/cb9 191322400 1 2024-03-23T08:23:26.945 INFO:tasks.workunit.client.1.smithi193.stdout:7/652: write d6/d9/d1f/d3e/d65/d70/f7f [420663,7274] 0 2024-03-23T08:23:26.947 INFO:tasks.workunit.client.1.smithi193.stdout:8/688: rename df/d63/d84/dc5/fe7 to df/feb 0 2024-03-23T08:23:26.951 INFO:tasks.workunit.client.1.smithi193.stdout:7/653: dread d6/f2d [0,4194304] 0 2024-03-23T08:23:26.952 INFO:tasks.workunit.client.1.smithi193.stdout:8/689: link df/f34 df/d10/d67/d7f/fec 0 2024-03-23T08:23:26.953 INFO:tasks.workunit.client.1.smithi193.stdout:7/654: creat d6/d9/d1f/d4d/d6b/fe2 x:0 0 0 2024-03-23T08:23:26.953 INFO:tasks.workunit.client.1.smithi193.stdout:6/570: sync 2024-03-23T08:23:26.953 INFO:tasks.workunit.client.1.smithi193.stdout:5/714: sync 2024-03-23T08:23:26.954 INFO:tasks.workunit.client.1.smithi193.stdout:2/733: sync 2024-03-23T08:23:26.954 INFO:tasks.workunit.client.1.smithi193.stdout:8/690: creat df/d63/d84/da8/fed x:0 0 0 2024-03-23T08:23:26.956 INFO:tasks.workunit.client.1.smithi193.stdout:6/571: write d0/d27/d81/d87/fa8 [467974,80251] 0 2024-03-23T08:23:26.957 INFO:tasks.workunit.client.1.smithi193.stdout:7/655: creat d6/d9/d1f/d3e/d65/d70/fe3 x:0 0 0 2024-03-23T08:23:26.960 INFO:tasks.workunit.client.1.smithi193.stdout:5/715: dread d0/d1c/d3b/f98 [0,4194304] 0 2024-03-23T08:23:26.960 INFO:tasks.workunit.client.1.smithi193.stdout:1/687: dread da/dd/df/f29 [0,4194304] 0 2024-03-23T08:23:26.963 INFO:tasks.workunit.client.1.smithi193.stdout:7/656: symlink d6/d9/d1f/d23/d33/d38/dc0/le4 0 2024-03-23T08:23:26.964 INFO:tasks.workunit.client.1.smithi193.stdout:8/691: write df/d10/d3e/d54/dd2/f1b [442737,96513] 0 2024-03-23T08:23:26.965 INFO:tasks.workunit.client.1.smithi193.stdout:2/734: dwrite d0/d1b/da7/fb3 [0,4194304] 0 2024-03-23T08:23:26.965 INFO:tasks.workunit.client.1.smithi193.stdout:1/688: creat da/dd/df/d7e/fd9 x:0 0 0 2024-03-23T08:23:26.966 INFO:tasks.workunit.client.1.smithi193.stdout:5/716: fsync d0/d1f/f22 0 2024-03-23T08:23:26.968 INFO:tasks.workunit.client.1.smithi193.stdout:2/735: dread - d0/d1b/d3b/f79 zero size 2024-03-23T08:23:26.968 INFO:tasks.workunit.client.1.smithi193.stdout:1/689: chown da/dd/df/d7e/fcb 964 1 2024-03-23T08:23:26.968 INFO:tasks.workunit.client.1.smithi193.stdout:5/717: stat d0/d1f/fbb 0 2024-03-23T08:23:26.968 INFO:tasks.workunit.client.1.smithi193.stdout:8/692: mkdir df/d4b/d4d/d9c/dee 0 2024-03-23T08:23:26.970 INFO:tasks.workunit.client.1.smithi193.stdout:2/736: chown d0/d28/f8c 66 1 2024-03-23T08:23:26.970 INFO:tasks.workunit.client.1.smithi193.stdout:8/693: chown df/d10/f29 6716832 1 2024-03-23T08:23:26.971 INFO:tasks.workunit.client.1.smithi193.stdout:6/572: link d0/d27/d81/df/d11/f30 d0/d27/d81/df/d11/d36/d61/d3a/fb4 0 2024-03-23T08:23:26.971 INFO:tasks.workunit.client.1.smithi193.stdout:7/657: mknod d6/d9/d1f/d23/d33/d3c/d41/ce5 0 2024-03-23T08:23:26.972 INFO:tasks.workunit.client.1.smithi193.stdout:5/718: creat d0/d1c/d7a/d86/f103 x:0 0 0 2024-03-23T08:23:26.973 INFO:tasks.workunit.client.1.smithi193.stdout:1/690: symlink da/dd/df/d7e/d7f/lda 0 2024-03-23T08:23:26.975 INFO:tasks.workunit.client.1.smithi193.stdout:8/694: symlink df/dc6/lef 0 2024-03-23T08:23:26.976 INFO:tasks.workunit.client.1.smithi193.stdout:1/691: write da/d13/d3f/d5a/f8c [4113801,58863] 0 2024-03-23T08:23:26.977 INFO:tasks.workunit.client.1.smithi193.stdout:5/719: rmdir d0/d1c/d7a/db1/d72/d73/d45/daa 39 2024-03-23T08:23:26.980 INFO:tasks.workunit.client.1.smithi193.stdout:8/695: unlink df/d4b/d4d/l52 0 2024-03-23T08:23:26.982 INFO:tasks.workunit.client.1.smithi193.stdout:1/692: mkdir da/dd/df/d7e/d7f/db2/ddb 0 2024-03-23T08:23:26.982 INFO:tasks.workunit.client.1.smithi193.stdout:7/658: dwrite d6/d9/d1f/d3e/d42/f60 [4194304,4194304] 0 2024-03-23T08:23:26.982 INFO:tasks.workunit.client.1.smithi193.stdout:8/696: stat df/dc6/d8b 0 2024-03-23T08:23:26.983 INFO:tasks.workunit.client.1.smithi193.stdout:8/697: read - df/d4b/d4d/f99 zero size 2024-03-23T08:23:26.988 INFO:tasks.workunit.client.1.smithi193.stdout:5/720: link d0/d1f/fc4 d0/d1c/d3b/f104 0 2024-03-23T08:23:26.988 INFO:tasks.workunit.client.1.smithi193.stdout:7/659: readlink d6/d9/d1f/d3e/d42/l43 0 2024-03-23T08:23:26.990 INFO:tasks.workunit.client.1.smithi193.stdout:8/698: truncate df/d10/d83/fc1 765181 0 2024-03-23T08:23:26.992 INFO:tasks.workunit.client.1.smithi193.stdout:2/737: dread d0/d28/d33/d72/fa5 [0,4194304] 0 2024-03-23T08:23:26.993 INFO:tasks.workunit.client.1.smithi193.stdout:8/699: dread - df/d63/d76/fe5 zero size 2024-03-23T08:23:26.993 INFO:tasks.workunit.client.1.smithi193.stdout:6/573: dwrite d0/d27/d81/f4e [0,4194304] 0 2024-03-23T08:23:26.994 INFO:tasks.workunit.client.1.smithi193.stdout:7/660: symlink d6/d9/d1f/d23/d33/d3c/le6 0 2024-03-23T08:23:26.995 INFO:tasks.workunit.client.1.smithi193.stdout:5/721: rename d0/d1c/d3b/f94 to d0/d1f/d93/de1/f105 0 2024-03-23T08:23:26.998 INFO:tasks.workunit.client.1.smithi193.stdout:2/738: write d0/d1b/d14/d66/d4e/fc4 [5092984,9822] 0 2024-03-23T08:23:26.999 INFO:tasks.workunit.client.1.smithi193.stdout:1/693: dwrite da/d13/d3f/f5e [0,4194304] 0 2024-03-23T08:23:27.002 INFO:tasks.workunit.client.1.smithi193.stdout:5/722: read - d0/d1c/d7a/db1/d72/fef zero size 2024-03-23T08:23:27.002 INFO:tasks.workunit.client.1.smithi193.stdout:6/574: write d0/d27/d81/df/f9b [1270091,50525] 0 2024-03-23T08:23:27.004 INFO:tasks.workunit.client.1.smithi193.stdout:5/723: write d0/d1c/d7a/db1/d72/d99/dd8/fd1 [135551,119324] 0 2024-03-23T08:23:27.005 INFO:tasks.workunit.client.1.smithi193.stdout:7/661: mkdir d6/d9/d1f/d23/d33/d38/d3b/d6c/de7 0 2024-03-23T08:23:27.008 INFO:tasks.workunit.client.1.smithi193.stdout:8/700: rmdir df/d63/d84/da8/dd4 0 2024-03-23T08:23:27.008 INFO:tasks.workunit.client.1.smithi193.stdout:4/737: sync 2024-03-23T08:23:27.008 INFO:tasks.workunit.client.1.smithi193.stdout:9/724: sync 2024-03-23T08:23:27.008 INFO:tasks.workunit.client.1.smithi193.stdout:3/600: sync 2024-03-23T08:23:27.008 INFO:tasks.workunit.client.1.smithi193.stdout:1/694: mkdir da/dd/df/ddc 0 2024-03-23T08:23:27.008 INFO:tasks.workunit.client.1.smithi193.stdout:6/575: creat d0/d27/d81/da1/fb5 x:0 0 0 2024-03-23T08:23:27.010 INFO:tasks.workunit.client.1.smithi193.stdout:3/601: readlink d1/de/d28/l8a 0 2024-03-23T08:23:27.012 INFO:tasks.workunit.client.1.smithi193.stdout:5/724: fdatasync d0/f43 0 2024-03-23T08:23:27.012 INFO:tasks.workunit.client.1.smithi193.stdout:7/662: mkdir d6/d9/d1f/de8 0 2024-03-23T08:23:27.012 INFO:tasks.workunit.client.1.smithi193.stdout:8/701: write df/d10/d3e/d54/dd2/f72 [1903802,96676] 0 2024-03-23T08:23:27.014 INFO:tasks.workunit.client.1.smithi193.stdout:4/738: symlink d11/d2d/d58/d80/d9d/lf9 0 2024-03-23T08:23:27.014 INFO:tasks.workunit.client.1.smithi193.stdout:9/725: fdatasync d2/d7/db/d1b/d43/d71/db4/dec/d3c/fbe 0 2024-03-23T08:23:27.014 INFO:tasks.workunit.client.1.smithi193.stdout:2/739: link d0/c78 d0/d1b/d14/d56/d7c/cd4 0 2024-03-23T08:23:27.015 INFO:tasks.workunit.client.1.smithi193.stdout:1/695: write da/dd/df/f2d [1357271,70474] 0 2024-03-23T08:23:27.017 INFO:tasks.workunit.client.1.smithi193.stdout:3/602: chown d1/de/d13/d21/d25/d2b/f35 989358427 1 2024-03-23T08:23:27.017 INFO:tasks.workunit.client.1.smithi193.stdout:8/702: write df/d10/d3e/f5f [970119,45489] 0 2024-03-23T08:23:27.017 INFO:tasks.workunit.client.1.smithi193.stdout:7/663: write d6/d9/d1f/d5f/f8a [862086,1905] 0 2024-03-23T08:23:27.018 INFO:tasks.workunit.client.1.smithi193.stdout:5/725: rmdir d0/d1c 39 2024-03-23T08:23:27.020 INFO:tasks.workunit.client.1.smithi193.stdout:6/576: creat d0/d27/d81/df/d11/d3b/d51/daa/fb6 x:0 0 0 2024-03-23T08:23:27.020 INFO:tasks.workunit.client.1.smithi193.stdout:9/726: chown d2/d7/db/d1b/d43/d71/db4/dec/d3c/d79 1046 1 2024-03-23T08:23:27.021 INFO:tasks.workunit.client.1.smithi193.stdout:1/696: dread - da/dd/df/d7c/fca zero size 2024-03-23T08:23:27.025 INFO:tasks.workunit.client.1.smithi193.stdout:7/664: creat d6/d9/d1f/d3e/d65/fe9 x:0 0 0 2024-03-23T08:23:27.025 INFO:tasks.workunit.client.1.smithi193.stdout:8/703: chown df/d10/cc4 1 1 2024-03-23T08:23:27.026 INFO:tasks.workunit.client.1.smithi193.stdout:4/739: dread d11/d1a/d33/d44/f6e [0,4194304] 0 2024-03-23T08:23:27.026 INFO:tasks.workunit.client.1.smithi193.stdout:3/603: creat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fce x:0 0 0 2024-03-23T08:23:27.029 INFO:tasks.workunit.client.1.smithi193.stdout:9/727: chown d2/d7/db/d1b/d43/d71/db4/dec/d3c/l47 11786715 1 2024-03-23T08:23:27.030 INFO:tasks.workunit.client.1.smithi193.stdout:5/726: truncate d0/d1c/d3b/f98 14272 0 2024-03-23T08:23:27.030 INFO:tasks.workunit.client.1.smithi193.stdout:2/740: dwrite d0/d28/f47 [0,4194304] 0 2024-03-23T08:23:27.031 INFO:tasks.workunit.client.1.smithi193.stdout:9/728: readlink d2/d7/l16 0 2024-03-23T08:23:27.033 INFO:tasks.workunit.client.1.smithi193.stdout:4/740: write d11/d2d/d58/d80/ff0 [1045327,103895] 0 2024-03-23T08:23:27.033 INFO:tasks.workunit.client.1.smithi193.stdout:8/704: chown df/d10/d67/cb4 1368 1 2024-03-23T08:23:27.038 INFO:tasks.workunit.client.1.smithi193.stdout:8/705: chown df/dc6/d8b 988 1 2024-03-23T08:23:27.039 INFO:tasks.workunit.client.1.smithi193.stdout:7/665: mknod d6/d9/d1f/d23/d33/d38/d66/dac/cea 0 2024-03-23T08:23:27.040 INFO:tasks.workunit.client.1.smithi193.stdout:2/741: rmdir d0/d1b/d77 39 2024-03-23T08:23:27.040 INFO:tasks.workunit.client.1.smithi193.stdout:1/697: dwrite da/d13/d3f/f54 [4194304,4194304] 0 2024-03-23T08:23:27.044 INFO:tasks.workunit.client.1.smithi193.stdout:4/741: creat d11/d8e/dac/ffa x:0 0 0 2024-03-23T08:23:27.045 INFO:tasks.workunit.client.1.smithi193.stdout:7/666: chown d6/d9/d1f/d23/d33/d38/dc0 312599 1 2024-03-23T08:23:27.046 INFO:tasks.workunit.client.1.smithi193.stdout:8/706: fdatasync df/dc6/d8b/dae/f90 0 2024-03-23T08:23:27.048 INFO:tasks.workunit.client.1.smithi193.stdout:6/577: dread d0/d27/d81/df/d11/d3b/d51/d9c/f96 [0,4194304] 0 2024-03-23T08:23:27.049 INFO:tasks.workunit.client.1.smithi193.stdout:1/698: unlink da/d13/c2a 0 2024-03-23T08:23:27.049 INFO:tasks.workunit.client.1.smithi193.stdout:5/727: dwrite d0/ff5 [0,4194304] 0 2024-03-23T08:23:27.050 INFO:tasks.workunit.client.1.smithi193.stdout:7/667: dread - d6/d9/d1f/d4d/fa2 zero size 2024-03-23T08:23:27.052 INFO:tasks.workunit.client.1.smithi193.stdout:9/729: getdents d2/d7/db/d1b/d43/d71/db4/dec/d3c 0 2024-03-23T08:23:27.052 INFO:tasks.workunit.client.1.smithi193.stdout:2/742: rename d0/d28/d33 to d0/d1b/d14/da1/dd5 0 2024-03-23T08:23:27.054 INFO:tasks.workunit.client.1.smithi193.stdout:8/707: write df/d4b/d4d/d9c/fa6 [123763,86254] 0 2024-03-23T08:23:27.056 INFO:tasks.workunit.client.1.smithi193.stdout:4/742: read d11/d1a/d33/d8f/db8/f2b [11586847,100085] 0 2024-03-23T08:23:27.057 INFO:tasks.workunit.client.1.smithi193.stdout:1/699: creat da/dd/df/d52/d82/db0/fdd x:0 0 0 2024-03-23T08:23:27.060 INFO:tasks.workunit.client.1.smithi193.stdout:8/708: unlink cb 0 2024-03-23T08:23:27.062 INFO:tasks.workunit.client.1.smithi193.stdout:6/578: mknod d0/d27/d81/df/d11/d56/cb7 0 2024-03-23T08:23:27.063 INFO:tasks.workunit.client.1.smithi193.stdout:9/730: symlink d2/d7/db/d17/lf1 0 2024-03-23T08:23:27.064 INFO:tasks.workunit.client.1.smithi193.stdout:7/668: truncate d6/d9/f15 592231 0 2024-03-23T08:23:27.064 INFO:tasks.workunit.client.1.smithi193.stdout:7/669: chown d6/d9/d1f 0 1 2024-03-23T08:23:27.065 INFO:tasks.workunit.client.1.smithi193.stdout:2/743: fdatasync d0/f2d 0 2024-03-23T08:23:27.067 INFO:tasks.workunit.client.1.smithi193.stdout:8/709: mkdir df/d63/d84/df0 0 2024-03-23T08:23:27.067 INFO:tasks.workunit.client.1.smithi193.stdout:6/579: mkdir d0/d27/d81/da1/db8 0 2024-03-23T08:23:27.068 INFO:tasks.workunit.client.1.smithi193.stdout:8/710: write df/f86 [490229,56072] 0 2024-03-23T08:23:27.069 INFO:tasks.workunit.client.1.smithi193.stdout:4/743: mkdir d11/d1a/d33/d42/db2/dbe/d50/dfb 0 2024-03-23T08:23:27.071 INFO:tasks.workunit.client.1.smithi193.stdout:1/700: getdents da/d13/d24 0 2024-03-23T08:23:27.072 INFO:tasks.workunit.client.1.smithi193.stdout:5/728: getdents d0/d1c/d7a/d86 0 2024-03-23T08:23:27.073 INFO:tasks.workunit.client.1.smithi193.stdout:6/580: stat d0/d27/d81/df/d11/f37 0 2024-03-23T08:23:27.073 INFO:tasks.workunit.client.1.smithi193.stdout:9/731: rename d2/d7/db/d1b/f4d to d2/ff2 0 2024-03-23T08:23:27.074 INFO:tasks.workunit.client.1.smithi193.stdout:8/711: truncate df/d4b/fe4 356924 0 2024-03-23T08:23:27.076 INFO:tasks.workunit.client.1.smithi193.stdout:4/744: creat d11/d2d/d58/d80/db9/ffc x:0 0 0 2024-03-23T08:23:27.076 INFO:tasks.workunit.client.1.smithi193.stdout:5/729: mkdir d0/d5a/dbf/d106 0 2024-03-23T08:23:27.078 INFO:tasks.workunit.client.1.smithi193.stdout:2/744: dwrite d0/d1b/d14/d66/d50/f52 [0,4194304] 0 2024-03-23T08:23:27.079 INFO:tasks.workunit.client.1.smithi193.stdout:9/732: write d2/d7/db/d1b/d5c/fe9 [975617,111172] 0 2024-03-23T08:23:27.081 INFO:tasks.workunit.client.1.smithi193.stdout:4/745: readlink d11/d2d/d58/d80/db9/ldc 0 2024-03-23T08:23:27.084 INFO:tasks.workunit.client.1.smithi193.stdout:6/581: write d0/d27/d81/df/d11/d36/d61/f8e [1357062,119977] 0 2024-03-23T08:23:27.084 INFO:tasks.workunit.client.1.smithi193.stdout:7/670: dread d6/d9/d1f/d23/d33/d38/d3b/f55 [0,4194304] 0 2024-03-23T08:23:27.085 INFO:tasks.workunit.client.1.smithi193.stdout:5/730: chown d0/d1c/d7a/db1/d72/d73/c39 1456 1 2024-03-23T08:23:27.086 INFO:tasks.workunit.client.1.smithi193.stdout:1/701: dread da/f12 [0,4194304] 0 2024-03-23T08:23:27.089 INFO:tasks.workunit.client.1.smithi193.stdout:4/746: mkdir d11/d1a/d65/dfd 0 2024-03-23T08:23:27.089 INFO:tasks.workunit.client.1.smithi193.stdout:1/702: fsync da/dd/df/f29 0 2024-03-23T08:23:27.090 INFO:tasks.workunit.client.1.smithi193.stdout:8/712: creat df/d10/d3e/d54/dd2/dca/ff1 x:0 0 0 2024-03-23T08:23:27.092 INFO:tasks.workunit.client.1.smithi193.stdout:7/671: creat d6/d9/d1f/d3e/d42/feb x:0 0 0 2024-03-23T08:23:27.093 INFO:tasks.workunit.client.1.smithi193.stdout:6/582: symlink d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/d9d/lb9 0 2024-03-23T08:23:27.100 INFO:tasks.workunit.client.1.smithi193.stdout:7/672: fdatasync d6/d9/d1f/daf/fe1 0 2024-03-23T08:23:27.101 INFO:tasks.workunit.client.1.smithi193.stdout:1/703: mkdir da/dd/df/d7e/d7f/dde 0 2024-03-23T08:23:27.105 INFO:tasks.workunit.client.1.smithi193.stdout:5/731: dwrite d0/d1c/d7a/d86/f103 [0,4194304] 0 2024-03-23T08:23:27.109 INFO:tasks.workunit.client.1.smithi193.stdout:7/673: mkdir d6/d9/d1f/d23/d33/d3c/dec 0 2024-03-23T08:23:27.110 INFO:tasks.workunit.client.1.smithi193.stdout:9/733: dread d2/d7/db/d7a/d95/fa7 [0,4194304] 0 2024-03-23T08:23:27.111 INFO:tasks.workunit.client.1.smithi193.stdout:8/713: dwrite df/d10/d3e/d54/dd2/fa5 [0,4194304] 0 2024-03-23T08:23:27.112 INFO:tasks.workunit.client.1.smithi193.stdout:6/583: creat d0/d27/d81/df/d11/d36/fba x:0 0 0 2024-03-23T08:23:27.113 INFO:tasks.workunit.client.1.smithi193.stdout:1/704: symlink da/dd/df/d7e/d7f/db2/ldf 0 2024-03-23T08:23:27.113 INFO:tasks.workunit.client.1.smithi193.stdout:2/745: dread d0/d1b/d14/f41 [0,4194304] 0 2024-03-23T08:23:27.114 INFO:tasks.workunit.client.1.smithi193.stdout:5/732: chown d0/d1c/d7a/db1/d72/d73/d57 6335 1 2024-03-23T08:23:27.118 INFO:tasks.workunit.client.1.smithi193.stdout:1/705: symlink da/d13/d24/le0 0 2024-03-23T08:23:27.118 INFO:tasks.workunit.client.1.smithi193.stdout:7/674: readlink d6/d9/d1f/d3e/d42/l5d 0 2024-03-23T08:23:27.118 INFO:tasks.workunit.client.1.smithi193.stdout:9/734: chown d2/d7/db/d32/lf0 56147 1 2024-03-23T08:23:27.120 INFO:tasks.workunit.client.1.smithi193.stdout:4/747: dwrite d11/d1a/d33/d7f/de0/f7e [0,4194304] 0 2024-03-23T08:23:27.120 INFO:tasks.workunit.client.1.smithi193.stdout:0/679: sync 2024-03-23T08:23:27.122 INFO:tasks.workunit.client.1.smithi193.stdout:0/680: readlink db/d52/laf 0 2024-03-23T08:23:27.123 INFO:tasks.workunit.client.1.smithi193.stdout:9/735: chown d2/d7/db/d1b/f4c 66562076 1 2024-03-23T08:23:27.125 INFO:tasks.workunit.client.1.smithi193.stdout:6/584: write d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f7e [1349830,41023] 0 2024-03-23T08:23:27.127 INFO:tasks.workunit.client.1.smithi193.stdout:7/675: creat d6/d9/d1f/d23/d33/d38/d46/d82/fed x:0 0 0 2024-03-23T08:23:27.127 INFO:tasks.workunit.client.1.smithi193.stdout:1/706: truncate da/dd/df/f2d 2208997 0 2024-03-23T08:23:27.130 INFO:tasks.workunit.client.1.smithi193.stdout:4/748: creat d11/d1a/d33/ffe x:0 0 0 2024-03-23T08:23:27.130 INFO:tasks.workunit.client.1.smithi193.stdout:5/733: dread d0/d1c/d3b/d4b/f95 [0,4194304] 0 2024-03-23T08:23:27.132 INFO:tasks.workunit.client.1.smithi193.stdout:0/681: creat db/d11/d14/d24/dc9/fed x:0 0 0 2024-03-23T08:23:27.132 INFO:tasks.workunit.client.1.smithi193.stdout:9/736: mkdir d2/d7/db/d32/d6b/dde/df3 0 2024-03-23T08:23:27.133 INFO:tasks.workunit.client.1.smithi193.stdout:2/746: dwrite d0/f2d [4194304,4194304] 0 2024-03-23T08:23:27.133 INFO:tasks.workunit.client.1.smithi193.stdout:6/585: chown d0/d27/d81/df/d11/d36/d61/d12 0 1 2024-03-23T08:23:27.134 INFO:tasks.workunit.client.1.smithi193.stdout:7/676: symlink d6/d9/d1f/d23/d33/d38/d46/lee 0 2024-03-23T08:23:27.136 INFO:tasks.workunit.client.1.smithi193.stdout:5/734: fdatasync d0/d1c/d7a/db1/d72/d99/fb5 0 2024-03-23T08:23:27.138 INFO:tasks.workunit.client.1.smithi193.stdout:0/682: fsync db/d11/d14/d24/d2a/fbe 0 2024-03-23T08:23:27.140 INFO:tasks.workunit.client.1.smithi193.stdout:2/747: fsync d0/d37/d48/f4a 0 2024-03-23T08:23:27.145 INFO:tasks.workunit.client.1.smithi193.stdout:6/586: readlink d0/d27/d81/df/d11/d36/d61/d7c/d69/l8c 0 2024-03-23T08:23:27.145 INFO:tasks.workunit.client.1.smithi193.stdout:7/677: creat d6/d9/d1f/daf/fef x:0 0 0 2024-03-23T08:23:27.147 INFO:tasks.workunit.client.1.smithi193.stdout:9/737: creat d2/d7/db/d17/d4b/d59/d97/ff4 x:0 0 0 2024-03-23T08:23:27.149 INFO:tasks.workunit.client.1.smithi193.stdout:0/683: creat db/d11/d14/d24/ddd/fee x:0 0 0 2024-03-23T08:23:27.149 INFO:tasks.workunit.client.1.smithi193.stdout:1/707: dwrite da/d13/d24/f79 [0,4194304] 0 2024-03-23T08:23:27.151 INFO:tasks.workunit.client.1.smithi193.stdout:4/749: dwrite d11/d2d/d58/d80/db9/fdf [0,4194304] 0 2024-03-23T08:23:27.151 INFO:tasks.workunit.client.1.smithi193.stdout:5/735: rename d0/d1c/d7a/db1/d72/d73/d57/c52 to d0/d1c/d7a/db1/d72/d73/c107 0 2024-03-23T08:23:27.152 INFO:tasks.workunit.client.1.smithi193.stdout:6/587: rmdir d0/d27/d81/df/d11/d36/d61/d3a 39 2024-03-23T08:23:27.154 INFO:tasks.workunit.client.1.smithi193.stdout:7/678: mkdir d6/d9/d1f/d23/d33/d3c/df0 0 2024-03-23T08:23:27.156 INFO:tasks.workunit.client.1.smithi193.stdout:9/738: creat d2/d7/db/d1b/d5c/ff5 x:0 0 0 2024-03-23T08:23:27.156 INFO:tasks.workunit.client.1.smithi193.stdout:1/708: symlink da/d13/le1 0 2024-03-23T08:23:27.157 INFO:tasks.workunit.client.1.smithi193.stdout:0/684: fdatasync db/d11/d14/d24/d47/d59/f79 0 2024-03-23T08:23:27.157 INFO:tasks.workunit.client.1.smithi193.stdout:8/714: dread df/d10/d67/d7f/fec [0,4194304] 0 2024-03-23T08:23:27.162 INFO:tasks.workunit.client.1.smithi193.stdout:8/715: chown df/d63/d84/dc5 1 1 2024-03-23T08:23:27.162 INFO:tasks.workunit.client.1.smithi193.stdout:4/750: unlink d11/d1a/d33/d42/f71 0 2024-03-23T08:23:27.163 INFO:tasks.workunit.client.1.smithi193.stdout:9/739: read d2/d7/db/d1b/f4c [3709775,123640] 0 2024-03-23T08:23:27.163 INFO:tasks.workunit.client.1.smithi193.stdout:2/748: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:27.165 INFO:tasks.workunit.client.1.smithi193.stdout:1/709: write da/dd/df/d7e/fd9 [602378,86039] 0 2024-03-23T08:23:27.166 INFO:tasks.workunit.client.1.smithi193.stdout:0/685: creat db/d11/d14/d24/ddd/fef x:0 0 0 2024-03-23T08:23:27.167 INFO:tasks.workunit.client.1.smithi193.stdout:4/751: fdatasync d11/d1a/d33/d42/f29 0 2024-03-23T08:23:27.167 INFO:tasks.workunit.client.1.smithi193.stdout:9/740: fsync d2/d7/db/d32/fd2 0 2024-03-23T08:23:27.169 INFO:tasks.workunit.client.1.smithi193.stdout:2/749: chown d0/d37/c4d 233517 1 2024-03-23T08:23:27.171 INFO:tasks.workunit.client.1.smithi193.stdout:2/750: write d0/d1b/d14/d66/d50/f52 [1120785,77027] 0 2024-03-23T08:23:27.172 INFO:tasks.workunit.client.1.smithi193.stdout:2/751: fsync d0/d1b/d14/d66/d50/f91 0 2024-03-23T08:23:27.173 INFO:tasks.workunit.client.1.smithi193.stdout:7/679: link d6/d9/d1f/d4d/d7c/d92/fde d6/d9/d4e/d81/ff1 0 2024-03-23T08:23:27.173 INFO:tasks.workunit.client.1.smithi193.stdout:4/752: rmdir d11/d8d 39 2024-03-23T08:23:27.174 INFO:tasks.workunit.client.1.smithi193.stdout:1/710: creat da/dd/df/d7c/fe2 x:0 0 0 2024-03-23T08:23:27.175 INFO:tasks.workunit.client.1.smithi193.stdout:0/686: mkdir db/d11/d14/d24/ddd/df0 0 2024-03-23T08:23:27.177 INFO:tasks.workunit.client.1.smithi193.stdout:0/687: write db/d11/d14/fad [5111402,54499] 0 2024-03-23T08:23:27.178 INFO:tasks.workunit.client.1.smithi193.stdout:7/680: symlink d6/d9/d4e/d81/lf2 0 2024-03-23T08:23:27.182 INFO:tasks.workunit.client.1.smithi193.stdout:4/753: rename d11/d8d/fa3 to d11/d27/df2/fff 0 2024-03-23T08:23:27.187 INFO:tasks.workunit.client.1.smithi193.stdout:7/681: rename d6/d9/d1f/d9d to d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/df3 0 2024-03-23T08:23:27.188 INFO:tasks.workunit.client.1.smithi193.stdout:8/716: dread df/d4b/f66 [0,4194304] 0 2024-03-23T08:23:27.191 INFO:tasks.workunit.client.1.smithi193.stdout:7/682: read d6/d9/d1f/d23/d33/d38/d46/d82/f8c [676521,120089] 0 2024-03-23T08:23:27.191 INFO:tasks.workunit.client.1.smithi193.stdout:9/741: dread d2/d7/db/d1b/faf [4194304,4194304] 0 2024-03-23T08:23:27.193 INFO:tasks.workunit.client.1.smithi193.stdout:1/711: getdents da/d13/d3f/d5a 0 2024-03-23T08:23:27.193 INFO:tasks.workunit.client.1.smithi193.stdout:8/717: mkdir df/d4b/d4d/d9c/df2 0 2024-03-23T08:23:27.194 INFO:tasks.workunit.client.1.smithi193.stdout:7/683: write d6/d9/d1f/d3e/d42/f59 [624617,51309] 0 2024-03-23T08:23:27.194 INFO:tasks.workunit.client.1.smithi193.stdout:6/588: truncate d0/d27/d81/f4e 1984333 0 2024-03-23T08:23:27.195 INFO:tasks.workunit.client.1.smithi193.stdout:4/754: truncate d11/d8e/dac/fbd 931083 0 2024-03-23T08:23:27.196 INFO:tasks.workunit.client.1.smithi193.stdout:1/712: read da/d13/d3f/fa7 [2578186,35050] 0 2024-03-23T08:23:27.196 INFO:tasks.workunit.client.1.smithi193.stdout:8/718: readlink df/d10/d67/d7f/lab 0 2024-03-23T08:23:27.196 INFO:tasks.workunit.client.1.smithi193.stdout:2/752: dread d0/d1b/d14/f40 [0,4194304] 0 2024-03-23T08:23:27.198 INFO:tasks.workunit.client.1.smithi193.stdout:6/589: fdatasync d0/d27/d81/d87/fa8 0 2024-03-23T08:23:27.199 INFO:tasks.workunit.client.1.smithi193.stdout:7/684: dread d6/d9/d1f/f50 [0,4194304] 0 2024-03-23T08:23:27.199 INFO:tasks.workunit.client.1.smithi193.stdout:8/719: chown df/d10/d3e/d54/dd2/f72 76 1 2024-03-23T08:23:27.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/753: dread - d0/d1b/d14/d56/d7c/fd2 zero size 2024-03-23T08:23:27.200 INFO:tasks.workunit.client.1.smithi193.stdout:9/742: mknod d2/d7/db/d1b/d43/d71/db4/dec/d5f/de3/cf6 0 2024-03-23T08:23:27.202 INFO:tasks.workunit.client.1.smithi193.stdout:6/590: chown d0/d27/d81/df/d11/d36/d61/d12/l2b 58368 1 2024-03-23T08:23:27.202 INFO:tasks.workunit.client.1.smithi193.stdout:8/720: chown df/d4b/d4d/c16 978603482 1 2024-03-23T08:23:27.202 INFO:tasks.workunit.client.1.smithi193.stdout:2/754: chown d0/d1b/d14/cae 432346 1 2024-03-23T08:23:27.203 INFO:tasks.workunit.client.1.smithi193.stdout:1/713: mknod da/dd/df/d7e/d7f/dde/ce3 0 2024-03-23T08:23:27.203 INFO:tasks.workunit.client.1.smithi193.stdout:9/743: read - d2/d7/db/d1b/d43/d71/db4/dec/d5f/fb1 zero size 2024-03-23T08:23:27.204 INFO:tasks.workunit.client.1.smithi193.stdout:7/685: symlink d6/d9/d1f/d23/d33/d3c/d41/d47/lf4 0 2024-03-23T08:23:27.204 INFO:tasks.workunit.client.1.smithi193.stdout:6/591: write d0/d27/d81/da1/fb5 [1036410,103941] 0 2024-03-23T08:23:27.205 INFO:tasks.workunit.client.1.smithi193.stdout:4/755: getdents d11/d2d/d58/d80/db9 0 2024-03-23T08:23:27.206 INFO:tasks.workunit.client.1.smithi193.stdout:8/721: creat df/dc6/d8b/dae/ff3 x:0 0 0 2024-03-23T08:23:27.207 INFO:tasks.workunit.client.1.smithi193.stdout:2/755: write d0/d1b/d14/d56/d7c/d7e/fca [482126,99414] 0 2024-03-23T08:23:27.207 INFO:tasks.workunit.client.1.smithi193.stdout:7/686: read - d6/d9/d1f/fd3 zero size 2024-03-23T08:23:27.209 INFO:tasks.workunit.client.1.smithi193.stdout:1/714: mkdir da/d13/d24/dac/de4 0 2024-03-23T08:23:27.210 INFO:tasks.workunit.client.1.smithi193.stdout:8/722: write df/d4b/d4d/f5c [3391254,618] 0 2024-03-23T08:23:27.210 INFO:tasks.workunit.client.1.smithi193.stdout:9/744: mkdir d2/df7 0 2024-03-23T08:23:27.211 INFO:tasks.workunit.client.1.smithi193.stdout:2/756: fdatasync d0/d1b/f13 0 2024-03-23T08:23:27.211 INFO:tasks.workunit.client.1.smithi193.stdout:7/687: fsync d6/d9/d1f/d23/f2e 0 2024-03-23T08:23:27.213 INFO:tasks.workunit.client.1.smithi193.stdout:7/688: readlink d6/l6e 0 2024-03-23T08:23:27.214 INFO:tasks.workunit.client.1.smithi193.stdout:8/723: write df/d10/d3e/d54/dd2/dca/ff1 [636960,15652] 0 2024-03-23T08:23:27.216 INFO:tasks.workunit.client.1.smithi193.stdout:9/745: truncate d2/d7/db/d1b/d43/d71/db4/dec/d5f/fb1 193264 0 2024-03-23T08:23:27.217 INFO:tasks.workunit.client.1.smithi193.stdout:1/715: creat da/dd/df/d7c/fe5 x:0 0 0 2024-03-23T08:23:27.218 INFO:tasks.workunit.client.1.smithi193.stdout:9/746: truncate d2/d7/db/d1b/d43/d71/db4/d78/fbc 680363 0 2024-03-23T08:23:27.218 INFO:tasks.workunit.client.1.smithi193.stdout:1/716: read da/dd/df/f25 [2398987,26796] 0 2024-03-23T08:23:27.219 INFO:tasks.workunit.client.1.smithi193.stdout:4/756: truncate d11/d2d/d3a/f5e 985592 0 2024-03-23T08:23:27.219 INFO:tasks.workunit.client.1.smithi193.stdout:9/747: chown d2/d7/db/d17/d33/f45 1674871 1 2024-03-23T08:23:27.220 INFO:tasks.workunit.client.1.smithi193.stdout:9/748: stat d2 0 2024-03-23T08:23:27.222 INFO:tasks.workunit.client.1.smithi193.stdout:7/689: mknod d6/d9/d1f/d23/d33/d38/d46/d82/cf5 0 2024-03-23T08:23:27.223 INFO:tasks.workunit.client.1.smithi193.stdout:6/592: creat d0/d27/d81/df/d11/fbb x:0 0 0 2024-03-23T08:23:27.225 INFO:tasks.workunit.client.1.smithi193.stdout:4/757: creat d11/d27/d85/f100 x:0 0 0 2024-03-23T08:23:27.227 INFO:tasks.workunit.client.1.smithi193.stdout:2/757: dwrite d0/d1b/d3b/f79 [0,4194304] 0 2024-03-23T08:23:27.229 INFO:tasks.workunit.client.1.smithi193.stdout:2/758: dread - d0/d1b/d14/d56/d7c/fd2 zero size 2024-03-23T08:23:27.229 INFO:tasks.workunit.client.1.smithi193.stdout:8/724: write df/d4b/f66 [114421,17601] 0 2024-03-23T08:23:27.232 INFO:tasks.workunit.client.1.smithi193.stdout:7/690: rename d6/d9/d1f/d3e/l40 to d6/d9/d1f/d3e/d65/lf6 0 2024-03-23T08:23:27.235 INFO:tasks.workunit.client.1.smithi193.stdout:2/759: rmdir d0/d1b/d14/d56 39 2024-03-23T08:23:27.239 INFO:tasks.workunit.client.1.smithi193.stdout:2/760: readlink d0/d1b/d14/da1/dd5/lad 0 2024-03-23T08:23:27.240 INFO:tasks.workunit.client.1.smithi193.stdout:3/604: sync 2024-03-23T08:23:27.240 INFO:tasks.workunit.client.1.smithi193.stdout:8/725: mkdir df/d63/d84/df4 0 2024-03-23T08:23:27.241 INFO:tasks.workunit.client.1.smithi193.stdout:1/717: link da/dd/df/d52/c66 da/d13/ce6 0 2024-03-23T08:23:27.244 INFO:tasks.workunit.client.1.smithi193.stdout:6/593: getdents d0/d27/d81/df/d11/d3b/d6c 0 2024-03-23T08:23:27.246 INFO:tasks.workunit.client.1.smithi193.stdout:4/758: dwrite d11/d8d/f97 [0,4194304] 0 2024-03-23T08:23:27.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/761: fsync d0/d1b/d3b/f58 0 2024-03-23T08:23:27.248 INFO:tasks.workunit.client.1.smithi193.stdout:7/691: dwrite d6/d9/d1f/d4d/d6b/fd5 [0,4194304] 0 2024-03-23T08:23:27.255 INFO:tasks.workunit.client.1.smithi193.stdout:1/718: write da/f59 [59678,129353] 0 2024-03-23T08:23:27.258 INFO:tasks.workunit.client.1.smithi193.stdout:1/719: fsync da/d13/d3f/d5a/f8b 0 2024-03-23T08:23:27.259 INFO:tasks.workunit.client.1.smithi193.stdout:4/759: symlink d11/d8d/l101 0 2024-03-23T08:23:27.260 INFO:tasks.workunit.client.1.smithi193.stdout:8/726: dwrite df/f33 [0,4194304] 0 2024-03-23T08:23:27.261 INFO:tasks.workunit.client.1.smithi193.stdout:2/762: mknod d0/d1b/d77/d9b/cd6 0 2024-03-23T08:23:27.263 INFO:tasks.workunit.client.1.smithi193.stdout:3/605: dread d1/de/d13/d21/d50/d44/f4f [0,4194304] 0 2024-03-23T08:23:27.266 INFO:tasks.workunit.client.1.smithi193.stdout:6/594: dwrite d0/d27/d81/d87/fa8 [0,4194304] 0 2024-03-23T08:23:27.269 INFO:tasks.workunit.client.1.smithi193.stdout:1/720: mkdir da/d13/d3f/d5a/d76/d77/de7 0 2024-03-23T08:23:27.269 INFO:tasks.workunit.client.1.smithi193.stdout:4/760: creat d11/d1a/d33/dce/f102 x:0 0 0 2024-03-23T08:23:27.272 INFO:tasks.workunit.client.1.smithi193.stdout:2/763: rmdir d0/d1b/d14/d66/d50/d92 39 2024-03-23T08:23:27.273 INFO:tasks.workunit.client.1.smithi193.stdout:3/606: readlink d1/de/d13/d21/d25/d34/l49 0 2024-03-23T08:23:27.273 INFO:tasks.workunit.client.1.smithi193.stdout:1/721: stat da/d13/d3f/d5a/f65 0 2024-03-23T08:23:27.275 INFO:tasks.workunit.client.1.smithi193.stdout:2/764: write d0/d28/f8c [770594,54483] 0 2024-03-23T08:23:27.276 INFO:tasks.workunit.client.1.smithi193.stdout:3/607: symlink d1/de/lcf 0 2024-03-23T08:23:27.278 INFO:tasks.workunit.client.1.smithi193.stdout:8/727: rename df/d4b/d4d/d2a/l4c to df/d10/lf5 0 2024-03-23T08:23:27.278 INFO:tasks.workunit.client.1.smithi193.stdout:6/595: dwrite d0/d27/d81/df/d11/d36/d61/f8e [0,4194304] 0 2024-03-23T08:23:27.281 INFO:tasks.workunit.client.1.smithi193.stdout:4/761: dread d11/d1a/d33/d7f/de0/fa0 [0,4194304] 0 2024-03-23T08:23:27.283 INFO:tasks.workunit.client.1.smithi193.stdout:7/692: write d6/d9/d1f/d4d/d7c/d92/fde [1014945,11847] 0 2024-03-23T08:23:27.284 INFO:tasks.workunit.client.1.smithi193.stdout:4/762: chown d11/d1a/d33/d42/db2 1507390 1 2024-03-23T08:23:27.286 INFO:tasks.workunit.client.1.smithi193.stdout:1/722: dread da/dd/df/f29 [0,4194304] 0 2024-03-23T08:23:27.288 INFO:tasks.workunit.client.1.smithi193.stdout:2/765: dread d0/d1b/f9e [0,4194304] 0 2024-03-23T08:23:27.288 INFO:tasks.workunit.client.1.smithi193.stdout:3/608: truncate d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fce 600004 0 2024-03-23T08:23:27.289 INFO:tasks.workunit.client.1.smithi193.stdout:1/723: stat da/d13/d3f/d5a/f9d 0 2024-03-23T08:23:27.291 INFO:tasks.workunit.client.1.smithi193.stdout:7/693: truncate d6/d9/d1f/d23/f2e 1860643 0 2024-03-23T08:23:27.293 INFO:tasks.workunit.client.1.smithi193.stdout:8/728: dwrite df/d10/d67/f7d [0,4194304] 0 2024-03-23T08:23:27.294 INFO:tasks.workunit.client.1.smithi193.stdout:4/763: link d11/d2d/d58/d80/ff0 d11/d27/df2/f103 0 2024-03-23T08:23:27.295 INFO:tasks.workunit.client.1.smithi193.stdout:1/724: creat da/dd/da9/fe8 x:0 0 0 2024-03-23T08:23:27.295 INFO:tasks.workunit.client.1.smithi193.stdout:2/766: symlink d0/d1b/d14/d66/d4e/dce/ld7 0 2024-03-23T08:23:27.297 INFO:tasks.workunit.client.1.smithi193.stdout:3/609: unlink d1/de/d13/d21/d25/d34/f91 0 2024-03-23T08:23:27.305 INFO:tasks.workunit.client.1.smithi193.stdout:4/764: rename d11/cb4 to d11/d1a/d33/d7f/de0/c104 0 2024-03-23T08:23:27.305 INFO:tasks.workunit.client.1.smithi193.stdout:1/725: truncate da/dd/f14 1338509 0 2024-03-23T08:23:27.311 INFO:tasks.workunit.client.1.smithi193.stdout:7/694: dwrite d6/f39 [0,4194304] 0 2024-03-23T08:23:27.313 INFO:tasks.workunit.client.1.smithi193.stdout:1/726: write da/d13/d3f/d5a/f67 [4376049,106209] 0 2024-03-23T08:23:27.315 INFO:tasks.workunit.client.1.smithi193.stdout:8/729: dwrite df/fe3 [0,4194304] 0 2024-03-23T08:23:27.316 INFO:tasks.workunit.client.1.smithi193.stdout:4/765: mkdir d11/d95/de4/d105 0 2024-03-23T08:23:27.322 INFO:tasks.workunit.client.1.smithi193.stdout:2/767: dwrite d0/d1b/d14/d56/d7c/fd2 [0,4194304] 0 2024-03-23T08:23:27.323 INFO:tasks.workunit.client.1.smithi193.stdout:6/596: dread d0/d27/d81/f20 [0,4194304] 0 2024-03-23T08:23:27.324 INFO:tasks.workunit.client.1.smithi193.stdout:1/727: write da/dd/df/d2e/f48 [622646,85620] 0 2024-03-23T08:23:27.326 INFO:tasks.workunit.client.1.smithi193.stdout:7/695: write d6/d9/d1f/d23/d33/d3c/d41/d47/fcc [328011,25546] 0 2024-03-23T08:23:27.326 INFO:tasks.workunit.client.1.smithi193.stdout:3/610: dwrite d1/de/d13/d21/d25/d2b/fb1 [0,4194304] 0 2024-03-23T08:23:27.327 INFO:tasks.workunit.client.1.smithi193.stdout:4/766: creat d11/d1a/d33/d7f/f106 x:0 0 0 2024-03-23T08:23:27.328 INFO:tasks.workunit.client.1.smithi193.stdout:5/736: sync 2024-03-23T08:23:27.331 INFO:tasks.workunit.client.1.smithi193.stdout:2/768: write d0/d1b/d14/d56/da3/db0/fb6 [4891959,114664] 0 2024-03-23T08:23:27.332 INFO:tasks.workunit.client.1.smithi193.stdout:7/696: readlink d6/d9/d1f/d3e/d42/l96 0 2024-03-23T08:23:27.333 INFO:tasks.workunit.client.1.smithi193.stdout:4/767: mkdir d11/d95/d107 0 2024-03-23T08:23:27.336 INFO:tasks.workunit.client.1.smithi193.stdout:4/768: read - d11/d1a/d33/d44/faa zero size 2024-03-23T08:23:27.337 INFO:tasks.workunit.client.1.smithi193.stdout:1/728: rename da/dd/df/d7e/d7f/db2/ldf to da/dd/df/d7e/d86/d95/le9 0 2024-03-23T08:23:27.338 INFO:tasks.workunit.client.1.smithi193.stdout:6/597: creat d0/d27/d81/df/d11/d3b/fbc x:0 0 0 2024-03-23T08:23:27.341 INFO:tasks.workunit.client.1.smithi193.stdout:7/697: rename d6/d9/d1f/d23/d33/f6d to d6/d9/d4e/d81/dca/ff7 0 2024-03-23T08:23:27.342 INFO:tasks.workunit.client.1.smithi193.stdout:0/688: sync 2024-03-23T08:23:27.342 INFO:tasks.workunit.client.1.smithi193.stdout:9/749: sync 2024-03-23T08:23:27.342 INFO:tasks.workunit.client.1.smithi193.stdout:1/729: write da/dd/df/f4b [947135,11096] 0 2024-03-23T08:23:27.342 INFO:tasks.workunit.client.1.smithi193.stdout:4/769: write d11/d2d/d58/d80/db9/ffc [614954,76906] 0 2024-03-23T08:23:27.344 INFO:tasks.workunit.client.1.smithi193.stdout:8/730: truncate df/d63/f81 2121800 0 2024-03-23T08:23:27.345 INFO:tasks.workunit.client.1.smithi193.stdout:3/611: dwrite d1/de/d13/d21/f23 [4194304,4194304] 0 2024-03-23T08:23:27.346 INFO:tasks.workunit.client.1.smithi193.stdout:9/750: stat d2/d7/f37 0 2024-03-23T08:23:27.350 INFO:tasks.workunit.client.1.smithi193.stdout:2/769: truncate d0/d1b/d14/da1/dd5/d72/f97 302567 0 2024-03-23T08:23:27.350 INFO:tasks.workunit.client.1.smithi193.stdout:5/737: write d0/d1c/d3b/f98 [522399,66400] 0 2024-03-23T08:23:27.350 INFO:tasks.workunit.client.1.smithi193.stdout:1/730: chown da/d13/d3f/d5a/f9d 55136 1 2024-03-23T08:23:27.351 INFO:tasks.workunit.client.1.smithi193.stdout:4/770: dread d11/d1a/d33/d7f/f81 [0,4194304] 0 2024-03-23T08:23:27.353 INFO:tasks.workunit.client.1.smithi193.stdout:3/612: creat d1/de/fd0 x:0 0 0 2024-03-23T08:23:27.354 INFO:tasks.workunit.client.1.smithi193.stdout:6/598: write d0/d27/d81/df/d11/d36/d61/d3a/f45 [365122,80546] 0 2024-03-23T08:23:27.356 INFO:tasks.workunit.client.1.smithi193.stdout:9/751: mkdir d2/d7/db/d17/d4b/d59/df8 0 2024-03-23T08:23:27.359 INFO:tasks.workunit.client.1.smithi193.stdout:5/738: chown d0/d1c/d7a/db1/d72/d73/d45/daa 31253 1 2024-03-23T08:23:27.359 INFO:tasks.workunit.client.1.smithi193.stdout:1/731: dread - da/dd/df/d52/d82/db0/fdd zero size 2024-03-23T08:23:27.360 INFO:tasks.workunit.client.1.smithi193.stdout:4/771: fdatasync d11/d1a/d33/d42/db2/dbe/d50/d49/f5c 0 2024-03-23T08:23:27.361 INFO:tasks.workunit.client.1.smithi193.stdout:3/613: rmdir d1/de/d13/d21 39 2024-03-23T08:23:27.362 INFO:tasks.workunit.client.1.smithi193.stdout:8/731: rename df/dc6/c73 to df/d10/d67/d7f/cf6 0 2024-03-23T08:23:27.362 INFO:tasks.workunit.client.1.smithi193.stdout:9/752: chown d2/l18 107593 1 2024-03-23T08:23:27.363 INFO:tasks.workunit.client.1.smithi193.stdout:0/689: link db/d11/d14/d3b/l89 db/d11/d14/d3b/d71/lf1 0 2024-03-23T08:23:27.364 INFO:tasks.workunit.client.1.smithi193.stdout:6/599: write d0/d27/d81/df/d11/d36/d61/f6d [525030,49486] 0 2024-03-23T08:23:27.364 INFO:tasks.workunit.client.1.smithi193.stdout:5/739: rmdir d0/d1f 39 2024-03-23T08:23:27.367 INFO:tasks.workunit.client.1.smithi193.stdout:1/732: rename da/d13/d3f/d5a to da/d13/d24/dac/dea 0 2024-03-23T08:23:27.371 INFO:tasks.workunit.client.1.smithi193.stdout:5/740: fsync d0/d1c/f3a 0 2024-03-23T08:23:27.373 INFO:tasks.workunit.client.1.smithi193.stdout:7/698: dread d6/f32 [4194304,4194304] 0 2024-03-23T08:23:27.373 INFO:tasks.workunit.client.1.smithi193.stdout:1/733: dread - da/d13/f94 zero size 2024-03-23T08:23:27.374 INFO:tasks.workunit.client.1.smithi193.stdout:4/772: creat d11/d1a/d33/d42/db2/dbe/d50/d49/f108 x:0 0 0 2024-03-23T08:23:27.375 INFO:tasks.workunit.client.1.smithi193.stdout:0/690: mkdir db/d11/d14/d21/d33/d69/d9b/df2 0 2024-03-23T08:23:27.377 INFO:tasks.workunit.client.1.smithi193.stdout:2/770: dwrite d0/d1b/d14/da1/dd5/d72/fcd [0,4194304] 0 2024-03-23T08:23:27.377 INFO:tasks.workunit.client.1.smithi193.stdout:9/753: truncate d2/d7/db/d32/d6b/f6c 877071 0 2024-03-23T08:23:27.384 INFO:tasks.workunit.client.1.smithi193.stdout:8/732: dwrite df/d4b/d4d/f22 [0,4194304] 0 2024-03-23T08:23:27.385 INFO:tasks.workunit.client.1.smithi193.stdout:7/699: chown d6/d9/d1f/d23/d33/d38/d66/fa3 28 1 2024-03-23T08:23:27.386 INFO:tasks.workunit.client.1.smithi193.stdout:5/741: rename d0/d1c/d7a/f7d to d0/d1c/d7a/db1/d72/d99/dd8/f108 0 2024-03-23T08:23:27.387 INFO:tasks.workunit.client.1.smithi193.stdout:0/691: write db/d11/d14/d24/d2a/fbe [120041,90098] 0 2024-03-23T08:23:27.389 INFO:tasks.workunit.client.1.smithi193.stdout:2/771: fsync d0/d1b/d14/d66/d50/d92/da9/fab 0 2024-03-23T08:23:27.391 INFO:tasks.workunit.client.1.smithi193.stdout:3/614: dwrite d1/de/d13/d21/d50/d6f/fa7 [0,4194304] 0 2024-03-23T08:23:27.393 INFO:tasks.workunit.client.1.smithi193.stdout:9/754: symlink d2/d7/db/d1b/d43/d71/db4/d78/lf9 0 2024-03-23T08:23:27.394 INFO:tasks.workunit.client.1.smithi193.stdout:7/700: readlink d6/d9/d1f/d23/d33/d3c/le6 0 2024-03-23T08:23:27.402 INFO:tasks.workunit.client.1.smithi193.stdout:6/600: dread d0/d27/d81/df/d11/d36/f44 [0,4194304] 0 2024-03-23T08:23:27.402 INFO:tasks.workunit.client.1.smithi193.stdout:1/734: dwrite da/fa8 [0,4194304] 0 2024-03-23T08:23:27.404 INFO:tasks.workunit.client.1.smithi193.stdout:0/692: mkdir db/d52/db8/dcf/df3 0 2024-03-23T08:23:27.406 INFO:tasks.workunit.client.1.smithi193.stdout:4/773: dwrite d11/d1a/d33/d42/db2/dbe/d50/d49/f108 [0,4194304] 0 2024-03-23T08:23:27.407 INFO:tasks.workunit.client.1.smithi193.stdout:8/733: dread df/d4b/d4d/d9c/fe2 [0,4194304] 0 2024-03-23T08:23:27.407 INFO:tasks.workunit.client.1.smithi193.stdout:9/755: mknod d2/d7/db/d32/cfa 0 2024-03-23T08:23:27.409 INFO:tasks.workunit.client.1.smithi193.stdout:7/701: mknod d6/d9/d1f/d4d/d7c/cf8 0 2024-03-23T08:23:27.415 INFO:tasks.workunit.client.1.smithi193.stdout:0/693: chown db/d11/d14/d24/d47/f61 1 1 2024-03-23T08:23:27.416 INFO:tasks.workunit.client.1.smithi193.stdout:7/702: fdatasync d6/d9/d1f/d23/f45 0 2024-03-23T08:23:27.418 INFO:tasks.workunit.client.1.smithi193.stdout:9/756: write d2/d7/db/d1b/d43/d71/db4/dec/d3c/f91 [282109,130711] 0 2024-03-23T08:23:27.421 INFO:tasks.workunit.client.1.smithi193.stdout:2/772: dwrite d0/d1b/d14/d25/f5a [0,4194304] 0 2024-03-23T08:23:27.421 INFO:tasks.workunit.client.1.smithi193.stdout:7/703: chown d6/d9/d1f/d23/d33/d38/f53 69372669 1 2024-03-23T08:23:27.422 INFO:tasks.workunit.client.1.smithi193.stdout:7/704: readlink d6/d9/d1f/daf/lce 0 2024-03-23T08:23:27.423 INFO:tasks.workunit.client.1.smithi193.stdout:2/773: truncate d0/d1b/d77/d9b/fbc 351580 0 2024-03-23T08:23:27.428 INFO:tasks.workunit.client.1.smithi193.stdout:4/774: dread d11/d1a/d33/d42/db2/dbe/d50/f61 [0,4194304] 0 2024-03-23T08:23:27.429 INFO:tasks.workunit.client.1.smithi193.stdout:4/775: chown d11/d1a/d33/d42/d52/f87 31 1 2024-03-23T08:23:27.445 INFO:tasks.workunit.client.1.smithi193.stdout:5/742: dread d0/f2a [0,4194304] 0 2024-03-23T08:23:27.471 INFO:tasks.workunit.client.1.smithi193.stdout:0/694: write db/d11/d14/d21/f41 [1672123,45892] 0 2024-03-23T08:23:27.473 INFO:tasks.workunit.client.1.smithi193.stdout:0/695: dread - db/d11/d14/d24/f97 zero size 2024-03-23T08:23:27.475 INFO:tasks.workunit.client.1.smithi193.stdout:9/757: mknod d2/d7/db/d17/d4b/d59/cfb 0 2024-03-23T08:23:27.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/774: fdatasync d0/d1b/d19/f5c 0 2024-03-23T08:23:27.478 INFO:tasks.workunit.client.1.smithi193.stdout:4/776: mkdir d11/d2d/da8/d109 0 2024-03-23T08:23:27.484 INFO:tasks.workunit.client.1.smithi193.stdout:8/734: mkdir df/d63/d84/df4/df7 0 2024-03-23T08:23:27.486 INFO:tasks.workunit.client.1.smithi193.stdout:5/743: dwrite d0/d1f/d93/feb [0,4194304] 0 2024-03-23T08:23:27.486 INFO:tasks.workunit.client.1.smithi193.stdout:8/735: stat df/dc6/d8b/fc0 0 2024-03-23T08:23:27.487 INFO:tasks.workunit.client.1.smithi193.stdout:8/736: readlink df/dc6/d8b/dae/l5e 0 2024-03-23T08:23:27.490 INFO:tasks.workunit.client.1.smithi193.stdout:8/737: write df/d10/f3b [1384684,113581] 0 2024-03-23T08:23:27.492 INFO:tasks.workunit.client.1.smithi193.stdout:8/738: fdatasync df/d4b/fe4 0 2024-03-23T08:23:27.493 INFO:tasks.workunit.client.1.smithi193.stdout:9/758: mknod d2/d7/db/d1b/d5c/cfc 0 2024-03-23T08:23:27.493 INFO:tasks.workunit.client.1.smithi193.stdout:1/735: rename da/dd/df/d7e/d7f/db2/fc1 to da/dd/df/d7e/d7f/db2/ddb/feb 0 2024-03-23T08:23:27.495 INFO:tasks.workunit.client.1.smithi193.stdout:4/777: mkdir d11/d1a/d33/dce/d10a 0 2024-03-23T08:23:27.497 INFO:tasks.workunit.client.1.smithi193.stdout:5/744: dwrite d0/d1c/f3a [0,4194304] 0 2024-03-23T08:23:27.502 INFO:tasks.workunit.client.1.smithi193.stdout:2/775: mknod d0/d1b/d14/d56/da3/db0/dc6/cd8 0 2024-03-23T08:23:27.507 INFO:tasks.workunit.client.1.smithi193.stdout:8/739: dwrite df/d10/d67/f7d [0,4194304] 0 2024-03-23T08:23:27.513 INFO:tasks.workunit.client.1.smithi193.stdout:2/776: symlink d0/d1b/d14/d56/da3/db0/ld9 0 2024-03-23T08:23:27.513 INFO:tasks.workunit.client.1.smithi193.stdout:9/759: dwrite d2/d7/db/d7a/f87 [0,4194304] 0 2024-03-23T08:23:27.515 INFO:tasks.workunit.client.1.smithi193.stdout:4/778: symlink d11/d1a/d33/d42/db2/dbe/d50/dfb/l10b 0 2024-03-23T08:23:27.516 INFO:tasks.workunit.client.1.smithi193.stdout:8/740: truncate df/d63/d84/fd8 4746519 0 2024-03-23T08:23:27.519 INFO:tasks.workunit.client.1.smithi193.stdout:8/741: read df/d10/d67/f7d [3100550,52733] 0 2024-03-23T08:23:27.520 INFO:tasks.workunit.client.1.smithi193.stdout:2/777: mknod d0/d1b/d14/d56/da3/cda 0 2024-03-23T08:23:27.522 INFO:tasks.workunit.client.1.smithi193.stdout:2/778: chown d0/d1b/d14/d66/d50/d92/da9/dba/fc7 195 1 2024-03-23T08:23:27.524 INFO:tasks.workunit.client.1.smithi193.stdout:9/760: link d2/d7/db/d32/f4e d2/d7/db/d17/d33/db2/ffd 0 2024-03-23T08:23:27.526 INFO:tasks.workunit.client.1.smithi193.stdout:7/705: sync 2024-03-23T08:23:27.528 INFO:tasks.workunit.client.1.smithi193.stdout:2/779: mknod d0/d1b/d14/da1/cdb 0 2024-03-23T08:23:27.530 INFO:tasks.workunit.client.1.smithi193.stdout:9/761: mkdir d2/d7/db/d7a/d95/da9/dfe 0 2024-03-23T08:23:27.533 INFO:tasks.workunit.client.1.smithi193.stdout:7/706: write d6/d9/f19 [1480649,98743] 0 2024-03-23T08:23:27.534 INFO:tasks.workunit.client.1.smithi193.stdout:2/780: unlink d0/d1b/d77/lb1 0 2024-03-23T08:23:27.535 INFO:tasks.workunit.client.1.smithi193.stdout:9/762: mkdir d2/d7/dff 0 2024-03-23T08:23:27.537 INFO:tasks.workunit.client.1.smithi193.stdout:9/763: write d2/d7/db/d1b/d43/d71/fc7 [731081,88168] 0 2024-03-23T08:23:27.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/781: mkdir d0/d1b/d14/d56/d7c/ddc 0 2024-03-23T08:23:27.542 INFO:tasks.workunit.client.1.smithi193.stdout:2/782: read d0/d1b/d3b/f58 [2713343,125477] 0 2024-03-23T08:23:27.543 INFO:tasks.workunit.client.1.smithi193.stdout:9/764: unlink d2/d7/db/c41 0 2024-03-23T08:23:27.546 INFO:tasks.workunit.client.1.smithi193.stdout:9/765: chown d2/d7/cca 1494 1 2024-03-23T08:23:27.550 INFO:tasks.workunit.client.1.smithi193.stdout:2/783: dwrite d0/d1b/d14/d56/da3/fb4 [0,4194304] 0 2024-03-23T08:23:27.552 INFO:tasks.workunit.client.1.smithi193.stdout:4/779: dread f9 [0,4194304] 0 2024-03-23T08:23:27.554 INFO:tasks.workunit.client.1.smithi193.stdout:9/766: dread d2/d7/db/d32/d6b/f6c [0,4194304] 0 2024-03-23T08:23:27.555 INFO:tasks.workunit.client.1.smithi193.stdout:9/767: chown d2/d7/db/d17/d33/f64 1824678 1 2024-03-23T08:23:27.556 INFO:tasks.workunit.client.1.smithi193.stdout:4/780: write d11/d1a/d33/d7f/de0/fa0 [133579,94488] 0 2024-03-23T08:23:27.561 INFO:tasks.workunit.client.1.smithi193.stdout:9/768: write d2/d7/f36 [8527232,6452] 0 2024-03-23T08:23:27.562 INFO:tasks.workunit.client.1.smithi193.stdout:3/615: truncate d1/de/d13/f24 1262752 0 2024-03-23T08:23:27.562 INFO:tasks.workunit.client.1.smithi193.stdout:9/769: write d2/d7/f8e [1041449,37396] 0 2024-03-23T08:23:27.565 INFO:tasks.workunit.client.1.smithi193.stdout:9/770: write d2/d7/db/d32/f4e [2844117,75167] 0 2024-03-23T08:23:27.567 INFO:tasks.workunit.client.1.smithi193.stdout:6/601: truncate d0/d27/d81/f86 2251319 0 2024-03-23T08:23:27.570 INFO:tasks.workunit.client.1.smithi193.stdout:9/771: write d2/d7/db/d1b/f30 [290532,33789] 0 2024-03-23T08:23:27.570 INFO:tasks.workunit.client.1.smithi193.stdout:6/602: chown d0/d27/d81/df/c59 18 1 2024-03-23T08:23:27.571 INFO:tasks.workunit.client.1.smithi193.stdout:3/616: dwrite d1/de/d13/d21/d25/d2b/d3c/faa [0,4194304] 0 2024-03-23T08:23:27.573 INFO:tasks.workunit.client.1.smithi193.stdout:9/772: write d2/d7/db/d17/f53 [2513244,113401] 0 2024-03-23T08:23:27.575 INFO:tasks.workunit.client.1.smithi193.stdout:3/617: dread - d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/fb5 zero size 2024-03-23T08:23:27.578 INFO:tasks.workunit.client.1.smithi193.stdout:9/773: symlink d2/d7/db/d7a/d95/l100 0 2024-03-23T08:23:27.580 INFO:tasks.workunit.client.1.smithi193.stdout:6/603: unlink d0/d27/d81/df/d11/d36/d61/d12/d21/f73 0 2024-03-23T08:23:27.580 INFO:tasks.workunit.client.1.smithi193.stdout:0/696: truncate db/d11/d14/d21/f22 158194 0 2024-03-23T08:23:27.580 INFO:tasks.workunit.client.1.smithi193.stdout:9/774: fsync d2/d7/db/d1b/d43/f80 0 2024-03-23T08:23:27.582 INFO:tasks.workunit.client.1.smithi193.stdout:6/604: chown d0/d27/d81/df/d11/d36/d61/d12/d21/l43 2949 1 2024-03-23T08:23:27.583 INFO:tasks.workunit.client.1.smithi193.stdout:0/697: mknod db/d11/d14/d21/d33/d69/d9b/dc1/cf4 0 2024-03-23T08:23:27.584 INFO:tasks.workunit.client.1.smithi193.stdout:0/698: chown db/d11/d14/d24/d2a/d4d/f81 4 1 2024-03-23T08:23:27.584 INFO:tasks.workunit.client.1.smithi193.stdout:6/605: mknod d0/d27/d81/df/d11/d36/d61/d7c/cbd 0 2024-03-23T08:23:27.587 INFO:tasks.workunit.client.1.smithi193.stdout:6/606: truncate d0/d27/d81/df/d11/d56/f8d 1855619 0 2024-03-23T08:23:27.588 INFO:tasks.workunit.client.1.smithi193.stdout:9/775: getdents d2/d7/db/d1b/d5c 0 2024-03-23T08:23:27.589 INFO:tasks.workunit.client.1.smithi193.stdout:1/736: truncate da/f3c 3316940 0 2024-03-23T08:23:27.591 INFO:tasks.workunit.client.1.smithi193.stdout:6/607: creat d0/d27/d81/df/d11/d3b/d6c/fbe x:0 0 0 2024-03-23T08:23:27.592 INFO:tasks.workunit.client.1.smithi193.stdout:9/776: fdatasync d2/d7/db/d32/f3f 0 2024-03-23T08:23:27.592 INFO:tasks.workunit.client.1.smithi193.stdout:0/699: dwrite db/d11/d14/d24/d2a/d4d/f50 [0,4194304] 0 2024-03-23T08:23:27.593 INFO:tasks.workunit.client.1.smithi193.stdout:1/737: rmdir da/d13/d24/dac/dea 39 2024-03-23T08:23:27.595 INFO:tasks.workunit.client.1.smithi193.stdout:1/738: dread - da/dd/df/d7e/d86/fb3 zero size 2024-03-23T08:23:27.597 INFO:tasks.workunit.client.1.smithi193.stdout:5/745: truncate d0/d1f/f51 1105468 0 2024-03-23T08:23:27.597 INFO:tasks.workunit.client.1.smithi193.stdout:9/777: creat d2/d7/db/d7a/f101 x:0 0 0 2024-03-23T08:23:27.597 INFO:tasks.workunit.client.1.smithi193.stdout:0/700: readlink db/d11/d14/d21/l8a 0 2024-03-23T08:23:27.598 INFO:tasks.workunit.client.1.smithi193.stdout:9/778: chown d2/d7/db/d1b/d43/d71/db4/d78/lf9 3609 1 2024-03-23T08:23:27.599 INFO:tasks.workunit.client.1.smithi193.stdout:1/739: readlink da/d13/d24/dac/dea/d76/d77/lc0 0 2024-03-23T08:23:27.600 INFO:tasks.workunit.client.1.smithi193.stdout:2/784: rename d0/d1b/d19/f7f to d0/d1b/d14/d56/fdd 0 2024-03-23T08:23:27.602 INFO:tasks.workunit.client.1.smithi193.stdout:0/701: fdatasync db/d11/d14/d24/d47/d59/fd7 0 2024-03-23T08:23:27.602 INFO:tasks.workunit.client.1.smithi193.stdout:5/746: dread d0/d1c/d7a/db1/d72/d73/d45/fa0 [0,4194304] 0 2024-03-23T08:23:27.603 INFO:tasks.workunit.client.1.smithi193.stdout:1/740: chown da/dd/df/l27 2985287 1 2024-03-23T08:23:27.604 INFO:tasks.workunit.client.1.smithi193.stdout:6/608: getdents d0/d27/d81/df/d11/d36 0 2024-03-23T08:23:27.604 INFO:tasks.workunit.client.1.smithi193.stdout:8/742: write df/d4b/d4d/d2a/f2b [1984979,111990] 0 2024-03-23T08:23:27.606 INFO:tasks.workunit.client.1.smithi193.stdout:2/785: mknod d0/d1b/d14/d25/dc2/cde 0 2024-03-23T08:23:27.608 INFO:tasks.workunit.client.1.smithi193.stdout:0/702: write db/d11/d14/d21/d33/d48/d85/fa7 [769848,24782] 0 2024-03-23T08:23:27.608 INFO:tasks.workunit.client.1.smithi193.stdout:5/747: creat d0/d1c/df7/f109 x:0 0 0 2024-03-23T08:23:27.612 INFO:tasks.workunit.client.1.smithi193.stdout:1/741: creat da/d13/d24/dac/dea/d76/fec x:0 0 0 2024-03-23T08:23:27.612 INFO:tasks.workunit.client.1.smithi193.stdout:4/781: rmdir d11/d1a/d33/d44 39 2024-03-23T08:23:27.612 INFO:tasks.workunit.client.1.smithi193.stdout:5/748: fsync d0/d1c/d7a/db1/d72/d73/d57/d6d/d8e/fa5 0 2024-03-23T08:23:27.614 INFO:tasks.workunit.client.1.smithi193.stdout:9/779: dwrite d2/d7/db/d1b/d43/d71/db4/dec/d5f/f83 [0,4194304] 0 2024-03-23T08:23:27.614 INFO:tasks.workunit.client.1.smithi193.stdout:2/786: truncate d0/d1b/d14/d56/f96 814888 0 2024-03-23T08:23:27.614 INFO:tasks.workunit.client.1.smithi193.stdout:6/609: creat d0/d27/d81/df/d11/d36/d61/d7c/fbf x:0 0 0 2024-03-23T08:23:27.615 INFO:tasks.workunit.client.1.smithi193.stdout:0/703: stat db/d11/d14/d24/d2a/de3/ddb/cdf 0 2024-03-23T08:23:27.616 INFO:tasks.workunit.client.1.smithi193.stdout:7/707: creat d6/d9/d1f/d23/d33/d38/ff9 x:0 0 0 2024-03-23T08:23:27.616 INFO:tasks.workunit.client.1.smithi193.stdout:8/743: getdents df/d63 0 2024-03-23T08:23:27.619 INFO:tasks.workunit.client.1.smithi193.stdout:4/782: unlink d11/d2d/d58/d80/d9d/lb6 0 2024-03-23T08:23:27.622 INFO:tasks.workunit.client.1.smithi193.stdout:6/610: truncate d0/d27/d72/fac 181185 0 2024-03-23T08:23:27.622 INFO:tasks.workunit.client.1.smithi193.stdout:7/708: chown d6/d9/d1f/d3e/lb9 526120 1 2024-03-23T08:23:27.623 INFO:tasks.workunit.client.1.smithi193.stdout:8/744: creat df/d63/d84/da8/ff8 x:0 0 0 2024-03-23T08:23:27.630 INFO:tasks.workunit.client.1.smithi193.stdout:5/749: rename d0/d1c/d7a/fed to d0/f10a 0 2024-03-23T08:23:27.632 INFO:tasks.workunit.client.1.smithi193.stdout:1/742: dwrite da/dd/df/d7c/fca [0,4194304] 0 2024-03-23T08:23:27.636 INFO:tasks.workunit.client.1.smithi193.stdout:0/704: dwrite db/f1e [4194304,4194304] 0 2024-03-23T08:23:27.642 INFO:tasks.workunit.client.1.smithi193.stdout:9/780: rename d2/d7/db/d32/fd6 to d2/d7/db/d17/d4b/d59/d75/f102 0 2024-03-23T08:23:27.642 INFO:tasks.workunit.client.1.smithi193.stdout:7/709: truncate d6/d9/d1f/d23/d33/d38/d66/fa3 554650 0 2024-03-23T08:23:27.643 INFO:tasks.workunit.client.1.smithi193.stdout:2/787: dwrite d0/d1b/fd3 [4194304,4194304] 0 2024-03-23T08:23:27.646 INFO:tasks.workunit.client.1.smithi193.stdout:0/705: creat db/d11/d14/d51/ff5 x:0 0 0 2024-03-23T08:23:27.647 INFO:tasks.workunit.client.1.smithi193.stdout:5/750: mknod d0/d1c/d3b/d4b/df9/c10b 0 2024-03-23T08:23:27.648 INFO:tasks.workunit.client.1.smithi193.stdout:8/745: dwrite df/d10/d3e/d54/dd2/fa5 [0,4194304] 0 2024-03-23T08:23:27.650 INFO:tasks.workunit.client.1.smithi193.stdout:5/751: dread - d0/f10a zero size 2024-03-23T08:23:27.653 INFO:tasks.workunit.client.1.smithi193.stdout:0/706: chown db/d11/d14/d24/d2a/d4b/f84 1814656 1 2024-03-23T08:23:27.653 INFO:tasks.workunit.client.1.smithi193.stdout:9/781: symlink d2/d7/db/d1b/d5c/l103 0 2024-03-23T08:23:27.656 INFO:tasks.workunit.client.1.smithi193.stdout:6/611: getdents d0/d27/d81/df/d11/d3b/d6c 0 2024-03-23T08:23:27.658 INFO:tasks.workunit.client.1.smithi193.stdout:2/788: symlink d0/d28/ldf 0 2024-03-23T08:23:27.662 INFO:tasks.workunit.client.1.smithi193.stdout:7/710: dwrite d6/d9/d1f/d4d/fa9 [0,4194304] 0 2024-03-23T08:23:27.670 INFO:tasks.workunit.client.1.smithi193.stdout:0/707: creat db/d11/d14/d21/d33/d69/d9b/df2/ff6 x:0 0 0 2024-03-23T08:23:27.674 INFO:tasks.workunit.client.1.smithi193.stdout:8/746: rename df/d4b/d4d/l61 to df/d10/d3e/lf9 0 2024-03-23T08:23:27.676 INFO:tasks.workunit.client.1.smithi193.stdout:9/782: rmdir d2/d7/db/d7a/d95/da9/dfe 0 2024-03-23T08:23:27.676 INFO:tasks.workunit.client.1.smithi193.stdout:5/752: dwrite d0/d1c/d7a/db1/d72/d73/d45/fdc [0,4194304] 0 2024-03-23T08:23:27.679 INFO:tasks.workunit.client.1.smithi193.stdout:4/783: dwrite d11/d1a/d33/d42/f28 [0,4194304] 0 2024-03-23T08:23:27.679 INFO:tasks.workunit.client.1.smithi193.stdout:8/747: chown df/f86 9428015 1 2024-03-23T08:23:27.680 INFO:tasks.workunit.client.1.smithi193.stdout:1/743: dwrite da/d13/d3f/f87 [0,4194304] 0 2024-03-23T08:23:27.686 INFO:tasks.workunit.client.1.smithi193.stdout:6/612: link d0/d27/d81/df/d11/d36/d61/l22 d0/d27/d81/df/d11/d3b/d6c/lc0 0 2024-03-23T08:23:27.690 INFO:tasks.workunit.client.1.smithi193.stdout:2/789: rename d0/d1b/c3e to d0/d1b/d77/d9b/ce0 0 2024-03-23T08:23:27.696 INFO:tasks.workunit.client.1.smithi193.stdout:5/753: rmdir d0/d1c/d7a/db1/d72/d73/d45/daa 39 2024-03-23T08:23:27.699 INFO:tasks.workunit.client.1.smithi193.stdout:2/790: dwrite d0/d37/f8a [0,4194304] 0 2024-03-23T08:23:27.699 INFO:tasks.workunit.client.1.smithi193.stdout:5/754: dread - d0/d1c/d3b/d81/df2/d97/fad zero size 2024-03-23T08:23:27.700 INFO:tasks.workunit.client.1.smithi193.stdout:5/755: readlink d0/d1c/d7a/db1/d72/l7e 0 2024-03-23T08:23:27.707 INFO:tasks.workunit.client.1.smithi193.stdout:7/711: truncate d6/d9/d1f/d23/d33/d38/d46/f5b 3329835 0 2024-03-23T08:23:27.709 INFO:tasks.workunit.client.1.smithi193.stdout:4/784: truncate d11/d1a/d33/f7d 253838 0 2024-03-23T08:23:27.710 INFO:tasks.workunit.client.1.smithi193.stdout:7/712: write d6/d9/d1f/fa4 [234588,68826] 0 2024-03-23T08:23:27.712 INFO:tasks.workunit.client.1.smithi193.stdout:1/744: creat da/d13/d24/dac/fed x:0 0 0 2024-03-23T08:23:27.715 INFO:tasks.workunit.client.1.smithi193.stdout:6/613: link d0/d27/d81/df/l5e d0/d27/d81/df/d11/d36/d61/d3a/d8f/lc1 0 2024-03-23T08:23:27.715 INFO:tasks.workunit.client.1.smithi193.stdout:8/748: truncate df/d10/d3e/d54/dd2/fa5 4086753 0 2024-03-23T08:23:27.716 INFO:tasks.workunit.client.1.smithi193.stdout:0/708: dwrite db/d11/d14/d21/f49 [0,4194304] 0 2024-03-23T08:23:27.720 INFO:tasks.workunit.client.1.smithi193.stdout:9/783: getdents d2/d7/db/d1b/d43/d71/db4/dec/d5f 0 2024-03-23T08:23:27.722 INFO:tasks.workunit.client.1.smithi193.stdout:6/614: read d0/d27/d81/df/d11/d3b/f49 [461380,38413] 0 2024-03-23T08:23:27.722 INFO:tasks.workunit.client.1.smithi193.stdout:7/713: write d6/d9/d1f/d23/d33/d3c/f4b [598239,64259] 0 2024-03-23T08:23:27.724 INFO:tasks.workunit.client.1.smithi193.stdout:5/756: getdents d0/d1c/d7a 0 2024-03-23T08:23:27.725 INFO:tasks.workunit.client.1.smithi193.stdout:2/791: dwrite d0/d1b/d14/d25/f29 [0,4194304] 0 2024-03-23T08:23:27.734 INFO:tasks.workunit.client.1.smithi193.stdout:4/785: getdents d11/d27 0 2024-03-23T08:23:27.734 INFO:tasks.workunit.client.1.smithi193.stdout:9/784: rename d2/d7/db/d32/d6b/dde/df3 to d2/d7/db/d17/d4b/d59/d97/d104 0 2024-03-23T08:23:27.735 INFO:tasks.workunit.client.1.smithi193.stdout:6/615: write d0/d27/d81/df/d11/d3b/d51/d9c/f8a [294792,44697] 0 2024-03-23T08:23:27.736 INFO:tasks.workunit.client.1.smithi193.stdout:5/757: creat d0/d1c/d7a/db1/d72/d99/dd8/f10c x:0 0 0 2024-03-23T08:23:27.737 INFO:tasks.workunit.client.1.smithi193.stdout:8/749: creat df/dc6/d8b/ffa x:0 0 0 2024-03-23T08:23:27.738 INFO:tasks.workunit.client.1.smithi193.stdout:0/709: dread db/d11/d14/d24/d47/d59/f79 [4194304,4194304] 0 2024-03-23T08:23:27.738 INFO:tasks.workunit.client.1.smithi193.stdout:4/786: fsync d11/d1a/d33/d8f/db8/f2b 0 2024-03-23T08:23:27.740 INFO:tasks.workunit.client.1.smithi193.stdout:7/714: dread d6/d9/d1f/d23/d33/d38/d3b/f55 [0,4194304] 0 2024-03-23T08:23:27.741 INFO:tasks.workunit.client.1.smithi193.stdout:1/745: truncate da/d13/f90 2468918 0 2024-03-23T08:23:27.742 INFO:tasks.workunit.client.1.smithi193.stdout:9/785: symlink d2/d7/db/l105 0 2024-03-23T08:23:27.744 INFO:tasks.workunit.client.1.smithi193.stdout:2/792: getdents d0/d1b/d14/d56/da3 0 2024-03-23T08:23:27.746 INFO:tasks.workunit.client.1.smithi193.stdout:5/758: unlink d0/d1c/d7a/db1/d72/d73/d45/d6e/lac 0 2024-03-23T08:23:27.746 INFO:tasks.workunit.client.1.smithi193.stdout:8/750: mknod df/d63/d84/df0/cfb 0 2024-03-23T08:23:27.748 INFO:tasks.workunit.client.1.smithi193.stdout:7/715: mkdir d6/d9/d1f/d23/d33/d38/d66/dac/dfa 0 2024-03-23T08:23:27.750 INFO:tasks.workunit.client.1.smithi193.stdout:9/786: dread - d2/d7/db/d17/d4b/d8b/fda zero size 2024-03-23T08:23:27.754 INFO:tasks.workunit.client.1.smithi193.stdout:8/751: write df/d10/d3e/d54/f85 [733126,94915] 0 2024-03-23T08:23:27.759 INFO:tasks.workunit.client.1.smithi193.stdout:0/710: dwrite db/d52/db8/dcf/fd6 [0,4194304] 0 2024-03-23T08:23:27.760 INFO:tasks.workunit.client.1.smithi193.stdout:3/618: sync 2024-03-23T08:23:27.760 INFO:tasks.workunit.client.1.smithi193.stdout:6/616: dwrite d0/d27/d81/df/d11/d36/d61/d3a/f45 [4194304,4194304] 0 2024-03-23T08:23:27.761 INFO:tasks.workunit.client.1.smithi193.stdout:5/759: symlink d0/d1c/d3b/d4b/dff/l10d 0 2024-03-23T08:23:27.762 INFO:tasks.workunit.client.1.smithi193.stdout:4/787: dwrite d11/d1a/d33/f39 [0,4194304] 0 2024-03-23T08:23:27.763 INFO:tasks.workunit.client.1.smithi193.stdout:7/716: truncate d6/d9/d1f/d23/d69/fd0 1511711 0 2024-03-23T08:23:27.764 INFO:tasks.workunit.client.1.smithi193.stdout:1/746: write da/d13/d24/dac/dea/f65 [120634,93388] 0 2024-03-23T08:23:27.764 INFO:tasks.workunit.client.1.smithi193.stdout:9/787: creat d2/d7/db/d1b/d5c/f106 x:0 0 0 2024-03-23T08:23:27.767 INFO:tasks.workunit.client.1.smithi193.stdout:5/760: read - d0/d1f/d93/dcd/ffe zero size 2024-03-23T08:23:27.769 INFO:tasks.workunit.client.1.smithi193.stdout:0/711: chown db/d11/d14/d21/f2c 27516 1 2024-03-23T08:23:27.770 INFO:tasks.workunit.client.1.smithi193.stdout:8/752: rename df/d63/f81 to df/d4b/d4d/d9c/df2/ffc 0 2024-03-23T08:23:27.771 INFO:tasks.workunit.client.1.smithi193.stdout:1/747: read da/f59 [3540720,102131] 0 2024-03-23T08:23:27.771 INFO:tasks.workunit.client.1.smithi193.stdout:4/788: mknod d11/d2d/d3a/c10c 0 2024-03-23T08:23:27.772 INFO:tasks.workunit.client.1.smithi193.stdout:2/793: dread d0/d1b/d14/d25/f29 [0,4194304] 0 2024-03-23T08:23:27.774 INFO:tasks.workunit.client.1.smithi193.stdout:6/617: mknod d0/d27/d81/df/d11/d56/cc2 0 2024-03-23T08:23:27.775 INFO:tasks.workunit.client.1.smithi193.stdout:7/717: creat d6/d9/d1f/d5f/ffb x:0 0 0 2024-03-23T08:23:27.776 INFO:tasks.workunit.client.1.smithi193.stdout:5/761: creat d0/d1c/d3b/d4b/f10e x:0 0 0 2024-03-23T08:23:27.777 INFO:tasks.workunit.client.1.smithi193.stdout:9/788: mknod d2/d7/db/d17/d33/d70/c107 0 2024-03-23T08:23:27.778 INFO:tasks.workunit.client.1.smithi193.stdout:9/789: read d2/f8f [120826,27484] 0 2024-03-23T08:23:27.779 INFO:tasks.workunit.client.1.smithi193.stdout:9/790: fsync d2/d7/db/d1b/d5c/fed 0 2024-03-23T08:23:27.780 INFO:tasks.workunit.client.1.smithi193.stdout:8/753: unlink df/d10/d67/d7f/fec 0 2024-03-23T08:23:27.781 INFO:tasks.workunit.client.1.smithi193.stdout:8/754: write df/d63/d84/dc5/fd3 [73263,2495] 0 2024-03-23T08:23:27.781 INFO:tasks.workunit.client.1.smithi193.stdout:3/619: mkdir d1/de/d13/d21/d27/daf/dd1 0 2024-03-23T08:23:27.782 INFO:tasks.workunit.client.1.smithi193.stdout:8/755: fsync df/d10/d67/fa9 0 2024-03-23T08:23:27.783 INFO:tasks.workunit.client.1.smithi193.stdout:4/789: creat d11/d8e/dac/f10d x:0 0 0 2024-03-23T08:23:27.783 INFO:tasks.workunit.client.1.smithi193.stdout:2/794: symlink d0/d1b/d14/da1/le1 0 2024-03-23T08:23:27.783 INFO:tasks.workunit.client.1.smithi193.stdout:1/748: mknod da/dd/df/d7e/d86/cee 0 2024-03-23T08:23:27.785 INFO:tasks.workunit.client.1.smithi193.stdout:7/718: creat d6/d9/d1f/d3e/d65/d70/ffc x:0 0 0 2024-03-23T08:23:27.785 INFO:tasks.workunit.client.1.smithi193.stdout:6/618: symlink d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/lc3 0 2024-03-23T08:23:27.785 INFO:tasks.workunit.client.1.smithi193.stdout:5/762: write d0/fe [582903,43275] 0 2024-03-23T08:23:27.788 INFO:tasks.workunit.client.1.smithi193.stdout:2/795: creat d0/d1b/d19/fe2 x:0 0 0 2024-03-23T08:23:27.789 INFO:tasks.workunit.client.1.smithi193.stdout:0/712: rmdir db/d11/d14/d24/ddd/df0 0 2024-03-23T08:23:27.791 INFO:tasks.workunit.client.1.smithi193.stdout:7/719: chown d6/d9/d1f/fa4 231339789 1 2024-03-23T08:23:27.792 INFO:tasks.workunit.client.1.smithi193.stdout:8/756: dread df/f21 [0,4194304] 0 2024-03-23T08:23:27.793 INFO:tasks.workunit.client.1.smithi193.stdout:1/749: truncate da/d13/d24/dac/fed 420262 0 2024-03-23T08:23:27.793 INFO:tasks.workunit.client.1.smithi193.stdout:9/791: creat d2/d7/db/f108 x:0 0 0 2024-03-23T08:23:27.794 INFO:tasks.workunit.client.1.smithi193.stdout:3/620: dwrite d1/de/d13/d21/f81 [0,4194304] 0 2024-03-23T08:23:27.797 INFO:tasks.workunit.client.1.smithi193.stdout:5/763: mknod d0/d1c/d3b/d4b/de2/c10f 0 2024-03-23T08:23:27.797 INFO:tasks.workunit.client.1.smithi193.stdout:8/757: mknod df/d63/cfd 0 2024-03-23T08:23:27.799 INFO:tasks.workunit.client.1.smithi193.stdout:6/619: dread d0/d27/d81/df/d11/f30 [0,4194304] 0 2024-03-23T08:23:27.799 INFO:tasks.workunit.client.1.smithi193.stdout:9/792: fsync d2/fc 0 2024-03-23T08:23:27.803 INFO:tasks.workunit.client.1.smithi193.stdout:9/793: chown d2/d7/db/d1b/l1c 655189 1 2024-03-23T08:23:27.804 INFO:tasks.workunit.client.1.smithi193.stdout:7/720: truncate d6/d9/fa 248990 0 2024-03-23T08:23:27.807 INFO:tasks.workunit.client.1.smithi193.stdout:9/794: unlink d2/d7/l16 0 2024-03-23T08:23:27.809 INFO:tasks.workunit.client.1.smithi193.stdout:5/764: mknod d0/d1c/d7a/db1/d72/d73/c110 0 2024-03-23T08:23:27.810 INFO:tasks.workunit.client.1.smithi193.stdout:3/621: write d1/de/d13/d21/d50/d44/f79 [4948615,86016] 0 2024-03-23T08:23:27.811 INFO:tasks.workunit.client.1.smithi193.stdout:8/758: dwrite df/d4b/d4d/f99 [0,4194304] 0 2024-03-23T08:23:27.815 INFO:tasks.workunit.client.1.smithi193.stdout:2/796: truncate d0/d28/fa8 1302017 0 2024-03-23T08:23:27.820 INFO:tasks.workunit.client.1.smithi193.stdout:8/759: creat df/d10/d67/ffe x:0 0 0 2024-03-23T08:23:27.820 INFO:tasks.workunit.client.1.smithi193.stdout:4/790: dread d11/d1a/d33/d42/d52/f6c [0,4194304] 0 2024-03-23T08:23:27.823 INFO:tasks.workunit.client.1.smithi193.stdout:2/797: unlink d0/d1b/f21 0 2024-03-23T08:23:27.823 INFO:tasks.workunit.client.1.smithi193.stdout:1/750: dread da/d13/d24/dac/dea/f8b [0,4194304] 0 2024-03-23T08:23:27.830 INFO:tasks.workunit.client.1.smithi193.stdout:7/721: truncate d6/d9/d1f/d23/d33/d38/d3b/f55 711326 0 2024-03-23T08:23:27.833 INFO:tasks.workunit.client.1.smithi193.stdout:2/798: write d0/d1b/d19/f80 [339583,56651] 0 2024-03-23T08:23:27.834 INFO:tasks.workunit.client.1.smithi193.stdout:9/795: dwrite d2/d7/db/d17/d33/db2/ffd [0,4194304] 0 2024-03-23T08:23:27.834 INFO:tasks.workunit.client.1.smithi193.stdout:0/713: dwrite db/d11/d14/d21/d33/d48/fc3 [0,4194304] 0 2024-03-23T08:23:27.835 INFO:tasks.workunit.client.1.smithi193.stdout:4/791: dread d11/d2d/d3a/fc1 [0,4194304] 0 2024-03-23T08:23:27.837 INFO:tasks.workunit.client.1.smithi193.stdout:3/622: dwrite d1/de/d13/d21/d50/fc8 [0,4194304] 0 2024-03-23T08:23:27.842 INFO:tasks.workunit.client.1.smithi193.stdout:9/796: write d2/d7/db/d1b/fbb [1070637,4638] 0 2024-03-23T08:23:27.846 INFO:tasks.workunit.client.1.smithi193.stdout:0/714: symlink db/d11/d14/d24/d2a/de3/lf7 0 2024-03-23T08:23:27.847 INFO:tasks.workunit.client.1.smithi193.stdout:7/722: symlink d6/d9/d1f/d23/d33/d38/d66/dac/dfa/lfd 0 2024-03-23T08:23:27.850 INFO:tasks.workunit.client.1.smithi193.stdout:5/765: dread d0/d1c/d7a/d86/f103 [0,4194304] 0 2024-03-23T08:23:27.853 INFO:tasks.workunit.client.1.smithi193.stdout:8/760: dwrite df/feb [0,4194304] 0 2024-03-23T08:23:27.853 INFO:tasks.workunit.client.1.smithi193.stdout:3/623: rename d1/de/d13/d96/d99/f2d to d1/de/d13/d21/d59/d60/fd2 0 2024-03-23T08:23:27.854 INFO:tasks.workunit.client.1.smithi193.stdout:1/751: link da/dd/df/d7e/d7f/cbb da/dd/df/d52/dce/cef 0 2024-03-23T08:23:27.861 INFO:tasks.workunit.client.1.smithi193.stdout:0/715: mknod db/d11/d14/d21/d33/d69/cf8 0 2024-03-23T08:23:27.861 INFO:tasks.workunit.client.1.smithi193.stdout:5/766: fsync d0/d1f/fc4 0 2024-03-23T08:23:27.862 INFO:tasks.workunit.client.1.smithi193.stdout:4/792: dwrite d11/d27/d85/f100 [0,4194304] 0 2024-03-23T08:23:27.862 INFO:tasks.workunit.client.1.smithi193.stdout:3/624: creat d1/de/fd3 x:0 0 0 2024-03-23T08:23:27.866 INFO:tasks.workunit.client.1.smithi193.stdout:1/752: rename da/l57 to da/dd/df/d2e/db4/lf0 0 2024-03-23T08:23:27.876 INFO:tasks.workunit.client.1.smithi193.stdout:9/797: getdents d2/d7/db/d17/d33 0 2024-03-23T08:23:27.877 INFO:tasks.workunit.client.1.smithi193.stdout:5/767: creat d0/d5a/dbf/f111 x:0 0 0 2024-03-23T08:23:27.879 INFO:tasks.workunit.client.1.smithi193.stdout:0/716: write db/d11/d14/d24/d2a/d4d/fa1 [554243,60142] 0 2024-03-23T08:23:27.880 INFO:tasks.workunit.client.1.smithi193.stdout:8/761: dwrite df/d63/d76/fe5 [0,4194304] 0 2024-03-23T08:23:27.886 INFO:tasks.workunit.client.1.smithi193.stdout:9/798: mkdir d2/d7/de5/d109 0 2024-03-23T08:23:27.887 INFO:tasks.workunit.client.1.smithi193.stdout:7/723: dwrite d6/d9/d1f/d23/d33/d38/f68 [0,4194304] 0 2024-03-23T08:23:27.887 INFO:tasks.workunit.client.1.smithi193.stdout:8/762: fsync df/d4b/d4d/f99 0 2024-03-23T08:23:27.888 INFO:tasks.workunit.client.1.smithi193.stdout:1/753: dread da/d13/d24/dac/dea/f65 [0,4194304] 0 2024-03-23T08:23:27.891 INFO:tasks.workunit.client.1.smithi193.stdout:3/625: dwrite d1/de/f89 [0,4194304] 0 2024-03-23T08:23:27.896 INFO:tasks.workunit.client.1.smithi193.stdout:7/724: unlink d6/d9/d1f/d23/d33/d38/d3b/ldc 0 2024-03-23T08:23:27.897 INFO:tasks.workunit.client.1.smithi193.stdout:5/768: getdents d0/d1c/d7a/db1/d72/d73/d57/d6d/dd9 0 2024-03-23T08:23:27.899 INFO:tasks.workunit.client.1.smithi193.stdout:0/717: dwrite db/d11/d14/d24/d2a/d4d/d93/fc5 [0,4194304] 0 2024-03-23T08:23:27.900 INFO:tasks.workunit.client.1.smithi193.stdout:4/793: dread d11/d1a/d33/d42/f29 [0,4194304] 0 2024-03-23T08:23:27.902 INFO:tasks.workunit.client.1.smithi193.stdout:4/794: chown d11/d2d/d58/d80/db9/ce3 1563023 1 2024-03-23T08:23:27.902 INFO:tasks.workunit.client.1.smithi193.stdout:4/795: stat d11 0 2024-03-23T08:23:27.903 INFO:tasks.workunit.client.1.smithi193.stdout:1/754: creat da/dd/df/d7e/d7f/db2/ff1 x:0 0 0 2024-03-23T08:23:27.904 INFO:tasks.workunit.client.1.smithi193.stdout:1/755: readlink da/d13/d24/l31 0 2024-03-23T08:23:27.905 INFO:tasks.workunit.client.1.smithi193.stdout:0/718: write db/d11/d14/d21/d33/d69/d9b/fce [214097,30926] 0 2024-03-23T08:23:27.905 INFO:tasks.workunit.client.1.smithi193.stdout:4/796: chown d11/d1a/d33/d42/db2/dbe/cda 463021 1 2024-03-23T08:23:27.905 INFO:tasks.workunit.client.1.smithi193.stdout:7/725: mknod d6/d9/d1f/d4d/d7c/d92/cfe 0 2024-03-23T08:23:27.911 INFO:tasks.workunit.client.1.smithi193.stdout:4/797: dread d11/d1a/d33/d42/d52/f6c [0,4194304] 0 2024-03-23T08:23:27.912 INFO:tasks.workunit.client.1.smithi193.stdout:0/719: symlink db/d11/d14/d21/d33/d69/lf9 0 2024-03-23T08:23:27.912 INFO:tasks.workunit.client.1.smithi193.stdout:7/726: dwrite d6/f14 [0,4194304] 0 2024-03-23T08:23:27.913 INFO:tasks.workunit.client.1.smithi193.stdout:4/798: stat d11/d2d/d58/d8b/dbc 0 2024-03-23T08:23:27.914 INFO:tasks.workunit.client.1.smithi193.stdout:3/626: truncate d1/de/d13/f33 1133414 0 2024-03-23T08:23:27.916 INFO:tasks.workunit.client.1.smithi193.stdout:4/799: write d11/d2d/d3a/f76 [3016620,74313] 0 2024-03-23T08:23:27.917 INFO:tasks.workunit.client.1.smithi193.stdout:3/627: mkdir d1/de/d13/d21/d59/dd4 0 2024-03-23T08:23:27.919 INFO:tasks.workunit.client.1.smithi193.stdout:0/720: symlink db/d11/d14/d24/d47/d59/dd4/lfa 0 2024-03-23T08:23:27.920 INFO:tasks.workunit.client.1.smithi193.stdout:4/800: rmdir d11/d2d/d58/d80 39 2024-03-23T08:23:27.923 INFO:tasks.workunit.client.1.smithi193.stdout:3/628: creat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fd5 x:0 0 0 2024-03-23T08:23:27.960 INFO:tasks.workunit.client.1.smithi193.stdout:6/620: sync 2024-03-23T08:23:27.962 INFO:tasks.workunit.client.1.smithi193.stdout:7/727: write d6/d9/d1f/d4d/d6b/f80 [386817,108664] 0 2024-03-23T08:23:27.965 INFO:tasks.workunit.client.1.smithi193.stdout:6/621: mknod d0/d27/cc4 0 2024-03-23T08:23:27.968 INFO:tasks.workunit.client.1.smithi193.stdout:4/801: dwrite d11/d95/f9a [0,4194304] 0 2024-03-23T08:23:27.969 INFO:tasks.workunit.client.1.smithi193.stdout:1/756: dwrite da/d13/d24/f45 [0,4194304] 0 2024-03-23T08:23:27.975 INFO:tasks.workunit.client.1.smithi193.stdout:6/622: creat d0/d27/d81/fc5 x:0 0 0 2024-03-23T08:23:27.975 INFO:tasks.workunit.client.1.smithi193.stdout:4/802: stat d11/d27/d85/de1/d99/ded/fef 0 2024-03-23T08:23:27.975 INFO:tasks.workunit.client.1.smithi193.stdout:1/757: read - da/dd/da9/fe8 zero size 2024-03-23T08:23:27.976 INFO:tasks.workunit.client.1.smithi193.stdout:3/629: dwrite d1/de/d13/d21/d25/d34/f47 [0,4194304] 0 2024-03-23T08:23:27.980 INFO:tasks.workunit.client.1.smithi193.stdout:1/758: unlink da/l88 0 2024-03-23T08:23:27.980 INFO:tasks.workunit.client.1.smithi193.stdout:6/623: read d0/d27/d81/df/d11/d3b/d6c/f90 [587248,125602] 0 2024-03-23T08:23:27.984 INFO:tasks.workunit.client.1.smithi193.stdout:4/803: rename d11/d1a/d33/d8f to d11/d2d/da8/d109/d10e 0 2024-03-23T08:23:27.986 INFO:tasks.workunit.client.1.smithi193.stdout:3/630: mknod d1/cd6 0 2024-03-23T08:23:27.987 INFO:tasks.workunit.client.1.smithi193.stdout:1/759: creat da/dd/df/d75/d98/ff2 x:0 0 0 2024-03-23T08:23:27.990 INFO:tasks.workunit.client.1.smithi193.stdout:3/631: stat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fbb 0 2024-03-23T08:23:27.990 INFO:tasks.workunit.client.1.smithi193.stdout:6/624: creat d0/fc6 x:0 0 0 2024-03-23T08:23:27.990 INFO:tasks.workunit.client.1.smithi193.stdout:1/760: write da/dd/da9/fe8 [542727,4730] 0 2024-03-23T08:23:27.992 INFO:tasks.workunit.client.1.smithi193.stdout:4/804: dwrite d11/d2d/d3a/f76 [4194304,4194304] 0 2024-03-23T08:23:27.993 INFO:tasks.workunit.client.1.smithi193.stdout:1/761: write da/d13/fc3 [635342,54100] 0 2024-03-23T08:23:27.994 INFO:tasks.workunit.client.1.smithi193.stdout:4/805: readlink d11/d8e/dac/lc9 0 2024-03-23T08:23:27.996 INFO:tasks.workunit.client.1.smithi193.stdout:7/728: truncate d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/df3/fa0 3531703 0 2024-03-23T08:23:27.998 INFO:tasks.workunit.client.1.smithi193.stdout:1/762: mkdir da/dd/df/d75/d98/df3 0 2024-03-23T08:23:27.999 INFO:tasks.workunit.client.1.smithi193.stdout:7/729: readlink d6/d9/d1f/d23/ld7 0 2024-03-23T08:23:27.999 INFO:tasks.workunit.client.1.smithi193.stdout:3/632: rename d1/de/d13/d21/d25/d34/da8/cc9 to d1/de/d13/d21/d50/d6f/d6e/cd7 0 2024-03-23T08:23:28.003 INFO:tasks.workunit.client.1.smithi193.stdout:1/763: creat da/dd/df/d52/d82/db0/ff4 x:0 0 0 2024-03-23T08:23:28.003 INFO:tasks.workunit.client.1.smithi193.stdout:6/625: dwrite d0/d27/d81/df/d11/d3b/d51/d9c/f96 [0,4194304] 0 2024-03-23T08:23:28.004 INFO:tasks.workunit.client.1.smithi193.stdout:3/633: chown d1/de/d13/d21/d59/cb3 666881875 1 2024-03-23T08:23:28.006 INFO:tasks.workunit.client.1.smithi193.stdout:1/764: write da/d13/d24/f7b [860025,117812] 0 2024-03-23T08:23:28.006 INFO:tasks.workunit.client.1.smithi193.stdout:4/806: rename d11/d2d/da8/d109/d10e/db8/f25 to d11/d2d/d58/d8b/dbc/f10f 0 2024-03-23T08:23:28.008 INFO:tasks.workunit.client.1.smithi193.stdout:6/626: symlink d0/d27/d72/lc7 0 2024-03-23T08:23:28.011 INFO:tasks.workunit.client.1.smithi193.stdout:3/634: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d85/fd8 x:0 0 0 2024-03-23T08:23:28.012 INFO:tasks.workunit.client.1.smithi193.stdout:4/807: creat d11/d27/d85/de1/d99/f110 x:0 0 0 2024-03-23T08:23:28.016 INFO:tasks.workunit.client.1.smithi193.stdout:3/635: read d1/de/d13/d21/d25/d2b/fb1 [2470647,21818] 0 2024-03-23T08:23:28.018 INFO:tasks.workunit.client.1.smithi193.stdout:7/730: write d6/d9/f15 [586805,28657] 0 2024-03-23T08:23:28.019 INFO:tasks.workunit.client.1.smithi193.stdout:6/627: dwrite d0/d27/d81/df/d11/d56/f8d [0,4194304] 0 2024-03-23T08:23:28.022 INFO:tasks.workunit.client.1.smithi193.stdout:3/636: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/dae/dd9 0 2024-03-23T08:23:28.023 INFO:tasks.workunit.client.1.smithi193.stdout:3/637: stat d1/de/d28/l8a 0 2024-03-23T08:23:28.029 INFO:tasks.workunit.client.1.smithi193.stdout:2/799: sync 2024-03-23T08:23:28.032 INFO:tasks.workunit.client.1.smithi193.stdout:1/765: dwrite da/d13/d24/f58 [0,4194304] 0 2024-03-23T08:23:28.035 INFO:tasks.workunit.client.1.smithi193.stdout:7/731: dwrite d6/d9/d1f/d4d/d7c/d92/fde [0,4194304] 0 2024-03-23T08:23:28.037 INFO:tasks.workunit.client.1.smithi193.stdout:1/766: chown da/d13/d3f/f44 2083063 1 2024-03-23T08:23:28.037 INFO:tasks.workunit.client.1.smithi193.stdout:4/808: dwrite d11/d27/df2/f103 [0,4194304] 0 2024-03-23T08:23:28.040 INFO:tasks.workunit.client.1.smithi193.stdout:2/800: rmdir d0/d1b/d14/d66/d50/d60 0 2024-03-23T08:23:28.045 INFO:tasks.workunit.client.1.smithi193.stdout:2/801: dread - d0/d1b/d19/fe2 zero size 2024-03-23T08:23:28.047 INFO:tasks.workunit.client.1.smithi193.stdout:3/638: dwrite d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/fc4 [0,4194304] 0 2024-03-23T08:23:28.051 INFO:tasks.workunit.client.1.smithi193.stdout:8/763: sync 2024-03-23T08:23:28.051 INFO:tasks.workunit.client.1.smithi193.stdout:9/799: sync 2024-03-23T08:23:28.051 INFO:tasks.workunit.client.1.smithi193.stdout:5/769: sync 2024-03-23T08:23:28.051 INFO:tasks.workunit.client.1.smithi193.stdout:1/767: dread da/dd/df/d52/f6e [0,4194304] 0 2024-03-23T08:23:28.053 INFO:tasks.workunit.client.1.smithi193.stdout:0/721: sync 2024-03-23T08:23:28.054 INFO:tasks.workunit.client.1.smithi193.stdout:2/802: unlink d0/d1b/ca 0 2024-03-23T08:23:28.057 INFO:tasks.workunit.client.1.smithi193.stdout:6/628: dread d0/d27/d81/f86 [0,4194304] 0 2024-03-23T08:23:28.057 INFO:tasks.workunit.client.1.smithi193.stdout:9/800: mknod d2/d7/db/d17/d33/db2/c10a 0 2024-03-23T08:23:28.059 INFO:tasks.workunit.client.1.smithi193.stdout:0/722: symlink db/d11/d14/d21/d33/d69/d9b/dc1/lfb 0 2024-03-23T08:23:28.061 INFO:tasks.workunit.client.1.smithi193.stdout:6/629: chown d0/d27/d81/df/d11/d3b/d6c/fbe 1886 1 2024-03-23T08:23:28.061 INFO:tasks.workunit.client.1.smithi193.stdout:1/768: rename da/c32 to da/dd/df/d7e/d7f/db2/cf5 0 2024-03-23T08:23:28.063 INFO:tasks.workunit.client.1.smithi193.stdout:4/809: link d11/d1a/d33/d42/db2/dbe/l55 d11/d2d/da8/d109/d10e/db8/l111 0 2024-03-23T08:23:28.064 INFO:tasks.workunit.client.1.smithi193.stdout:9/801: mkdir d2/d7/db/d1b/d5c/d10b 0 2024-03-23T08:23:28.065 INFO:tasks.workunit.client.1.smithi193.stdout:4/810: chown d11/d27/df2/f103 0 1 2024-03-23T08:23:28.066 INFO:tasks.workunit.client.1.smithi193.stdout:2/803: truncate d0/d1b/d3b/f58 639699 0 2024-03-23T08:23:28.069 INFO:tasks.workunit.client.1.smithi193.stdout:0/723: mkdir db/d11/d14/d24/d47/d59/dd4/dfc 0 2024-03-23T08:23:28.069 INFO:tasks.workunit.client.1.smithi193.stdout:2/804: stat d0/d1b/d14/d56 0 2024-03-23T08:23:28.070 INFO:tasks.workunit.client.1.smithi193.stdout:5/770: rename d0/d1c/d7a/db1/d72/d73/d57/d6d to d0/d1c/d3b/dbc/d112 0 2024-03-23T08:23:28.072 INFO:tasks.workunit.client.1.smithi193.stdout:9/802: dread d2/d7/db/d7a/d95/fc0 [0,4194304] 0 2024-03-23T08:23:28.074 INFO:tasks.workunit.client.1.smithi193.stdout:7/732: dwrite d6/d9/d1f/d23/d33/d38/d46/d82/fd1 [0,4194304] 0 2024-03-23T08:23:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:27 smithi193 ceph-mon[48370]: pgmap v23: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 40 MiB/s rd, 158 MiB/s wr, 296 op/s 2024-03-23T08:23:28.078 INFO:tasks.workunit.client.1.smithi193.stdout:8/764: dread df/f18 [0,4194304] 0 2024-03-23T08:23:28.078 INFO:tasks.workunit.client.1.smithi193.stdout:4/811: creat d11/d1a/d33/d42/d52/f112 x:0 0 0 2024-03-23T08:23:28.083 INFO:tasks.workunit.client.1.smithi193.stdout:9/803: mknod d2/d7/db/d17/d33/c10c 0 2024-03-23T08:23:28.084 INFO:tasks.workunit.client.1.smithi193.stdout:1/769: dwrite da/f59 [0,4194304] 0 2024-03-23T08:23:28.084 INFO:tasks.workunit.client.1.smithi193.stdout:3/639: dread d1/de/d13/d21/d50/d6f/fa7 [0,4194304] 0 2024-03-23T08:23:28.084 INFO:tasks.workunit.client.1.smithi193.stdout:4/812: mknod d11/d95/de4/c113 0 2024-03-23T08:23:28.086 INFO:tasks.workunit.client.1.smithi193.stdout:8/765: chown df/dc6/d8b/dae/l5e 6751 1 2024-03-23T08:23:28.091 INFO:tasks.workunit.client.1.smithi193.stdout:4/813: truncate d11/d1a/d33/d42/db2/dbe/d50/fa5 843941 0 2024-03-23T08:23:28.092 INFO:tasks.workunit.client.1.smithi193.stdout:9/804: truncate d2/d7/db/d1b/d43/d71/db4/dec/d5f/fd1 13801 0 2024-03-23T08:23:28.093 INFO:tasks.workunit.client.1.smithi193.stdout:1/770: mkdir da/dd/df/d7e/d7f/df6 0 2024-03-23T08:23:28.094 INFO:tasks.workunit.client.1.smithi193.stdout:2/805: truncate d0/d37/d48/f55 2272243 0 2024-03-23T08:23:28.095 INFO:tasks.workunit.client.1.smithi193.stdout:3/640: fsync d1/fbd 0 2024-03-23T08:23:28.096 INFO:tasks.workunit.client.1.smithi193.stdout:6/630: dread d0/d27/d81/df/d11/d36/d61/d7c/d69/f7b [0,4194304] 0 2024-03-23T08:23:28.096 INFO:tasks.workunit.client.1.smithi193.stdout:5/771: truncate d0/d1c/d7a/db1/d72/d99/dd8/fd1 149861 0 2024-03-23T08:23:28.097 INFO:tasks.workunit.client.1.smithi193.stdout:2/806: chown d0/d1b/d14/d66/fa0 371071 1 2024-03-23T08:23:28.097 INFO:tasks.workunit.client.1.smithi193.stdout:7/733: rename d6/d9/d1f/d23/c27 to d6/d9/d1f/d3e/cff 0 2024-03-23T08:23:28.098 INFO:tasks.workunit.client.1.smithi193.stdout:8/766: mknod df/cff 0 2024-03-23T08:23:28.100 INFO:tasks.workunit.client.1.smithi193.stdout:8/767: stat df/f86 0 2024-03-23T08:23:28.101 INFO:tasks.workunit.client.1.smithi193.stdout:5/772: creat d0/d1c/d7a/f113 x:0 0 0 2024-03-23T08:23:28.101 INFO:tasks.workunit.client.1.smithi193.stdout:1/771: read da/dd/df/d7e/d7f/db2/ddb/feb [1326189,104650] 0 2024-03-23T08:23:28.101 INFO:tasks.workunit.client.1.smithi193.stdout:2/807: mknod d0/d1b/d3b/ce3 0 2024-03-23T08:23:28.102 INFO:tasks.workunit.client.1.smithi193.stdout:4/814: creat d11/d2d/d58/d80/d9d/f114 x:0 0 0 2024-03-23T08:23:28.104 INFO:tasks.workunit.client.1.smithi193.stdout:6/631: mknod d0/d27/d81/df/cc8 0 2024-03-23T08:23:28.105 INFO:tasks.workunit.client.1.smithi193.stdout:0/724: dread db/d11/d14/d21/dd2/fe1 [0,4194304] 0 2024-03-23T08:23:28.105 INFO:tasks.workunit.client.1.smithi193.stdout:8/768: dread df/d10/d83/fc1 [0,4194304] 0 2024-03-23T08:23:28.105 INFO:tasks.workunit.client.1.smithi193.stdout:9/805: rmdir d2/d7/dff 0 2024-03-23T08:23:28.108 INFO:tasks.workunit.client.1.smithi193.stdout:0/725: readlink db/d11/d14/d24/l86 0 2024-03-23T08:23:28.110 INFO:tasks.workunit.client.1.smithi193.stdout:5/773: write d0/fa [4439138,26869] 0 2024-03-23T08:23:28.112 INFO:tasks.workunit.client.1.smithi193.stdout:8/769: dread df/d4b/d4d/d9c/f9f [0,4194304] 0 2024-03-23T08:23:28.113 INFO:tasks.workunit.client.1.smithi193.stdout:4/815: unlink d11/d27/d85/f100 0 2024-03-23T08:23:28.113 INFO:tasks.workunit.client.1.smithi193.stdout:8/770: fdatasync df/d63/d84/da8/ff8 0 2024-03-23T08:23:28.118 INFO:tasks.workunit.client.1.smithi193.stdout:6/632: creat d0/d27/d81/df/d11/d3b/d6c/fc9 x:0 0 0 2024-03-23T08:23:28.120 INFO:tasks.workunit.client.1.smithi193.stdout:9/806: dread d2/d7/f11 [4194304,4194304] 0 2024-03-23T08:23:28.122 INFO:tasks.workunit.client.1.smithi193.stdout:5/774: readlink d0/l46 0 2024-03-23T08:23:28.125 INFO:tasks.workunit.client.1.smithi193.stdout:8/771: symlink df/d63/l100 0 2024-03-23T08:23:28.126 INFO:tasks.workunit.client.1.smithi193.stdout:3/641: dwrite d1/de/d13/f33 [0,4194304] 0 2024-03-23T08:23:28.128 INFO:tasks.workunit.client.1.smithi193.stdout:0/726: dwrite db/d11/d14/d21/f49 [0,4194304] 0 2024-03-23T08:23:28.133 INFO:tasks.workunit.client.1.smithi193.stdout:1/772: dread f5 [0,4194304] 0 2024-03-23T08:23:28.133 INFO:tasks.workunit.client.1.smithi193.stdout:7/734: truncate d6/d9/d1f/d4d/fa9 2191547 0 2024-03-23T08:23:28.136 INFO:tasks.workunit.client.1.smithi193.stdout:2/808: dread d0/d1b/da7/fb3 [0,4194304] 0 2024-03-23T08:23:28.137 INFO:tasks.workunit.client.1.smithi193.stdout:5/775: mknod d0/d1c/d3b/d4b/d55/c114 0 2024-03-23T08:23:28.140 INFO:tasks.workunit.client.1.smithi193.stdout:6/633: chown d0/d27/d81/df/d11/d36/d61/d12/d21/c3f 491153977 1 2024-03-23T08:23:28.140 INFO:tasks.workunit.client.1.smithi193.stdout:3/642: symlink d1/de/d13/d21/d27/daf/lda 0 2024-03-23T08:23:28.140 INFO:tasks.workunit.client.1.smithi193.stdout:9/807: getdents d2/d7/db/d1b/d5c/d10b 0 2024-03-23T08:23:28.141 INFO:tasks.workunit.client.1.smithi193.stdout:0/727: mknod db/d11/d14/d24/d2a/de3/ddb/cfd 0 2024-03-23T08:23:28.141 INFO:tasks.workunit.client.1.smithi193.stdout:1/773: chown da/f3c 3822065 1 2024-03-23T08:23:28.141 INFO:tasks.workunit.client.1.smithi193.stdout:5/776: chown d0/d1c/d3b/d4b/f95 7060114 1 2024-03-23T08:23:28.142 INFO:tasks.workunit.client.1.smithi193.stdout:7/735: mkdir d6/d9/d1f/d23/d33/d38/d66/dac/d100 0 2024-03-23T08:23:28.142 INFO:tasks.workunit.client.1.smithi193.stdout:4/816: getdents d11/d2d/da8 0 2024-03-23T08:23:28.144 INFO:tasks.workunit.client.1.smithi193.stdout:3/643: readlink d1/de/d13/d96/d99/l48 0 2024-03-23T08:23:28.146 INFO:tasks.workunit.client.1.smithi193.stdout:1/774: mknod da/d13/d3f/cf7 0 2024-03-23T08:23:28.148 INFO:tasks.workunit.client.1.smithi193.stdout:8/772: dwrite df/d10/d83/fe6 [0,4194304] 0 2024-03-23T08:23:28.149 INFO:tasks.workunit.client.1.smithi193.stdout:5/777: rmdir d0/d1c/d3b/d4b/de2 39 2024-03-23T08:23:28.151 INFO:tasks.workunit.client.1.smithi193.stdout:5/778: chown d0/d1c/d3b 4909926 1 2024-03-23T08:23:28.151 INFO:tasks.workunit.client.1.smithi193.stdout:1/775: write da/dd/df/d2e/f48 [1592929,20218] 0 2024-03-23T08:23:28.153 INFO:tasks.workunit.client.1.smithi193.stdout:2/809: write d0/d37/d48/f4a [441054,77127] 0 2024-03-23T08:23:28.155 INFO:tasks.workunit.client.1.smithi193.stdout:3/644: rmdir d1/de/d13/d21/d25/d2b 39 2024-03-23T08:23:28.155 INFO:tasks.workunit.client.1.smithi193.stdout:0/728: truncate db/d11/d14/d24/d47/d59/f79 6653981 0 2024-03-23T08:23:28.156 INFO:tasks.workunit.client.1.smithi193.stdout:5/779: read d0/d1f/d93/feb [2672545,8303] 0 2024-03-23T08:23:28.158 INFO:tasks.workunit.client.1.smithi193.stdout:6/634: write d0/d27/d81/df/d11/d36/f41 [4960929,47928] 0 2024-03-23T08:23:28.159 INFO:tasks.workunit.client.1.smithi193.stdout:4/817: write d11/d1a/d33/d44/f6e [1692382,9839] 0 2024-03-23T08:23:28.163 INFO:tasks.workunit.client.1.smithi193.stdout:1/776: dread da/d13/d24/dac/dea/f9d [0,4194304] 0 2024-03-23T08:23:28.165 INFO:tasks.workunit.client.1.smithi193.stdout:7/736: truncate d6/d9/d1f/d4d/fc6 3615454 0 2024-03-23T08:23:28.165 INFO:tasks.workunit.client.1.smithi193.stdout:0/729: chown db/d11/d14/d24/cae 7 1 2024-03-23T08:23:28.168 INFO:tasks.workunit.client.1.smithi193.stdout:9/808: dwrite d2/d7/db/d1b/d43/d71/db4/dec/d3c/d79/fd3 [0,4194304] 0 2024-03-23T08:23:28.170 INFO:tasks.workunit.client.1.smithi193.stdout:2/810: getdents d0/d1b/d14/d66/d4e 0 2024-03-23T08:23:28.172 INFO:tasks.workunit.client.1.smithi193.stdout:7/737: write d6/f39 [3044066,78939] 0 2024-03-23T08:23:28.176 INFO:tasks.workunit.client.1.smithi193.stdout:9/809: fsync d2/d7/f11 0 2024-03-23T08:23:28.180 INFO:tasks.workunit.client.1.smithi193.stdout:9/810: readlink d2/d7/db/d32/l8c 0 2024-03-23T08:23:28.182 INFO:tasks.workunit.client.1.smithi193.stdout:3/645: truncate d1/de/d13/d21/d50/fc8 1149916 0 2024-03-23T08:23:28.182 INFO:tasks.workunit.client.1.smithi193.stdout:8/773: dread df/d4b/d4d/f45 [0,4194304] 0 2024-03-23T08:23:28.182 INFO:tasks.workunit.client.1.smithi193.stdout:5/780: dwrite d0/d1c/df7/f109 [0,4194304] 0 2024-03-23T08:23:28.183 INFO:tasks.workunit.client.1.smithi193.stdout:0/730: symlink db/d11/d14/d24/d2a/d96/lfe 0 2024-03-23T08:23:28.185 INFO:tasks.workunit.client.1.smithi193.stdout:4/818: link d11/d1a/d33/d42/db2/cee d11/d2d/d58/d80/c115 0 2024-03-23T08:23:28.187 INFO:tasks.workunit.client.1.smithi193.stdout:7/738: chown d6/d9/d1f/d23/d33/d38/cda 10 1 2024-03-23T08:23:28.188 INFO:tasks.workunit.client.1.smithi193.stdout:1/777: rename da/d13/d24/dac/dea/d76/d77/lc0 to da/dd/df/d52/d82/db0/lf8 0 2024-03-23T08:23:28.188 INFO:tasks.workunit.client.1.smithi193.stdout:9/811: write d2/d7/db/d1b/d5c/fe9 [618161,105242] 0 2024-03-23T08:23:28.190 INFO:tasks.workunit.client.1.smithi193.stdout:0/731: write db/d11/d14/d24/f5f [2270471,130460] 0 2024-03-23T08:23:28.193 INFO:tasks.workunit.client.1.smithi193.stdout:2/811: dwrite d0/d1b/d14/d25/f89 [0,4194304] 0 2024-03-23T08:23:28.194 INFO:tasks.workunit.client.1.smithi193.stdout:6/635: dread d0/d27/f91 [0,4194304] 0 2024-03-23T08:23:28.195 INFO:tasks.workunit.client.1.smithi193.stdout:8/774: mkdir df/d10/d3e/d4e/d101 0 2024-03-23T08:23:28.196 INFO:tasks.workunit.client.1.smithi193.stdout:2/812: chown d0/d1b/d14/d25/f83 74 1 2024-03-23T08:23:28.196 INFO:tasks.workunit.client.1.smithi193.stdout:1/778: unlink da/d13/fc3 0 2024-03-23T08:23:28.202 INFO:tasks.workunit.client.1.smithi193.stdout:1/779: readlink da/l47 0 2024-03-23T08:23:28.203 INFO:tasks.workunit.client.1.smithi193.stdout:8/775: write df/d10/d3e/d54/dd2/f72 [2055459,2703] 0 2024-03-23T08:23:28.203 INFO:tasks.workunit.client.1.smithi193.stdout:7/739: write d6/d9/d1f/daf/fe1 [543133,71095] 0 2024-03-23T08:23:28.203 INFO:tasks.workunit.client.1.smithi193.stdout:7/740: chown d6/d9/d1f/d3e/d65 955607 1 2024-03-23T08:23:28.203 INFO:tasks.workunit.client.1.smithi193.stdout:4/819: mknod d11/d2d/d58/d80/d9d/df5/c116 0 2024-03-23T08:23:28.204 INFO:tasks.workunit.client.1.smithi193.stdout:0/732: readlink db/d11/d14/d21/l55 0 2024-03-23T08:23:28.204 INFO:tasks.workunit.client.1.smithi193.stdout:6/636: truncate d0/d27/d81/fa6 1474376 0 2024-03-23T08:23:28.204 INFO:tasks.workunit.client.1.smithi193.stdout:2/813: unlink d0/d1b/d14/d66/fa0 0 2024-03-23T08:23:28.206 INFO:tasks.workunit.client.1.smithi193.stdout:8/776: unlink df/d4b/d4d/d2a/f2b 0 2024-03-23T08:23:28.207 INFO:tasks.workunit.client.1.smithi193.stdout:1/780: write da/dd/fa6 [3954673,118864] 0 2024-03-23T08:23:28.207 INFO:tasks.workunit.client.1.smithi193.stdout:0/733: mknod db/d11/d14/d21/dd2/cff 0 2024-03-23T08:23:28.212 INFO:tasks.workunit.client.1.smithi193.stdout:9/812: dread d2/d7/f15 [0,4194304] 0 2024-03-23T08:23:28.212 INFO:tasks.workunit.client.1.smithi193.stdout:6/637: creat d0/d27/d81/df/d11/d3b/d51/daa/fca x:0 0 0 2024-03-23T08:23:28.213 INFO:tasks.workunit.client.1.smithi193.stdout:2/814: creat d0/d1b/d14/d56/d7c/fe4 x:0 0 0 2024-03-23T08:23:28.215 INFO:tasks.workunit.client.1.smithi193.stdout:4/820: rename d11/d1a/d33/d42/db2/dbe/d50/d49/c64 to d11/d27/c117 0 2024-03-23T08:23:28.217 INFO:tasks.workunit.client.1.smithi193.stdout:1/781: write da/d13/f49 [1142612,54849] 0 2024-03-23T08:23:28.219 INFO:tasks.workunit.client.1.smithi193.stdout:8/777: creat df/d4b/d4d/d9c/dee/f102 x:0 0 0 2024-03-23T08:23:28.220 INFO:tasks.workunit.client.1.smithi193.stdout:3/646: dwrite d1/de/d13/d96/d99/f30 [0,4194304] 0 2024-03-23T08:23:28.220 INFO:tasks.workunit.client.1.smithi193.stdout:6/638: fsync d0/d27/d81/df/d11/d36/fba 0 2024-03-23T08:23:28.222 INFO:tasks.workunit.client.1.smithi193.stdout:0/734: read db/d11/d14/d24/d47/d59/fd9 [322087,104390] 0 2024-03-23T08:23:28.223 INFO:tasks.workunit.client.1.smithi193.stdout:1/782: write da/dd/df/d2e/f48 [1663031,51211] 0 2024-03-23T08:23:28.224 INFO:tasks.workunit.client.1.smithi193.stdout:7/741: dread d6/d9/d1f/d23/d33/d38/d46/d82/f8c [0,4194304] 0 2024-03-23T08:23:28.226 INFO:tasks.workunit.client.1.smithi193.stdout:5/781: dread d0/d1c/fb0 [0,4194304] 0 2024-03-23T08:23:28.226 INFO:tasks.workunit.client.1.smithi193.stdout:6/639: read d0/d27/d81/df/f9b [2803326,110254] 0 2024-03-23T08:23:28.227 INFO:tasks.workunit.client.1.smithi193.stdout:1/783: readlink da/dd/df/d7c/ld6 0 2024-03-23T08:23:28.227 INFO:tasks.workunit.client.1.smithi193.stdout:8/778: mkdir df/d10/d67/d7f/d103 0 2024-03-23T08:23:28.228 INFO:tasks.workunit.client.1.smithi193.stdout:1/784: chown da/c9c 35998787 1 2024-03-23T08:23:28.228 INFO:tasks.workunit.client.1.smithi193.stdout:9/813: link d2/ff2 d2/d7/db/d32/f10d 0 2024-03-23T08:23:28.229 INFO:tasks.workunit.client.1.smithi193.stdout:1/785: write da/dd/df/d7e/fcb [1237307,43903] 0 2024-03-23T08:23:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:27 smithi070 ceph-mon[39699]: pgmap v23: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 40 MiB/s rd, 158 MiB/s wr, 296 op/s 2024-03-23T08:23:28.234 INFO:tasks.workunit.client.1.smithi193.stdout:3/647: write d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fd5 [923036,39597] 0 2024-03-23T08:23:28.235 INFO:tasks.workunit.client.1.smithi193.stdout:7/742: write d6/f32 [136587,1268] 0 2024-03-23T08:23:28.235 INFO:tasks.workunit.client.1.smithi193.stdout:5/782: dread - d0/d1f/f2e zero size 2024-03-23T08:23:28.236 INFO:tasks.workunit.client.1.smithi193.stdout:6/640: rmdir d0/d27/d81/df/d11/d56 39 2024-03-23T08:23:28.236 INFO:tasks.workunit.client.1.smithi193.stdout:5/783: chown d0/d1c/l4e 112535913 1 2024-03-23T08:23:28.237 INFO:tasks.workunit.client.1.smithi193.stdout:7/743: fsync d6/d9/d1f/daf/fef 0 2024-03-23T08:23:28.238 INFO:tasks.workunit.client.1.smithi193.stdout:5/784: chown d0/d1c/d3b/d4b/dff 1837 1 2024-03-23T08:23:28.239 INFO:tasks.workunit.client.1.smithi193.stdout:9/814: creat d2/d7/db/d1b/d43/d71/db4/dec/f10e x:0 0 0 2024-03-23T08:23:28.240 INFO:tasks.workunit.client.1.smithi193.stdout:1/786: rename da/d13/d3f/f5e to da/dd/df/d75/d98/ff9 0 2024-03-23T08:23:28.241 INFO:tasks.workunit.client.1.smithi193.stdout:1/787: write da/dd/df/d52/d82/db0/ff4 [250559,56155] 0 2024-03-23T08:23:28.242 INFO:tasks.workunit.client.1.smithi193.stdout:4/821: rmdir d11/d1a/d65/dfd 0 2024-03-23T08:23:28.242 INFO:tasks.workunit.client.1.smithi193.stdout:3/648: creat d1/de/d13/d21/d27/d7c/fdb x:0 0 0 2024-03-23T08:23:28.245 INFO:tasks.workunit.client.1.smithi193.stdout:6/641: unlink d0/d27/d81/df/d11/d36/d61/d3a/c63 0 2024-03-23T08:23:28.246 INFO:tasks.workunit.client.1.smithi193.stdout:4/822: dread d11/d1a/d33/d42/f4d [0,4194304] 0 2024-03-23T08:23:28.247 INFO:tasks.workunit.client.1.smithi193.stdout:6/642: dread - d0/d27/d81/df/faf zero size 2024-03-23T08:23:28.248 INFO:tasks.workunit.client.1.smithi193.stdout:9/815: creat d2/d7/db/d17/d33/f10f x:0 0 0 2024-03-23T08:23:28.248 INFO:tasks.workunit.client.1.smithi193.stdout:1/788: symlink da/dd/df/d52/d82/db0/lfa 0 2024-03-23T08:23:28.249 INFO:tasks.workunit.client.1.smithi193.stdout:9/816: chown d2/d7/db/d7a/f101 24 1 2024-03-23T08:23:28.252 INFO:tasks.workunit.client.1.smithi193.stdout:3/649: rename d1/de/d28/c6d to d1/de/d13/d21/d59/dc6/cdc 0 2024-03-23T08:23:28.255 INFO:tasks.workunit.client.1.smithi193.stdout:4/823: symlink d11/d2d/d58/d80/db9/l118 0 2024-03-23T08:23:28.255 INFO:tasks.workunit.client.1.smithi193.stdout:9/817: creat d2/d7/db/d32/d6b/dde/f110 x:0 0 0 2024-03-23T08:23:28.257 INFO:tasks.workunit.client.1.smithi193.stdout:1/789: rename da/dd/df/f25 to da/dd/df/d7e/d86/d95/dc4/ffb 0 2024-03-23T08:23:28.260 INFO:tasks.workunit.client.1.smithi193.stdout:3/650: symlink d1/de/d13/d21/d27/ldd 0 2024-03-23T08:23:28.262 INFO:tasks.workunit.client.1.smithi193.stdout:9/818: mknod d2/d7/db/d1b/d43/d71/db4/dec/d3c/c111 0 2024-03-23T08:23:28.263 INFO:tasks.workunit.client.1.smithi193.stdout:9/819: fsync d2/d7/db/f108 0 2024-03-23T08:23:28.265 INFO:tasks.workunit.client.1.smithi193.stdout:6/643: dread d0/f40 [0,4194304] 0 2024-03-23T08:23:28.265 INFO:tasks.workunit.client.1.smithi193.stdout:4/824: link d11/d1a/d33/d44/f82 d11/d8d/f119 0 2024-03-23T08:23:28.288 INFO:tasks.workunit.client.1.smithi193.stdout:3/651: rename d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fcd to d1/de/d13/d21/d59/fde 0 2024-03-23T08:23:28.289 INFO:tasks.workunit.client.1.smithi193.stdout:9/820: mkdir d2/d7/db/d1b/d43/d9a/d112 0 2024-03-23T08:23:28.289 INFO:tasks.workunit.client.1.smithi193.stdout:8/779: dwrite df/d10/d3e/d54/dd2/fa5 [0,4194304] 0 2024-03-23T08:23:28.291 INFO:tasks.workunit.client.1.smithi193.stdout:5/785: write d0/d1f/f51 [1916476,50997] 0 2024-03-23T08:23:28.296 INFO:tasks.workunit.client.1.smithi193.stdout:6/644: truncate d0/d27/d81/df/d11/fbb 856565 0 2024-03-23T08:23:28.298 INFO:tasks.workunit.client.1.smithi193.stdout:7/744: dwrite d6/d9/f36 [0,4194304] 0 2024-03-23T08:23:28.303 INFO:tasks.workunit.client.1.smithi193.stdout:5/786: dwrite d0/d1c/d3b/d4b/f10e [0,4194304] 0 2024-03-23T08:23:28.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/815: sync 2024-03-23T08:23:28.360 INFO:tasks.workunit.client.1.smithi193.stdout:0/735: sync 2024-03-23T08:23:28.365 INFO:tasks.workunit.client.1.smithi193.stdout:2/816: dwrite d0/d1b/d14/f45 [0,4194304] 0 2024-03-23T08:23:28.385 INFO:tasks.workunit.client.1.smithi193.stdout:9/821: mkdir d2/d7/db/d7a/d95/da9/d113 0 2024-03-23T08:23:28.386 INFO:tasks.workunit.client.1.smithi193.stdout:8/780: creat df/d10/d67/d7f/f104 x:0 0 0 2024-03-23T08:23:28.390 INFO:tasks.workunit.client.1.smithi193.stdout:3/652: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/ddf 0 2024-03-23T08:23:28.390 INFO:tasks.workunit.client.1.smithi193.stdout:3/653: write d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/fb5 [272824,99443] 0 2024-03-23T08:23:28.395 INFO:tasks.workunit.client.1.smithi193.stdout:3/654: dwrite d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fbb [0,4194304] 0 2024-03-23T08:23:28.434 INFO:tasks.workunit.client.1.smithi193.stdout:0/736: symlink db/d11/d14/d24/d2a/de3/l100 0 2024-03-23T08:23:28.434 INFO:tasks.workunit.client.1.smithi193.stdout:6/645: dwrite d0/d27/d81/df/d11/d36/d61/f1d [0,4194304] 0 2024-03-23T08:23:28.437 INFO:tasks.workunit.client.1.smithi193.stdout:1/790: link da/dd/df/d52/c66 da/dd/df/d7e/d86/cfc 0 2024-03-23T08:23:28.439 INFO:tasks.workunit.client.1.smithi193.stdout:9/822: mkdir d2/d7/db/d32/d114 0 2024-03-23T08:23:28.444 INFO:tasks.workunit.client.1.smithi193.stdout:6/646: dwrite d0/d27/d81/fa6 [0,4194304] 0 2024-03-23T08:23:28.445 INFO:tasks.workunit.client.1.smithi193.stdout:6/647: chown d0/d27/d81/df/d11/d3b 19 1 2024-03-23T08:23:28.445 INFO:tasks.workunit.client.1.smithi193.stdout:6/648: stat d0/d27/f91 0 2024-03-23T08:23:28.449 INFO:tasks.workunit.client.1.smithi193.stdout:0/737: creat db/d11/d14/d21/d33/d48/d85/f101 x:0 0 0 2024-03-23T08:23:28.450 INFO:tasks.workunit.client.1.smithi193.stdout:5/787: truncate d0/d1f/f49 771162 0 2024-03-23T08:23:28.454 INFO:tasks.workunit.client.1.smithi193.stdout:9/823: rename d2/d7/db/d1b/d43/d71/db4/d78/lf9 to d2/d7/db/d7a/d95/da9/l115 0 2024-03-23T08:23:28.455 INFO:tasks.workunit.client.1.smithi193.stdout:1/791: mkdir da/dd/df/ddc/dfd 0 2024-03-23T08:23:28.456 INFO:tasks.workunit.client.1.smithi193.stdout:8/781: creat df/dc6/d8b/f105 x:0 0 0 2024-03-23T08:23:28.456 INFO:tasks.workunit.client.1.smithi193.stdout:5/788: dread - d0/d1c/d7a/db1/d72/d73/d57/fb4 zero size 2024-03-23T08:23:28.456 INFO:tasks.workunit.client.1.smithi193.stdout:6/649: truncate d0/d27/d81/df/d11/d3b/d6c/fc9 44084 0 2024-03-23T08:23:28.457 INFO:tasks.workunit.client.1.smithi193.stdout:7/745: getdents d6/d9/d1f/d23/d33/d38/d46/d82 0 2024-03-23T08:23:28.457 INFO:tasks.workunit.client.1.smithi193.stdout:0/738: symlink db/d11/d14/d21/d33/d69/d9b/l102 0 2024-03-23T08:23:28.460 INFO:tasks.workunit.client.1.smithi193.stdout:9/824: readlink d2/d7/db/d17/d4b/d8d/ldb 0 2024-03-23T08:23:28.461 INFO:tasks.workunit.client.1.smithi193.stdout:4/825: dwrite d11/d1a/d33/d42/db2/dbe/f5d [0,4194304] 0 2024-03-23T08:23:28.461 INFO:tasks.workunit.client.1.smithi193.stdout:1/792: write da/d13/d24/f43 [100810,25690] 0 2024-03-23T08:23:28.463 INFO:tasks.workunit.client.1.smithi193.stdout:2/817: dwrite d0/d37/d48/f55 [0,4194304] 0 2024-03-23T08:23:28.463 INFO:tasks.workunit.client.1.smithi193.stdout:7/746: fdatasync d6/d9/d1f/d3e/d42/f7a 0 2024-03-23T08:23:28.472 INFO:tasks.workunit.client.1.smithi193.stdout:1/793: fdatasync da/dd/df/d7c/fe2 0 2024-03-23T08:23:28.472 INFO:tasks.workunit.client.1.smithi193.stdout:0/739: truncate db/d11/fd8 164526 0 2024-03-23T08:23:28.473 INFO:tasks.workunit.client.1.smithi193.stdout:4/826: fdatasync d11/d1a/d33/d7f/f106 0 2024-03-23T08:23:28.474 INFO:tasks.workunit.client.1.smithi193.stdout:9/825: creat d2/d7/de5/f116 x:0 0 0 2024-03-23T08:23:28.477 INFO:tasks.workunit.client.1.smithi193.stdout:8/782: truncate df/d10/d3e/d54/dd2/f8c 1888680 0 2024-03-23T08:23:28.478 INFO:tasks.workunit.client.1.smithi193.stdout:5/789: dwrite d0/f12 [0,4194304] 0 2024-03-23T08:23:28.479 INFO:tasks.workunit.client.1.smithi193.stdout:7/747: mknod d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/c101 0 2024-03-23T08:23:28.479 INFO:tasks.workunit.client.1.smithi193.stdout:1/794: chown da/dd/df/d7e/ca3 109 1 2024-03-23T08:23:28.481 INFO:tasks.workunit.client.1.smithi193.stdout:9/826: write d2/d7/db/d1b/d43/f84 [105618,65564] 0 2024-03-23T08:23:28.482 INFO:tasks.workunit.client.1.smithi193.stdout:2/818: rmdir d0/d1b/d14/d66/d50/d8f 39 2024-03-23T08:23:28.485 INFO:tasks.workunit.client.1.smithi193.stdout:7/748: readlink d6/d9/d1f/d4d/d6b/lc3 0 2024-03-23T08:23:28.485 INFO:tasks.workunit.client.1.smithi193.stdout:3/655: dwrite d1/de/d13/d21/f23 [0,4194304] 0 2024-03-23T08:23:28.488 INFO:tasks.workunit.client.1.smithi193.stdout:9/827: write d2/d7/de5/f116 [777430,123610] 0 2024-03-23T08:23:28.489 INFO:tasks.workunit.client.1.smithi193.stdout:8/783: symlink df/d10/d3e/d4e/l106 0 2024-03-23T08:23:28.492 INFO:tasks.workunit.client.1.smithi193.stdout:1/795: dread da/dd/df/d52/f6e [0,4194304] 0 2024-03-23T08:23:28.496 INFO:tasks.workunit.client.1.smithi193.stdout:4/827: dread d11/d8e/dac/fbd [0,4194304] 0 2024-03-23T08:23:28.496 INFO:tasks.workunit.client.1.smithi193.stdout:3/656: write d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fd5 [1925799,108143] 0 2024-03-23T08:23:28.496 INFO:tasks.workunit.client.1.smithi193.stdout:5/790: rename d0/f10a to d0/d1c/df7/f115 0 2024-03-23T08:23:28.497 INFO:tasks.workunit.client.1.smithi193.stdout:8/784: symlink df/d10/d83/l107 0 2024-03-23T08:23:28.498 INFO:tasks.workunit.client.1.smithi193.stdout:2/819: mkdir d0/d1b/d14/d56/d7c/ddc/de5 0 2024-03-23T08:23:28.498 INFO:tasks.workunit.client.1.smithi193.stdout:6/650: dread d0/d27/d81/df/d11/d36/d61/f1d [4194304,4194304] 0 2024-03-23T08:23:28.499 INFO:tasks.workunit.client.1.smithi193.stdout:1/796: creat da/d13/d24/dac/dea/d76/ffe x:0 0 0 2024-03-23T08:23:28.501 INFO:tasks.workunit.client.1.smithi193.stdout:5/791: fdatasync d0/d1c/d7a/db1/d72/d73/d45/da2/fdd 0 2024-03-23T08:23:28.501 INFO:tasks.workunit.client.1.smithi193.stdout:8/785: mknod df/dcc/c108 0 2024-03-23T08:23:28.505 INFO:tasks.workunit.client.1.smithi193.stdout:4/828: rename d11/d1a/d33/d7f/f81 to d11/d2d/d58/d8b/dbc/de9/f11a 0 2024-03-23T08:23:28.506 INFO:tasks.workunit.client.1.smithi193.stdout:1/797: mkdir da/dd/df/d52/d82/dff 0 2024-03-23T08:23:28.507 INFO:tasks.workunit.client.1.smithi193.stdout:7/749: dwrite d6/d9/d1f/d23/d33/d38/d46/d82/fed [0,4194304] 0 2024-03-23T08:23:28.508 INFO:tasks.workunit.client.1.smithi193.stdout:3/657: truncate d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/fa0 506560 0 2024-03-23T08:23:28.509 INFO:tasks.workunit.client.1.smithi193.stdout:6/651: rmdir d0/d27/d81/df 39 2024-03-23T08:23:28.510 INFO:tasks.workunit.client.1.smithi193.stdout:7/750: chown d6/d9/d1f/d3e/d65/d70/f83 55 1 2024-03-23T08:23:28.512 INFO:tasks.workunit.client.1.smithi193.stdout:5/792: symlink d0/d1c/d7a/l116 0 2024-03-23T08:23:28.512 INFO:tasks.workunit.client.1.smithi193.stdout:7/751: chown d6/d9/d1f/d4d/d7c/d92 169922007 1 2024-03-23T08:23:28.513 INFO:tasks.workunit.client.1.smithi193.stdout:5/793: write d0/d1f/fbb [763679,36074] 0 2024-03-23T08:23:28.514 INFO:tasks.workunit.client.1.smithi193.stdout:8/786: creat df/d63/d84/df4/f109 x:0 0 0 2024-03-23T08:23:28.514 INFO:tasks.workunit.client.1.smithi193.stdout:4/829: dwrite d11/d2d/fde [0,4194304] 0 2024-03-23T08:23:28.517 INFO:tasks.workunit.client.1.smithi193.stdout:4/830: readlink d11/d2d/d58/d80/db9/lc4 0 2024-03-23T08:23:28.520 INFO:tasks.workunit.client.1.smithi193.stdout:4/831: write d11/d1a/d33/d44/f54 [1862539,74116] 0 2024-03-23T08:23:28.521 INFO:tasks.workunit.client.1.smithi193.stdout:1/798: mknod da/d13/d24/dac/c100 0 2024-03-23T08:23:28.521 INFO:tasks.workunit.client.1.smithi193.stdout:2/820: dread d0/d1b/d14/f41 [0,4194304] 0 2024-03-23T08:23:28.522 INFO:tasks.workunit.client.1.smithi193.stdout:6/652: getdents d0/d27/d81/da1/da9 0 2024-03-23T08:23:28.525 INFO:tasks.workunit.client.1.smithi193.stdout:9/828: rename d2/d7/db/d32/d6b/dde/fdf to d2/d7/db/d1b/d43/d71/db4/dec/d3c/f117 0 2024-03-23T08:23:28.529 INFO:tasks.workunit.client.1.smithi193.stdout:2/821: rename d0/d37/d48/fbe to d0/d1b/d3b/fe6 0 2024-03-23T08:23:28.530 INFO:tasks.workunit.client.1.smithi193.stdout:5/794: link d0/d1f/d93/fa3 d0/d5a/dbf/d106/f117 0 2024-03-23T08:23:28.533 INFO:tasks.workunit.client.1.smithi193.stdout:4/832: creat d11/d2d/da8/d109/d10e/f11b x:0 0 0 2024-03-23T08:23:28.534 INFO:tasks.workunit.client.1.smithi193.stdout:2/822: stat d0/d1b/d14/da1/dd5/l4b 0 2024-03-23T08:23:28.534 INFO:tasks.workunit.client.1.smithi193.stdout:4/833: readlink d11/l15 0 2024-03-23T08:23:28.536 INFO:tasks.workunit.client.1.smithi193.stdout:1/799: dwrite da/dd/df/d52/d82/f9f [0,4194304] 0 2024-03-23T08:23:28.537 INFO:tasks.workunit.client.1.smithi193.stdout:6/653: creat d0/d27/d81/df/d11/d36/d61/fcb x:0 0 0 2024-03-23T08:23:28.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/823: rmdir d0/d1b 39 2024-03-23T08:23:28.540 INFO:tasks.workunit.client.1.smithi193.stdout:8/787: dread df/d10/d67/d7f/fc2 [0,4194304] 0 2024-03-23T08:23:28.542 INFO:tasks.workunit.client.1.smithi193.stdout:1/800: rmdir da/dd/df/d7e/d86 39 2024-03-23T08:23:28.544 INFO:tasks.workunit.client.1.smithi193.stdout:6/654: creat d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/fcc x:0 0 0 2024-03-23T08:23:28.546 INFO:tasks.workunit.client.1.smithi193.stdout:6/655: stat d0/fc6 0 2024-03-23T08:23:28.547 INFO:tasks.workunit.client.1.smithi193.stdout:2/824: fdatasync d0/d1b/d14/d66/d50/d92/da9/fab 0 2024-03-23T08:23:28.549 INFO:tasks.workunit.client.1.smithi193.stdout:8/788: rename df/dcc/ccf to df/d4b/d4d/d9c/dee/c10a 0 2024-03-23T08:23:28.551 INFO:tasks.workunit.client.1.smithi193.stdout:5/795: dwrite d0/d1c/d3b/d4b/f95 [0,4194304] 0 2024-03-23T08:23:28.553 INFO:tasks.workunit.client.1.smithi193.stdout:8/789: chown df/d4b/d4d 3776446 1 2024-03-23T08:23:28.553 INFO:tasks.workunit.client.1.smithi193.stdout:9/829: dread d2/d7/db/d1b/f30 [0,4194304] 0 2024-03-23T08:23:28.553 INFO:tasks.workunit.client.1.smithi193.stdout:1/801: rename da/dd/df/d75/d98/ff2 to da/d13/d24/f101 0 2024-03-23T08:23:28.557 INFO:tasks.workunit.client.1.smithi193.stdout:9/830: write d2/d7/db/d17/d33/d70/fd0 [2361182,63276] 0 2024-03-23T08:23:28.558 INFO:tasks.workunit.client.1.smithi193.stdout:6/656: creat d0/d27/d81/df/d11/d36/d61/d3a/d8f/fcd x:0 0 0 2024-03-23T08:23:28.558 INFO:tasks.workunit.client.1.smithi193.stdout:5/796: mknod d0/d1c/d7a/db1/d72/d73/c118 0 2024-03-23T08:23:28.560 INFO:tasks.workunit.client.1.smithi193.stdout:1/802: mknod da/d13/c102 0 2024-03-23T08:23:28.562 INFO:tasks.workunit.client.1.smithi193.stdout:6/657: write d0/d27/d81/df/d11/d3b/d51/daa/fca [306781,122367] 0 2024-03-23T08:23:28.564 INFO:tasks.workunit.client.1.smithi193.stdout:1/803: truncate da/f4e 5353815 0 2024-03-23T08:23:28.565 INFO:tasks.workunit.client.1.smithi193.stdout:9/831: truncate d2/d7/db/d32/f67 1082735 0 2024-03-23T08:23:28.566 INFO:tasks.workunit.client.1.smithi193.stdout:5/797: mkdir d0/d1c/d3b/d4b/dff/d119 0 2024-03-23T08:23:28.568 INFO:tasks.workunit.client.1.smithi193.stdout:1/804: fsync da/d13/d24/dac/fed 0 2024-03-23T08:23:28.569 INFO:tasks.workunit.client.1.smithi193.stdout:4/834: dread f9 [4194304,4194304] 0 2024-03-23T08:23:28.571 INFO:tasks.workunit.client.1.smithi193.stdout:5/798: rmdir d0/d1c/d3b/dbc 39 2024-03-23T08:23:28.572 INFO:tasks.workunit.client.1.smithi193.stdout:2/825: dwrite d0/d28/fa8 [0,4194304] 0 2024-03-23T08:23:28.573 INFO:tasks.workunit.client.1.smithi193.stdout:9/832: chown d2/d7/db/d1b/d5c/cfc 53915 1 2024-03-23T08:23:28.575 INFO:tasks.workunit.client.1.smithi193.stdout:1/805: readlink da/dd/df/d52/d82/l92 0 2024-03-23T08:23:28.578 INFO:tasks.workunit.client.1.smithi193.stdout:4/835: creat d11/d1a/d33/d42/d52/d70/f11c x:0 0 0 2024-03-23T08:23:28.578 INFO:tasks.workunit.client.1.smithi193.stdout:5/799: chown d0/d1c/d3b/d81/df2/ff8 3 1 2024-03-23T08:23:28.580 INFO:tasks.workunit.client.1.smithi193.stdout:6/658: dwrite d0/d27/d81/df/d11/d36/f41 [0,4194304] 0 2024-03-23T08:23:28.582 INFO:tasks.workunit.client.1.smithi193.stdout:6/659: chown d0 233755 1 2024-03-23T08:23:28.586 INFO:tasks.workunit.client.1.smithi193.stdout:5/800: mknod d0/d1c/d3b/d81/c11a 0 2024-03-23T08:23:28.594 INFO:tasks.workunit.client.1.smithi193.stdout:2/826: dwrite d0/d1b/d3b/f58 [0,4194304] 0 2024-03-23T08:23:28.597 INFO:tasks.workunit.client.1.smithi193.stdout:6/660: unlink d0/d27/d81/df/d11/d36/d61/c9 0 2024-03-23T08:23:28.597 INFO:tasks.workunit.client.1.smithi193.stdout:9/833: truncate d2/d7/db/d17/f53 2253448 0 2024-03-23T08:23:28.600 INFO:tasks.workunit.client.1.smithi193.stdout:2/827: readlink d0/l10 0 2024-03-23T08:23:28.600 INFO:tasks.workunit.client.1.smithi193.stdout:9/834: readlink d2/d7/db/d17/d1f/l99 0 2024-03-23T08:23:28.604 INFO:tasks.workunit.client.1.smithi193.stdout:4/836: dread d11/d1a/d33/d42/db2/dbe/d50/fab [0,4194304] 0 2024-03-23T08:23:28.604 INFO:tasks.workunit.client.1.smithi193.stdout:6/661: chown d0/d27/d81/df/d11/d3b/f66 3484 1 2024-03-23T08:23:28.605 INFO:tasks.workunit.client.1.smithi193.stdout:5/801: dread d0/d1f/f5b [0,4194304] 0 2024-03-23T08:23:28.606 INFO:tasks.workunit.client.1.smithi193.stdout:9/835: getdents d2/d7/db/d17/d33/d70/de1 0 2024-03-23T08:23:28.608 INFO:tasks.workunit.client.1.smithi193.stdout:9/836: read d2/d7/db/d1b/d43/d71/db4/dec/d5f/fb1 [66171,43419] 0 2024-03-23T08:23:28.609 INFO:tasks.workunit.client.1.smithi193.stdout:4/837: truncate d11/d8e/dac/fbd 1828151 0 2024-03-23T08:23:28.612 INFO:tasks.workunit.client.1.smithi193.stdout:6/662: mkdir d0/d27/d81/dce 0 2024-03-23T08:23:28.613 INFO:tasks.workunit.client.1.smithi193.stdout:2/828: link d0/d1b/d14/d66/d4e/fcb d0/d1b/d14/d66/d50/d92/da9/dba/fe7 0 2024-03-23T08:23:28.615 INFO:tasks.workunit.client.1.smithi193.stdout:4/838: unlink d11/d2d/d58/d8b/dbc/cec 0 2024-03-23T08:23:28.615 INFO:tasks.workunit.client.1.smithi193.stdout:4/839: read - d11/d1a/d33/dce/fd7 zero size 2024-03-23T08:23:28.619 INFO:tasks.workunit.client.1.smithi193.stdout:2/829: stat d0/d1b/d14/d66/d50/d8f/la4 0 2024-03-23T08:23:28.622 INFO:tasks.workunit.client.1.smithi193.stdout:9/837: rmdir d2/d7/db/d1b/d43/d71/db4/d78/ddc 0 2024-03-23T08:23:28.623 INFO:tasks.workunit.client.1.smithi193.stdout:6/663: getdents d0/d27/d81/da1/da9 0 2024-03-23T08:23:28.624 INFO:tasks.workunit.client.1.smithi193.stdout:6/664: chown d0/d27/d81/df/d11/d36/d61/d12/l2b 898 1 2024-03-23T08:23:28.626 INFO:tasks.workunit.client.1.smithi193.stdout:4/840: creat d11/d95/d107/f11d x:0 0 0 2024-03-23T08:23:28.631 INFO:tasks.workunit.client.1.smithi193.stdout:5/802: dwrite d0/d1c/d7a/db1/d72/d73/f6c [0,4194304] 0 2024-03-23T08:23:28.632 INFO:tasks.workunit.client.1.smithi193.stdout:0/740: sync 2024-03-23T08:23:28.633 INFO:tasks.workunit.client.1.smithi193.stdout:4/841: symlink d11/d2d/d58/d8b/dbc/l11e 0 2024-03-23T08:23:28.636 INFO:tasks.workunit.client.1.smithi193.stdout:0/741: fdatasync db/d11/d14/d24/d47/dc7/fb2 0 2024-03-23T08:23:28.643 INFO:tasks.workunit.client.1.smithi193.stdout:9/838: dwrite d2/d7/db/f60 [0,4194304] 0 2024-03-23T08:23:28.645 INFO:tasks.workunit.client.1.smithi193.stdout:2/830: dwrite d0/fd [0,4194304] 0 2024-03-23T08:23:28.647 INFO:tasks.workunit.client.1.smithi193.stdout:5/803: write d0/d1c/d3b/dbc/d112/f87 [541555,104003] 0 2024-03-23T08:23:28.649 INFO:tasks.workunit.client.1.smithi193.stdout:6/665: dwrite d0/d27/d81/df/f4f [0,4194304] 0 2024-03-23T08:23:28.649 INFO:tasks.workunit.client.1.smithi193.stdout:5/804: dread - d0/d1f/fca zero size 2024-03-23T08:23:28.653 INFO:tasks.workunit.client.1.smithi193.stdout:2/831: creat d0/d1b/d14/d25/fe8 x:0 0 0 2024-03-23T08:23:28.654 INFO:tasks.workunit.client.1.smithi193.stdout:4/842: dread d11/d2d/d58/d80/ff0 [0,4194304] 0 2024-03-23T08:23:28.658 INFO:tasks.workunit.client.1.smithi193.stdout:9/839: dwrite d2/d7/f8e [0,4194304] 0 2024-03-23T08:23:28.660 INFO:tasks.workunit.client.1.smithi193.stdout:6/666: mknod d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/ccf 0 2024-03-23T08:23:28.661 INFO:tasks.workunit.client.1.smithi193.stdout:4/843: fdatasync d11/d27/d85/f8a 0 2024-03-23T08:23:28.662 INFO:tasks.workunit.client.1.smithi193.stdout:4/844: write d11/d1a/d65/fe8 [1257203,20320] 0 2024-03-23T08:23:28.662 INFO:tasks.workunit.client.1.smithi193.stdout:6/667: chown d0/d27/d81/df/d11/d36/d61/d3a/f45 22743287 1 2024-03-23T08:23:28.663 INFO:tasks.workunit.client.1.smithi193.stdout:6/668: chown d0/fc6 15834231 1 2024-03-23T08:23:28.665 INFO:tasks.workunit.client.1.smithi193.stdout:6/669: write d0/f8b [827207,111559] 0 2024-03-23T08:23:28.666 INFO:tasks.workunit.client.1.smithi193.stdout:9/840: rename d2/d7/db/d1b/d43/d71/db4/dec/d5f/f68 to d2/d7/db/d7a/d95/f118 0 2024-03-23T08:23:28.671 INFO:tasks.workunit.client.1.smithi193.stdout:4/845: dwrite fb [0,4194304] 0 2024-03-23T08:23:28.675 INFO:tasks.workunit.client.1.smithi193.stdout:6/670: creat d0/d27/d81/df/d11/d56/fd0 x:0 0 0 2024-03-23T08:23:28.681 INFO:tasks.workunit.client.1.smithi193.stdout:4/846: dwrite d11/f8c [0,4194304] 0 2024-03-23T08:23:28.685 INFO:tasks.workunit.client.1.smithi193.stdout:2/832: write d0/d1b/d14/da1/dd5/d72/fa5 [2979617,16011] 0 2024-03-23T08:23:28.688 INFO:tasks.workunit.client.1.smithi193.stdout:9/841: dwrite d2/d7/db/d1b/d43/d71/db4/dec/d3c/f117 [0,4194304] 0 2024-03-23T08:23:28.692 INFO:tasks.workunit.client.1.smithi193.stdout:5/805: truncate d0/d1c/d7a/db1/d72/d73/d45/daa/fe5 677435 0 2024-03-23T08:23:28.695 INFO:tasks.workunit.client.1.smithi193.stdout:4/847: dwrite d11/d1a/d33/d44/f4a [0,4194304] 0 2024-03-23T08:23:28.695 INFO:tasks.workunit.client.1.smithi193.stdout:5/806: dread - d0/d1c/df7/f115 zero size 2024-03-23T08:23:28.696 INFO:tasks.workunit.client.1.smithi193.stdout:9/842: symlink d2/d7/db/d17/d33/db2/l119 0 2024-03-23T08:23:28.700 INFO:tasks.workunit.client.1.smithi193.stdout:4/848: mknod d11/d27/d85/c11f 0 2024-03-23T08:23:28.702 INFO:tasks.workunit.client.1.smithi193.stdout:5/807: truncate d0/d1c/d7a/db1/d72/d73/d45/f77 116167 0 2024-03-23T08:23:28.703 INFO:tasks.workunit.client.1.smithi193.stdout:4/849: write d11/d2d/d3a/f76 [1631201,15384] 0 2024-03-23T08:23:28.706 INFO:tasks.workunit.client.1.smithi193.stdout:9/843: dwrite d2/d7/db/d7a/f101 [0,4194304] 0 2024-03-23T08:23:28.706 INFO:tasks.workunit.client.1.smithi193.stdout:4/850: write d11/f8c [4811449,87787] 0 2024-03-23T08:23:28.707 INFO:tasks.workunit.client.1.smithi193.stdout:0/742: dread db/d11/d14/d24/f25 [0,4194304] 0 2024-03-23T08:23:28.708 INFO:tasks.workunit.client.1.smithi193.stdout:5/808: fdatasync d0/d1c/d7a/db1/d72/d73/d45/fa0 0 2024-03-23T08:23:28.710 INFO:tasks.workunit.client.1.smithi193.stdout:2/833: write d0/d37/f4c [230025,45750] 0 2024-03-23T08:23:28.712 INFO:tasks.workunit.client.1.smithi193.stdout:0/743: symlink db/d11/d14/d51/l103 0 2024-03-23T08:23:28.712 INFO:tasks.workunit.client.1.smithi193.stdout:4/851: write d11/d2d/da8/fd8 [901683,8079] 0 2024-03-23T08:23:28.722 INFO:tasks.workunit.client.1.smithi193.stdout:2/834: dwrite d0/d1b/d77/d9b/fbc [0,4194304] 0 2024-03-23T08:23:28.722 INFO:tasks.workunit.client.1.smithi193.stdout:4/852: chown d11/d1a/d33/d42/db2/dbe/d50/l43 24793 1 2024-03-23T08:23:28.723 INFO:tasks.workunit.client.1.smithi193.stdout:5/809: dread d0/d1c/d3b/f3c [0,4194304] 0 2024-03-23T08:23:28.725 INFO:tasks.workunit.client.1.smithi193.stdout:4/853: chown d11/d2d/d58/d80/df3 107 1 2024-03-23T08:23:28.727 INFO:tasks.workunit.client.1.smithi193.stdout:5/810: fdatasync d0/d1c/df7/f115 0 2024-03-23T08:23:28.729 INFO:tasks.workunit.client.1.smithi193.stdout:4/854: symlink d11/d95/l120 0 2024-03-23T08:23:28.730 INFO:tasks.workunit.client.1.smithi193.stdout:2/835: creat d0/d1b/d19/fe9 x:0 0 0 2024-03-23T08:23:28.732 INFO:tasks.workunit.client.1.smithi193.stdout:0/744: dwrite db/d11/d14/d24/d47/d59/d8c/fb7 [0,4194304] 0 2024-03-23T08:23:28.733 INFO:tasks.workunit.client.1.smithi193.stdout:9/844: dread d2/d7/db/d7a/f87 [0,4194304] 0 2024-03-23T08:23:28.733 INFO:tasks.workunit.client.1.smithi193.stdout:2/836: dread - d0/d1b/d14/da1/dd5/d72/fc9 zero size 2024-03-23T08:23:28.736 INFO:tasks.workunit.client.1.smithi193.stdout:4/855: symlink d11/d1a/d33/d42/d52/l121 0 2024-03-23T08:23:28.738 INFO:tasks.workunit.client.1.smithi193.stdout:5/811: symlink d0/d1c/d7a/db1/d72/d73/d45/d6e/d91/dfd/l11b 0 2024-03-23T08:23:28.741 INFO:tasks.workunit.client.1.smithi193.stdout:2/837: creat d0/d28/fea x:0 0 0 2024-03-23T08:23:28.744 INFO:tasks.workunit.client.1.smithi193.stdout:5/812: dwrite d0/f12 [4194304,4194304] 0 2024-03-23T08:23:28.746 INFO:tasks.workunit.client.1.smithi193.stdout:9/845: truncate d2/d7/f36 3401835 0 2024-03-23T08:23:28.750 INFO:tasks.workunit.client.1.smithi193.stdout:5/813: rename d0/d1f/d93/de1/ce3 to d0/d1c/d7a/db1/d72/d73/d45/c11c 0 2024-03-23T08:23:28.754 INFO:tasks.workunit.client.1.smithi193.stdout:0/745: dread db/d11/d14/d3b/d71/fcb [0,4194304] 0 2024-03-23T08:23:28.756 INFO:tasks.workunit.client.1.smithi193.stdout:9/846: dwrite d2/d7/db/d7a/f9f [4194304,4194304] 0 2024-03-23T08:23:28.758 INFO:tasks.workunit.client.1.smithi193.stdout:4/856: dwrite d11/d2d/da8/d109/d10e/db8/f26 [0,4194304] 0 2024-03-23T08:23:28.761 INFO:tasks.workunit.client.1.smithi193.stdout:5/814: creat d0/d1c/d7a/db1/d72/f11d x:0 0 0 2024-03-23T08:23:28.762 INFO:tasks.workunit.client.1.smithi193.stdout:0/746: getdents db/d11/d14/d21/d33/d69/dde 0 2024-03-23T08:23:28.764 INFO:tasks.workunit.client.1.smithi193.stdout:9/847: symlink d2/d7/db/d1b/d43/d71/db4/d78/l11a 0 2024-03-23T08:23:28.765 INFO:tasks.workunit.client.1.smithi193.stdout:9/848: stat d2/d7/db/f60 0 2024-03-23T08:23:28.766 INFO:tasks.workunit.client.1.smithi193.stdout:9/849: stat d2/d7/db/d1b/d5c/d10b 0 2024-03-23T08:23:28.770 INFO:tasks.workunit.client.1.smithi193.stdout:5/815: unlink d0/d1c/d7a/db1/d72/d99/fc5 0 2024-03-23T08:23:28.771 INFO:tasks.workunit.client.1.smithi193.stdout:4/857: mkdir d11/d95/de4/d105/d122 0 2024-03-23T08:23:28.772 INFO:tasks.workunit.client.1.smithi193.stdout:4/858: read d11/d1a/d33/d42/db2/dbe/f5d [2065559,49171] 0 2024-03-23T08:23:28.773 INFO:tasks.workunit.client.1.smithi193.stdout:4/859: readlink d11/d2d/d58/d80/db9/lc4 0 2024-03-23T08:23:28.774 INFO:tasks.workunit.client.1.smithi193.stdout:9/850: mkdir d2/d7/db/d1b/d43/d71/d11b 0 2024-03-23T08:23:28.774 INFO:tasks.workunit.client.1.smithi193.stdout:4/860: stat d11/d2d/d58/d80/d9f/fa2 0 2024-03-23T08:23:28.775 INFO:tasks.workunit.client.1.smithi193.stdout:3/658: sync 2024-03-23T08:23:28.777 INFO:tasks.workunit.client.1.smithi193.stdout:3/659: write d1/de/d13/d21/d25/d2b/d3c/f6a [891587,42453] 0 2024-03-23T08:23:28.777 INFO:tasks.workunit.client.1.smithi193.stdout:0/747: getdents db/d11/d14/d24/d2a/d4d 0 2024-03-23T08:23:28.778 INFO:tasks.workunit.client.1.smithi193.stdout:4/861: write d11/d1a/d33/d42/db2/dbe/d50/f61 [973705,65884] 0 2024-03-23T08:23:28.781 INFO:tasks.workunit.client.1.smithi193.stdout:3/660: symlink d1/de/d13/d96/le0 0 2024-03-23T08:23:28.785 INFO:tasks.workunit.client.1.smithi193.stdout:9/851: dwrite d2/d7/db/d1b/d43/d71/db4/dec/d3c/fbe [0,4194304] 0 2024-03-23T08:23:28.785 INFO:tasks.workunit.client.1.smithi193.stdout:0/748: rename db/d11/d14/d21/d33/d48/d85/cdc to db/d11/d14/d24/d47/dc7/c104 0 2024-03-23T08:23:28.786 INFO:tasks.workunit.client.1.smithi193.stdout:3/661: dread d1/de/d13/d96/d99/f55 [0,4194304] 0 2024-03-23T08:23:28.788 INFO:tasks.workunit.client.1.smithi193.stdout:4/862: rename d11/d1a/d33/d44/f6e to d11/d1a/d33/d7f/f123 0 2024-03-23T08:23:28.789 INFO:tasks.workunit.client.1.smithi193.stdout:0/749: mkdir db/d11/d14/d24/d2a/d4b/d105 0 2024-03-23T08:23:28.789 INFO:tasks.workunit.client.1.smithi193.stdout:9/852: mknod d2/d7/db/d7a/d95/da9/c11c 0 2024-03-23T08:23:28.791 INFO:tasks.workunit.client.1.smithi193.stdout:9/853: chown d2/d7/db/d7a/d95/fa7 709903341 1 2024-03-23T08:23:28.791 INFO:tasks.workunit.client.1.smithi193.stdout:4/863: symlink d11/d27/d85/de1/d99/ded/l124 0 2024-03-23T08:23:28.792 INFO:tasks.workunit.client.1.smithi193.stdout:3/662: creat d1/fe1 x:0 0 0 2024-03-23T08:23:28.793 INFO:tasks.workunit.client.1.smithi193.stdout:0/750: symlink db/d11/d14/d24/d47/d59/dd4/l106 0 2024-03-23T08:23:28.793 INFO:tasks.workunit.client.1.smithi193.stdout:4/864: dread - d11/d2d/d58/d8b/dbc/d84/fe5 zero size 2024-03-23T08:23:28.794 INFO:tasks.workunit.client.1.smithi193.stdout:9/854: symlink d2/d7/db/d1b/d43/d71/db4/dec/d5f/de3/l11d 0 2024-03-23T08:23:28.801 INFO:tasks.workunit.client.1.smithi193.stdout:9/855: creat d2/d7/db/d1b/d43/d71/db4/dec/d3c/d79/f11e x:0 0 0 2024-03-23T08:23:28.805 INFO:tasks.workunit.client.1.smithi193.stdout:9/856: readlink d2/d7/db/l105 0 2024-03-23T08:23:28.806 INFO:tasks.workunit.client.1.smithi193.stdout:0/751: dwrite db/f1e [4194304,4194304] 0 2024-03-23T08:23:28.808 INFO:tasks.workunit.client.1.smithi193.stdout:3/663: dwrite d1/de/d13/d21/d25/d2b/fb1 [4194304,4194304] 0 2024-03-23T08:23:28.808 INFO:tasks.workunit.client.1.smithi193.stdout:4/865: dwrite d11/d1a/d33/d42/f59 [0,4194304] 0 2024-03-23T08:23:28.809 INFO:tasks.workunit.client.1.smithi193.stdout:0/752: write db/d11/d14/d3b/d71/f7f [1009989,83139] 0 2024-03-23T08:23:28.812 INFO:tasks.workunit.client.1.smithi193.stdout:9/857: symlink d2/d7/db/l11f 0 2024-03-23T08:23:28.813 INFO:tasks.workunit.client.1.smithi193.stdout:4/866: stat d11/d2d/da8/d109/d10e/db8/f6b 0 2024-03-23T08:23:28.814 INFO:tasks.workunit.client.1.smithi193.stdout:9/858: fdatasync d2/d7/db/d7a/f101 0 2024-03-23T08:23:28.814 INFO:tasks.workunit.client.1.smithi193.stdout:0/753: mkdir db/d11/d14/d24/ddd/d107 0 2024-03-23T08:23:28.817 INFO:tasks.workunit.client.1.smithi193.stdout:4/867: chown d11/d8e/dac/fbd 5 1 2024-03-23T08:23:28.819 INFO:tasks.workunit.client.1.smithi193.stdout:0/754: rename db/d11/d14/d21/l8a to db/d11/d14/d24/d2a/d4d/l108 0 2024-03-23T08:23:28.820 INFO:tasks.workunit.client.1.smithi193.stdout:4/868: write d11/d2d/d3a/fc1 [1016956,1833] 0 2024-03-23T08:23:28.822 INFO:tasks.workunit.client.1.smithi193.stdout:0/755: creat db/d52/db8/dcf/f109 x:0 0 0 2024-03-23T08:23:28.823 INFO:tasks.workunit.client.1.smithi193.stdout:4/869: chown d11/d1a/d33/f7d 2 1 2024-03-23T08:23:28.825 INFO:tasks.workunit.client.1.smithi193.stdout:0/756: write db/d11/d14/d24/d47/d59/f79 [5828527,27547] 0 2024-03-23T08:23:28.827 INFO:tasks.workunit.client.1.smithi193.stdout:0/757: creat db/d11/d14/d51/f10a x:0 0 0 2024-03-23T08:23:28.832 INFO:tasks.workunit.client.1.smithi193.stdout:4/870: dwrite d11/d1a/d33/d7f/de0/f7e [0,4194304] 0 2024-03-23T08:23:28.832 INFO:tasks.workunit.client.1.smithi193.stdout:0/758: link db/f8f db/d11/d14/d24/d2a/de3/f10b 0 2024-03-23T08:23:28.835 INFO:tasks.workunit.client.1.smithi193.stdout:0/759: mkdir db/d11/d14/d3b/d10c 0 2024-03-23T08:23:28.836 INFO:tasks.workunit.client.1.smithi193.stdout:0/760: chown db/d11/d14/d24/d47/d59/d8c 12069 1 2024-03-23T08:23:28.841 INFO:tasks.workunit.client.1.smithi193.stdout:0/761: mknod db/d11/d14/d21/d33/d69/dde/c10d 0 2024-03-23T08:23:28.841 INFO:tasks.workunit.client.1.smithi193.stdout:0/762: fdatasync db/d11/d14/d51/f10a 0 2024-03-23T08:23:28.844 INFO:tasks.workunit.client.1.smithi193.stdout:9/859: dread d2/fce [0,4194304] 0 2024-03-23T08:23:28.847 INFO:tasks.workunit.client.1.smithi193.stdout:3/664: dread d1/de/d13/d21/d25/d2b/d3c/d4a/f93 [0,4194304] 0 2024-03-23T08:23:28.853 INFO:tasks.workunit.client.1.smithi193.stdout:3/665: getdents d1/de/d13/d21/d50/d6f 0 2024-03-23T08:23:28.856 INFO:tasks.workunit.client.1.smithi193.stdout:3/666: mknod d1/de/d13/d21/d25/d2b/d3c/d4a/d85/ce2 0 2024-03-23T08:23:28.856 INFO:tasks.workunit.client.1.smithi193.stdout:9/860: write d2/d7/db/d32/f67 [1421879,69661] 0 2024-03-23T08:23:28.858 INFO:tasks.workunit.client.1.smithi193.stdout:9/861: chown d2/d7/db/f34 1312868 1 2024-03-23T08:23:28.861 INFO:tasks.workunit.client.1.smithi193.stdout:3/667: dread d1/f2f [0,4194304] 0 2024-03-23T08:23:28.861 INFO:tasks.workunit.client.1.smithi193.stdout:0/763: dwrite db/d11/d14/d21/d33/f68 [0,4194304] 0 2024-03-23T08:23:28.862 INFO:tasks.workunit.client.1.smithi193.stdout:9/862: creat d2/d7/db/d7a/d95/da9/f120 x:0 0 0 2024-03-23T08:23:28.864 INFO:tasks.workunit.client.1.smithi193.stdout:9/863: rmdir d2/d7/db/d17 39 2024-03-23T08:23:28.866 INFO:tasks.workunit.client.1.smithi193.stdout:0/764: creat db/d11/d14/d21/d33/d48/f10e x:0 0 0 2024-03-23T08:23:28.867 INFO:tasks.workunit.client.1.smithi193.stdout:0/765: dread - db/d11/d14/d24/d2a/d4d/d93/fd1 zero size 2024-03-23T08:23:28.867 INFO:tasks.workunit.client.1.smithi193.stdout:3/668: rename d1/de/d13/d21/d25/d2b/d3c/d4a/dae/lc1 to d1/de/d28/le3 0 2024-03-23T08:23:28.870 INFO:tasks.workunit.client.1.smithi193.stdout:9/864: rename d2/d7/db/d1b/f6a to d2/d7/f121 0 2024-03-23T08:23:28.872 INFO:tasks.workunit.client.1.smithi193.stdout:0/766: rename db/d11/d14/d21/f2c to db/d11/d14/d24/dc9/f10f 0 2024-03-23T08:23:28.874 INFO:tasks.workunit.client.1.smithi193.stdout:3/669: rename d1/de/d13/d21/d27/daf to d1/de/d13/d21/d25/d2b/d3c/d4a/de4 0 2024-03-23T08:23:28.875 INFO:tasks.workunit.client.1.smithi193.stdout:9/865: symlink d2/d7/db/d1b/d43/d9a/d112/l122 0 2024-03-23T08:23:28.881 INFO:tasks.workunit.client.1.smithi193.stdout:0/767: creat db/d11/d14/d24/d47/d59/f110 x:0 0 0 2024-03-23T08:23:28.891 INFO:tasks.workunit.client.1.smithi193.stdout:9/866: creat d2/d7/db/d17/d4b/f123 x:0 0 0 2024-03-23T08:23:28.895 INFO:tasks.workunit.client.1.smithi193.stdout:0/768: creat db/d11/d14/f111 x:0 0 0 2024-03-23T08:23:28.898 INFO:tasks.workunit.client.1.smithi193.stdout:0/769: rename db/d11/l9f to db/d11/d14/d24/d2a/d4d/l112 0 2024-03-23T08:23:28.900 INFO:tasks.workunit.client.1.smithi193.stdout:9/867: getdents d2/d7/db/d7a 0 2024-03-23T08:23:28.901 INFO:tasks.workunit.client.1.smithi193.stdout:0/770: creat db/d11/d14/d21/d33/d48/f113 x:0 0 0 2024-03-23T08:23:28.901 INFO:tasks.workunit.client.1.smithi193.stdout:9/868: write d2/d7/db/d1b/d5c/ff5 [980751,114336] 0 2024-03-23T08:23:28.902 INFO:tasks.workunit.client.1.smithi193.stdout:0/771: chown db/d11/d14/d24/d2a/d96/lfe 3930033 1 2024-03-23T08:23:28.904 INFO:tasks.workunit.client.1.smithi193.stdout:0/772: dread - db/d11/d14/d21/d33/f80 zero size 2024-03-23T08:23:28.904 INFO:tasks.workunit.client.1.smithi193.stdout:9/869: getdents d2/d7/db/d17/d33/d70/de1 0 2024-03-23T08:23:28.906 INFO:tasks.workunit.client.1.smithi193.stdout:0/773: write db/d11/d14/d21/f78 [1328898,30453] 0 2024-03-23T08:23:28.908 INFO:tasks.workunit.client.1.smithi193.stdout:7/752: sync 2024-03-23T08:23:28.908 INFO:tasks.workunit.client.1.smithi193.stdout:8/790: sync 2024-03-23T08:23:28.909 INFO:tasks.workunit.client.1.smithi193.stdout:7/753: write d6/d9/d1f/d23/d33/d38/d66/fa3 [743600,69846] 0 2024-03-23T08:23:28.913 INFO:tasks.workunit.client.1.smithi193.stdout:3/670: write d1/f20 [212116,78803] 0 2024-03-23T08:23:28.915 INFO:tasks.workunit.client.1.smithi193.stdout:8/791: mkdir df/d4b/d4d/d2a/d10b 0 2024-03-23T08:23:28.917 INFO:tasks.workunit.client.1.smithi193.stdout:7/754: mknod d6/d9/d1f/d3e/d65/c102 0 2024-03-23T08:23:28.918 INFO:tasks.workunit.client.1.smithi193.stdout:3/671: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/dae/de5 0 2024-03-23T08:23:28.918 INFO:tasks.workunit.client.1.smithi193.stdout:0/774: rmdir db 39 2024-03-23T08:23:28.919 INFO:tasks.workunit.client.1.smithi193.stdout:9/870: link d2/d7/db/d32/lf0 d2/d7/db/d17/d1f/l124 0 2024-03-23T08:23:28.921 INFO:tasks.workunit.client.1.smithi193.stdout:3/672: symlink d1/de/d13/d96/le6 0 2024-03-23T08:23:28.922 INFO:tasks.workunit.client.1.smithi193.stdout:7/755: creat d6/d9/d1f/d23/d33/d3c/dec/f103 x:0 0 0 2024-03-23T08:23:28.926 INFO:tasks.workunit.client.1.smithi193.stdout:0/775: mknod db/d11/d14/d21/d33/d69/d9b/dc1/c114 0 2024-03-23T08:23:28.927 INFO:tasks.workunit.client.1.smithi193.stdout:9/871: dwrite d2/d7/db/d1b/fef [0,4194304] 0 2024-03-23T08:23:28.929 INFO:tasks.workunit.client.1.smithi193.stdout:0/776: fsync db/d11/d14/d24/d47/d59/f79 0 2024-03-23T08:23:28.930 INFO:tasks.workunit.client.1.smithi193.stdout:3/673: truncate d1/de/f16 3762167 0 2024-03-23T08:23:28.932 INFO:tasks.workunit.client.1.smithi193.stdout:9/872: creat d2/d7/db/d17/d1f/f125 x:0 0 0 2024-03-23T08:23:28.933 INFO:tasks.workunit.client.1.smithi193.stdout:7/756: link d6/d9/d1f/d3e/d65/d70/fe3 d6/d9/d1f/d23/d33/d38/d3b/d6c/de7/f104 0 2024-03-23T08:23:28.935 INFO:tasks.workunit.client.1.smithi193.stdout:8/792: dwrite df/d10/d83/fc1 [0,4194304] 0 2024-03-23T08:23:28.935 INFO:tasks.workunit.client.1.smithi193.stdout:0/777: stat db/d11/d14/d21/l3d 0 2024-03-23T08:23:28.936 INFO:tasks.workunit.client.1.smithi193.stdout:3/674: write d1/de/d13/d21/d27/f1f [7992851,109099] 0 2024-03-23T08:23:28.936 INFO:tasks.workunit.client.1.smithi193.stdout:9/873: dread - d2/d7/db/d17/d4b/f123 zero size 2024-03-23T08:23:28.937 INFO:tasks.workunit.client.1.smithi193.stdout:7/757: chown d6/d9/d1f/d23/d33/d38/f91 27275 1 2024-03-23T08:23:28.937 INFO:tasks.workunit.client.1.smithi193.stdout:1/806: sync 2024-03-23T08:23:28.937 INFO:tasks.workunit.client.1.smithi193.stdout:6/671: sync 2024-03-23T08:23:28.937 INFO:tasks.workunit.client.1.smithi193.stdout:5/816: sync 2024-03-23T08:23:28.937 INFO:tasks.workunit.client.1.smithi193.stdout:2/838: sync 2024-03-23T08:23:28.939 INFO:tasks.workunit.client.1.smithi193.stdout:5/817: stat d0/f4d 0 2024-03-23T08:23:28.941 INFO:tasks.workunit.client.1.smithi193.stdout:9/874: chown d2/d7/db/d17/d33/d70/lb9 15 1 2024-03-23T08:23:28.941 INFO:tasks.workunit.client.1.smithi193.stdout:7/758: read - d6/d9/d1f/d3e/fab zero size 2024-03-23T08:23:28.942 INFO:tasks.workunit.client.1.smithi193.stdout:2/839: write d0/d1b/d14/d56/d7c/d7e/fca [99486,70002] 0 2024-03-23T08:23:28.943 INFO:tasks.workunit.client.1.smithi193.stdout:1/807: chown da/d13/c6d 11732284 1 2024-03-23T08:23:28.943 INFO:tasks.workunit.client.1.smithi193.stdout:8/793: dread df/dc6/d8b/dae/f90 [0,4194304] 0 2024-03-23T08:23:28.945 INFO:tasks.workunit.client.1.smithi193.stdout:7/759: stat d6/d9/d1f/d23/d33/d38/dc0/fd9 0 2024-03-23T08:23:28.946 INFO:tasks.workunit.client.1.smithi193.stdout:0/778: getdents db 0 2024-03-23T08:23:28.947 INFO:tasks.workunit.client.1.smithi193.stdout:3/675: creat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/de5/fe7 x:0 0 0 2024-03-23T08:23:28.948 INFO:tasks.workunit.client.1.smithi193.stdout:2/840: rmdir d0/d28/d9f 39 2024-03-23T08:23:28.949 INFO:tasks.workunit.client.1.smithi193.stdout:6/672: rename d0/d27/d81/df/d11/d36/d61/d12/f1e to d0/d27/d81/df/d11/d56/fd1 0 2024-03-23T08:23:28.949 INFO:tasks.workunit.client.1.smithi193.stdout:8/794: write df/d10/d3e/d54/dd2/dca/ff1 [601955,129044] 0 2024-03-23T08:23:28.950 INFO:tasks.workunit.client.1.smithi193.stdout:9/875: symlink d2/d7/db/d17/d4b/d8b/l126 0 2024-03-23T08:23:28.951 INFO:tasks.workunit.client.1.smithi193.stdout:2/841: dread - d0/d1b/d14/d56/d7c/fe4 zero size 2024-03-23T08:23:28.954 INFO:tasks.workunit.client.1.smithi193.stdout:1/808: symlink da/dd/df/ddc/dfd/l103 0 2024-03-23T08:23:28.954 INFO:tasks.workunit.client.1.smithi193.stdout:0/779: chown db/d11/d14/l5e 87724 1 2024-03-23T08:23:28.954 INFO:tasks.workunit.client.1.smithi193.stdout:3/676: mkdir d1/de/d13/d21/d50/d6f/de8 0 2024-03-23T08:23:28.955 INFO:tasks.workunit.client.1.smithi193.stdout:7/760: mknod d6/dbd/c105 0 2024-03-23T08:23:28.956 INFO:tasks.workunit.client.1.smithi193.stdout:5/818: dwrite d0/d1c/d7a/db1/d72/d73/d45/fdc [0,4194304] 0 2024-03-23T08:23:28.957 INFO:tasks.workunit.client.1.smithi193.stdout:1/809: chown da/d13/d24/dac/dea/f9d 32 1 2024-03-23T08:23:28.958 INFO:tasks.workunit.client.1.smithi193.stdout:0/780: write db/d11/d14/d24/d47/dc7/fb2 [256981,26771] 0 2024-03-23T08:23:28.958 INFO:tasks.workunit.client.1.smithi193.stdout:6/673: rename d0/d27/d81/d87/lab to d0/d27/ld2 0 2024-03-23T08:23:28.960 INFO:tasks.workunit.client.1.smithi193.stdout:2/842: unlink d0/d1b/d14/da1/dd5/l4b 0 2024-03-23T08:23:28.963 INFO:tasks.workunit.client.1.smithi193.stdout:5/819: mknod d0/d1c/d3b/d4b/d55/c11e 0 2024-03-23T08:23:28.964 INFO:tasks.workunit.client.1.smithi193.stdout:2/843: chown d0/d1b/d14/d66/d4e/dce/ld7 3743147 1 2024-03-23T08:23:28.967 INFO:tasks.workunit.client.1.smithi193.stdout:6/674: creat d0/d27/d81/da1/fd3 x:0 0 0 2024-03-23T08:23:28.967 INFO:tasks.workunit.client.1.smithi193.stdout:1/810: dwrite da/dd/df/d52/d82/f83 [0,4194304] 0 2024-03-23T08:23:28.968 INFO:tasks.workunit.client.1.smithi193.stdout:5/820: read d0/d1c/df7/f109 [2931870,100542] 0 2024-03-23T08:23:28.969 INFO:tasks.workunit.client.1.smithi193.stdout:0/781: mkdir db/d11/d14/d24/d2a/d96/d115 0 2024-03-23T08:23:28.970 INFO:tasks.workunit.client.1.smithi193.stdout:7/761: fsync d6/d9/d1f/d4d/fc6 0 2024-03-23T08:23:28.972 INFO:tasks.workunit.client.1.smithi193.stdout:0/782: fsync db/d11/d14/d24/d2a/fbe 0 2024-03-23T08:23:28.976 INFO:tasks.workunit.client.1.smithi193.stdout:8/795: link df/d10/d3e/d4e/l106 df/d63/l10c 0 2024-03-23T08:23:28.978 INFO:tasks.workunit.client.1.smithi193.stdout:8/796: stat df/f5a 0 2024-03-23T08:23:28.979 INFO:tasks.workunit.client.1.smithi193.stdout:0/783: dread db/d11/d14/d21/d33/f68 [0,4194304] 0 2024-03-23T08:23:28.980 INFO:tasks.workunit.client.1.smithi193.stdout:5/821: mkdir d0/d1f/d93/de1/d11f 0 2024-03-23T08:23:28.982 INFO:tasks.workunit.client.1.smithi193.stdout:7/762: dread d6/d9/d1f/d23/d33/d3c/f4c [0,4194304] 0 2024-03-23T08:23:28.983 INFO:tasks.workunit.client.1.smithi193.stdout:9/876: getdents d2/d7/db/d17/d33 0 2024-03-23T08:23:28.984 INFO:tasks.workunit.client.1.smithi193.stdout:0/784: read db/d11/d14/d24/f4a [2211114,62850] 0 2024-03-23T08:23:28.984 INFO:tasks.workunit.client.1.smithi193.stdout:4/871: sync 2024-03-23T08:23:28.985 INFO:tasks.workunit.client.1.smithi193.stdout:1/811: creat da/dd/df/d2e/dd5/f104 x:0 0 0 2024-03-23T08:23:28.985 INFO:tasks.workunit.client.1.smithi193.stdout:3/677: dwrite d1/de/d13/f5b [0,4194304] 0 2024-03-23T08:23:28.986 INFO:tasks.workunit.client.1.smithi193.stdout:5/822: rmdir d0/d1c/d3b/d81 39 2024-03-23T08:23:28.987 INFO:tasks.workunit.client.1.smithi193.stdout:2/844: dread d0/d1b/fd3 [4194304,4194304] 0 2024-03-23T08:23:28.988 INFO:tasks.workunit.client.1.smithi193.stdout:6/675: rename d0/d27/d81/df/d11/d36/f97 to d0/fd4 0 2024-03-23T08:23:28.988 INFO:tasks.workunit.client.1.smithi193.stdout:7/763: fdatasync d6/d9/d1f/d23/d33/d38/d3b/f48 0 2024-03-23T08:23:28.989 INFO:tasks.workunit.client.1.smithi193.stdout:6/676: stat d0/d27/d81/df/l28 0 2024-03-23T08:23:28.989 INFO:tasks.workunit.client.1.smithi193.stdout:0/785: rmdir db/d11/d14/d21/d33/d69/d9b/df2 39 2024-03-23T08:23:28.990 INFO:tasks.workunit.client.1.smithi193.stdout:1/812: creat da/dd/df/d52/d82/f105 x:0 0 0 2024-03-23T08:23:28.991 INFO:tasks.workunit.client.1.smithi193.stdout:5/823: chown d0/d5a/ddf/cf0 62 1 2024-03-23T08:23:28.991 INFO:tasks.workunit.client.1.smithi193.stdout:8/797: rename df/d10/d3e/d4e to df/d10/d3e/d54/d10d 0 2024-03-23T08:23:28.994 INFO:tasks.workunit.client.1.smithi193.stdout:7/764: creat d6/d9/d1f/d5f/f106 x:0 0 0 2024-03-23T08:23:28.995 INFO:tasks.workunit.client.1.smithi193.stdout:8/798: fdatasync df/d63/d84/dc5/fd3 0 2024-03-23T08:23:28.996 INFO:tasks.workunit.client.1.smithi193.stdout:3/678: mknod d1/de/d13/d21/d50/d6f/d6e/ce9 0 2024-03-23T08:23:29.001 INFO:tasks.workunit.client.1.smithi193.stdout:3/679: write d1/de/d13/d21/d25/d34/f47 [1290333,32171] 0 2024-03-23T08:23:29.003 INFO:tasks.workunit.client.1.smithi193.stdout:1/813: mknod da/dd/df/d75/d98/df3/c106 0 2024-03-23T08:23:29.007 INFO:tasks.workunit.client.1.smithi193.stdout:0/786: rmdir db/d52/db8/dcf/df3 0 2024-03-23T08:23:29.009 INFO:tasks.workunit.client.1.smithi193.stdout:3/680: symlink d1/de/d13/d96/lea 0 2024-03-23T08:23:29.011 INFO:tasks.workunit.client.1.smithi193.stdout:1/814: symlink da/dd/df/d7c/dd2/l107 0 2024-03-23T08:23:29.011 INFO:tasks.workunit.client.1.smithi193.stdout:2/845: dread d0/d37/f8a [0,4194304] 0 2024-03-23T08:23:29.012 INFO:tasks.workunit.client.1.smithi193.stdout:9/877: dread d2/d7/db/d1b/d43/d71/db4/dec/d5f/f83 [0,4194304] 0 2024-03-23T08:23:29.013 INFO:tasks.workunit.client.1.smithi193.stdout:2/846: readlink d0/d1b/d14/l7d 0 2024-03-23T08:23:29.013 INFO:tasks.workunit.client.1.smithi193.stdout:6/677: dread d0/d27/d81/df/d11/d56/f8d [0,4194304] 0 2024-03-23T08:23:29.014 INFO:tasks.workunit.client.1.smithi193.stdout:7/765: rename d6/d9/d1f/d23/d33/d38/dc0/fd9 to d6/d9/d1f/d23/d33/f107 0 2024-03-23T08:23:29.016 INFO:tasks.workunit.client.1.smithi193.stdout:0/787: mknod db/d11/d14/d24/d2a/d4b/d105/c116 0 2024-03-23T08:23:29.016 INFO:tasks.workunit.client.1.smithi193.stdout:7/766: write d6/d9/d1f/fa4 [1292115,32774] 0 2024-03-23T08:23:29.017 INFO:tasks.workunit.client.1.smithi193.stdout:3/681: mkdir d1/de/d13/d21/d59/dd4/deb 0 2024-03-23T08:23:29.017 INFO:tasks.workunit.client.1.smithi193.stdout:5/824: write d0/d1c/d3b/dbc/d112/d8e/f92 [1673656,101355] 0 2024-03-23T08:23:29.018 INFO:tasks.workunit.client.1.smithi193.stdout:4/872: truncate d11/d1a/d33/d7f/de0/f7e 3547074 0 2024-03-23T08:23:29.019 INFO:tasks.workunit.client.1.smithi193.stdout:8/799: dread df/f56 [0,4194304] 0 2024-03-23T08:23:29.020 INFO:tasks.workunit.client.1.smithi193.stdout:9/878: mkdir d2/d7/db/d17/d4b/d59/d97/d127 0 2024-03-23T08:23:29.025 INFO:tasks.workunit.client.1.smithi193.stdout:1/815: write da/f3c [1383140,18900] 0 2024-03-23T08:23:29.027 INFO:tasks.workunit.client.1.smithi193.stdout:7/767: write d6/d9/d1f/d23/d33/f107 [680540,57175] 0 2024-03-23T08:23:29.027 INFO:tasks.workunit.client.1.smithi193.stdout:8/800: mkdir df/d10/d3e/d54/dd2/dca/d10e 0 2024-03-23T08:23:29.029 INFO:tasks.workunit.client.1.smithi193.stdout:9/879: unlink d2/d7/db/d17/d4b/d59/d75/lcf 0 2024-03-23T08:23:29.033 INFO:tasks.workunit.client.1.smithi193.stdout:0/788: rename db/dc6/ce5 to db/d11/d14/d3b/c117 0 2024-03-23T08:23:29.037 INFO:tasks.workunit.client.1.smithi193.stdout:5/825: dread d0/d1c/d7a/db1/f67 [0,4194304] 0 2024-03-23T08:23:29.037 INFO:tasks.workunit.client.1.smithi193.stdout:8/801: creat df/d10/d3e/d54/d10d/f10f x:0 0 0 2024-03-23T08:23:29.038 INFO:tasks.workunit.client.1.smithi193.stdout:0/789: write db/d11/d14/d24/d2a/d4b/fbd [1034313,65859] 0 2024-03-23T08:23:29.040 INFO:tasks.workunit.client.1.smithi193.stdout:7/768: dwrite d6/d9/d1f/d3e/d65/d70/ffc [0,4194304] 0 2024-03-23T08:23:29.043 INFO:tasks.workunit.client.1.smithi193.stdout:4/873: dwrite d11/d1a/f72 [0,4194304] 0 2024-03-23T08:23:29.043 INFO:tasks.workunit.client.1.smithi193.stdout:3/682: rename d1/fe1 to d1/de/d28/fec 0 2024-03-23T08:23:29.044 INFO:tasks.workunit.client.1.smithi193.stdout:6/678: truncate d0/d27/d81/df/d11/d3b/f76 2164475 0 2024-03-23T08:23:29.049 INFO:tasks.workunit.client.1.smithi193.stdout:2/847: link d0/d1b/d14/da1/cdb d0/d28/d9f/ceb 0 2024-03-23T08:23:29.049 INFO:tasks.workunit.client.1.smithi193.stdout:4/874: chown d11/d27/d85/de1/d99/f110 25552069 1 2024-03-23T08:23:29.053 INFO:tasks.workunit.client.1.smithi193.stdout:9/880: write d2/d7/db/d32/f10d [5002185,91936] 0 2024-03-23T08:23:29.058 INFO:tasks.workunit.client.1.smithi193.stdout:0/790: dwrite db/d11/d14/d21/f41 [0,4194304] 0 2024-03-23T08:23:29.059 INFO:tasks.workunit.client.1.smithi193.stdout:0/791: stat db/d11/d14/d21/d33/d69/d9b/dc1 0 2024-03-23T08:23:29.060 INFO:tasks.workunit.client.1.smithi193.stdout:1/816: truncate da/dd/df/f4b 1828610 0 2024-03-23T08:23:29.062 INFO:tasks.workunit.client.1.smithi193.stdout:3/683: chown d1/de/d13/d21/d59/dc6/cdc 0 1 2024-03-23T08:23:29.064 INFO:tasks.workunit.client.1.smithi193.stdout:2/848: chown d0/d1b/d14/d25/c53 5 1 2024-03-23T08:23:29.066 INFO:tasks.workunit.client.1.smithi193.stdout:4/875: symlink d11/d1a/d33/d42/db2/dbe/d50/db3/l125 0 2024-03-23T08:23:29.069 INFO:tasks.workunit.client.1.smithi193.stdout:0/792: symlink db/d11/d14/l118 0 2024-03-23T08:23:29.069 INFO:tasks.workunit.client.1.smithi193.stdout:2/849: chown d0/d1b/d3b/ce3 845 1 2024-03-23T08:23:29.070 INFO:tasks.workunit.client.1.smithi193.stdout:3/684: mknod d1/de/d13/d21/d50/d44/ced 0 2024-03-23T08:23:29.070 INFO:tasks.workunit.client.1.smithi193.stdout:8/802: dread df/d4b/d4d/f22 [0,4194304] 0 2024-03-23T08:23:29.072 INFO:tasks.workunit.client.1.smithi193.stdout:0/793: readlink db/d11/d14/d21/lea 0 2024-03-23T08:23:29.073 INFO:tasks.workunit.client.1.smithi193.stdout:5/826: dread d0/d1f/f78 [0,4194304] 0 2024-03-23T08:23:29.074 INFO:tasks.workunit.client.1.smithi193.stdout:4/876: symlink d11/d2d/d58/d80/d9d/l126 0 2024-03-23T08:23:29.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:28 smithi193 ceph-mon[48370]: pgmap v24: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 85 MiB/s rd, 257 MiB/s wr, 497 op/s 2024-03-23T08:23:29.075 INFO:tasks.workunit.client.1.smithi193.stdout:2/850: creat d0/d1b/d3b/fec x:0 0 0 2024-03-23T08:23:29.076 INFO:tasks.workunit.client.1.smithi193.stdout:8/803: mknod df/d63/d84/df0/c110 0 2024-03-23T08:23:29.077 INFO:tasks.workunit.client.1.smithi193.stdout:5/827: dread d0/d1c/d7a/db1/d72/d99/dd8/f108 [0,4194304] 0 2024-03-23T08:23:29.078 INFO:tasks.workunit.client.1.smithi193.stdout:9/881: write d2/d7/db/d17/f53 [3281778,14679] 0 2024-03-23T08:23:29.079 INFO:tasks.workunit.client.1.smithi193.stdout:0/794: rename db/d11/d14/d24/d47/d59/f7c to db/d11/d14/d51/f119 0 2024-03-23T08:23:29.081 INFO:tasks.workunit.client.1.smithi193.stdout:3/685: link d1/de/d28/fc5 d1/de/d13/d96/d99/fee 0 2024-03-23T08:23:29.082 INFO:tasks.workunit.client.1.smithi193.stdout:8/804: read df/d63/fbc [722825,37711] 0 2024-03-23T08:23:29.082 INFO:tasks.workunit.client.1.smithi193.stdout:2/851: dread - d0/d1b/d14/da1/dd5/d72/fc9 zero size 2024-03-23T08:23:29.083 INFO:tasks.workunit.client.1.smithi193.stdout:6/679: write d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f84 [193068,37002] 0 2024-03-23T08:23:29.087 INFO:tasks.workunit.client.1.smithi193.stdout:4/877: getdents d11/d2d/da8 0 2024-03-23T08:23:29.087 INFO:tasks.workunit.client.1.smithi193.stdout:9/882: creat d2/d7/db/d32/d6b/f128 x:0 0 0 2024-03-23T08:23:29.088 INFO:tasks.workunit.client.1.smithi193.stdout:3/686: stat d1/de/d13/d21/d50/d44/f79 0 2024-03-23T08:23:29.089 INFO:tasks.workunit.client.1.smithi193.stdout:0/795: rename db/lb6 to db/d11/d14/d24/dc9/l11a 0 2024-03-23T08:23:29.089 INFO:tasks.workunit.client.1.smithi193.stdout:7/769: dwrite d6/d9/d1f/d23/d33/d3c/d41/f79 [0,4194304] 0 2024-03-23T08:23:29.089 INFO:tasks.workunit.client.1.smithi193.stdout:1/817: dwrite f5 [0,4194304] 0 2024-03-23T08:23:29.093 INFO:tasks.workunit.client.1.smithi193.stdout:2/852: symlink d0/d1b/d14/d56/d7c/d7e/led 0 2024-03-23T08:23:29.099 INFO:tasks.workunit.client.1.smithi193.stdout:8/805: rename df/c3a to df/d10/d3e/d54/dd2/dca/c111 0 2024-03-23T08:23:29.099 INFO:tasks.workunit.client.1.smithi193.stdout:5/828: truncate d0/d1c/d7a/db1/d72/d73/f44 411238 0 2024-03-23T08:23:29.099 INFO:tasks.workunit.client.1.smithi193.stdout:9/883: mknod d2/d7/db/d32/d6b/c129 0 2024-03-23T08:23:29.099 INFO:tasks.workunit.client.1.smithi193.stdout:2/853: fdatasync d0/d1b/d14/d66/d4e/fc4 0 2024-03-23T08:23:29.100 INFO:tasks.workunit.client.1.smithi193.stdout:7/770: unlink d6/f39 0 2024-03-23T08:23:29.101 INFO:tasks.workunit.client.1.smithi193.stdout:3/687: mknod d1/de/d13/d21/d59/d60/cef 0 2024-03-23T08:23:29.102 INFO:tasks.workunit.client.1.smithi193.stdout:8/806: fdatasync df/d63/d84/da8/fed 0 2024-03-23T08:23:29.103 INFO:tasks.workunit.client.1.smithi193.stdout:6/680: write d0/d27/d81/df/d11/d36/d61/d12/f35 [1902946,51919] 0 2024-03-23T08:23:29.104 INFO:tasks.workunit.client.1.smithi193.stdout:2/854: mkdir d0/d1b/d14/d25/dc2/dee 0 2024-03-23T08:23:29.106 INFO:tasks.workunit.client.1.smithi193.stdout:8/807: write df/d10/d3e/d54/dd2/dca/ff1 [507664,46091] 0 2024-03-23T08:23:29.107 INFO:tasks.workunit.client.1.smithi193.stdout:2/855: readlink d0/d1b/d14/d25/l3d 0 2024-03-23T08:23:29.109 INFO:tasks.workunit.client.1.smithi193.stdout:3/688: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/d85/db9/df0 0 2024-03-23T08:23:29.110 INFO:tasks.workunit.client.1.smithi193.stdout:1/818: getdents da/d13/d24/dac/dea 0 2024-03-23T08:23:29.112 INFO:tasks.workunit.client.1.smithi193.stdout:2/856: symlink d0/d1b/d14/d56/da3/lef 0 2024-03-23T08:23:29.113 INFO:tasks.workunit.client.1.smithi193.stdout:6/681: mknod d0/d27/d81/dce/cd5 0 2024-03-23T08:23:29.113 INFO:tasks.workunit.client.1.smithi193.stdout:5/829: getdents d0/d1c/d3b/d81/df2/d97 0 2024-03-23T08:23:29.115 INFO:tasks.workunit.client.1.smithi193.stdout:3/689: unlink d1/de/d13/f62 0 2024-03-23T08:23:29.117 INFO:tasks.workunit.client.1.smithi193.stdout:3/690: dread - d1/de/d28/fec zero size 2024-03-23T08:23:29.117 INFO:tasks.workunit.client.1.smithi193.stdout:5/830: write d0/d1c/d3b/dbc/d112/d8e/fa7 [3746974,7933] 0 2024-03-23T08:23:29.120 INFO:tasks.workunit.client.1.smithi193.stdout:3/691: chown d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/cb6 0 1 2024-03-23T08:23:29.121 INFO:tasks.workunit.client.1.smithi193.stdout:6/682: mkdir d0/d27/d81/df/d11/d3b/d51/dd6 0 2024-03-23T08:23:29.122 INFO:tasks.workunit.client.1.smithi193.stdout:0/796: dread db/d11/d14/d21/d33/f43 [0,4194304] 0 2024-03-23T08:23:29.124 INFO:tasks.workunit.client.1.smithi193.stdout:7/771: truncate d6/d9/d1f/f50 209096 0 2024-03-23T08:23:29.125 INFO:tasks.workunit.client.1.smithi193.stdout:5/831: chown d0/d1c/d7a/db1/c83 24 1 2024-03-23T08:23:29.126 INFO:tasks.workunit.client.1.smithi193.stdout:8/808: getdents df/d10/d83 0 2024-03-23T08:23:29.126 INFO:tasks.workunit.client.1.smithi193.stdout:9/884: truncate d2/d7/db/d1b/d43/f84 676514 0 2024-03-23T08:23:29.127 INFO:tasks.workunit.client.1.smithi193.stdout:9/885: chown d2/d7/db/d1b/d5c 409 1 2024-03-23T08:23:29.127 INFO:tasks.workunit.client.1.smithi193.stdout:8/809: dread - df/d63/d84/da8/fed zero size 2024-03-23T08:23:29.132 INFO:tasks.workunit.client.1.smithi193.stdout:1/819: write da/dd/df/d7e/d86/f89 [533075,59795] 0 2024-03-23T08:23:29.134 INFO:tasks.workunit.client.1.smithi193.stdout:2/857: truncate d0/d1b/d14/d56/f96 161747 0 2024-03-23T08:23:29.135 INFO:tasks.workunit.client.1.smithi193.stdout:6/683: creat d0/d27/d81/fd7 x:0 0 0 2024-03-23T08:23:29.136 INFO:tasks.workunit.client.1.smithi193.stdout:2/858: fsync d0/d1b/f8 0 2024-03-23T08:23:29.138 INFO:tasks.workunit.client.1.smithi193.stdout:0/797: link db/d11/d14/d24/d2a/d96/lfe db/d11/d14/d24/d47/d59/l11b 0 2024-03-23T08:23:29.140 INFO:tasks.workunit.client.1.smithi193.stdout:7/772: write d6/d9/d1f/d23/d33/d38/d46/d82/f8c [291581,127506] 0 2024-03-23T08:23:29.141 INFO:tasks.workunit.client.1.smithi193.stdout:8/810: write df/d4b/d4d/d9c/fe2 [1079482,111704] 0 2024-03-23T08:23:29.146 INFO:tasks.workunit.client.1.smithi193.stdout:5/832: getdents d0/d5a/dbf/d106 0 2024-03-23T08:23:29.146 INFO:tasks.workunit.client.1.smithi193.stdout:6/684: dwrite d0/d27/d81/df/d11/d36/d61/f6d [0,4194304] 0 2024-03-23T08:23:29.147 INFO:tasks.workunit.client.1.smithi193.stdout:8/811: fdatasync df/f18 0 2024-03-23T08:23:29.149 INFO:tasks.workunit.client.1.smithi193.stdout:9/886: link d2/d7/c3e d2/d7/db/d1b/d43/d71/db4/dec/d5f/de3/c12a 0 2024-03-23T08:23:29.152 INFO:tasks.workunit.client.1.smithi193.stdout:6/685: write d0/d27/d81/d87/fa8 [1907167,109046] 0 2024-03-23T08:23:29.152 INFO:tasks.workunit.client.1.smithi193.stdout:0/798: fdatasync db/d11/d14/d21/f26 0 2024-03-23T08:23:29.152 INFO:tasks.workunit.client.1.smithi193.stdout:1/820: rename da/dd/df/d2e/l85 to da/dd/df/d7e/l108 0 2024-03-23T08:23:29.155 INFO:tasks.workunit.client.1.smithi193.stdout:8/812: rmdir df/d4b/d4d 39 2024-03-23T08:23:29.159 INFO:tasks.workunit.client.1.smithi193.stdout:9/887: mknod d2/d7/db/d17/d4b/d59/d97/d104/c12b 0 2024-03-23T08:23:29.160 INFO:tasks.workunit.client.1.smithi193.stdout:1/821: symlink da/dd/df/d7e/d7f/l109 0 2024-03-23T08:23:29.163 INFO:tasks.workunit.client.1.smithi193.stdout:2/859: rename d0/d1b/d14/da1/dd5/d72/f97 to d0/d28/d9f/ff0 0 2024-03-23T08:23:29.166 INFO:tasks.workunit.client.1.smithi193.stdout:7/773: dread d6/d9/d1f/f94 [0,4194304] 0 2024-03-23T08:23:29.167 INFO:tasks.workunit.client.1.smithi193.stdout:6/686: getdents d0/d27/d81/df/d11/d56 0 2024-03-23T08:23:29.168 INFO:tasks.workunit.client.1.smithi193.stdout:2/860: mknod d0/d1b/d3b/cf1 0 2024-03-23T08:23:29.170 INFO:tasks.workunit.client.1.smithi193.stdout:9/888: truncate d2/d7/db/d17/d33/f64 1225364 0 2024-03-23T08:23:29.177 INFO:tasks.workunit.client.1.smithi193.stdout:6/687: fdatasync d0/d27/d81/df/f9b 0 2024-03-23T08:23:29.177 INFO:tasks.workunit.client.1.smithi193.stdout:2/861: write d0/d1b/da7/fb3 [877609,69937] 0 2024-03-23T08:23:29.179 INFO:tasks.workunit.client.1.smithi193.stdout:7/774: chown d6/d9/d1f/d23/d33/d38/d46/d82/f8c 6122 1 2024-03-23T08:23:29.179 INFO:tasks.workunit.client.1.smithi193.stdout:8/813: dread df/d10/d67/d7f/fc2 [0,4194304] 0 2024-03-23T08:23:29.179 INFO:tasks.workunit.client.1.smithi193.stdout:5/833: dwrite d0/d1c/d3b/dbc/d112/f7c [0,4194304] 0 2024-03-23T08:23:29.183 INFO:tasks.workunit.client.1.smithi193.stdout:9/889: creat d2/d7/db/d1b/f12c x:0 0 0 2024-03-23T08:23:29.184 INFO:tasks.workunit.client.1.smithi193.stdout:1/822: dwrite da/f12 [8388608,4194304] 0 2024-03-23T08:23:29.187 INFO:tasks.workunit.client.1.smithi193.stdout:0/799: dwrite db/d11/d14/d24/d47/d59/fd9 [0,4194304] 0 2024-03-23T08:23:29.188 INFO:tasks.workunit.client.1.smithi193.stdout:6/688: unlink d0/d27/d81/f33 0 2024-03-23T08:23:29.190 INFO:tasks.workunit.client.1.smithi193.stdout:1/823: write da/dd/f39 [2618644,32280] 0 2024-03-23T08:23:29.193 INFO:tasks.workunit.client.1.smithi193.stdout:1/824: readlink da/d13/d24/dac/dea/d76/ld4 0 2024-03-23T08:23:29.194 INFO:tasks.workunit.client.1.smithi193.stdout:0/800: stat db/d11/d14/d24/d47/dc7/fb2 0 2024-03-23T08:23:29.196 INFO:tasks.workunit.client.1.smithi193.stdout:8/814: rename df/d10/d3e/d54/d10d/f10f to df/d63/d84/df4/f112 0 2024-03-23T08:23:29.197 INFO:tasks.workunit.client.1.smithi193.stdout:6/689: creat d0/d27/d81/da1/fd8 x:0 0 0 2024-03-23T08:23:29.202 INFO:tasks.workunit.client.1.smithi193.stdout:9/890: dwrite d2/d7/db/d1b/d43/d71/db4/dec/f4f [0,4194304] 0 2024-03-23T08:23:29.205 INFO:tasks.workunit.client.1.smithi193.stdout:1/825: unlink da/dd/df/d75/d98/c9a 0 2024-03-23T08:23:29.206 INFO:tasks.workunit.client.1.smithi193.stdout:0/801: truncate db/f1e 9198383 0 2024-03-23T08:23:29.206 INFO:tasks.workunit.client.1.smithi193.stdout:8/815: dwrite f3 [4194304,4194304] 0 2024-03-23T08:23:29.207 INFO:tasks.workunit.client.1.smithi193.stdout:0/802: write db/d11/d14/d24/f39 [1702566,68411] 0 2024-03-23T08:23:29.209 INFO:tasks.workunit.client.1.smithi193.stdout:7/775: link d6/d9/d1f/d23/d33/d38/d46/d82/cf5 d6/d9/d1f/d4d/d7c/c108 0 2024-03-23T08:23:29.212 INFO:tasks.workunit.client.1.smithi193.stdout:6/690: symlink d0/d27/d81/df/d11/ld9 0 2024-03-23T08:23:29.214 INFO:tasks.workunit.client.1.smithi193.stdout:8/816: rename df/d63/d84/c9a to df/d63/d76/c113 0 2024-03-23T08:23:29.215 INFO:tasks.workunit.client.1.smithi193.stdout:0/803: mknod db/d11/d14/d24/d47/d59/d8c/c11c 0 2024-03-23T08:23:29.220 INFO:tasks.workunit.client.1.smithi193.stdout:7/776: mknod d6/d9/d1f/d3e/d65/d7d/c109 0 2024-03-23T08:23:29.221 INFO:tasks.workunit.client.1.smithi193.stdout:6/691: write d0/d27/d81/df/d11/d36/d61/f1d [2578138,128798] 0 2024-03-23T08:23:29.225 INFO:tasks.workunit.client.1.smithi193.stdout:7/777: creat d6/d9/d1f/d23/d33/d38/d3b/d6c/f10a x:0 0 0 2024-03-23T08:23:29.226 INFO:tasks.workunit.client.1.smithi193.stdout:2/862: write d0/d1b/d14/d56/d7c/d7e/f88 [854727,92807] 0 2024-03-23T08:23:29.227 INFO:tasks.workunit.client.1.smithi193.stdout:0/804: unlink db/d11/d14/d21/d33/d69/d9b/dc1/lc8 0 2024-03-23T08:23:29.229 INFO:tasks.workunit.client.1.smithi193.stdout:6/692: rename d0/d27/d72 to d0/d27/d81/df/d11/d3b/d51/daa/dda 0 2024-03-23T08:23:29.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:28 smithi070 ceph-mon[39699]: pgmap v24: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 85 MiB/s rd, 257 MiB/s wr, 497 op/s 2024-03-23T08:23:29.233 INFO:tasks.workunit.client.1.smithi193.stdout:0/805: write db/d11/d14/d24/d47/f61 [2695683,9598] 0 2024-03-23T08:23:29.233 INFO:tasks.workunit.client.1.smithi193.stdout:2/863: truncate d0/d1b/d14/d25/fe8 875966 0 2024-03-23T08:23:29.234 INFO:tasks.workunit.client.1.smithi193.stdout:5/834: dwrite d0/d1c/d7a/db1/d72/d99/dd8/fd1 [0,4194304] 0 2024-03-23T08:23:29.235 INFO:tasks.workunit.client.1.smithi193.stdout:1/826: truncate da/fa8 3985311 0 2024-03-23T08:23:29.237 INFO:tasks.workunit.client.1.smithi193.stdout:0/806: fsync db/d11/d14/d24/d2a/d4d/f81 0 2024-03-23T08:23:29.238 INFO:tasks.workunit.client.1.smithi193.stdout:5/835: fsync d0/d1c/d7a/db1/d72/f11d 0 2024-03-23T08:23:29.241 INFO:tasks.workunit.client.1.smithi193.stdout:8/817: write df/d10/f7b [1801010,34520] 0 2024-03-23T08:23:29.242 INFO:tasks.workunit.client.1.smithi193.stdout:5/836: dread - d0/d1f/fc4 zero size 2024-03-23T08:23:29.242 INFO:tasks.workunit.client.1.smithi193.stdout:7/778: symlink d6/d9/d1f/d23/d33/d38/d3b/db7/l10b 0 2024-03-23T08:23:29.245 INFO:tasks.workunit.client.1.smithi193.stdout:2/864: rename d0/d28/fa8 to d0/d1b/d14/da1/ff2 0 2024-03-23T08:23:29.246 INFO:tasks.workunit.client.1.smithi193.stdout:0/807: mkdir db/d11/d14/d24/d2a/d96/d11d 0 2024-03-23T08:23:29.246 INFO:tasks.workunit.client.1.smithi193.stdout:8/818: read df/d63/f68 [3981850,677] 0 2024-03-23T08:23:29.247 INFO:tasks.workunit.client.1.smithi193.stdout:8/819: chown df/d10/d67/fa9 39794600 1 2024-03-23T08:23:29.249 INFO:tasks.workunit.client.1.smithi193.stdout:1/827: mknod da/dd/df/ddc/dfd/c10a 0 2024-03-23T08:23:29.249 INFO:tasks.workunit.client.1.smithi193.stdout:7/779: dread - d6/d9/d1f/d23/d33/d38/f53 zero size 2024-03-23T08:23:29.254 INFO:tasks.workunit.client.1.smithi193.stdout:8/820: chown df/l55 6 1 2024-03-23T08:23:29.255 INFO:tasks.workunit.client.1.smithi193.stdout:1/828: dread da/dd/f78 [0,4194304] 0 2024-03-23T08:23:29.255 INFO:tasks.workunit.client.1.smithi193.stdout:2/865: creat d0/dcf/ff3 x:0 0 0 2024-03-23T08:23:29.255 INFO:tasks.workunit.client.1.smithi193.stdout:8/821: fsync df/d4b/fe4 0 2024-03-23T08:23:29.256 INFO:tasks.workunit.client.1.smithi193.stdout:0/808: dwrite db/d11/d14/d24/d47/d59/fd9 [0,4194304] 0 2024-03-23T08:23:29.256 INFO:tasks.workunit.client.1.smithi193.stdout:6/693: link d0/l18 d0/d27/d81/da1/db8/ldb 0 2024-03-23T08:23:29.258 INFO:tasks.workunit.client.1.smithi193.stdout:7/780: symlink d6/d9/l10c 0 2024-03-23T08:23:29.259 INFO:tasks.workunit.client.1.smithi193.stdout:6/694: write d0/d27/d81/df/faf [804437,68633] 0 2024-03-23T08:23:29.260 INFO:tasks.workunit.client.1.smithi193.stdout:2/866: write d0/d1b/d14/da1/dd5/d72/fc9 [517210,127422] 0 2024-03-23T08:23:29.261 INFO:tasks.workunit.client.1.smithi193.stdout:1/829: chown da/d13/lb7 339 1 2024-03-23T08:23:29.263 INFO:tasks.workunit.client.1.smithi193.stdout:7/781: mknod d6/dbd/c10d 0 2024-03-23T08:23:29.263 INFO:tasks.workunit.client.1.smithi193.stdout:2/867: fsync d0/fd 0 2024-03-23T08:23:29.264 INFO:tasks.workunit.client.1.smithi193.stdout:0/809: getdents db/d11/d14/d24/d2a/d96/d115 0 2024-03-23T08:23:29.265 INFO:tasks.workunit.client.1.smithi193.stdout:8/822: mknod df/dc6/d8b/c114 0 2024-03-23T08:23:29.265 INFO:tasks.workunit.client.1.smithi193.stdout:1/830: fdatasync da/d13/d24/dac/dea/f8c 0 2024-03-23T08:23:29.267 INFO:tasks.workunit.client.1.smithi193.stdout:8/823: fdatasync df/d63/d84/f8a 0 2024-03-23T08:23:29.268 INFO:tasks.workunit.client.1.smithi193.stdout:2/868: write d0/d1b/d14/f41 [816689,20321] 0 2024-03-23T08:23:29.269 INFO:tasks.workunit.client.1.smithi193.stdout:6/695: getdents d0/d27/d81/df/d11/d3b/d51/dd6 0 2024-03-23T08:23:29.270 INFO:tasks.workunit.client.1.smithi193.stdout:7/782: fsync d6/d9/d1f/d23/d33/d38/d3b/d6c/f8d 0 2024-03-23T08:23:29.271 INFO:tasks.workunit.client.1.smithi193.stdout:1/831: unlink da/dd/df/d7e/d7f/dde/ce3 0 2024-03-23T08:23:29.272 INFO:tasks.workunit.client.1.smithi193.stdout:0/810: truncate db/d11/d14/d24/d2a/d96/fb9 716673 0 2024-03-23T08:23:29.272 INFO:tasks.workunit.client.1.smithi193.stdout:8/824: write df/fc8 [274529,28271] 0 2024-03-23T08:23:29.275 INFO:tasks.workunit.client.1.smithi193.stdout:5/837: dwrite d0/d1c/d3b/f3c [0,4194304] 0 2024-03-23T08:23:29.277 INFO:tasks.workunit.client.1.smithi193.stdout:2/869: write d0/d1b/d14/d66/d50/d92/da9/fab [734747,35897] 0 2024-03-23T08:23:29.277 INFO:tasks.workunit.client.1.smithi193.stdout:6/696: creat d0/d27/d81/df/d11/d3b/d51/fdc x:0 0 0 2024-03-23T08:23:29.278 INFO:tasks.workunit.client.1.smithi193.stdout:7/783: write d6/d9/d1f/d4d/d7c/d8e/fbf [634539,127543] 0 2024-03-23T08:23:29.280 INFO:tasks.workunit.client.1.smithi193.stdout:5/838: write d0/fe [4329882,45363] 0 2024-03-23T08:23:29.280 INFO:tasks.workunit.client.1.smithi193.stdout:2/870: readlink d0/d1b/d14/d66/d4e/l67 0 2024-03-23T08:23:29.282 INFO:tasks.workunit.client.1.smithi193.stdout:8/825: creat df/d10/d67/d7f/f115 x:0 0 0 2024-03-23T08:23:29.283 INFO:tasks.workunit.client.1.smithi193.stdout:7/784: chown d6/d9/d1f/d23/d33/l3a 635701 1 2024-03-23T08:23:29.283 INFO:tasks.workunit.client.1.smithi193.stdout:2/871: write d0/d1b/f13 [924508,42843] 0 2024-03-23T08:23:29.283 INFO:tasks.workunit.client.1.smithi193.stdout:1/832: mkdir da/d13/d24/dac/dea/d76/d77/d10b 0 2024-03-23T08:23:29.284 INFO:tasks.workunit.client.1.smithi193.stdout:5/839: write d0/d1c/d7a/f113 [816479,10416] 0 2024-03-23T08:23:29.284 INFO:tasks.workunit.client.1.smithi193.stdout:0/811: getdents db/dc6 0 2024-03-23T08:23:29.286 INFO:tasks.workunit.client.1.smithi193.stdout:7/785: fdatasync d6/d9/f3d 0 2024-03-23T08:23:29.288 INFO:tasks.workunit.client.1.smithi193.stdout:8/826: getdents df/d63/d76 0 2024-03-23T08:23:29.289 INFO:tasks.workunit.client.1.smithi193.stdout:2/872: fsync d0/d1b/d14/d66/d50/d92/f9a 0 2024-03-23T08:23:29.291 INFO:tasks.workunit.client.1.smithi193.stdout:6/697: link d0/d27/d81/df/d11/d3b/d51/c98 d0/d27/d81/da1/cdd 0 2024-03-23T08:23:29.292 INFO:tasks.workunit.client.1.smithi193.stdout:7/786: rmdir d6/d9/d1f/d23/d33/d38/d3b 39 2024-03-23T08:23:29.293 INFO:tasks.workunit.client.1.smithi193.stdout:8/827: write df/d4b/d4d/d2a/f3c [4455758,18260] 0 2024-03-23T08:23:29.294 INFO:tasks.workunit.client.1.smithi193.stdout:3/692: sync 2024-03-23T08:23:29.294 INFO:tasks.workunit.client.1.smithi193.stdout:4/878: sync 2024-03-23T08:23:29.295 INFO:tasks.workunit.client.1.smithi193.stdout:0/812: rename db/d11/d14/d24/f8d to db/d11/d14/d3b/f11e 0 2024-03-23T08:23:29.297 INFO:tasks.workunit.client.1.smithi193.stdout:2/873: dread d0/d1b/d14/d25/fe8 [0,4194304] 0 2024-03-23T08:23:29.298 INFO:tasks.workunit.client.1.smithi193.stdout:1/833: dwrite da/d13/d24/dac/dea/f61 [0,4194304] 0 2024-03-23T08:23:29.298 INFO:tasks.workunit.client.1.smithi193.stdout:4/879: chown d11/d27/d85 803292085 1 2024-03-23T08:23:29.299 INFO:tasks.workunit.client.1.smithi193.stdout:8/828: fdatasync df/d63/d76/fe5 0 2024-03-23T08:23:29.299 INFO:tasks.workunit.client.1.smithi193.stdout:6/698: mknod d0/d27/d81/d87/cde 0 2024-03-23T08:23:29.301 INFO:tasks.workunit.client.1.smithi193.stdout:5/840: creat d0/d1c/d7a/db1/d72/d73/f120 x:0 0 0 2024-03-23T08:23:29.301 INFO:tasks.workunit.client.1.smithi193.stdout:4/880: read f5 [3736592,60277] 0 2024-03-23T08:23:29.304 INFO:tasks.workunit.client.1.smithi193.stdout:6/699: write d0/d27/d81/df/d11/d3b/f66 [500937,127946] 0 2024-03-23T08:23:29.306 INFO:tasks.workunit.client.1.smithi193.stdout:4/881: truncate d11/d1a/d33/d42/db2/dbe/d50/f61 1472579 0 2024-03-23T08:23:29.310 INFO:tasks.workunit.client.1.smithi193.stdout:1/834: rename da/dd/df/d2e/f48 to da/dd/df/d7e/d7f/db2/f10c 0 2024-03-23T08:23:29.310 INFO:tasks.workunit.client.1.smithi193.stdout:5/841: chown d0/d1f/f49 22171029 1 2024-03-23T08:23:29.311 INFO:tasks.workunit.client.1.smithi193.stdout:0/813: creat db/d52/f11f x:0 0 0 2024-03-23T08:23:29.313 INFO:tasks.workunit.client.1.smithi193.stdout:6/700: fdatasync d0/d27/d81/df/d11/d3b/f66 0 2024-03-23T08:23:29.314 INFO:tasks.workunit.client.1.smithi193.stdout:0/814: write db/d52/f11f [501177,73710] 0 2024-03-23T08:23:29.316 INFO:tasks.workunit.client.1.smithi193.stdout:1/835: mknod da/dd/df/d7e/d7f/df6/c10d 0 2024-03-23T08:23:29.317 INFO:tasks.workunit.client.1.smithi193.stdout:5/842: mkdir d0/d1c/d7a/db1/d72/d73/d45/d6e/d121 0 2024-03-23T08:23:29.320 INFO:tasks.workunit.client.1.smithi193.stdout:2/874: dwrite d0/d37/f8a [0,4194304] 0 2024-03-23T08:23:29.321 INFO:tasks.workunit.client.1.smithi193.stdout:6/701: creat d0/d27/d81/da1/db8/fdf x:0 0 0 2024-03-23T08:23:29.328 INFO:tasks.workunit.client.1.smithi193.stdout:3/693: dwrite d1/de/d28/fc5 [0,4194304] 0 2024-03-23T08:23:29.330 INFO:tasks.workunit.client.1.smithi193.stdout:8/829: dwrite df/d10/f2f [0,4194304] 0 2024-03-23T08:23:29.330 INFO:tasks.workunit.client.1.smithi193.stdout:5/843: rename d0/d5a/ddf/cf0 to d0/d1c/d3b/d4b/dff/c122 0 2024-03-23T08:23:29.332 INFO:tasks.workunit.client.1.smithi193.stdout:7/787: dread d6/d9/d1f/d23/d33/d38/d3b/f5a [0,4194304] 0 2024-03-23T08:23:29.332 INFO:tasks.workunit.client.1.smithi193.stdout:1/836: truncate da/dd/f14 310939 0 2024-03-23T08:23:29.333 INFO:tasks.workunit.client.1.smithi193.stdout:8/830: readlink df/dc6/lef 0 2024-03-23T08:23:29.334 INFO:tasks.workunit.client.1.smithi193.stdout:6/702: fdatasync d0/d27/d81/df/d11/f30 0 2024-03-23T08:23:29.336 INFO:tasks.workunit.client.1.smithi193.stdout:2/875: truncate d0/f1d 1967707 0 2024-03-23T08:23:29.337 INFO:tasks.workunit.client.1.smithi193.stdout:3/694: fdatasync d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fbb 0 2024-03-23T08:23:29.339 INFO:tasks.workunit.client.1.smithi193.stdout:5/844: creat d0/d1c/d7a/d86/f123 x:0 0 0 2024-03-23T08:23:29.339 INFO:tasks.workunit.client.1.smithi193.stdout:8/831: chown df/d63/f68 466953 1 2024-03-23T08:23:29.340 INFO:tasks.workunit.client.1.smithi193.stdout:1/837: write da/d13/d24/f43 [257050,113982] 0 2024-03-23T08:23:29.341 INFO:tasks.workunit.client.1.smithi193.stdout:2/876: dread - d0/d1b/d3b/fec zero size 2024-03-23T08:23:29.342 INFO:tasks.workunit.client.1.smithi193.stdout:0/815: dwrite db/d11/d14/d21/d33/d48/f54 [4194304,4194304] 0 2024-03-23T08:23:29.342 INFO:tasks.workunit.client.1.smithi193.stdout:5/845: dread - d0/d1f/f2e zero size 2024-03-23T08:23:29.343 INFO:tasks.workunit.client.1.smithi193.stdout:7/788: chown d6/d9/d1f/d23/d33/d3c/d41/c98 398938 1 2024-03-23T08:23:29.343 INFO:tasks.workunit.client.1.smithi193.stdout:3/695: chown d1/de/d28/fec 1061 1 2024-03-23T08:23:29.348 INFO:tasks.workunit.client.1.smithi193.stdout:5/846: readlink d0/d1f/la9 0 2024-03-23T08:23:29.348 INFO:tasks.workunit.client.1.smithi193.stdout:6/703: chown d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f7e 1219 1 2024-03-23T08:23:29.349 INFO:tasks.workunit.client.1.smithi193.stdout:1/838: readlink da/dd/df/d7e/d86/d95/dc4/lcd 0 2024-03-23T08:23:29.352 INFO:tasks.workunit.client.1.smithi193.stdout:2/877: creat d0/d37/d48/ff4 x:0 0 0 2024-03-23T08:23:29.352 INFO:tasks.workunit.client.1.smithi193.stdout:7/789: write d6/d9/d1f/d23/d33/d38/d3b/f48 [1018775,57018] 0 2024-03-23T08:23:29.355 INFO:tasks.workunit.client.1.smithi193.stdout:5/847: read d0/d1c/d3b/dbc/d112/d8e/fa5 [153954,105636] 0 2024-03-23T08:23:29.355 INFO:tasks.workunit.client.1.smithi193.stdout:1/839: fsync da/dd/df/d2e/f4f 0 2024-03-23T08:23:29.357 INFO:tasks.workunit.client.1.smithi193.stdout:1/840: write da/dd/df/d7c/fd8 [1040548,29310] 0 2024-03-23T08:23:29.357 INFO:tasks.workunit.client.1.smithi193.stdout:0/816: write db/d11/d14/d21/f27 [4481688,14942] 0 2024-03-23T08:23:29.361 INFO:tasks.workunit.client.1.smithi193.stdout:8/832: truncate df/d10/d3e/d54/dd2/fb6 1609629 0 2024-03-23T08:23:29.365 INFO:tasks.workunit.client.1.smithi193.stdout:3/696: link d1/de/d13/d21/d25/d2b/d3c/d4a/d85/fd8 d1/de/d13/d21/d25/d2b/d3c/d4a/de4/ff1 0 2024-03-23T08:23:29.366 INFO:tasks.workunit.client.1.smithi193.stdout:6/704: rename d0/d27/f91 to d0/d27/d81/fe0 0 2024-03-23T08:23:29.367 INFO:tasks.workunit.client.1.smithi193.stdout:1/841: dwrite da/d13/d24/dac/dea/f67 [0,4194304] 0 2024-03-23T08:23:29.370 INFO:tasks.workunit.client.1.smithi193.stdout:5/848: creat d0/d1c/d7a/db1/d72/d73/f124 x:0 0 0 2024-03-23T08:23:29.372 INFO:tasks.workunit.client.1.smithi193.stdout:0/817: rename db/d11/d14/d21/c57 to db/d11/d14/d24/d47/d59/d8c/c120 0 2024-03-23T08:23:29.372 INFO:tasks.workunit.client.1.smithi193.stdout:3/697: rmdir d1/de/d13/d21/d25/d34 39 2024-03-23T08:23:29.375 INFO:tasks.workunit.client.1.smithi193.stdout:5/849: rmdir d0/d1c/d3b/dbc 39 2024-03-23T08:23:29.378 INFO:tasks.workunit.client.1.smithi193.stdout:8/833: dwrite df/f2c [4194304,4194304] 0 2024-03-23T08:23:29.379 INFO:tasks.workunit.client.1.smithi193.stdout:8/834: stat df/d10/d3e/f5f 0 2024-03-23T08:23:29.382 INFO:tasks.workunit.client.1.smithi193.stdout:3/698: mkdir d1/de/d13/d21/d59/df2 0 2024-03-23T08:23:29.382 INFO:tasks.workunit.client.1.smithi193.stdout:7/790: dread d6/d9/f36 [0,4194304] 0 2024-03-23T08:23:29.382 INFO:tasks.workunit.client.1.smithi193.stdout:8/835: creat df/d10/d67/f116 x:0 0 0 2024-03-23T08:23:29.383 INFO:tasks.workunit.client.1.smithi193.stdout:2/878: dwrite d0/d1b/d19/fe2 [0,4194304] 0 2024-03-23T08:23:29.384 INFO:tasks.workunit.client.1.smithi193.stdout:0/818: creat db/d11/d14/d21/d33/d69/d9b/dc1/f121 x:0 0 0 2024-03-23T08:23:29.387 INFO:tasks.workunit.client.1.smithi193.stdout:3/699: read d1/de/d13/f5b [1164161,90944] 0 2024-03-23T08:23:29.389 INFO:tasks.workunit.client.1.smithi193.stdout:8/836: dread - df/d10/d67/d7f/f104 zero size 2024-03-23T08:23:29.391 INFO:tasks.workunit.client.1.smithi193.stdout:0/819: write db/d11/d14/d51/f56 [2139444,105273] 0 2024-03-23T08:23:29.394 INFO:tasks.workunit.client.1.smithi193.stdout:9/891: sync 2024-03-23T08:23:29.396 INFO:tasks.workunit.client.1.smithi193.stdout:6/705: dwrite d0/d27/d81/df/d11/d36/d61/d3a/f45 [4194304,4194304] 0 2024-03-23T08:23:29.398 INFO:tasks.workunit.client.1.smithi193.stdout:1/842: truncate da/d13/d3f/f87 1513772 0 2024-03-23T08:23:29.403 INFO:tasks.workunit.client.1.smithi193.stdout:3/700: unlink d1/de/d13/d21/d25/l42 0 2024-03-23T08:23:29.404 INFO:tasks.workunit.client.1.smithi193.stdout:8/837: truncate df/f5a 315984 0 2024-03-23T08:23:29.405 INFO:tasks.workunit.client.1.smithi193.stdout:5/850: dread d0/d1c/d7a/db1/f8b [0,4194304] 0 2024-03-23T08:23:29.407 INFO:tasks.workunit.client.1.smithi193.stdout:7/791: dwrite d6/d9/d4e/fae [0,4194304] 0 2024-03-23T08:23:29.408 INFO:tasks.workunit.client.1.smithi193.stdout:1/843: mknod da/dd/df/d2e/c10e 0 2024-03-23T08:23:29.411 INFO:tasks.workunit.client.1.smithi193.stdout:2/879: dwrite d0/d28/daf/fb9 [0,4194304] 0 2024-03-23T08:23:29.411 INFO:tasks.workunit.client.1.smithi193.stdout:8/838: write df/f18 [2570665,104453] 0 2024-03-23T08:23:29.412 INFO:tasks.workunit.client.1.smithi193.stdout:5/851: chown d0/d1c/d7a/db1/d72/d73/d45/d6e 24 1 2024-03-23T08:23:29.412 INFO:tasks.workunit.client.1.smithi193.stdout:2/880: chown d0/dcf 0 1 2024-03-23T08:23:29.413 INFO:tasks.workunit.client.1.smithi193.stdout:9/892: rename d2/d7/db/d32/d6b to d2/d7/db/d12d 0 2024-03-23T08:23:29.414 INFO:tasks.workunit.client.1.smithi193.stdout:7/792: unlink d6/d9/d4e/c4f 0 2024-03-23T08:23:29.415 INFO:tasks.workunit.client.1.smithi193.stdout:3/701: mknod d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/cf3 0 2024-03-23T08:23:29.419 INFO:tasks.workunit.client.1.smithi193.stdout:6/706: dwrite d0/fd4 [0,4194304] 0 2024-03-23T08:23:29.421 INFO:tasks.workunit.client.1.smithi193.stdout:8/839: creat df/d10/d83/f117 x:0 0 0 2024-03-23T08:23:29.423 INFO:tasks.workunit.client.1.smithi193.stdout:9/893: dread d2/d7/db/d1b/d43/d71/db4/dec/d3c/fbe [0,4194304] 0 2024-03-23T08:23:29.424 INFO:tasks.workunit.client.1.smithi193.stdout:9/894: write d2/d7/db/d7a/d95/fc6 [744062,78544] 0 2024-03-23T08:23:29.429 INFO:tasks.workunit.client.1.smithi193.stdout:0/820: write db/d11/d14/d24/d47/f74 [1714765,111799] 0 2024-03-23T08:23:29.431 INFO:tasks.workunit.client.1.smithi193.stdout:5/852: rename d0/d1c/d7a/db1/c83 to d0/d1c/d7a/db1/d72/d73/d45/daa/c125 0 2024-03-23T08:23:29.432 INFO:tasks.workunit.client.1.smithi193.stdout:6/707: mkdir d0/d27/d81/df/d11/d36/d61/d7c/d69/de1 0 2024-03-23T08:23:29.433 INFO:tasks.workunit.client.1.smithi193.stdout:6/708: fdatasync d0/d27/d81/fd7 0 2024-03-23T08:23:29.435 INFO:tasks.workunit.client.1.smithi193.stdout:6/709: truncate d0/d27/d81/da1/fd3 963423 0 2024-03-23T08:23:29.435 INFO:tasks.workunit.client.1.smithi193.stdout:9/895: unlink d2/d7/fcb 0 2024-03-23T08:23:29.436 INFO:tasks.workunit.client.1.smithi193.stdout:6/710: stat d0/d27/d81/df/d11/d36/d61/f48 0 2024-03-23T08:23:29.436 INFO:tasks.workunit.client.1.smithi193.stdout:9/896: readlink d2/d7/db/d17/d33/db2/l119 0 2024-03-23T08:23:29.436 INFO:tasks.workunit.client.1.smithi193.stdout:2/881: fsync d0/d28/daf/fb9 0 2024-03-23T08:23:29.438 INFO:tasks.workunit.client.1.smithi193.stdout:9/897: fdatasync d2/d7/db/d1b/d5c/ff5 0 2024-03-23T08:23:29.440 INFO:tasks.workunit.client.1.smithi193.stdout:0/821: mkdir db/d11/d14/d3b/d71/d122 0 2024-03-23T08:23:29.443 INFO:tasks.workunit.client.1.smithi193.stdout:0/822: chown db/d11/d14/d24/ddd/fef 90841 1 2024-03-23T08:23:29.444 INFO:tasks.workunit.client.1.smithi193.stdout:6/711: write d0/d27/d81/df/d11/d56/f8d [3205078,102284] 0 2024-03-23T08:23:29.444 INFO:tasks.workunit.client.1.smithi193.stdout:5/853: rmdir d0/d1c/d7a/db1/d72/d73 39 2024-03-23T08:23:29.447 INFO:tasks.workunit.client.1.smithi193.stdout:5/854: read - d0/d1f/fca zero size 2024-03-23T08:23:29.448 INFO:tasks.workunit.client.1.smithi193.stdout:2/882: dwrite d0/d37/d48/ff4 [0,4194304] 0 2024-03-23T08:23:29.450 INFO:tasks.workunit.client.1.smithi193.stdout:6/712: fdatasync d0/d27/d81/df/d11/d3b/f66 0 2024-03-23T08:23:29.450 INFO:tasks.workunit.client.1.smithi193.stdout:7/793: write d6/d9/f31 [1861599,99995] 0 2024-03-23T08:23:29.451 INFO:tasks.workunit.client.1.smithi193.stdout:9/898: dwrite d2/d7/db/d1b/fef [0,4194304] 0 2024-03-23T08:23:29.451 INFO:tasks.workunit.client.1.smithi193.stdout:8/840: creat df/d10/f118 x:0 0 0 2024-03-23T08:23:29.452 INFO:tasks.workunit.client.1.smithi193.stdout:3/702: getdents d1/de/d13 0 2024-03-23T08:23:29.453 INFO:tasks.workunit.client.1.smithi193.stdout:2/883: fdatasync d0/d28/daf/fb9 0 2024-03-23T08:23:29.455 INFO:tasks.workunit.client.1.smithi193.stdout:0/823: creat db/d11/d14/d21/d33/d69/d9b/dc1/f123 x:0 0 0 2024-03-23T08:23:29.456 INFO:tasks.workunit.client.1.smithi193.stdout:7/794: creat d6/d9/d1f/d3e/d65/d70/f10e x:0 0 0 2024-03-23T08:23:29.457 INFO:tasks.workunit.client.1.smithi193.stdout:8/841: creat df/d10/d67/f119 x:0 0 0 2024-03-23T08:23:29.459 INFO:tasks.workunit.client.1.smithi193.stdout:6/713: truncate d0/d27/d81/df/d11/d3b/f66 745108 0 2024-03-23T08:23:29.460 INFO:tasks.workunit.client.1.smithi193.stdout:9/899: dread d2/d7/db/d1b/d43/d71/db4/d78/fbc [0,4194304] 0 2024-03-23T08:23:29.461 INFO:tasks.workunit.client.1.smithi193.stdout:9/900: readlink d2/d7/db/d1b/l1c 0 2024-03-23T08:23:29.461 INFO:tasks.workunit.client.1.smithi193.stdout:3/703: creat d1/de/d13/d21/ff4 x:0 0 0 2024-03-23T08:23:29.463 INFO:tasks.workunit.client.1.smithi193.stdout:8/842: unlink df/d4b/d4d/l51 0 2024-03-23T08:23:29.464 INFO:tasks.workunit.client.1.smithi193.stdout:8/843: truncate df/dc6/d8b/fc0 594499 0 2024-03-23T08:23:29.465 INFO:tasks.workunit.client.1.smithi193.stdout:9/901: read d2/d7/db/f34 [163791,28082] 0 2024-03-23T08:23:29.466 INFO:tasks.workunit.client.1.smithi193.stdout:0/824: creat db/d11/d14/d21/d33/d69/d9b/dc1/f124 x:0 0 0 2024-03-23T08:23:29.466 INFO:tasks.workunit.client.1.smithi193.stdout:5/855: write d0/d1f/f22 [1791974,98049] 0 2024-03-23T08:23:29.469 INFO:tasks.workunit.client.1.smithi193.stdout:4/882: sync 2024-03-23T08:23:29.469 INFO:tasks.workunit.client.1.smithi193.stdout:3/704: readlink d1/de/d13/d21/d25/d34/l49 0 2024-03-23T08:23:29.469 INFO:tasks.workunit.client.1.smithi193.stdout:5/856: chown d0/d1c/d7a/db1/d72/d99/lae 82 1 2024-03-23T08:23:29.471 INFO:tasks.workunit.client.1.smithi193.stdout:0/825: creat db/d11/d14/d21/dd2/f125 x:0 0 0 2024-03-23T08:23:29.471 INFO:tasks.workunit.client.1.smithi193.stdout:4/883: readlink d11/d2d/d58/d80/d9d/lf9 0 2024-03-23T08:23:29.471 INFO:tasks.workunit.client.1.smithi193.stdout:3/705: chown d1/de/d13/d21/d25/d34/da8 10972884 1 2024-03-23T08:23:29.472 INFO:tasks.workunit.client.1.smithi193.stdout:8/844: creat df/d10/d83/dcb/f11a x:0 0 0 2024-03-23T08:23:29.473 INFO:tasks.workunit.client.1.smithi193.stdout:0/826: chown db/d11/d14/d3b/f11e 331 1 2024-03-23T08:23:29.474 INFO:tasks.workunit.client.1.smithi193.stdout:9/902: mknod d2/d7/db/d17/d4b/d59/df8/c12e 0 2024-03-23T08:23:29.477 INFO:tasks.workunit.client.1.smithi193.stdout:7/795: dread d6/d9/d1f/d4d/fa9 [0,4194304] 0 2024-03-23T08:23:29.477 INFO:tasks.workunit.client.1.smithi193.stdout:0/827: chown db/d11/d14/d51/f56 29110517 1 2024-03-23T08:23:29.478 INFO:tasks.workunit.client.1.smithi193.stdout:8/845: creat df/d63/d84/dc5/f11b x:0 0 0 2024-03-23T08:23:29.479 INFO:tasks.workunit.client.1.smithi193.stdout:3/706: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/dae/df5 0 2024-03-23T08:23:29.480 INFO:tasks.workunit.client.1.smithi193.stdout:6/714: truncate d0/d27/d81/d87/fa8 379083 0 2024-03-23T08:23:29.480 INFO:tasks.workunit.client.1.smithi193.stdout:7/796: fdatasync d6/d9/d1f/fa4 0 2024-03-23T08:23:29.480 INFO:tasks.workunit.client.1.smithi193.stdout:5/857: dwrite d0/fe [0,4194304] 0 2024-03-23T08:23:29.482 INFO:tasks.workunit.client.1.smithi193.stdout:3/707: write d1/de/d13/d21/ff4 [825631,33548] 0 2024-03-23T08:23:29.484 INFO:tasks.workunit.client.1.smithi193.stdout:8/846: fdatasync df/d10/d3e/fb0 0 2024-03-23T08:23:29.488 INFO:tasks.workunit.client.1.smithi193.stdout:5/858: write d0/d1c/fb0 [180123,58779] 0 2024-03-23T08:23:29.490 INFO:tasks.workunit.client.1.smithi193.stdout:0/828: creat db/d11/d14/f126 x:0 0 0 2024-03-23T08:23:29.493 INFO:tasks.workunit.client.1.smithi193.stdout:9/903: dread d2/d7/db/d1b/d43/d71/db4/dec/f40 [0,4194304] 0 2024-03-23T08:23:29.494 INFO:tasks.workunit.client.1.smithi193.stdout:7/797: dread d6/d9/d1f/d23/d33/d38/d66/fc1 [0,4194304] 0 2024-03-23T08:23:29.496 INFO:tasks.workunit.client.1.smithi193.stdout:8/847: rename df/cce to df/d63/d84/da8/c11c 0 2024-03-23T08:23:29.496 INFO:tasks.workunit.client.1.smithi193.stdout:5/859: chown d0/d1c/d3b/d81/df2/d97/faf 32 1 2024-03-23T08:23:29.498 INFO:tasks.workunit.client.1.smithi193.stdout:6/715: truncate d0/d27/d81/fe0 4321558 0 2024-03-23T08:23:29.499 INFO:tasks.workunit.client.1.smithi193.stdout:4/884: dwrite d11/d2d/d58/f86 [0,4194304] 0 2024-03-23T08:23:29.501 INFO:tasks.workunit.client.1.smithi193.stdout:7/798: symlink d6/d9/d4e/d81/l10f 0 2024-03-23T08:23:29.502 INFO:tasks.workunit.client.1.smithi193.stdout:8/848: mkdir df/d11d 0 2024-03-23T08:23:29.508 INFO:tasks.workunit.client.1.smithi193.stdout:4/885: chown d11/d1a/d33/d42/db2/dbe/d50/f61 115 1 2024-03-23T08:23:29.512 INFO:tasks.workunit.client.1.smithi193.stdout:4/886: fdatasync d11/d1a/d33/d7f/de0/fa0 0 2024-03-23T08:23:29.513 INFO:tasks.workunit.client.1.smithi193.stdout:0/829: dwrite db/d11/d14/d24/d2a/d4d/f50 [0,4194304] 0 2024-03-23T08:23:29.513 INFO:tasks.workunit.client.1.smithi193.stdout:5/860: dwrite d0/f7 [0,4194304] 0 2024-03-23T08:23:29.514 INFO:tasks.workunit.client.1.smithi193.stdout:9/904: dread d2/d7/db/d17/d4b/d8b/fb7 [0,4194304] 0 2024-03-23T08:23:29.518 INFO:tasks.workunit.client.1.smithi193.stdout:4/887: readlink d11/d2d/da8/d109/d10e/db8/l111 0 2024-03-23T08:23:29.518 INFO:tasks.workunit.client.1.smithi193.stdout:3/708: dread d1/de/d13/d96/d99/f30 [0,4194304] 0 2024-03-23T08:23:29.519 INFO:tasks.workunit.client.1.smithi193.stdout:8/849: getdents df/d4b/d4d/d9c/df2 0 2024-03-23T08:23:29.520 INFO:tasks.workunit.client.1.smithi193.stdout:5/861: mkdir d0/d1c/d7a/db1/d72/d73/d126 0 2024-03-23T08:23:29.521 INFO:tasks.workunit.client.1.smithi193.stdout:8/850: chown df/d63/d76 1 1 2024-03-23T08:23:29.522 INFO:tasks.workunit.client.1.smithi193.stdout:3/709: chown d1/de/f16 43 1 2024-03-23T08:23:29.523 INFO:tasks.workunit.client.1.smithi193.stdout:8/851: read - df/d63/d84/da8/fed zero size 2024-03-23T08:23:29.524 INFO:tasks.workunit.client.1.smithi193.stdout:7/799: dwrite d6/d9/d1f/d4d/d7c/d92/fde [4194304,4194304] 0 2024-03-23T08:23:29.525 INFO:tasks.workunit.client.1.smithi193.stdout:8/852: stat df/d63/d76/cea 0 2024-03-23T08:23:29.526 INFO:tasks.workunit.client.1.smithi193.stdout:5/862: creat d0/d1f/d93/f127 x:0 0 0 2024-03-23T08:23:29.534 INFO:tasks.workunit.client.1.smithi193.stdout:5/863: rmdir d0/d1c/d7a/db1/d72 39 2024-03-23T08:23:29.536 INFO:tasks.workunit.client.1.smithi193.stdout:7/800: link d6/f32 d6/d9/d1f/d23/d33/d38/dc0/f110 0 2024-03-23T08:23:29.536 INFO:tasks.workunit.client.1.smithi193.stdout:6/716: dread d0/d27/d81/df/d11/d56/fd1 [0,4194304] 0 2024-03-23T08:23:29.540 INFO:tasks.workunit.client.1.smithi193.stdout:5/864: link d0/d1c/d7a/d86/f123 d0/d1f/f128 0 2024-03-23T08:23:29.540 INFO:tasks.workunit.client.1.smithi193.stdout:7/801: mknod d6/d9/d4e/d81/c111 0 2024-03-23T08:23:29.542 INFO:tasks.workunit.client.1.smithi193.stdout:6/717: mkdir d0/d27/d81/df/d11/d36/d61/d3a/d8f/de2 0 2024-03-23T08:23:29.542 INFO:tasks.workunit.client.1.smithi193.stdout:5/865: symlink d0/d1c/d3b/d4b/d55/l129 0 2024-03-23T08:23:29.544 INFO:tasks.workunit.client.1.smithi193.stdout:8/853: truncate df/d4b/d4d/f1e 1869313 0 2024-03-23T08:23:29.546 INFO:tasks.workunit.client.1.smithi193.stdout:6/718: symlink d0/d27/d81/df/d11/d36/d61/d3a/d8f/le3 0 2024-03-23T08:23:29.549 INFO:tasks.workunit.client.1.smithi193.stdout:7/802: rename d6/d9/d1f/d23/d33/d3c/dec to d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/d112 0 2024-03-23T08:23:29.549 INFO:tasks.workunit.client.1.smithi193.stdout:9/905: dread d2/f4 [4194304,4194304] 0 2024-03-23T08:23:29.549 INFO:tasks.workunit.client.1.smithi193.stdout:5/866: truncate d0/ff 3810121 0 2024-03-23T08:23:29.551 INFO:tasks.workunit.client.1.smithi193.stdout:5/867: stat d0/d1f/d93/cd5 0 2024-03-23T08:23:29.551 INFO:tasks.workunit.client.1.smithi193.stdout:9/906: rename d2/d7 to d2/d7/db/d1b/d43/d71/d12f 22 2024-03-23T08:23:29.551 INFO:tasks.workunit.client.1.smithi193.stdout:6/719: mkdir d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/db1/de4 0 2024-03-23T08:23:29.593 INFO:tasks.workunit.client.1.smithi193.stdout:1/844: sync 2024-03-23T08:23:29.593 INFO:tasks.workunit.client.1.smithi193.stdout:6/720: read d0/f8b [4458168,63151] 0 2024-03-23T08:23:29.594 INFO:tasks.workunit.client.1.smithi193.stdout:7/803: mkdir d6/d9/d1f/d23/d33/d38/d66/d113 0 2024-03-23T08:23:29.595 INFO:tasks.workunit.client.1.smithi193.stdout:8/854: link df/d10/d3e/d54/dd2/cbd df/d10/c11e 0 2024-03-23T08:23:29.596 INFO:tasks.workunit.client.1.smithi193.stdout:3/710: dread d1/de/d13/d21/d59/d60/f6b [0,4194304] 0 2024-03-23T08:23:29.599 INFO:tasks.workunit.client.1.smithi193.stdout:5/868: symlink d0/d1c/d3b/d4b/dff/d119/l12a 0 2024-03-23T08:23:29.599 INFO:tasks.workunit.client.1.smithi193.stdout:1/845: mknod da/d13/d24/dac/dea/c10f 0 2024-03-23T08:23:29.600 INFO:tasks.workunit.client.1.smithi193.stdout:7/804: dread - d6/d9/d1f/d23/d33/d38/d3b/d6c/de7/f104 zero size 2024-03-23T08:23:29.602 INFO:tasks.workunit.client.1.smithi193.stdout:8/855: getdents df/d4b/d4d/d2a/d10b 0 2024-03-23T08:23:29.605 INFO:tasks.workunit.client.1.smithi193.stdout:3/711: rmdir d1/de/d13/d21/d25/d2b/d3c/d4a/de4 39 2024-03-23T08:23:29.606 INFO:tasks.workunit.client.1.smithi193.stdout:7/805: creat d6/d9/d1f/d5f/f114 x:0 0 0 2024-03-23T08:23:29.606 INFO:tasks.workunit.client.1.smithi193.stdout:9/907: dwrite d2/d7/db/d1b/d43/d71/db4/d78/fd7 [0,4194304] 0 2024-03-23T08:23:29.607 INFO:tasks.workunit.client.1.smithi193.stdout:1/846: creat da/dd/df/d52/d82/db0/f110 x:0 0 0 2024-03-23T08:23:29.608 INFO:tasks.workunit.client.1.smithi193.stdout:8/856: symlink df/d10/d67/d7f/d103/l11f 0 2024-03-23T08:23:29.608 INFO:tasks.workunit.client.1.smithi193.stdout:5/869: fsync d0/d1c/d7a/db1/d72/d99/dd8/f10c 0 2024-03-23T08:23:29.610 INFO:tasks.workunit.client.1.smithi193.stdout:5/870: readlink d0/d1c/l4e 0 2024-03-23T08:23:29.611 INFO:tasks.workunit.client.1.smithi193.stdout:9/908: write d2/d7/db/d1b/fef [1218180,89560] 0 2024-03-23T08:23:29.612 INFO:tasks.workunit.client.1.smithi193.stdout:1/847: mknod da/dd/df/ddc/dfd/c111 0 2024-03-23T08:23:29.613 INFO:tasks.workunit.client.1.smithi193.stdout:6/721: dread d0/d27/d81/f20 [0,4194304] 0 2024-03-23T08:23:29.614 INFO:tasks.workunit.client.1.smithi193.stdout:7/806: mkdir d6/d9/d1f/d23/d33/d3c/d41/d47/d115 0 2024-03-23T08:23:29.614 INFO:tasks.workunit.client.1.smithi193.stdout:3/712: read d1/de/d13/d21/d25/d2b/d3c/faa [1599786,75774] 0 2024-03-23T08:23:29.615 INFO:tasks.workunit.client.1.smithi193.stdout:8/857: mknod df/d10/d67/c120 0 2024-03-23T08:23:29.615 INFO:tasks.workunit.client.1.smithi193.stdout:1/848: write da/dd/f64 [5226805,123791] 0 2024-03-23T08:23:29.619 INFO:tasks.workunit.client.1.smithi193.stdout:7/807: mkdir d6/d9/d116 0 2024-03-23T08:23:29.619 INFO:tasks.workunit.client.1.smithi193.stdout:6/722: mkdir d0/d27/d81/da1/de5 0 2024-03-23T08:23:29.621 INFO:tasks.workunit.client.1.smithi193.stdout:3/713: mknod d1/de/d13/d21/d59/dc3/cf6 0 2024-03-23T08:23:29.622 INFO:tasks.workunit.client.1.smithi193.stdout:6/723: write d0/d27/d81/fc5 [878783,102580] 0 2024-03-23T08:23:29.623 INFO:tasks.workunit.client.1.smithi193.stdout:5/871: dwrite d0/d1f/fb3 [0,4194304] 0 2024-03-23T08:23:29.624 INFO:tasks.workunit.client.1.smithi193.stdout:1/849: rename da/dd/c9e to da/dd/df/d7c/dd2/c112 0 2024-03-23T08:23:29.625 INFO:tasks.workunit.client.1.smithi193.stdout:6/724: fdatasync d0/d27/d81/fa6 0 2024-03-23T08:23:29.625 INFO:tasks.workunit.client.1.smithi193.stdout:5/872: stat d0/d1f/fb7 0 2024-03-23T08:23:29.626 INFO:tasks.workunit.client.1.smithi193.stdout:9/909: dread d2/d7/db/d1b/d43/d71/db4/dec/d3c/fbe [0,4194304] 0 2024-03-23T08:23:29.633 INFO:tasks.workunit.client.1.smithi193.stdout:5/873: creat d0/d1c/d3b/d4b/df9/f12b x:0 0 0 2024-03-23T08:23:29.634 INFO:tasks.workunit.client.1.smithi193.stdout:6/725: rename d0/d27/d81/df/c59 to d0/d27/d81/df/d11/d3b/d6c/ce6 0 2024-03-23T08:23:29.636 INFO:tasks.workunit.client.1.smithi193.stdout:6/726: chown d0/fd4 361659361 1 2024-03-23T08:23:29.636 INFO:tasks.workunit.client.1.smithi193.stdout:9/910: read d2/d7/db/d17/d33/d70/f7b [153261,78100] 0 2024-03-23T08:23:29.637 INFO:tasks.workunit.client.1.smithi193.stdout:3/714: write d1/de/d13/d21/f82 [891561,80788] 0 2024-03-23T08:23:29.639 INFO:tasks.workunit.client.1.smithi193.stdout:5/874: rename d0/d1c/d7a/db1/d72/d73/d45/fdc to d0/d1c/d3b/d81/df2/d97/f12c 0 2024-03-23T08:23:29.642 INFO:tasks.workunit.client.1.smithi193.stdout:6/727: fdatasync d0/d27/d81/df/d11/d3b/d6c/fbe 0 2024-03-23T08:23:29.642 INFO:tasks.workunit.client.1.smithi193.stdout:1/850: dread da/d13/d3f/f87 [0,4194304] 0 2024-03-23T08:23:29.645 INFO:tasks.workunit.client.1.smithi193.stdout:3/715: creat d1/de/d13/d96/d99/ff7 x:0 0 0 2024-03-23T08:23:29.647 INFO:tasks.workunit.client.1.smithi193.stdout:7/808: dwrite d6/d9/d1f/d23/d33/d38/d3b/d6c/de7/f104 [0,4194304] 0 2024-03-23T08:23:29.648 INFO:tasks.workunit.client.1.smithi193.stdout:5/875: truncate d0/d1c/d3b/d4b/df9/f12b 303196 0 2024-03-23T08:23:29.650 INFO:tasks.workunit.client.1.smithi193.stdout:1/851: mkdir da/dd/df/d7e/d113 0 2024-03-23T08:23:29.650 INFO:tasks.workunit.client.1.smithi193.stdout:6/728: write d0/d27/d81/df/d11/d36/fa3 [187936,60300] 0 2024-03-23T08:23:29.651 INFO:tasks.workunit.client.1.smithi193.stdout:9/911: getdents d2/d7/db/d1b/d43/d71/db4/dec 0 2024-03-23T08:23:29.653 INFO:tasks.workunit.client.1.smithi193.stdout:3/716: mknod d1/de/d13/d21/d27/d7c/cf8 0 2024-03-23T08:23:29.654 INFO:tasks.workunit.client.1.smithi193.stdout:3/717: fdatasync d1/de/fd0 0 2024-03-23T08:23:29.656 INFO:tasks.workunit.client.1.smithi193.stdout:3/718: chown d1/de/d13/d21/d25/d2b/d3c/d4a/dae/de5 982378 1 2024-03-23T08:23:29.657 INFO:tasks.workunit.client.1.smithi193.stdout:7/809: mkdir d6/d9/d1f/d23/d33/d38/d3b/d6c/de7/d117 0 2024-03-23T08:23:29.658 INFO:tasks.workunit.client.1.smithi193.stdout:6/729: mknod d0/d27/d81/df/d11/d3b/d51/dd6/ce7 0 2024-03-23T08:23:29.659 INFO:tasks.workunit.client.1.smithi193.stdout:1/852: rmdir da/dd/df/d52/d82/dff 0 2024-03-23T08:23:29.663 INFO:tasks.workunit.client.1.smithi193.stdout:1/853: fdatasync da/d13/d24/dac/dea/f67 0 2024-03-23T08:23:29.663 INFO:tasks.workunit.client.1.smithi193.stdout:6/730: creat d0/d27/d81/da1/fe8 x:0 0 0 2024-03-23T08:23:29.664 INFO:tasks.workunit.client.1.smithi193.stdout:7/810: getdents d6/d9/d4e 0 2024-03-23T08:23:29.665 INFO:tasks.workunit.client.1.smithi193.stdout:9/912: dwrite d2/d7/db/d1b/d43/d71/db4/dec/d3c/d79/fd3 [0,4194304] 0 2024-03-23T08:23:29.666 INFO:tasks.workunit.client.1.smithi193.stdout:6/731: dread - d0/d27/d81/df/d11/d3b/d51/fdc zero size 2024-03-23T08:23:29.667 INFO:tasks.workunit.client.1.smithi193.stdout:3/719: creat d1/de/d13/d21/d25/d2b/d3c/d4a/ff9 x:0 0 0 2024-03-23T08:23:29.669 INFO:tasks.workunit.client.1.smithi193.stdout:1/854: mkdir da/dd/df/d52/dce/d114 0 2024-03-23T08:23:29.669 INFO:tasks.workunit.client.1.smithi193.stdout:9/913: creat d2/d7/db/d17/d33/d70/f130 x:0 0 0 2024-03-23T08:23:29.670 INFO:tasks.workunit.client.1.smithi193.stdout:7/811: mknod d6/d9/d1f/d23/d33/c118 0 2024-03-23T08:23:29.671 INFO:tasks.workunit.client.1.smithi193.stdout:6/732: stat d0/d27/d81/df/d11/d36/d61/d12/c4c 0 2024-03-23T08:23:29.671 INFO:tasks.workunit.client.1.smithi193.stdout:1/855: dread - da/dd/df/d52/d82/db0/f110 zero size 2024-03-23T08:23:29.677 INFO:tasks.workunit.client.1.smithi193.stdout:1/856: creat da/dd/df/d7e/d7f/f115 x:0 0 0 2024-03-23T08:23:29.677 INFO:tasks.workunit.client.1.smithi193.stdout:7/812: symlink d6/d9/d1f/d23/d33/d38/d66/l119 0 2024-03-23T08:23:29.677 INFO:tasks.workunit.client.1.smithi193.stdout:2/884: sync 2024-03-23T08:23:29.678 INFO:tasks.workunit.client.1.smithi193.stdout:0/830: sync 2024-03-23T08:23:29.679 INFO:tasks.workunit.client.1.smithi193.stdout:1/857: symlink da/d13/d24/l116 0 2024-03-23T08:23:29.680 INFO:tasks.workunit.client.1.smithi193.stdout:7/813: creat d6/d9/d1f/d4d/d6b/f11a x:0 0 0 2024-03-23T08:23:29.681 INFO:tasks.workunit.client.1.smithi193.stdout:9/914: dwrite d2/d7/db/d1b/d5c/fed [0,4194304] 0 2024-03-23T08:23:29.681 INFO:tasks.workunit.client.1.smithi193.stdout:2/885: read d0/d1b/d3b/f79 [2749498,118743] 0 2024-03-23T08:23:29.683 INFO:tasks.workunit.client.1.smithi193.stdout:0/831: rmdir db/d11/d14/d21/d33/d69/dde 39 2024-03-23T08:23:29.685 INFO:tasks.workunit.client.1.smithi193.stdout:9/915: fsync d2/d7/db/d17/d4b/d59/d97/ff4 0 2024-03-23T08:23:29.685 INFO:tasks.workunit.client.1.smithi193.stdout:7/814: dread - d6/d9/d1f/d23/d33/d38/d3b/d6c/f10a zero size 2024-03-23T08:23:29.686 INFO:tasks.workunit.client.1.smithi193.stdout:2/886: creat d0/d1b/d14/d56/d7c/ddc/ff5 x:0 0 0 2024-03-23T08:23:29.689 INFO:tasks.workunit.client.1.smithi193.stdout:0/832: truncate db/d11/d14/d24/d47/d59/f110 361329 0 2024-03-23T08:23:29.689 INFO:tasks.workunit.client.1.smithi193.stdout:7/815: truncate d6/d9/d1f/d5f/f114 876361 0 2024-03-23T08:23:29.690 INFO:tasks.workunit.client.1.smithi193.stdout:9/916: mkdir d2/d7/db/d17/d4b/d59/d97/d131 0 2024-03-23T08:23:29.691 INFO:tasks.workunit.client.1.smithi193.stdout:1/858: getdents da/d13/d24/dac/dcf 0 2024-03-23T08:23:29.691 INFO:tasks.workunit.client.1.smithi193.stdout:2/887: rmdir d0/d1b/d14/d66/d50/d92 39 2024-03-23T08:23:29.693 INFO:tasks.workunit.client.1.smithi193.stdout:7/816: mkdir d6/d9/d1f/d23/d33/d38/d46/d82/d11b 0 2024-03-23T08:23:29.694 INFO:tasks.workunit.client.1.smithi193.stdout:1/859: chown da/d13/f90 430 1 2024-03-23T08:23:29.698 INFO:tasks.workunit.client.1.smithi193.stdout:2/888: rename d0/l4 to d0/d1b/d14/d56/d7c/d7e/lf6 0 2024-03-23T08:23:29.699 INFO:tasks.workunit.client.1.smithi193.stdout:7/817: truncate d6/d9/d1f/d4d/fa9 2762132 0 2024-03-23T08:23:29.702 INFO:tasks.workunit.client.1.smithi193.stdout:1/860: rename da/dd/df/d2e/l72 to da/d13/d24/dac/dea/l117 0 2024-03-23T08:23:29.704 INFO:tasks.workunit.client.1.smithi193.stdout:0/833: dwrite db/d11/d14/d24/d2a/d4d/d93/fcc [0,4194304] 0 2024-03-23T08:23:29.704 INFO:tasks.workunit.client.1.smithi193.stdout:9/917: dwrite d2/d7/db/d1b/d43/d71/db4/dec/f4f [0,4194304] 0 2024-03-23T08:23:29.708 INFO:tasks.workunit.client.1.smithi193.stdout:1/861: readlink da/dd/df/d7e/d86/d95/lb5 0 2024-03-23T08:23:29.710 INFO:tasks.workunit.client.1.smithi193.stdout:7/818: symlink d6/d9/d1f/d23/d33/d3c/d41/d47/l11c 0 2024-03-23T08:23:29.712 INFO:tasks.workunit.client.1.smithi193.stdout:0/834: dread db/d11/d14/d24/d47/dc7/fb2 [0,4194304] 0 2024-03-23T08:23:29.713 INFO:tasks.workunit.client.1.smithi193.stdout:1/862: mknod da/c118 0 2024-03-23T08:23:29.715 INFO:tasks.workunit.client.1.smithi193.stdout:7/819: chown d6/d9/d4e/d81/dca/ff7 5377 1 2024-03-23T08:23:29.716 INFO:tasks.workunit.client.1.smithi193.stdout:0/835: read db/d11/d14/d24/f5f [794587,41901] 0 2024-03-23T08:23:29.719 INFO:tasks.workunit.client.1.smithi193.stdout:9/918: dwrite d2/d7/db/d1b/d43/d71/db4/dec/d5f/f83 [0,4194304] 0 2024-03-23T08:23:29.719 INFO:tasks.workunit.client.1.smithi193.stdout:7/820: chown d6/d9/d1f/d23/c2b 164 1 2024-03-23T08:23:29.720 INFO:tasks.workunit.client.1.smithi193.stdout:2/889: truncate d0/d37/f8a 2920407 0 2024-03-23T08:23:29.722 INFO:tasks.workunit.client.1.smithi193.stdout:2/890: read - d0/d1b/d14/d56/d7c/ddc/ff5 zero size 2024-03-23T08:23:29.723 INFO:tasks.workunit.client.1.smithi193.stdout:1/863: creat da/dd/df/d7e/d86/d95/f119 x:0 0 0 2024-03-23T08:23:29.725 INFO:tasks.workunit.client.1.smithi193.stdout:9/919: mknod d2/d7/db/d17/c132 0 2024-03-23T08:23:29.726 INFO:tasks.workunit.client.1.smithi193.stdout:2/891: write d0/d1b/d14/d56/d7c/fd2 [929496,37405] 0 2024-03-23T08:23:29.728 INFO:tasks.workunit.client.1.smithi193.stdout:1/864: write da/dd/df/d52/f7a [543199,55969] 0 2024-03-23T08:23:29.729 INFO:tasks.workunit.client.1.smithi193.stdout:7/821: rename d6/d9/d1f/d3e/d65/d70/faa to d6/d9/d1f/d23/d33/d38/d46/d82/f11d 0 2024-03-23T08:23:29.729 INFO:tasks.workunit.client.1.smithi193.stdout:1/865: chown da/dd/df/d2e 368466 1 2024-03-23T08:23:29.731 INFO:tasks.workunit.client.1.smithi193.stdout:1/866: dread da/dd/f78 [0,4194304] 0 2024-03-23T08:23:29.732 INFO:tasks.workunit.client.1.smithi193.stdout:7/822: creat d6/d9/d1f/d23/d33/d3c/d41/f11e x:0 0 0 2024-03-23T08:23:29.732 INFO:tasks.workunit.client.1.smithi193.stdout:2/892: rmdir d0/d1b/d14/d66/d50/d92 39 2024-03-23T08:23:29.733 INFO:tasks.workunit.client.1.smithi193.stdout:2/893: chown d0/d1b/fd3 37 1 2024-03-23T08:23:29.734 INFO:tasks.workunit.client.1.smithi193.stdout:9/920: link d2/d7/db/d1b/d5c/cfc d2/d7/db/d17/d4b/d59/d97/d104/c133 0 2024-03-23T08:23:29.738 INFO:tasks.workunit.client.1.smithi193.stdout:7/823: creat d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/d112/f11f x:0 0 0 2024-03-23T08:23:29.740 INFO:tasks.workunit.client.1.smithi193.stdout:1/867: dwrite da/dd/df/d7c/fe5 [0,4194304] 0 2024-03-23T08:23:29.741 INFO:tasks.workunit.client.1.smithi193.stdout:9/921: mknod d2/df7/c134 0 2024-03-23T08:23:29.741 INFO:tasks.workunit.client.1.smithi193.stdout:9/922: chown d2 0 1 2024-03-23T08:23:29.746 INFO:tasks.workunit.client.1.smithi193.stdout:1/868: creat da/dd/df/d7e/d86/d95/dc4/f11a x:0 0 0 2024-03-23T08:23:29.747 INFO:tasks.workunit.client.1.smithi193.stdout:7/824: getdents d6/d9/d1f/d23/d33/d38/d3b 0 2024-03-23T08:23:29.748 INFO:tasks.workunit.client.1.smithi193.stdout:9/923: mkdir d2/d7/db/d17/d33/d70/d135 0 2024-03-23T08:23:29.750 INFO:tasks.workunit.client.1.smithi193.stdout:7/825: mkdir d6/d9/d1f/d23/d33/d3c/d120 0 2024-03-23T08:23:29.751 INFO:tasks.workunit.client.1.smithi193.stdout:1/869: getdents da/d13/d24/dac/dea/d76/d77 0 2024-03-23T08:23:29.752 INFO:tasks.workunit.client.1.smithi193.stdout:7/826: creat d6/d9/d1f/d23/d33/d38/dc0/f121 x:0 0 0 2024-03-23T08:23:29.753 INFO:tasks.workunit.client.1.smithi193.stdout:7/827: fdatasync d6/f14 0 2024-03-23T08:23:29.753 INFO:tasks.workunit.client.1.smithi193.stdout:1/870: mkdir da/d13/d24/dac/dea/d76/d77/de7/d11b 0 2024-03-23T08:23:29.755 INFO:tasks.workunit.client.1.smithi193.stdout:7/828: mkdir d6/d9/d1f/d4d/d7c/d122 0 2024-03-23T08:23:29.756 INFO:tasks.workunit.client.1.smithi193.stdout:7/829: write d6/d9/f1c [284950,101449] 0 2024-03-23T08:23:29.758 INFO:tasks.workunit.client.1.smithi193.stdout:2/894: dread d0/d1b/d14/d66/d50/d8f/f90 [0,4194304] 0 2024-03-23T08:23:29.758 INFO:tasks.workunit.client.1.smithi193.stdout:1/871: creat da/dd/df/d7e/f11c x:0 0 0 2024-03-23T08:23:29.760 INFO:tasks.workunit.client.1.smithi193.stdout:2/895: chown d0/d1b/d14/d56/d7c/ddc 2 1 2024-03-23T08:23:29.765 INFO:tasks.workunit.client.1.smithi193.stdout:2/896: link d0/d1b/d14/d66/d50/d8f/la4 d0/d1b/d14/d56/d7c/lf7 0 2024-03-23T08:23:29.765 INFO:tasks.workunit.client.1.smithi193.stdout:1/872: dwrite da/f59 [0,4194304] 0 2024-03-23T08:23:29.766 INFO:tasks.workunit.client.1.smithi193.stdout:2/897: dread - d0/d1b/d3b/fec zero size 2024-03-23T08:23:29.769 INFO:tasks.workunit.client.1.smithi193.stdout:1/873: dread da/d13/d24/dac/dea/f65 [0,4194304] 0 2024-03-23T08:23:29.771 INFO:tasks.workunit.client.1.smithi193.stdout:7/830: dread d6/d9/d1f/d3e/d42/f59 [0,4194304] 0 2024-03-23T08:23:29.771 INFO:tasks.workunit.client.1.smithi193.stdout:2/898: dwrite d0/d1b/d14/d66/d50/f52 [0,4194304] 0 2024-03-23T08:23:29.785 INFO:tasks.workunit.client.1.smithi193.stdout:4/888: sync 2024-03-23T08:23:29.819 INFO:tasks.workunit.client.1.smithi193.stdout:7/831: chown d6/d9/d1f/d23/d33/d38/d3b/f89 126619859 1 2024-03-23T08:23:29.822 INFO:tasks.workunit.client.1.smithi193.stdout:7/832: getdents d6/d9/d1f/d4d/d7c/d8e 0 2024-03-23T08:23:29.823 INFO:tasks.workunit.client.1.smithi193.stdout:7/833: readlink d6/d9/d4e/d81/lf2 0 2024-03-23T08:23:29.826 INFO:tasks.workunit.client.1.smithi193.stdout:7/834: dread d6/d9/fa [0,4194304] 0 2024-03-23T08:23:29.833 INFO:tasks.workunit.client.1.smithi193.stdout:2/899: dread d0/fd [0,4194304] 0 2024-03-23T08:23:29.833 INFO:tasks.workunit.client.1.smithi193.stdout:7/835: dwrite d6/d9/d1f/d3e/f5e [0,4194304] 0 2024-03-23T08:23:29.835 INFO:tasks.workunit.client.1.smithi193.stdout:2/900: read d0/d1b/d14/d66/d50/f91 [175474,7015] 0 2024-03-23T08:23:29.840 INFO:tasks.workunit.client.1.smithi193.stdout:2/901: creat d0/d1b/d14/d66/ff8 x:0 0 0 2024-03-23T08:23:29.842 INFO:tasks.workunit.client.1.smithi193.stdout:2/902: chown d0/d1b/d14/d56/da3/cda 412 1 2024-03-23T08:23:29.845 INFO:tasks.workunit.client.1.smithi193.stdout:7/836: dwrite d6/d9/d1f/d23/d33/d38/d3b/f89 [0,4194304] 0 2024-03-23T08:23:29.848 INFO:tasks.workunit.client.1.smithi193.stdout:7/837: symlink d6/d9/d1f/d23/d33/d3c/l123 0 2024-03-23T08:23:29.852 INFO:tasks.workunit.client.1.smithi193.stdout:7/838: symlink d6/d9/d1f/d23/d33/d38/d66/l124 0 2024-03-23T08:23:29.855 INFO:tasks.workunit.client.1.smithi193.stdout:7/839: rename d6/l3f to d6/d9/d1f/daf/l125 0 2024-03-23T08:23:29.856 INFO:tasks.workunit.client.1.smithi193.stdout:7/840: fsync d6/d9/fa5 0 2024-03-23T08:23:29.857 INFO:tasks.workunit.client.1.smithi193.stdout:7/841: chown d6/f14 96073617 1 2024-03-23T08:23:29.857 INFO:tasks.workunit.client.1.smithi193.stdout:2/903: dread d0/d1b/d14/f40 [0,4194304] 0 2024-03-23T08:23:29.861 INFO:tasks.workunit.client.1.smithi193.stdout:7/842: getdents d6/d9/d1f/d23/d33 0 2024-03-23T08:23:29.863 INFO:tasks.workunit.client.1.smithi193.stdout:7/843: write d6/d9/d1f/d23/d33/d38/d3b/d6c/f8d [63048,78951] 0 2024-03-23T08:23:29.867 INFO:tasks.workunit.client.1.smithi193.stdout:2/904: dwrite d0/d37/f8a [0,4194304] 0 2024-03-23T08:23:29.869 INFO:tasks.workunit.client.1.smithi193.stdout:7/844: write d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/df3/fa0 [2004172,61316] 0 2024-03-23T08:23:29.870 INFO:tasks.workunit.client.1.smithi193.stdout:2/905: write d0/d1b/fd3 [787021,127226] 0 2024-03-23T08:23:29.872 INFO:tasks.workunit.client.1.smithi193.stdout:7/845: write d6/fc [4165309,114311] 0 2024-03-23T08:23:29.876 INFO:tasks.workunit.client.1.smithi193.stdout:7/846: symlink d6/d9/d1f/d23/d33/d38/d3b/d6c/l126 0 2024-03-23T08:23:29.878 INFO:tasks.workunit.client.1.smithi193.stdout:2/906: truncate d0/f7 7210134 0 2024-03-23T08:23:29.881 INFO:tasks.workunit.client.1.smithi193.stdout:7/847: dread d6/d9/d4e/fae [0,4194304] 0 2024-03-23T08:23:29.883 INFO:tasks.workunit.client.1.smithi193.stdout:8/858: sync 2024-03-23T08:23:29.883 INFO:tasks.workunit.client.1.smithi193.stdout:5/876: sync 2024-03-23T08:23:29.883 INFO:tasks.workunit.client.1.smithi193.stdout:3/720: sync 2024-03-23T08:23:29.891 INFO:tasks.workunit.client.1.smithi193.stdout:7/848: dread - d6/d9/d1f/d3e/d65/d7d/fb4 zero size 2024-03-23T08:23:29.892 INFO:tasks.workunit.client.1.smithi193.stdout:2/907: dwrite d0/d37/d48/ff4 [0,4194304] 0 2024-03-23T08:23:29.896 INFO:tasks.workunit.client.1.smithi193.stdout:3/721: read - d1/de/d13/d21/d25/d2b/d3c/d4a/de4/ff1 zero size 2024-03-23T08:23:29.896 INFO:tasks.workunit.client.1.smithi193.stdout:2/908: creat d0/dcf/ff9 x:0 0 0 2024-03-23T08:23:29.897 INFO:tasks.workunit.client.1.smithi193.stdout:7/849: mknod d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/df3/da1/c127 0 2024-03-23T08:23:29.899 INFO:tasks.workunit.client.1.smithi193.stdout:8/859: mkdir df/d63/d84/df4/df7/d121 0 2024-03-23T08:23:29.900 INFO:tasks.workunit.client.1.smithi193.stdout:8/860: write df/d10/fbb [2021299,61137] 0 2024-03-23T08:23:29.902 INFO:tasks.workunit.client.1.smithi193.stdout:5/877: creat d0/d1c/d3b/f12d x:0 0 0 2024-03-23T08:23:29.903 INFO:tasks.workunit.client.1.smithi193.stdout:3/722: dwrite d1/de/d13/d21/d25/d2b/fb1 [0,4194304] 0 2024-03-23T08:23:29.905 INFO:tasks.workunit.client.1.smithi193.stdout:7/850: creat d6/d9/d1f/d23/d33/d38/d3b/d6c/de7/f128 x:0 0 0 2024-03-23T08:23:29.906 INFO:tasks.workunit.client.1.smithi193.stdout:7/851: dread - d6/d9/d1f/d23/d33/d38/dc0/f121 zero size 2024-03-23T08:23:29.907 INFO:tasks.workunit.client.1.smithi193.stdout:8/861: dread df/dc6/d8b/dae/f90 [0,4194304] 0 2024-03-23T08:23:29.907 INFO:tasks.workunit.client.1.smithi193.stdout:5/878: truncate d0/d1c/d7a/f8f 332082 0 2024-03-23T08:23:29.911 INFO:tasks.workunit.client.1.smithi193.stdout:3/723: rename d1/c65 to d1/de/d13/d21/d59/df2/cfa 0 2024-03-23T08:23:29.913 INFO:tasks.workunit.client.1.smithi193.stdout:5/879: mknod d0/d1c/d3b/d4b/d55/c12e 0 2024-03-23T08:23:29.914 INFO:tasks.workunit.client.1.smithi193.stdout:7/852: dwrite d6/d9/d1f/d23/d33/d38/d3b/f48 [0,4194304] 0 2024-03-23T08:23:29.915 INFO:tasks.workunit.client.1.smithi193.stdout:3/724: write d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fce [727895,98884] 0 2024-03-23T08:23:29.917 INFO:tasks.workunit.client.1.smithi193.stdout:5/880: unlink d0/f1a 0 2024-03-23T08:23:29.917 INFO:tasks.workunit.client.1.smithi193.stdout:3/725: stat d1/de/d13/d21/d25/d2b/d3c/c76 0 2024-03-23T08:23:29.921 INFO:tasks.workunit.client.1.smithi193.stdout:3/726: read d1/de/d13/d21/d50/d44/f79 [866459,21214] 0 2024-03-23T08:23:29.923 INFO:tasks.workunit.client.1.smithi193.stdout:7/853: dwrite d6/d9/d1f/d3e/d42/f7a [0,4194304] 0 2024-03-23T08:23:29.925 INFO:tasks.workunit.client.1.smithi193.stdout:7/854: readlink d6/d9/d1f/d23/d33/l62 0 2024-03-23T08:23:29.926 INFO:tasks.workunit.client.1.smithi193.stdout:3/727: mknod d1/de/d13/d96/cfb 0 2024-03-23T08:23:29.928 INFO:tasks.workunit.client.1.smithi193.stdout:3/728: readlink d1/de/d13/d21/d25/d34/l49 0 2024-03-23T08:23:29.929 INFO:tasks.workunit.client.1.smithi193.stdout:2/909: write d0/d1b/d14/d25/f5a [5035962,92411] 0 2024-03-23T08:23:29.929 INFO:tasks.workunit.client.1.smithi193.stdout:7/855: mkdir d6/d9/d1f/d23/d33/d129 0 2024-03-23T08:23:29.931 INFO:tasks.workunit.client.1.smithi193.stdout:2/910: truncate d0/d1b/d14/f40 5153705 0 2024-03-23T08:23:29.934 INFO:tasks.workunit.client.1.smithi193.stdout:2/911: unlink d0/d1b/d3b/ce3 0 2024-03-23T08:23:29.934 INFO:tasks.workunit.client.1.smithi193.stdout:5/881: write d0/ff [1570467,56805] 0 2024-03-23T08:23:29.935 INFO:tasks.workunit.client.1.smithi193.stdout:7/856: symlink d6/d9/d1f/l12a 0 2024-03-23T08:23:29.936 INFO:tasks.workunit.client.1.smithi193.stdout:2/912: read d0/d1b/d3b/f7a [2070218,76758] 0 2024-03-23T08:23:29.967 INFO:tasks.workunit.client.1.smithi193.stdout:0/836: sync 2024-03-23T08:23:29.967 INFO:tasks.workunit.client.1.smithi193.stdout:6/733: sync 2024-03-23T08:23:29.967 INFO:tasks.workunit.client.1.smithi193.stdout:0/837: chown db/d52/db8/dcf/fd6 25860 1 2024-03-23T08:23:29.997 INFO:tasks.workunit.client.1.smithi193.stdout:4/889: sync 2024-03-23T08:23:29.997 INFO:tasks.workunit.client.1.smithi193.stdout:9/924: sync 2024-03-23T08:23:29.998 INFO:tasks.workunit.client.1.smithi193.stdout:1/874: sync 2024-03-23T08:23:30.003 INFO:tasks.workunit.client.1.smithi193.stdout:9/925: dwrite d2/d7/db/d1b/d5c/fed [0,4194304] 0 2024-03-23T08:23:30.064 INFO:tasks.workunit.client.1.smithi193.stdout:0/838: creat db/d52/db8/f127 x:0 0 0 2024-03-23T08:23:30.065 INFO:tasks.workunit.client.1.smithi193.stdout:3/729: write d1/de/d13/d21/d25/d34/f8f [61064,82174] 0 2024-03-23T08:23:30.065 INFO:tasks.workunit.client.1.smithi193.stdout:0/839: readlink db/d11/d14/d24/d2a/d4d/l6a 0 2024-03-23T08:23:30.067 INFO:tasks.workunit.client.1.smithi193.stdout:0/840: read db/d11/d14/d24/d47/d59/f72 [3018390,22307] 0 2024-03-23T08:23:30.068 INFO:tasks.workunit.client.1.smithi193.stdout:0/841: dread - db/d11/d14/f111 zero size 2024-03-23T08:23:30.077 INFO:tasks.workunit.client.1.smithi193.stdout:8/862: sync 2024-03-23T08:23:30.101 INFO:tasks.workunit.client.1.smithi193.stdout:1/875: rename da/dd/df/d52/l63 to da/d13/d24/dac/dea/d76/d77/d10b/l11d 0 2024-03-23T08:23:30.103 INFO:tasks.workunit.client.1.smithi193.stdout:1/876: write da/dd/df/d52/d82/fc7 [498461,108749] 0 2024-03-23T08:23:30.103 INFO:tasks.workunit.client.1.smithi193.stdout:5/882: getdents d0/d1c/d7a/db1/d72 0 2024-03-23T08:23:30.105 INFO:tasks.workunit.client.1.smithi193.stdout:7/857: write d6/d9/f16 [3078990,34332] 0 2024-03-23T08:23:30.106 INFO:tasks.workunit.client.1.smithi193.stdout:6/734: creat d0/d27/d81/da1/da9/fe9 x:0 0 0 2024-03-23T08:23:30.106 INFO:tasks.workunit.client.1.smithi193.stdout:9/926: sync 2024-03-23T08:23:30.106 INFO:tasks.workunit.client.1.smithi193.stdout:1/877: dread da/d13/d24/dac/fed [0,4194304] 0 2024-03-23T08:23:30.108 INFO:tasks.workunit.client.1.smithi193.stdout:1/878: fdatasync da/d13/d24/dac/dea/f61 0 2024-03-23T08:23:30.109 INFO:tasks.workunit.client.1.smithi193.stdout:3/730: rename d1/de/d13/d21/d25/d2b/d3c/d40 to d1/de/d13/d21/d25/d2b/d3c/d4a/d85/db9/dfc 0 2024-03-23T08:23:30.111 INFO:tasks.workunit.client.1.smithi193.stdout:1/879: write da/dd/da9/fe8 [991800,130800] 0 2024-03-23T08:23:30.111 INFO:tasks.workunit.client.1.smithi193.stdout:4/890: link d11/d1a/d33/d42/db2/dbe/d50/d49/f108 d11/d1a/d33/d42/d52/db1/f127 0 2024-03-23T08:23:30.112 INFO:tasks.workunit.client.1.smithi193.stdout:2/913: link d0/d37/d48/c69 d0/d1b/d14/d56/da3/db0/cfa 0 2024-03-23T08:23:30.114 INFO:tasks.workunit.client.1.smithi193.stdout:6/735: dread d0/d27/d81/df/d11/d3b/d51/d9c/f8a [0,4194304] 0 2024-03-23T08:23:30.117 INFO:tasks.workunit.client.1.smithi193.stdout:3/731: rmdir d1/de/d13/d21/d59/dc3 39 2024-03-23T08:23:30.118 INFO:tasks.workunit.client.1.smithi193.stdout:8/863: dread df/d4b/d4d/f1e [0,4194304] 0 2024-03-23T08:23:30.119 INFO:tasks.workunit.client.1.smithi193.stdout:0/842: link db/d11/d14/d21/l3d db/d11/d14/d24/d47/d59/dd4/dfc/l128 0 2024-03-23T08:23:30.121 INFO:tasks.workunit.client.1.smithi193.stdout:5/883: dread d0/d1c/d7a/db1/f67 [0,4194304] 0 2024-03-23T08:23:30.121 INFO:tasks.workunit.client.1.smithi193.stdout:8/864: chown df/f33 116531 1 2024-03-23T08:23:30.121 INFO:tasks.workunit.client.1.smithi193.stdout:9/927: dread d2/d7/db/f2e [0,4194304] 0 2024-03-23T08:23:30.123 INFO:tasks.workunit.client.1.smithi193.stdout:3/732: chown d1/de/d13/d21/d25/d2b/d3c/c45 4067 1 2024-03-23T08:23:30.123 INFO:tasks.workunit.client.1.smithi193.stdout:8/865: fsync df/d10/d83/f117 0 2024-03-23T08:23:30.126 INFO:tasks.workunit.client.1.smithi193.stdout:1/880: link da/dd/df/d7e/d7f/f115 da/dd/df/d7e/d113/f11e 0 2024-03-23T08:23:30.128 INFO:tasks.workunit.client.1.smithi193.stdout:0/843: write db/d11/d14/d21/d33/d69/d9b/df2/ff6 [3621,36406] 0 2024-03-23T08:23:30.131 INFO:tasks.workunit.client.1.smithi193.stdout:8/866: mknod df/d63/d84/df4/df7/c122 0 2024-03-23T08:23:30.132 INFO:tasks.workunit.client.1.smithi193.stdout:1/881: unlink da/c19 0 2024-03-23T08:23:30.132 INFO:tasks.workunit.client.1.smithi193.stdout:6/736: dread d0/d27/d81/df/d11/d36/d61/f1d [0,4194304] 0 2024-03-23T08:23:30.132 INFO:tasks.workunit.client.1.smithi193.stdout:9/928: mkdir d2/d7/db/d7a/d95/da9/d113/d136 0 2024-03-23T08:23:30.134 INFO:tasks.workunit.client.1.smithi193.stdout:4/891: dwrite d11/d2d/f38 [0,4194304] 0 2024-03-23T08:23:30.135 INFO:tasks.workunit.client.1.smithi193.stdout:7/858: truncate d6/d9/d1f/d23/d33/d38/dc0/f110 7557915 0 2024-03-23T08:23:30.136 INFO:tasks.workunit.client.1.smithi193.stdout:2/914: truncate d0/d1b/d14/d56/d7c/fd2 3735 0 2024-03-23T08:23:30.138 INFO:tasks.workunit.client.1.smithi193.stdout:8/867: creat df/d10/d83/dcb/f123 x:0 0 0 2024-03-23T08:23:30.142 INFO:tasks.workunit.client.1.smithi193.stdout:1/882: unlink da/dd/df/d75/cb9 0 2024-03-23T08:23:30.143 INFO:tasks.workunit.client.1.smithi193.stdout:3/733: write d1/fc [1057866,115259] 0 2024-03-23T08:23:30.143 INFO:tasks.workunit.client.1.smithi193.stdout:4/892: rmdir d11/d1a/d65 39 2024-03-23T08:23:30.146 INFO:tasks.workunit.client.1.smithi193.stdout:5/884: rename d0/d1c/d3b/d81/df2/ff8 to d0/d1c/d3b/d4b/f12f 0 2024-03-23T08:23:30.147 INFO:tasks.workunit.client.1.smithi193.stdout:8/868: dwrite df/d10/d3e/f5f [0,4194304] 0 2024-03-23T08:23:30.147 INFO:tasks.workunit.client.1.smithi193.stdout:7/859: mkdir d6/d9/d1f/d23/d33/d38/d46/d82/d12b 0 2024-03-23T08:23:30.150 INFO:tasks.workunit.client.1.smithi193.stdout:6/737: symlink d0/d27/d81/df/d11/d3b/lea 0 2024-03-23T08:23:30.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/893: fdatasync d11/d1a/d33/d42/db2/dbe/d50/f5b 0 2024-03-23T08:23:30.157 INFO:tasks.workunit.client.1.smithi193.stdout:3/734: read d1/de/d13/d21/d50/d44/f79 [4314048,23899] 0 2024-03-23T08:23:30.158 INFO:tasks.workunit.client.1.smithi193.stdout:7/860: mknod d6/d9/d1f/d23/d69/c12c 0 2024-03-23T08:23:30.159 INFO:tasks.workunit.client.1.smithi193.stdout:8/869: rmdir df/d10/d3e/d54/d10d 39 2024-03-23T08:23:30.160 INFO:tasks.workunit.client.1.smithi193.stdout:9/929: dwrite d2/d7/db/d17/d33/f45 [4194304,4194304] 0 2024-03-23T08:23:30.161 INFO:tasks.workunit.client.1.smithi193.stdout:4/894: rmdir d11/d1a/d33/d42/db2 39 2024-03-23T08:23:30.161 INFO:tasks.workunit.client.1.smithi193.stdout:8/870: fdatasync df/d4b/fe4 0 2024-03-23T08:23:30.162 INFO:tasks.workunit.client.1.smithi193.stdout:6/738: mkdir d0/d27/d81/df/d11/d3b/d51/daa/deb 0 2024-03-23T08:23:30.163 INFO:tasks.workunit.client.1.smithi193.stdout:1/883: link da/dd/df/ddc/dfd/l103 da/dd/df/d2e/l11f 0 2024-03-23T08:23:30.165 INFO:tasks.workunit.client.1.smithi193.stdout:8/871: symlink df/d4b/d4d/d2a/l124 0 2024-03-23T08:23:30.167 INFO:tasks.workunit.client.1.smithi193.stdout:2/915: link d0/l5 d0/d1b/d14/d56/da3/db0/lfb 0 2024-03-23T08:23:30.168 INFO:tasks.workunit.client.1.smithi193.stdout:9/930: fdatasync d2/d7/db/d17/d33/d70/f7b 0 2024-03-23T08:23:30.172 INFO:tasks.workunit.client.1.smithi193.stdout:1/884: mkdir da/d13/d24/dac/dea/d76/d77/de7/d120 0 2024-03-23T08:23:30.172 INFO:tasks.workunit.client.1.smithi193.stdout:9/931: chown d2/ff2 1566 1 2024-03-23T08:23:30.174 INFO:tasks.workunit.client.1.smithi193.stdout:7/861: creat d6/d9/d1f/d23/d33/d38/d66/f12d x:0 0 0 2024-03-23T08:23:30.177 INFO:tasks.workunit.client.1.smithi193.stdout:2/916: mkdir d0/d1b/d14/d56/da3/db0/dc6/dfc 0 2024-03-23T08:23:30.177 INFO:tasks.workunit.client.1.smithi193.stdout:4/895: dwrite d11/d2d/d3a/f76 [4194304,4194304] 0 2024-03-23T08:23:30.179 INFO:tasks.workunit.client.1.smithi193.stdout:1/885: write da/d13/d24/dac/dea/d76/ffe [286944,58713] 0 2024-03-23T08:23:30.180 INFO:tasks.workunit.client.1.smithi193.stdout:6/739: rename d0/d27/d81/df/d11/d56/fd0 to d0/d27/d81/fec 0 2024-03-23T08:23:30.186 INFO:tasks.workunit.client.1.smithi193.stdout:3/735: dwrite d1/de/d13/d21/d25/d2b/d3c/d4a/f93 [4194304,4194304] 0 2024-03-23T08:23:30.190 INFO:tasks.workunit.client.1.smithi193.stdout:4/896: truncate d11/d2d/da8/d109/d10e/db8/f75 762497 0 2024-03-23T08:23:30.192 INFO:tasks.workunit.client.1.smithi193.stdout:1/886: mkdir da/d13/d24/dac/dea/d76/d77/d121 0 2024-03-23T08:23:30.196 INFO:tasks.workunit.client.1.smithi193.stdout:2/917: write d0/d28/d9f/fb5 [345702,7451] 0 2024-03-23T08:23:30.196 INFO:tasks.workunit.client.1.smithi193.stdout:3/736: read d1/de/d13/d21/d50/fc8 [296703,40421] 0 2024-03-23T08:23:30.198 INFO:tasks.workunit.client.1.smithi193.stdout:5/885: dread d0/ff5 [0,4194304] 0 2024-03-23T08:23:30.199 INFO:tasks.workunit.client.1.smithi193.stdout:1/887: creat da/dd/df/d2e/dd5/f122 x:0 0 0 2024-03-23T08:23:30.199 INFO:tasks.workunit.client.1.smithi193.stdout:4/897: symlink d11/d95/d107/l128 0 2024-03-23T08:23:30.199 INFO:tasks.workunit.client.1.smithi193.stdout:8/872: dwrite df/d10/f17 [0,4194304] 0 2024-03-23T08:23:30.203 INFO:tasks.workunit.client.1.smithi193.stdout:2/918: write d0/d37/d48/ff4 [4329096,88797] 0 2024-03-23T08:23:30.203 INFO:tasks.workunit.client.1.smithi193.stdout:9/932: dread d2/d7/db/d7a/f101 [0,4194304] 0 2024-03-23T08:23:30.205 INFO:tasks.workunit.client.1.smithi193.stdout:1/888: write da/dd/df/d52/d82/db0/f110 [206786,26308] 0 2024-03-23T08:23:30.205 INFO:tasks.workunit.client.1.smithi193.stdout:6/740: truncate d0/d27/d81/df/d11/d36/d61/f48 487754 0 2024-03-23T08:23:30.207 INFO:tasks.workunit.client.1.smithi193.stdout:9/933: symlink d2/d7/db/d7a/d95/da9/l137 0 2024-03-23T08:23:30.208 INFO:tasks.workunit.client.1.smithi193.stdout:4/898: rename d11/d27/c30 to d11/d1a/d33/d42/d52/c129 0 2024-03-23T08:23:30.211 INFO:tasks.workunit.client.1.smithi193.stdout:1/889: write da/dd/df/f2d [190498,18849] 0 2024-03-23T08:23:30.214 INFO:tasks.workunit.client.1.smithi193.stdout:9/934: unlink d2/d7/db/d17/l46 0 2024-03-23T08:23:30.215 INFO:tasks.workunit.client.1.smithi193.stdout:3/737: rename d1/de/l3b to d1/de/d13/d21/d50/d44/lfd 0 2024-03-23T08:23:30.216 INFO:tasks.workunit.client.1.smithi193.stdout:4/899: dread - d11/d8d/fcf zero size 2024-03-23T08:23:30.219 INFO:tasks.workunit.client.1.smithi193.stdout:9/935: fsync d2/d7/db/d7a/f9f 0 2024-03-23T08:23:30.219 INFO:tasks.workunit.client.1.smithi193.stdout:2/919: link d0/d37/d48/c84 d0/d1b/cfd 0 2024-03-23T08:23:30.226 INFO:tasks.workunit.client.1.smithi193.stdout:1/890: rename da/dd/df/d75/d98/df3 to da/dd/df/d2e/db4/d123 0 2024-03-23T08:23:30.226 INFO:tasks.workunit.client.1.smithi193.stdout:6/741: dwrite d0/d27/d81/df/d11/fbb [0,4194304] 0 2024-03-23T08:23:30.226 INFO:tasks.workunit.client.1.smithi193.stdout:9/936: fsync d2/d7/db/d12d/dde/f110 0 2024-03-23T08:23:30.226 INFO:tasks.workunit.client.1.smithi193.stdout:4/900: unlink d11/d2d/d3a/fc1 0 2024-03-23T08:23:30.226 INFO:tasks.workunit.client.1.smithi193.stdout:5/886: dwrite d0/d1c/d7a/db1/d72/d73/d45/f5f [0,4194304] 0 2024-03-23T08:23:30.229 INFO:tasks.workunit.client.1.smithi193.stdout:2/920: rmdir d0/d37 39 2024-03-23T08:23:30.229 INFO:tasks.workunit.client.1.smithi193.stdout:0/844: sync 2024-03-23T08:23:30.230 INFO:tasks.workunit.client.1.smithi193.stdout:3/738: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/dfe 0 2024-03-23T08:23:30.231 INFO:tasks.workunit.client.1.smithi193.stdout:8/873: dread df/d10/d3e/fb0 [0,4194304] 0 2024-03-23T08:23:30.231 INFO:tasks.workunit.client.1.smithi193.stdout:1/891: dread - da/d13/d24/dac/dea/d76/fec zero size 2024-03-23T08:23:30.233 INFO:tasks.workunit.client.1.smithi193.stdout:5/887: read d0/d1f/f41 [1219775,123191] 0 2024-03-23T08:23:30.235 INFO:tasks.workunit.client.1.smithi193.stdout:9/937: write d2/d7/db/d17/d33/db2/fcc [738134,16348] 0 2024-03-23T08:23:30.235 INFO:tasks.workunit.client.1.smithi193.stdout:6/742: creat d0/d27/d81/df/d11/d56/fed x:0 0 0 2024-03-23T08:23:30.236 INFO:tasks.workunit.client.1.smithi193.stdout:1/892: chown da/d13/cd3 248354 1 2024-03-23T08:23:30.238 INFO:tasks.workunit.client.1.smithi193.stdout:4/901: symlink d11/d2d/da8/d109/d10e/db8/l12a 0 2024-03-23T08:23:30.239 INFO:tasks.workunit.client.1.smithi193.stdout:9/938: write d2/d7/db/f108 [745192,102149] 0 2024-03-23T08:23:30.244 INFO:tasks.workunit.client.1.smithi193.stdout:5/888: rmdir d0/d1c/d7a/d86/d9f 39 2024-03-23T08:23:30.247 INFO:tasks.workunit.client.1.smithi193.stdout:1/893: unlink da/dd/df/d75/d98/ff9 0 2024-03-23T08:23:30.247 INFO:tasks.workunit.client.1.smithi193.stdout:3/739: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/dfe/fff x:0 0 0 2024-03-23T08:23:30.249 INFO:tasks.workunit.client.1.smithi193.stdout:6/743: write d0/f40 [3126885,27870] 0 2024-03-23T08:23:30.250 INFO:tasks.workunit.client.1.smithi193.stdout:0/845: dwrite db/d11/d14/d24/d47/fec [0,4194304] 0 2024-03-23T08:23:30.250 INFO:tasks.workunit.client.1.smithi193.stdout:9/939: read d2/d7/db/d12d/f6c [837045,42257] 0 2024-03-23T08:23:30.254 INFO:tasks.workunit.client.1.smithi193.stdout:3/740: unlink d1/de/c80 0 2024-03-23T08:23:30.255 INFO:tasks.workunit.client.1.smithi193.stdout:6/744: chown d0/d27/d81/fec 367783 1 2024-03-23T08:23:30.255 INFO:tasks.workunit.client.1.smithi193.stdout:8/874: link df/d4b/d4d/d9c/dee/c10a df/d10/d67/d7f/d103/c125 0 2024-03-23T08:23:30.256 INFO:tasks.workunit.client.1.smithi193.stdout:9/940: chown d2/d7/db/d7a/d95/fd4 592140 1 2024-03-23T08:23:30.256 INFO:tasks.workunit.client.1.smithi193.stdout:0/846: fdatasync db/d11/d14/d24/ddd/fef 0 2024-03-23T08:23:30.260 INFO:tasks.workunit.client.1.smithi193.stdout:8/875: unlink df/d10/d3e/d54/cc9 0 2024-03-23T08:23:30.261 INFO:tasks.workunit.client.1.smithi193.stdout:9/941: fsync d2/d7/db/d1b/d43/d71/db4/dec/d3c/d79/f11e 0 2024-03-23T08:23:30.262 INFO:tasks.workunit.client.1.smithi193.stdout:0/847: fdatasync db/d11/d14/d21/d33/f80 0 2024-03-23T08:23:30.267 INFO:tasks.workunit.client.1.smithi193.stdout:2/921: dread d0/d1b/d19/f1a [0,4194304] 0 2024-03-23T08:23:30.267 INFO:tasks.workunit.client.1.smithi193.stdout:8/876: symlink df/d63/d84/df4/l126 0 2024-03-23T08:23:30.269 INFO:tasks.workunit.client.1.smithi193.stdout:6/745: getdents d0/d27/d81/df/d11/d3b/d51 0 2024-03-23T08:23:30.270 INFO:tasks.workunit.client.1.smithi193.stdout:3/741: rename d1/de/d13/d21/d25/d2b/d3c/l8d to d1/de/d13/d21/d25/l100 0 2024-03-23T08:23:30.271 INFO:tasks.workunit.client.1.smithi193.stdout:4/902: dread d11/d1a/d33/d42/f28 [0,4194304] 0 2024-03-23T08:23:30.272 INFO:tasks.workunit.client.1.smithi193.stdout:5/889: dwrite d0/d1f/d93/dcd/ffe [0,4194304] 0 2024-03-23T08:23:30.272 INFO:tasks.workunit.client.1.smithi193.stdout:9/942: mkdir d2/d7/db/d1b/d43/d71/db4/dec/d5f/d9c/d138 0 2024-03-23T08:23:30.272 INFO:tasks.workunit.client.1.smithi193.stdout:7/862: sync 2024-03-23T08:23:30.275 INFO:tasks.workunit.client.1.smithi193.stdout:4/903: readlink d11/d8d/ldb 0 2024-03-23T08:23:30.276 INFO:tasks.workunit.client.1.smithi193.stdout:2/922: chown d0/d1b/d14/d56/d7c/d7e/led 15466 1 2024-03-23T08:23:30.277 INFO:tasks.workunit.client.1.smithi193.stdout:1/894: dread da/d13/d24/f79 [0,4194304] 0 2024-03-23T08:23:30.280 INFO:tasks.workunit.client.1.smithi193.stdout:3/742: stat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/fce 0 2024-03-23T08:23:30.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/877: write df/d10/d3e/d54/d10d/f5b [2685840,58399] 0 2024-03-23T08:23:30.282 INFO:tasks.workunit.client.1.smithi193.stdout:9/943: dread d2/d7/db/d17/fc4 [0,4194304] 0 2024-03-23T08:23:30.286 INFO:tasks.workunit.client.1.smithi193.stdout:2/923: unlink d0/d1b/d14/d56/d7c/d7e/fca 0 2024-03-23T08:23:30.289 INFO:tasks.workunit.client.1.smithi193.stdout:9/944: creat d2/d7/f139 x:0 0 0 2024-03-23T08:23:30.290 INFO:tasks.workunit.client.1.smithi193.stdout:5/890: rename d0/d1c/d3b/dbc/dd2 to d0/d1c/d3b/d130 0 2024-03-23T08:23:30.292 INFO:tasks.workunit.client.1.smithi193.stdout:7/863: dwrite d6/d9/d1f/daf/fe1 [0,4194304] 0 2024-03-23T08:23:30.292 INFO:tasks.workunit.client.1.smithi193.stdout:0/848: truncate db/d11/d14/d24/d2a/d4d/d93/fc5 923343 0 2024-03-23T08:23:30.295 INFO:tasks.workunit.client.1.smithi193.stdout:4/904: dwrite d11/d1a/d33/d42/db2/dbe/d50/f5b [4194304,4194304] 0 2024-03-23T08:23:30.295 INFO:tasks.workunit.client.1.smithi193.stdout:5/891: readlink d0/l6 0 2024-03-23T08:23:30.297 INFO:tasks.workunit.client.1.smithi193.stdout:3/743: creat d1/de/d13/d21/d25/f101 x:0 0 0 2024-03-23T08:23:30.300 INFO:tasks.workunit.client.1.smithi193.stdout:8/878: rename df/l15 to df/d63/d84/df0/l127 0 2024-03-23T08:23:30.302 INFO:tasks.workunit.client.1.smithi193.stdout:4/905: write d11/f8c [1617787,59480] 0 2024-03-23T08:23:30.303 INFO:tasks.workunit.client.1.smithi193.stdout:6/746: dread d0/d27/d81/df/f25 [0,4194304] 0 2024-03-23T08:23:30.304 INFO:tasks.workunit.client.1.smithi193.stdout:7/864: creat d6/d9/d1f/de8/f12e x:0 0 0 2024-03-23T08:23:30.306 INFO:tasks.workunit.client.1.smithi193.stdout:5/892: creat d0/d1c/d3b/dbc/d112/d8e/f131 x:0 0 0 2024-03-23T08:23:30.307 INFO:tasks.workunit.client.1.smithi193.stdout:8/879: symlink df/d4b/d4d/l128 0 2024-03-23T08:23:30.309 INFO:tasks.workunit.client.1.smithi193.stdout:9/945: getdents d2/d7/db/d1b/d43/d71/db4/dec 0 2024-03-23T08:23:30.310 INFO:tasks.workunit.client.1.smithi193.stdout:3/744: symlink d1/de/d13/d21/d27/l102 0 2024-03-23T08:23:30.314 INFO:tasks.workunit.client.1.smithi193.stdout:8/880: chown df/d10/d83/cdb 51 1 2024-03-23T08:23:30.314 INFO:tasks.workunit.client.1.smithi193.stdout:5/893: mknod d0/d5a/dbf/c132 0 2024-03-23T08:23:30.314 INFO:tasks.workunit.client.1.smithi193.stdout:1/895: dwrite da/d13/d24/f97 [0,4194304] 0 2024-03-23T08:23:30.317 INFO:tasks.workunit.client.1.smithi193.stdout:7/865: read d6/d9/d1f/d23/d33/d3c/f4c [394976,71581] 0 2024-03-23T08:23:30.318 INFO:tasks.workunit.client.1.smithi193.stdout:4/906: rename d11/d27/df2/f103 to d11/d1a/d33/d42/d52/d70/f12b 0 2024-03-23T08:23:30.319 INFO:tasks.workunit.client.1.smithi193.stdout:0/849: dwrite db/d11/d14/d51/f56 [0,4194304] 0 2024-03-23T08:23:30.322 INFO:tasks.workunit.client.1.smithi193.stdout:8/881: chown df/d4b/d4d/f5c 1 1 2024-03-23T08:23:30.322 INFO:tasks.workunit.client.1.smithi193.stdout:9/946: mkdir d2/d7/db/d12d/dde/d13a 0 2024-03-23T08:23:30.323 INFO:tasks.workunit.client.1.smithi193.stdout:1/896: mknod da/d13/d3f/c124 0 2024-03-23T08:23:30.324 INFO:tasks.workunit.client.1.smithi193.stdout:5/894: mkdir d0/d1f/d93/de1/d133 0 2024-03-23T08:23:30.327 INFO:tasks.workunit.client.1.smithi193.stdout:1/897: write f8 [2956302,124495] 0 2024-03-23T08:23:30.327 INFO:tasks.workunit.client.1.smithi193.stdout:4/907: stat d11/d1a/d33/d42/db2/dbe/d50/dfb/l10b 0 2024-03-23T08:23:30.327 INFO:tasks.workunit.client.1.smithi193.stdout:8/882: dread - df/d63/d84/df4/f112 zero size 2024-03-23T08:23:30.328 INFO:tasks.workunit.client.1.smithi193.stdout:3/745: rmdir d1/de/d13/d21/d50/d6f/dca 0 2024-03-23T08:23:30.329 INFO:tasks.workunit.client.1.smithi193.stdout:8/883: fsync df/d10/fbb 0 2024-03-23T08:23:30.330 INFO:tasks.workunit.client.1.smithi193.stdout:6/747: dwrite d0/d27/d81/df/d11/d36/d61/d7c/fbf [0,4194304] 0 2024-03-23T08:23:30.332 INFO:tasks.workunit.client.1.smithi193.stdout:1/898: truncate da/dd/f64 6218089 0 2024-03-23T08:23:30.334 INFO:tasks.workunit.client.1.smithi193.stdout:8/884: dread - df/d63/d84/df4/f112 zero size 2024-03-23T08:23:30.334 INFO:tasks.workunit.client.1.smithi193.stdout:5/895: unlink d0/d1c/d7a/db1/d72/d73/d57/fb4 0 2024-03-23T08:23:30.334 INFO:tasks.workunit.client.1.smithi193.stdout:9/947: mknod d2/d7/de5/d109/c13b 0 2024-03-23T08:23:30.334 INFO:tasks.workunit.client.1.smithi193.stdout:6/748: write d0/fc6 [365231,69236] 0 2024-03-23T08:23:30.335 INFO:tasks.workunit.client.1.smithi193.stdout:3/746: chown d1/de/d13/d21/d25/d2b/d3c/l3e 556531168 1 2024-03-23T08:23:30.336 INFO:tasks.workunit.client.1.smithi193.stdout:4/908: mknod d11/d2d/d58/d80/df3/c12c 0 2024-03-23T08:23:30.337 INFO:tasks.workunit.client.1.smithi193.stdout:9/948: write d2/d7/f1e [2983754,119286] 0 2024-03-23T08:23:30.339 INFO:tasks.workunit.client.1.smithi193.stdout:1/899: write da/dd/df/d52/f6e [476825,106450] 0 2024-03-23T08:23:30.341 INFO:tasks.workunit.client.1.smithi193.stdout:4/909: readlink d11/d1a/d33/d42/db2/dbe/d50/l43 0 2024-03-23T08:23:30.341 INFO:tasks.workunit.client.1.smithi193.stdout:0/850: getdents db/d11/d14/d21/d33 0 2024-03-23T08:23:30.342 INFO:tasks.workunit.client.1.smithi193.stdout:4/910: chown d11/d95/fdd 40262 1 2024-03-23T08:23:30.342 INFO:tasks.workunit.client.1.smithi193.stdout:0/851: dread - db/d52/db8/f127 zero size 2024-03-23T08:23:30.343 INFO:tasks.workunit.client.1.smithi193.stdout:8/885: creat df/d63/d84/dc5/f129 x:0 0 0 2024-03-23T08:23:30.343 INFO:tasks.workunit.client.1.smithi193.stdout:5/896: truncate d0/d1f/f49 750955 0 2024-03-23T08:23:30.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/747: mkdir d1/de/d13/d21/d59/dc6/d103 0 2024-03-23T08:23:30.346 INFO:tasks.workunit.client.1.smithi193.stdout:4/911: dread - d11/d27/d85/de1/d99/f110 zero size 2024-03-23T08:23:30.347 INFO:tasks.workunit.client.1.smithi193.stdout:1/900: creat da/dd/df/d7c/dd2/f125 x:0 0 0 2024-03-23T08:23:30.348 INFO:tasks.workunit.client.1.smithi193.stdout:5/897: stat d0/d1c/d3b/d4b/dff/c122 0 2024-03-23T08:23:30.349 INFO:tasks.workunit.client.1.smithi193.stdout:6/749: dread d0/d27/d81/f20 [0,4194304] 0 2024-03-23T08:23:30.350 INFO:tasks.workunit.client.1.smithi193.stdout:3/748: mkdir d1/de/d28/d104 0 2024-03-23T08:23:30.351 INFO:tasks.workunit.client.1.smithi193.stdout:0/852: mkdir db/d11/d14/d24/d2a/de3/d129 0 2024-03-23T08:23:30.352 INFO:tasks.workunit.client.1.smithi193.stdout:1/901: write da/d13/d24/dac/dea/f67 [2086680,90138] 0 2024-03-23T08:23:30.354 INFO:tasks.workunit.client.1.smithi193.stdout:6/750: creat d0/d27/fee x:0 0 0 2024-03-23T08:23:30.360 INFO:tasks.workunit.client.1.smithi193.stdout:0/853: read db/d11/d14/d21/d33/d48/d85/fa7 [171105,60483] 0 2024-03-23T08:23:30.360 INFO:tasks.workunit.client.1.smithi193.stdout:5/898: write d0/d1f/fc4 [21919,53922] 0 2024-03-23T08:23:30.362 INFO:tasks.workunit.client.1.smithi193.stdout:8/886: dwrite df/d10/d3e/d54/f89 [0,4194304] 0 2024-03-23T08:23:30.363 INFO:tasks.workunit.client.1.smithi193.stdout:6/751: rmdir d0/d27/d81/df/d11/d3b/d51 39 2024-03-23T08:23:30.364 INFO:tasks.workunit.client.1.smithi193.stdout:3/749: rename d1/de/d13/d21/d59/dd4 to d1/de/d13/d21/d25/d2b/d3c/d4a/de4/d105 0 2024-03-23T08:23:30.370 INFO:tasks.workunit.client.1.smithi193.stdout:9/949: dwrite d2/d7/db/d17/d33/f63 [0,4194304] 0 2024-03-23T08:23:30.371 INFO:tasks.workunit.client.1.smithi193.stdout:5/899: write d0/d1c/d7a/db1/f67 [988970,99430] 0 2024-03-23T08:23:30.372 INFO:tasks.workunit.client.1.smithi193.stdout:8/887: write df/d10/d67/d7f/f104 [215642,103944] 0 2024-03-23T08:23:30.372 INFO:tasks.workunit.client.1.smithi193.stdout:0/854: symlink db/d11/d14/d24/d2a/d96/l12a 0 2024-03-23T08:23:30.374 INFO:tasks.workunit.client.1.smithi193.stdout:5/900: write d0/f15 [158520,122792] 0 2024-03-23T08:23:30.380 INFO:tasks.workunit.client.1.smithi193.stdout:8/888: chown df/d10/d3e/d54/d10d/f88 781 1 2024-03-23T08:23:30.383 INFO:tasks.workunit.client.1.smithi193.stdout:3/750: getdents d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/ddf 0 2024-03-23T08:23:30.383 INFO:tasks.workunit.client.1.smithi193.stdout:1/902: dread f8 [0,4194304] 0 2024-03-23T08:23:30.388 INFO:tasks.workunit.client.1.smithi193.stdout:1/903: dread - da/dd/df/d7e/d113/f11e zero size 2024-03-23T08:23:30.389 INFO:tasks.workunit.client.1.smithi193.stdout:0/855: dread db/d11/d14/d21/f22 [0,4194304] 0 2024-03-23T08:23:30.389 INFO:tasks.workunit.client.1.smithi193.stdout:5/901: dwrite d0/d5a/dbf/f101 [0,4194304] 0 2024-03-23T08:23:30.392 INFO:tasks.workunit.client.1.smithi193.stdout:9/950: dwrite d2/d7/db/d1b/fbb [4194304,4194304] 0 2024-03-23T08:23:30.392 INFO:tasks.workunit.client.1.smithi193.stdout:8/889: truncate df/d4b/d4d/d9c/f9f 204424 0 2024-03-23T08:23:30.397 INFO:tasks.workunit.client.1.smithi193.stdout:1/904: mkdir da/d13/d24/dac/de4/d126 0 2024-03-23T08:23:30.399 INFO:tasks.workunit.client.1.smithi193.stdout:9/951: symlink d2/d7/db/d17/d4b/d59/l13c 0 2024-03-23T08:23:30.400 INFO:tasks.workunit.client.1.smithi193.stdout:8/890: creat df/dcc/f12a x:0 0 0 2024-03-23T08:23:30.401 INFO:tasks.workunit.client.1.smithi193.stdout:8/891: chown df/dc6/fa2 1 1 2024-03-23T08:23:30.402 INFO:tasks.workunit.client.1.smithi193.stdout:3/751: dwrite d1/de/f11 [0,4194304] 0 2024-03-23T08:23:30.403 INFO:tasks.workunit.client.1.smithi193.stdout:0/856: symlink db/d11/d14/d24/d47/d59/dd4/dfc/l12b 0 2024-03-23T08:23:30.404 INFO:tasks.workunit.client.1.smithi193.stdout:9/952: symlink d2/d7/db/d1b/l13d 0 2024-03-23T08:23:30.405 INFO:tasks.workunit.client.1.smithi193.stdout:8/892: mknod df/d63/c12b 0 2024-03-23T08:23:30.407 INFO:tasks.workunit.client.1.smithi193.stdout:1/905: dwrite da/f3c [0,4194304] 0 2024-03-23T08:23:30.410 INFO:tasks.workunit.client.1.smithi193.stdout:8/893: unlink df/d10/c11e 0 2024-03-23T08:23:30.410 INFO:tasks.workunit.client.1.smithi193.stdout:5/902: dread d0/d1c/d7a/db1/f8b [0,4194304] 0 2024-03-23T08:23:30.419 INFO:tasks.workunit.client.1.smithi193.stdout:8/894: link df/d10/d3e/d54/dd2/cb1 df/d63/d84/dc5/c12c 0 2024-03-23T08:23:30.419 INFO:tasks.workunit.client.1.smithi193.stdout:0/857: dwrite db/d11/d14/d21/d33/d69/d9b/df2/ff6 [0,4194304] 0 2024-03-23T08:23:30.422 INFO:tasks.workunit.client.1.smithi193.stdout:1/906: dwrite da/dd/df/d7e/fd9 [0,4194304] 0 2024-03-23T08:23:30.424 INFO:tasks.workunit.client.1.smithi193.stdout:5/903: dwrite d0/fe [0,4194304] 0 2024-03-23T08:23:30.426 INFO:tasks.workunit.client.1.smithi193.stdout:5/904: chown d0/d1c/d7a/cf6 2994 1 2024-03-23T08:23:30.427 INFO:tasks.workunit.client.1.smithi193.stdout:8/895: mknod df/d4b/c12d 0 2024-03-23T08:23:30.428 INFO:tasks.workunit.client.1.smithi193.stdout:1/907: rename da/dd/df/d7e/d7f/f8a to da/d13/d24/dac/dea/d76/d77/f127 0 2024-03-23T08:23:30.429 INFO:tasks.workunit.client.1.smithi193.stdout:9/953: dread d2/f98 [0,4194304] 0 2024-03-23T08:23:30.430 INFO:tasks.workunit.client.1.smithi193.stdout:0/858: symlink db/d11/d14/d24/l12c 0 2024-03-23T08:23:30.433 INFO:tasks.workunit.client.1.smithi193.stdout:1/908: chown da/dd/f14 1542147 1 2024-03-23T08:23:30.434 INFO:tasks.workunit.client.1.smithi193.stdout:8/896: link df/fb3 df/d10/d83/dcb/f12e 0 2024-03-23T08:23:30.435 INFO:tasks.workunit.client.1.smithi193.stdout:0/859: rmdir db/d52 39 2024-03-23T08:23:30.436 INFO:tasks.workunit.client.1.smithi193.stdout:9/954: creat d2/d7/db/d7a/d95/da9/f13e x:0 0 0 2024-03-23T08:23:30.438 INFO:tasks.workunit.client.1.smithi193.stdout:1/909: mkdir da/dd/df/d7c/dd2/d128 0 2024-03-23T08:23:30.440 INFO:tasks.workunit.client.1.smithi193.stdout:9/955: write d2/d7/db/d32/fd2 [32986,64457] 0 2024-03-23T08:23:30.442 INFO:tasks.workunit.client.1.smithi193.stdout:8/897: getdents df/d10/d3e/d54/dd2/dca/d10e 0 2024-03-23T08:23:30.442 INFO:tasks.workunit.client.1.smithi193.stdout:1/910: chown da/d13/d24/c36 357401851 1 2024-03-23T08:23:30.442 INFO:tasks.workunit.client.1.smithi193.stdout:5/905: dwrite d0/d1c/d7a/db1/f76 [4194304,4194304] 0 2024-03-23T08:23:30.445 INFO:tasks.workunit.client.1.smithi193.stdout:2/924: sync 2024-03-23T08:23:30.445 INFO:tasks.workunit.client.1.smithi193.stdout:8/898: fsync df/d10/d3e/d54/dd2/fa5 0 2024-03-23T08:23:30.445 INFO:tasks.workunit.client.1.smithi193.stdout:1/911: write f8 [3347574,113451] 0 2024-03-23T08:23:30.452 INFO:tasks.workunit.client.1.smithi193.stdout:5/906: dread - d0/d1c/d3b/d81/df2/d97/faf zero size 2024-03-23T08:23:30.453 INFO:tasks.workunit.client.1.smithi193.stdout:9/956: chown d2/d7/db/d17/d4b/d59/d97/ff4 189 1 2024-03-23T08:23:30.453 INFO:tasks.workunit.client.1.smithi193.stdout:2/925: mknod d0/d1b/d14/d56/cfe 0 2024-03-23T08:23:30.455 INFO:tasks.workunit.client.1.smithi193.stdout:1/912: symlink da/dd/df/d75/l129 0 2024-03-23T08:23:30.458 INFO:tasks.workunit.client.1.smithi193.stdout:5/907: chown d0/d1c/d7a/db1/d72/d73/d45/da2 391 1 2024-03-23T08:23:30.459 INFO:tasks.workunit.client.1.smithi193.stdout:8/899: unlink df/d63/d84/df4/l126 0 2024-03-23T08:23:30.460 INFO:tasks.workunit.client.1.smithi193.stdout:0/860: dwrite db/d11/d14/d24/d47/f74 [0,4194304] 0 2024-03-23T08:23:30.461 INFO:tasks.workunit.client.1.smithi193.stdout:2/926: creat d0/d1b/d14/d56/d7c/fff x:0 0 0 2024-03-23T08:23:30.466 INFO:tasks.workunit.client.1.smithi193.stdout:5/908: symlink d0/d1c/d7a/db1/d72/l134 0 2024-03-23T08:23:30.471 INFO:tasks.workunit.client.1.smithi193.stdout:9/957: rename d2/d7/db/d17/d4b/ccd to d2/d7/db/d1b/d43/d71/db4/dec/d5f/c13f 0 2024-03-23T08:23:30.472 INFO:tasks.workunit.client.1.smithi193.stdout:9/958: write d2/d7/db/d7a/d95/da9/f120 [758776,57655] 0 2024-03-23T08:23:30.475 INFO:tasks.workunit.client.1.smithi193.stdout:0/861: dread db/f87 [0,4194304] 0 2024-03-23T08:23:30.477 INFO:tasks.workunit.client.1.smithi193.stdout:0/862: write db/d11/d14/d24/d47/d59/fd9 [897586,33141] 0 2024-03-23T08:23:30.478 INFO:tasks.workunit.client.1.smithi193.stdout:6/752: dread d0/d27/d81/df/d11/d36/f44 [0,4194304] 0 2024-03-23T08:23:30.479 INFO:tasks.workunit.client.1.smithi193.stdout:9/959: getdents d2/d7/db/d1b 0 2024-03-23T08:23:30.480 INFO:tasks.workunit.client.1.smithi193.stdout:0/863: rmdir db/d11/d14/d21 39 2024-03-23T08:23:30.493 INFO:tasks.workunit.client.1.smithi193.stdout:5/909: dread d0/d1f/f41 [0,4194304] 0 2024-03-23T08:23:30.521 INFO:tasks.workunit.client.1.smithi193.stdout:9/960: creat d2/d7/db/d7a/d95/da9/d113/f140 x:0 0 0 2024-03-23T08:23:30.522 INFO:tasks.workunit.client.1.smithi193.stdout:0/864: creat db/d11/d14/d3b/d71/f12d x:0 0 0 2024-03-23T08:23:30.530 INFO:tasks.workunit.client.1.smithi193.stdout:9/961: dwrite d2/d7/db/d17/d33/f45 [8388608,4194304] 0 2024-03-23T08:23:30.531 INFO:tasks.workunit.client.1.smithi193.stdout:8/900: dwrite df/f21 [0,4194304] 0 2024-03-23T08:23:30.533 INFO:tasks.workunit.client.1.smithi193.stdout:5/910: dwrite d0/d1c/d3b/d81/df2/d97/fcc [0,4194304] 0 2024-03-23T08:23:30.542 INFO:tasks.workunit.client.1.smithi193.stdout:8/901: dwrite df/f2c [4194304,4194304] 0 2024-03-23T08:23:30.547 INFO:tasks.workunit.client.1.smithi193.stdout:7/866: sync 2024-03-23T08:23:30.555 INFO:tasks.workunit.client.1.smithi193.stdout:7/867: dwrite d6/d9/d1f/d23/d33/d3c/d41/f79 [4194304,4194304] 0 2024-03-23T08:23:30.564 INFO:tasks.workunit.client.1.smithi193.stdout:7/868: dwrite d6/d9/d1f/de8/f12e [0,4194304] 0 2024-03-23T08:23:30.571 INFO:tasks.workunit.client.1.smithi193.stdout:1/913: truncate da/dd/df/d7e/d7f/db2/f10c 168673 0 2024-03-23T08:23:30.573 INFO:tasks.workunit.client.1.smithi193.stdout:9/962: creat d2/d7/db/d17/d33/db2/f141 x:0 0 0 2024-03-23T08:23:30.574 INFO:tasks.workunit.client.1.smithi193.stdout:8/902: mknod df/d10/d3e/d54/dd2/c12f 0 2024-03-23T08:23:30.575 INFO:tasks.workunit.client.1.smithi193.stdout:2/927: dwrite d0/f1d [0,4194304] 0 2024-03-23T08:23:30.577 INFO:tasks.workunit.client.1.smithi193.stdout:7/869: rename d6/d9/d1f/d23/d33/d38/d46/d82/f8c to d6/d9/d1f/d23/d33/d38/d3b/db7/f12f 0 2024-03-23T08:23:30.580 INFO:tasks.workunit.client.1.smithi193.stdout:1/914: creat da/dd/df/d7e/d7f/db2/ddb/f12a x:0 0 0 2024-03-23T08:23:30.581 INFO:tasks.workunit.client.1.smithi193.stdout:6/753: getdents d0/d27/d81/df/d11/d36/d61/d75 0 2024-03-23T08:23:30.582 INFO:tasks.workunit.client.1.smithi193.stdout:2/928: creat d0/d1b/d14/d56/d7c/d7e/f100 x:0 0 0 2024-03-23T08:23:30.583 INFO:tasks.workunit.client.1.smithi193.stdout:8/903: dread df/dc6/fad [0,4194304] 0 2024-03-23T08:23:30.583 INFO:tasks.workunit.client.1.smithi193.stdout:1/915: chown da/d13/d3f/f54 5186020 1 2024-03-23T08:23:30.584 INFO:tasks.workunit.client.1.smithi193.stdout:4/912: sync 2024-03-23T08:23:30.584 INFO:tasks.workunit.client.1.smithi193.stdout:3/752: sync 2024-03-23T08:23:30.585 INFO:tasks.workunit.client.1.smithi193.stdout:5/911: link d0/l3f d0/d1f/d93/de1/d11f/l135 0 2024-03-23T08:23:30.587 INFO:tasks.workunit.client.1.smithi193.stdout:0/865: getdents db/d52/db8/dcf 0 2024-03-23T08:23:30.587 INFO:tasks.workunit.client.1.smithi193.stdout:2/929: chown d0/d1b/d14/d66/d4e 7 1 2024-03-23T08:23:30.588 INFO:tasks.workunit.client.1.smithi193.stdout:8/904: chown df/d10/lc7 969 1 2024-03-23T08:23:30.589 INFO:tasks.workunit.client.1.smithi193.stdout:3/753: dread - d1/de/d78/fc7 zero size 2024-03-23T08:23:30.589 INFO:tasks.workunit.client.1.smithi193.stdout:5/912: dread - d0/d1f/f2e zero size 2024-03-23T08:23:30.590 INFO:tasks.workunit.client.1.smithi193.stdout:7/870: link d6/d9/d1f/d23/d33/d38/d3b/db7/l10b d6/l130 0 2024-03-23T08:23:30.592 INFO:tasks.workunit.client.1.smithi193.stdout:4/913: creat d11/d1a/f12d x:0 0 0 2024-03-23T08:23:30.593 INFO:tasks.workunit.client.1.smithi193.stdout:0/866: mkdir db/d11/d14/d3b/d71/d12e 0 2024-03-23T08:23:30.594 INFO:tasks.workunit.client.1.smithi193.stdout:9/963: dwrite d2/d7/db/d1b/d43/d71/db4/dec/d3c/f91 [0,4194304] 0 2024-03-23T08:23:30.594 INFO:tasks.workunit.client.1.smithi193.stdout:2/930: creat d0/d1b/d14/d56/da3/f101 x:0 0 0 2024-03-23T08:23:30.595 INFO:tasks.workunit.client.1.smithi193.stdout:3/754: write d1/de/fd0 [240580,115377] 0 2024-03-23T08:23:30.600 INFO:tasks.workunit.client.1.smithi193.stdout:1/916: getdents da/dd/df/d7e/d86 0 2024-03-23T08:23:30.602 INFO:tasks.workunit.client.1.smithi193.stdout:7/871: stat d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/d112/f11f 0 2024-03-23T08:23:30.605 INFO:tasks.workunit.client.1.smithi193.stdout:3/755: unlink d1/de/d13/d21/f31 0 2024-03-23T08:23:30.605 INFO:tasks.workunit.client.1.smithi193.stdout:1/917: chown da/dd/df/d7e/ca3 30 1 2024-03-23T08:23:30.607 INFO:tasks.workunit.client.1.smithi193.stdout:8/905: dwrite df/d10/d83/dcb/f11a [0,4194304] 0 2024-03-23T08:23:30.609 INFO:tasks.workunit.client.1.smithi193.stdout:7/872: write d6/d9/d1f/d4d/d6b/fe2 [94191,85594] 0 2024-03-23T08:23:30.609 INFO:tasks.workunit.client.1.smithi193.stdout:4/914: truncate d11/d1a/d33/d42/d52/f87 198473 0 2024-03-23T08:23:30.610 INFO:tasks.workunit.client.1.smithi193.stdout:2/931: unlink d0/d37/d48/c6e 0 2024-03-23T08:23:30.614 INFO:tasks.workunit.client.1.smithi193.stdout:0/867: link db/d11/d14/d24/d47/f74 db/d11/d14/d24/d47/dc7/f12f 0 2024-03-23T08:23:30.615 INFO:tasks.workunit.client.1.smithi193.stdout:1/918: mkdir da/d13/d3f/d12b 0 2024-03-23T08:23:30.615 INFO:tasks.workunit.client.1.smithi193.stdout:8/906: chown df/d4b/d4d/l95 50526 1 2024-03-23T08:23:30.617 INFO:tasks.workunit.client.1.smithi193.stdout:6/754: dread d0/d27/d81/df/d11/d36/d61/d12/f1b [0,4194304] 0 2024-03-23T08:23:30.618 INFO:tasks.workunit.client.1.smithi193.stdout:8/907: dread - df/d63/d84/da8/fed zero size 2024-03-23T08:23:30.620 INFO:tasks.workunit.client.1.smithi193.stdout:4/915: creat d11/d1a/d33/d42/db2/dbe/d50/db3/f12e x:0 0 0 2024-03-23T08:23:30.622 INFO:tasks.workunit.client.1.smithi193.stdout:1/919: mknod da/dd/df/d7c/dd2/c12c 0 2024-03-23T08:23:30.624 INFO:tasks.workunit.client.1.smithi193.stdout:7/873: dread d6/d9/d1f/f50 [0,4194304] 0 2024-03-23T08:23:30.626 INFO:tasks.workunit.client.1.smithi193.stdout:6/755: read - d0/d27/d81/df/d11/d3b/fbc zero size 2024-03-23T08:23:30.628 INFO:tasks.workunit.client.1.smithi193.stdout:8/908: write df/d4b/d4d/f22 [433074,62239] 0 2024-03-23T08:23:30.628 INFO:tasks.workunit.client.1.smithi193.stdout:4/916: mkdir d11/d12f 0 2024-03-23T08:23:30.629 INFO:tasks.workunit.client.1.smithi193.stdout:5/913: dwrite d0/d1c/d7a/db1/d72/d99/dd8/f108 [0,4194304] 0 2024-03-23T08:23:30.631 INFO:tasks.workunit.client.1.smithi193.stdout:9/964: truncate d2/d7/db/d17/d4b/d8b/fb7 838941 0 2024-03-23T08:23:30.632 INFO:tasks.workunit.client.1.smithi193.stdout:3/756: dwrite d1/de/d13/d21/d27/f1f [4194304,4194304] 0 2024-03-23T08:23:30.634 INFO:tasks.workunit.client.1.smithi193.stdout:8/909: fdatasync df/d10/f118 0 2024-03-23T08:23:30.639 INFO:tasks.workunit.client.1.smithi193.stdout:7/874: stat d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/df3/da1 0 2024-03-23T08:23:30.639 INFO:tasks.workunit.client.1.smithi193.stdout:1/920: mkdir da/d13/d24/dac/dea/d76/d77/d12d 0 2024-03-23T08:23:30.639 INFO:tasks.workunit.client.1.smithi193.stdout:6/756: unlink d0/d27/d81/df/d11/d3b/f66 0 2024-03-23T08:23:30.640 INFO:tasks.workunit.client.1.smithi193.stdout:5/914: unlink d0/d1c/d3b/dbc/d112/fe6 0 2024-03-23T08:23:30.642 INFO:tasks.workunit.client.1.smithi193.stdout:4/917: rename d11/f8c to d11/d2d/da8/f130 0 2024-03-23T08:23:30.642 INFO:tasks.workunit.client.1.smithi193.stdout:3/757: dread d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/fb5 [0,4194304] 0 2024-03-23T08:23:30.643 INFO:tasks.workunit.client.1.smithi193.stdout:0/868: dread db/d11/d14/d21/d33/f68 [0,4194304] 0 2024-03-23T08:23:30.643 INFO:tasks.workunit.client.1.smithi193.stdout:3/758: fsync d1/de/d78/fc7 0 2024-03-23T08:23:30.644 INFO:tasks.workunit.client.1.smithi193.stdout:1/921: mkdir da/d13/d3f/d12e 0 2024-03-23T08:23:30.644 INFO:tasks.workunit.client.1.smithi193.stdout:6/757: truncate d0/d27/d81/df/d11/d36/d61/d12/f35 3784313 0 2024-03-23T08:23:30.645 INFO:tasks.workunit.client.1.smithi193.stdout:0/869: fdatasync db/d11/d14/d24/ddd/fef 0 2024-03-23T08:23:30.646 INFO:tasks.workunit.client.1.smithi193.stdout:8/910: creat df/d4b/d4d/d9c/f130 x:0 0 0 2024-03-23T08:23:30.646 INFO:tasks.workunit.client.1.smithi193.stdout:1/922: stat da/dd/df/d52/f7a 0 2024-03-23T08:23:30.647 INFO:tasks.workunit.client.1.smithi193.stdout:9/965: symlink d2/d7/db/d1b/d43/d71/db4/dec/d5f/l142 0 2024-03-23T08:23:30.647 INFO:tasks.workunit.client.1.smithi193.stdout:3/759: chown d1/de/d13/d21/d25/d2b/d3c/d4a/d85/fd8 659 1 2024-03-23T08:23:30.649 INFO:tasks.workunit.client.1.smithi193.stdout:9/966: fdatasync d2/d7/db/d7a/d95/da9/f120 0 2024-03-23T08:23:30.649 INFO:tasks.workunit.client.1.smithi193.stdout:8/911: read df/d63/d84/fde [378087,50141] 0 2024-03-23T08:23:30.651 INFO:tasks.workunit.client.1.smithi193.stdout:8/912: truncate df/dc6/d8b/ffa 975285 0 2024-03-23T08:23:30.651 INFO:tasks.workunit.client.1.smithi193.stdout:6/758: creat d0/d27/d81/df/d11/d3b/d6c/fef x:0 0 0 2024-03-23T08:23:30.653 INFO:tasks.workunit.client.1.smithi193.stdout:0/870: chown db/d11/d14/d24/d2a/d4b/c91 2 1 2024-03-23T08:23:30.653 INFO:tasks.workunit.client.1.smithi193.stdout:7/875: getdents d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/df3/da1/de0 0 2024-03-23T08:23:30.654 INFO:tasks.workunit.client.1.smithi193.stdout:0/871: chown db/d11/d14/d3b/f63 1 1 2024-03-23T08:23:30.655 INFO:tasks.workunit.client.1.smithi193.stdout:1/923: mknod da/dd/df/d7e/d7f/db2/c12f 0 2024-03-23T08:23:30.656 INFO:tasks.workunit.client.1.smithi193.stdout:3/760: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/f106 x:0 0 0 2024-03-23T08:23:30.658 INFO:tasks.workunit.client.1.smithi193.stdout:9/967: dwrite d2/d7/db/d17/f53 [0,4194304] 0 2024-03-23T08:23:30.658 INFO:tasks.workunit.client.1.smithi193.stdout:5/915: truncate d0/d1c/d7a/f8f 979046 0 2024-03-23T08:23:30.660 INFO:tasks.workunit.client.1.smithi193.stdout:3/761: truncate d1/de/d13/d21/d25/f7e 4757406 0 2024-03-23T08:23:30.663 INFO:tasks.workunit.client.1.smithi193.stdout:1/924: mknod da/d13/d24/dac/de4/c130 0 2024-03-23T08:23:30.664 INFO:tasks.workunit.client.1.smithi193.stdout:9/968: creat d2/d7/db/d1b/f143 x:0 0 0 2024-03-23T08:23:30.664 INFO:tasks.workunit.client.1.smithi193.stdout:7/876: write d6/d9/d4e/fae [1529010,25677] 0 2024-03-23T08:23:30.670 INFO:tasks.workunit.client.1.smithi193.stdout:1/925: rmdir da/dd 39 2024-03-23T08:23:30.672 INFO:tasks.workunit.client.1.smithi193.stdout:8/913: dwrite df/f56 [0,4194304] 0 2024-03-23T08:23:30.674 INFO:tasks.workunit.client.1.smithi193.stdout:6/759: write d0/d27/d81/df/d11/f30 [896144,113223] 0 2024-03-23T08:23:30.677 INFO:tasks.workunit.client.1.smithi193.stdout:5/916: dwrite d0/d1c/d7a/db1/d72/d73/f120 [0,4194304] 0 2024-03-23T08:23:30.679 INFO:tasks.workunit.client.1.smithi193.stdout:5/917: chown d0/d1c/d7a/db1/d72/d73/f120 11776 1 2024-03-23T08:23:30.680 INFO:tasks.workunit.client.1.smithi193.stdout:1/926: truncate da/dd/fa6 4522598 0 2024-03-23T08:23:30.681 INFO:tasks.workunit.client.1.smithi193.stdout:8/914: creat df/d63/d84/dc5/f131 x:0 0 0 2024-03-23T08:23:30.682 INFO:tasks.workunit.client.1.smithi193.stdout:0/872: link db/d11/d14/d3b/c117 db/d11/d14/d24/d47/d59/c130 0 2024-03-23T08:23:30.682 INFO:tasks.workunit.client.1.smithi193.stdout:8/915: chown df/d63/d84/c92 608 1 2024-03-23T08:23:30.683 INFO:tasks.workunit.client.1.smithi193.stdout:0/873: fsync db/d11/d14/d21/dd2/f125 0 2024-03-23T08:23:30.685 INFO:tasks.workunit.client.1.smithi193.stdout:6/760: write d0/d27/d81/df/f25 [3651697,109251] 0 2024-03-23T08:23:30.688 INFO:tasks.workunit.client.1.smithi193.stdout:7/877: link d6/d9/d4e/d81/dca/cd4 d6/d9/d1f/daf/dd2/c131 0 2024-03-23T08:23:30.690 INFO:tasks.workunit.client.1.smithi193.stdout:0/874: readlink db/d11/d14/d3b/l89 0 2024-03-23T08:23:30.691 INFO:tasks.workunit.client.1.smithi193.stdout:9/969: getdents d2/d7/db/d17 0 2024-03-23T08:23:30.694 INFO:tasks.workunit.client.1.smithi193.stdout:5/918: link d0/d1c/d7a/d86/d9f/fc8 d0/d1c/d3b/dbc/f136 0 2024-03-23T08:23:30.695 INFO:tasks.workunit.client.1.smithi193.stdout:8/916: dwrite df/dcc/f12a [0,4194304] 0 2024-03-23T08:23:30.695 INFO:tasks.workunit.client.1.smithi193.stdout:6/761: write d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f84 [441376,21728] 0 2024-03-23T08:23:30.696 INFO:tasks.workunit.client.1.smithi193.stdout:7/878: write d6/d9/d1f/d23/d33/d38/d66/f12d [985885,110141] 0 2024-03-23T08:23:30.700 INFO:tasks.workunit.client.1.smithi193.stdout:3/762: dread d1/de/d13/d96/d99/f30 [0,4194304] 0 2024-03-23T08:23:30.700 INFO:tasks.workunit.client.1.smithi193.stdout:0/875: mknod db/d11/d14/d21/dd2/c131 0 2024-03-23T08:23:30.700 INFO:tasks.workunit.client.1.smithi193.stdout:1/927: write da/dd/df/f4b [2466380,83841] 0 2024-03-23T08:23:30.702 INFO:tasks.workunit.client.1.smithi193.stdout:6/762: fsync d0/fd4 0 2024-03-23T08:23:30.703 INFO:tasks.workunit.client.1.smithi193.stdout:9/970: write d2/d7/db/d17/d33/d70/fd0 [1466838,88796] 0 2024-03-23T08:23:30.706 INFO:tasks.workunit.client.1.smithi193.stdout:3/763: chown d1/de/d13/d21/d50/d6f/d6e/fc0 741758645 1 2024-03-23T08:23:30.709 INFO:tasks.workunit.client.1.smithi193.stdout:6/763: unlink d0/d27/d81/df/d11/d36/d61/d12/d21/l5d 0 2024-03-23T08:23:30.711 INFO:tasks.workunit.client.1.smithi193.stdout:5/919: dwrite d0/d1f/d93/de1/f105 [4194304,4194304] 0 2024-03-23T08:23:30.712 INFO:tasks.workunit.client.1.smithi193.stdout:3/764: chown d1/de/d13/d21/d25/d2b/d3c/d4a/ff9 5 1 2024-03-23T08:23:30.714 INFO:tasks.workunit.client.1.smithi193.stdout:0/876: getdents db/d11/d14/d24/d47/d59/d8c 0 2024-03-23T08:23:30.715 INFO:tasks.workunit.client.1.smithi193.stdout:5/920: fdatasync d0/d1c/d7a/db1/f67 0 2024-03-23T08:23:30.717 INFO:tasks.workunit.client.1.smithi193.stdout:8/917: truncate df/d10/d3e/f5f 3241727 0 2024-03-23T08:23:30.717 INFO:tasks.workunit.client.1.smithi193.stdout:6/764: chown d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/l9e 3283856 1 2024-03-23T08:23:30.720 INFO:tasks.workunit.client.1.smithi193.stdout:9/971: dwrite d2/d7/db/d1b/d43/d71/db4/dec/f10e [0,4194304] 0 2024-03-23T08:23:30.721 INFO:tasks.workunit.client.1.smithi193.stdout:5/921: chown d0/d1c/d3b/d4b/d55/f80 0 1 2024-03-23T08:23:30.722 INFO:tasks.workunit.client.1.smithi193.stdout:0/877: chown db/d11/d14/d24/d47/d59/dd4/dfc/l12b 3031 1 2024-03-23T08:23:30.723 INFO:tasks.workunit.client.1.smithi193.stdout:9/972: chown d2/d7/db/d17/d33/d70 29 1 2024-03-23T08:23:30.723 INFO:tasks.workunit.client.1.smithi193.stdout:0/878: stat db/d11/d14/d24/d2a/d4d/fa1 0 2024-03-23T08:23:30.728 INFO:tasks.workunit.client.1.smithi193.stdout:3/765: getdents d1/de/d13/d21/d25/d2b/d3c 0 2024-03-23T08:23:30.728 INFO:tasks.workunit.client.1.smithi193.stdout:9/973: stat d2/d7/db/d17/d1f 0 2024-03-23T08:23:30.728 INFO:tasks.workunit.client.1.smithi193.stdout:2/932: sync 2024-03-23T08:23:30.729 INFO:tasks.workunit.client.1.smithi193.stdout:6/765: mkdir d0/d27/d81/df/d11/d3b/df0 0 2024-03-23T08:23:30.733 INFO:tasks.workunit.client.1.smithi193.stdout:1/928: dread da/d13/d24/f7b [0,4194304] 0 2024-03-23T08:23:30.734 INFO:tasks.workunit.client.1.smithi193.stdout:6/766: creat d0/d27/ff1 x:0 0 0 2024-03-23T08:23:30.735 INFO:tasks.workunit.client.1.smithi193.stdout:3/766: symlink d1/de/d13/d21/d25/d34/da8/l107 0 2024-03-23T08:23:30.736 INFO:tasks.workunit.client.1.smithi193.stdout:0/879: rename db/d11/d14/d24/d47/d59/dd4/dfc/l128 to db/d11/d14/d21/dd2/l132 0 2024-03-23T08:23:30.736 INFO:tasks.workunit.client.1.smithi193.stdout:2/933: stat d0/d1b/d14/d56/da3/db0/fb6 0 2024-03-23T08:23:30.741 INFO:tasks.workunit.client.1.smithi193.stdout:9/974: getdents d2/d7/db/d7a/d95/da9/d113/d136 0 2024-03-23T08:23:30.741 INFO:tasks.workunit.client.1.smithi193.stdout:8/918: dwrite df/d10/d3e/d54/dd2/f8c [0,4194304] 0 2024-03-23T08:23:30.744 INFO:tasks.workunit.client.1.smithi193.stdout:2/934: write d0/d1b/d14/d56/da3/f101 [128635,20544] 0 2024-03-23T08:23:30.745 INFO:tasks.workunit.client.1.smithi193.stdout:5/922: dwrite d0/d1c/d7a/db1/d72/d73/d45/fa0 [0,4194304] 0 2024-03-23T08:23:30.747 INFO:tasks.workunit.client.1.smithi193.stdout:3/767: write d1/de/d13/d21/d50/d6f/fa7 [2440451,39067] 0 2024-03-23T08:23:30.748 INFO:tasks.workunit.client.1.smithi193.stdout:8/919: unlink df/d63/d84/fd8 0 2024-03-23T08:23:30.748 INFO:tasks.workunit.client.1.smithi193.stdout:2/935: readlink d0/d1b/d14/l7d 0 2024-03-23T08:23:30.755 INFO:tasks.workunit.client.1.smithi193.stdout:8/920: truncate df/dc6/d8b/dae/ff3 126926 0 2024-03-23T08:23:30.755 INFO:tasks.workunit.client.1.smithi193.stdout:3/768: write d1/de/d13/d96/d99/f30 [601604,13630] 0 2024-03-23T08:23:30.757 INFO:tasks.workunit.client.1.smithi193.stdout:2/936: creat d0/d1b/d14/da1/dd5/d72/f102 x:0 0 0 2024-03-23T08:23:30.759 INFO:tasks.workunit.client.1.smithi193.stdout:3/769: fsync d1/de/f10 0 2024-03-23T08:23:30.759 INFO:tasks.workunit.client.1.smithi193.stdout:5/923: getdents d0/d1c/d7a/db1/d72 0 2024-03-23T08:23:30.761 INFO:tasks.workunit.client.1.smithi193.stdout:2/937: creat d0/d37/d48/f103 x:0 0 0 2024-03-23T08:23:30.761 INFO:tasks.workunit.client.1.smithi193.stdout:8/921: rmdir df/d11d 0 2024-03-23T08:23:30.762 INFO:tasks.workunit.client.1.smithi193.stdout:5/924: mkdir d0/d1c/d137 0 2024-03-23T08:23:30.764 INFO:tasks.workunit.client.1.smithi193.stdout:8/922: write df/d4b/d4d/d9c/dee/f102 [610338,72651] 0 2024-03-23T08:23:30.765 INFO:tasks.workunit.client.1.smithi193.stdout:1/929: dwrite f1 [0,4194304] 0 2024-03-23T08:23:30.766 INFO:tasks.workunit.client.1.smithi193.stdout:0/880: truncate db/d11/d14/d24/d2a/d4d/f50 1284598 0 2024-03-23T08:23:30.770 INFO:tasks.workunit.client.1.smithi193.stdout:5/925: mkdir d0/d1c/d7a/db1/d72/d73/d57/d138 0 2024-03-23T08:23:30.773 INFO:tasks.workunit.client.1.smithi193.stdout:0/881: creat db/f133 x:0 0 0 2024-03-23T08:23:30.773 INFO:tasks.workunit.client.1.smithi193.stdout:2/938: dwrite d0/d1b/d14/da1/dd5/d72/fc9 [0,4194304] 0 2024-03-23T08:23:30.775 INFO:tasks.workunit.client.1.smithi193.stdout:2/939: chown d0/d1b/d14/da1/dd5/d72/fc9 152698 1 2024-03-23T08:23:30.776 INFO:tasks.workunit.client.1.smithi193.stdout:1/930: getdents da/d13/d24/dac/dea/d76/d77/de7/d11b 0 2024-03-23T08:23:30.777 INFO:tasks.workunit.client.1.smithi193.stdout:0/882: chown db/l36 344 1 2024-03-23T08:23:30.777 INFO:tasks.workunit.client.1.smithi193.stdout:5/926: mkdir d0/d1c/d7a/db1/d72/d73/d139 0 2024-03-23T08:23:30.778 INFO:tasks.workunit.client.1.smithi193.stdout:1/931: chown da/dd/df/d7e/d86/d95/dc4/cc8 152748 1 2024-03-23T08:23:30.782 INFO:tasks.workunit.client.1.smithi193.stdout:0/883: creat db/d11/d14/f134 x:0 0 0 2024-03-23T08:23:30.785 INFO:tasks.workunit.client.1.smithi193.stdout:0/884: unlink db/d11/d14/d21/f27 0 2024-03-23T08:23:30.786 INFO:tasks.workunit.client.1.smithi193.stdout:0/885: write db/d11/d14/f134 [298444,31127] 0 2024-03-23T08:23:30.789 INFO:tasks.workunit.client.1.smithi193.stdout:0/886: symlink db/d11/d14/d3b/d10c/l135 0 2024-03-23T08:23:30.791 INFO:tasks.workunit.client.1.smithi193.stdout:0/887: mknod db/d11/d14/d3b/d10c/c136 0 2024-03-23T08:23:30.794 INFO:tasks.workunit.client.1.smithi193.stdout:0/888: mknod db/d11/c137 0 2024-03-23T08:23:30.796 INFO:tasks.workunit.client.1.smithi193.stdout:1/932: dread da/f4e [0,4194304] 0 2024-03-23T08:23:30.799 INFO:tasks.workunit.client.1.smithi193.stdout:0/889: mknod db/d11/d14/d3b/d71/d122/c138 0 2024-03-23T08:23:30.801 INFO:tasks.workunit.client.1.smithi193.stdout:1/933: creat da/dd/df/d7e/d86/f131 x:0 0 0 2024-03-23T08:23:30.803 INFO:tasks.workunit.client.1.smithi193.stdout:0/890: rmdir db/d11/d14/d24/d47/d59 39 2024-03-23T08:23:30.803 INFO:tasks.workunit.client.1.smithi193.stdout:5/927: dwrite d0/f3e [0,4194304] 0 2024-03-23T08:23:30.803 INFO:tasks.workunit.client.1.smithi193.stdout:2/940: dwrite d0/d1b/d19/f1a [0,4194304] 0 2024-03-23T08:23:30.804 INFO:tasks.workunit.client.1.smithi193.stdout:1/934: write da/dd/df/d52/d82/f9f [3795132,8708] 0 2024-03-23T08:23:30.812 INFO:tasks.workunit.client.1.smithi193.stdout:1/935: rename da/d13/d24/dac/dea/d8e/f91 to da/dd/df/d7e/d7f/f132 0 2024-03-23T08:23:30.814 INFO:tasks.workunit.client.1.smithi193.stdout:5/928: truncate d0/d1c/d3b/dbc/d112/fb8 516887 0 2024-03-23T08:23:30.815 INFO:tasks.workunit.client.1.smithi193.stdout:1/936: truncate da/dd/f64 6933839 0 2024-03-23T08:23:30.816 INFO:tasks.workunit.client.1.smithi193.stdout:1/937: write da/f3c [4177260,84615] 0 2024-03-23T08:23:30.818 INFO:tasks.workunit.client.1.smithi193.stdout:5/929: write d0/d1c/d3b/ff4 [762509,44855] 0 2024-03-23T08:23:30.818 INFO:tasks.workunit.client.1.smithi193.stdout:2/941: dwrite d0/d1b/d14/d66/d50/d92/da9/fab [0,4194304] 0 2024-03-23T08:23:30.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/938: fdatasync da/dd/df/d7e/d86/d95/f119 0 2024-03-23T08:23:30.820 INFO:tasks.workunit.client.1.smithi193.stdout:5/930: stat d0/d1c/d3b/d4b/f84 0 2024-03-23T08:23:30.822 INFO:tasks.workunit.client.1.smithi193.stdout:1/939: stat da/dd/df/d75 0 2024-03-23T08:23:30.823 INFO:tasks.workunit.client.1.smithi193.stdout:2/942: write d0/d1b/d19/f1a [473435,36456] 0 2024-03-23T08:23:30.824 INFO:tasks.workunit.client.1.smithi193.stdout:5/931: write d0/d1c/d7a/db1/f38 [3939270,26236] 0 2024-03-23T08:23:30.826 INFO:tasks.workunit.client.1.smithi193.stdout:2/943: truncate d0/d1b/d3b/fe6 509453 0 2024-03-23T08:23:30.829 INFO:tasks.workunit.client.1.smithi193.stdout:1/940: rename da/dd/df/d52/f7a to da/dd/df/d7e/d86/d95/f133 0 2024-03-23T08:23:30.829 INFO:tasks.workunit.client.1.smithi193.stdout:1/941: readlink da/d13/le1 0 2024-03-23T08:23:30.831 INFO:tasks.workunit.client.1.smithi193.stdout:1/942: symlink da/dd/df/ddc/dfd/l134 0 2024-03-23T08:23:30.832 INFO:tasks.workunit.client.1.smithi193.stdout:0/891: dread db/f6f [0,4194304] 0 2024-03-23T08:23:30.835 INFO:tasks.workunit.client.1.smithi193.stdout:1/943: rename da/d13/d24/dac/dea/f8c to da/dd/da9/f135 0 2024-03-23T08:23:30.835 INFO:tasks.workunit.client.1.smithi193.stdout:0/892: chown db/d11/d14/d24/d2a/d4d/d93/fc5 203 1 2024-03-23T08:23:30.836 INFO:tasks.workunit.client.1.smithi193.stdout:0/893: chown db/f133 233406641 1 2024-03-23T08:23:30.837 INFO:tasks.workunit.client.1.smithi193.stdout:1/944: creat da/dd/df/d7c/f136 x:0 0 0 2024-03-23T08:23:30.837 INFO:tasks.workunit.client.1.smithi193.stdout:0/894: stat db/d11/d14/d21/d33/c64 0 2024-03-23T08:23:30.838 INFO:tasks.workunit.client.1.smithi193.stdout:1/945: dread - da/dd/df/d2e/dd5/f122 zero size 2024-03-23T08:23:30.839 INFO:tasks.workunit.client.1.smithi193.stdout:0/895: stat db/d11/d14/d21/l3d 0 2024-03-23T08:23:30.840 INFO:tasks.workunit.client.1.smithi193.stdout:1/946: mkdir da/dd/df/d7e/d86/d137 0 2024-03-23T08:23:30.841 INFO:tasks.workunit.client.1.smithi193.stdout:0/896: rename db/d11/d14/d21/d33/f43 to db/d11/d14/d24/d2a/de3/ddb/f139 0 2024-03-23T08:23:30.842 INFO:tasks.workunit.client.1.smithi193.stdout:1/947: fdatasync da/dd/df/d2e/f93 0 2024-03-23T08:23:30.844 INFO:tasks.workunit.client.1.smithi193.stdout:0/897: creat db/d11/d14/d21/d33/d48/f13a x:0 0 0 2024-03-23T08:23:30.846 INFO:tasks.workunit.client.1.smithi193.stdout:1/948: creat da/d13/d3f/d12e/f138 x:0 0 0 2024-03-23T08:23:30.847 INFO:tasks.workunit.client.1.smithi193.stdout:0/898: creat db/d11/d14/d24/ddd/d107/f13b x:0 0 0 2024-03-23T08:23:30.876 INFO:tasks.workunit.client.1.smithi193.stdout:4/918: sync 2024-03-23T08:23:30.899 INFO:tasks.workunit.client.1.smithi193.stdout:1/949: dwrite f1 [0,4194304] 0 2024-03-23T08:23:30.899 INFO:tasks.workunit.client.1.smithi193.stdout:0/899: mkdir db/d11/d14/d24/d47/d59/dd4/d13c 0 2024-03-23T08:23:30.901 INFO:tasks.workunit.client.1.smithi193.stdout:0/900: fsync db/d11/d14/d24/d2a/de3/fe0 0 2024-03-23T08:23:30.903 INFO:tasks.workunit.client.1.smithi193.stdout:0/901: write db/d11/d14/d21/d33/d48/f54 [6211462,123591] 0 2024-03-23T08:23:30.911 INFO:tasks.workunit.client.1.smithi193.stdout:1/950: unlink da/dd/df/d2e/f4f 0 2024-03-23T08:23:30.912 INFO:tasks.workunit.client.1.smithi193.stdout:4/919: creat d11/d1a/d33/d42/db2/dbe/d50/db3/f131 x:0 0 0 2024-03-23T08:23:30.913 INFO:tasks.workunit.client.1.smithi193.stdout:4/920: readlink d11/d1a/d33/d42/db2/dbe/l55 0 2024-03-23T08:23:30.913 INFO:tasks.workunit.client.1.smithi193.stdout:0/902: mknod db/d11/d14/d21/d33/d69/d9b/c13d 0 2024-03-23T08:23:30.914 INFO:tasks.workunit.client.1.smithi193.stdout:1/951: mknod da/dd/df/d52/d82/db0/c139 0 2024-03-23T08:23:30.915 INFO:tasks.workunit.client.1.smithi193.stdout:4/921: creat d11/d1a/d33/d42/db2/f132 x:0 0 0 2024-03-23T08:23:30.917 INFO:tasks.workunit.client.1.smithi193.stdout:0/903: creat db/d11/d14/d24/d47/d59/dd4/d13c/f13e x:0 0 0 2024-03-23T08:23:30.917 INFO:tasks.workunit.client.1.smithi193.stdout:1/952: creat da/dd/df/d52/f13a x:0 0 0 2024-03-23T08:23:30.918 INFO:tasks.workunit.client.1.smithi193.stdout:0/904: write db/d52/db8/dcf/f109 [649669,114109] 0 2024-03-23T08:23:30.918 INFO:tasks.workunit.client.1.smithi193.stdout:4/922: mknod d11/d2d/da8/d109/d10e/c133 0 2024-03-23T08:23:30.924 INFO:tasks.workunit.client.1.smithi193.stdout:1/953: dwrite da/dd/f39 [0,4194304] 0 2024-03-23T08:23:30.926 INFO:tasks.workunit.client.1.smithi193.stdout:4/923: link d11/d2d/da8/d109/d10e/db8/f31 d11/d1a/d33/d42/db2/dbe/d50/f134 0 2024-03-23T08:23:30.927 INFO:tasks.workunit.client.1.smithi193.stdout:1/954: mknod da/d13/d24/dac/dea/d76/d77/de7/c13b 0 2024-03-23T08:23:30.929 INFO:tasks.workunit.client.1.smithi193.stdout:4/924: creat d11/d2d/d58/d80/f135 x:0 0 0 2024-03-23T08:23:30.931 INFO:tasks.workunit.client.1.smithi193.stdout:4/925: write d11/d1a/d33/d7f/de0/fa0 [536336,14496] 0 2024-03-23T08:23:30.934 INFO:tasks.workunit.client.1.smithi193.stdout:4/926: mkdir d11/d27/d85/d136 0 2024-03-23T08:23:30.936 INFO:tasks.workunit.client.1.smithi193.stdout:4/927: creat d11/d2d/d3a/f137 x:0 0 0 2024-03-23T08:23:30.936 INFO:tasks.workunit.client.1.smithi193.stdout:1/955: dread da/d13/d3f/f87 [0,4194304] 0 2024-03-23T08:23:30.938 INFO:tasks.workunit.client.1.smithi193.stdout:4/928: creat d11/d8d/f138 x:0 0 0 2024-03-23T08:23:30.938 INFO:tasks.workunit.client.1.smithi193.stdout:1/956: mkdir da/dd/df/d7e/d7f/df6/d13c 0 2024-03-23T08:23:30.940 INFO:tasks.workunit.client.1.smithi193.stdout:4/929: unlink d11/d8e/dac/f10d 0 2024-03-23T08:23:30.943 INFO:tasks.workunit.client.1.smithi193.stdout:0/905: truncate db/d11/d14/fad 351895 0 2024-03-23T08:23:30.946 INFO:tasks.workunit.client.1.smithi193.stdout:0/906: chown db/d11/d14/d3b/c117 2169 1 2024-03-23T08:23:30.947 INFO:tasks.workunit.client.1.smithi193.stdout:4/930: dwrite d11/d1a/d33/d42/d52/d70/f11c [0,4194304] 0 2024-03-23T08:23:30.948 INFO:tasks.workunit.client.1.smithi193.stdout:4/931: write fb [1430860,104140] 0 2024-03-23T08:23:30.949 INFO:tasks.workunit.client.1.smithi193.stdout:4/932: chown d11/d1a/d33/d42/d52/ld2 3493657 1 2024-03-23T08:23:30.951 INFO:tasks.workunit.client.1.smithi193.stdout:1/957: truncate da/d13/d24/f7b 3601275 0 2024-03-23T08:23:30.954 INFO:tasks.workunit.client.1.smithi193.stdout:0/907: rmdir db/d11/d14 39 2024-03-23T08:23:30.958 INFO:tasks.workunit.client.1.smithi193.stdout:1/958: dread da/d13/d24/dac/fed [0,4194304] 0 2024-03-23T08:23:30.958 INFO:tasks.workunit.client.1.smithi193.stdout:9/975: sync 2024-03-23T08:23:30.958 INFO:tasks.workunit.client.1.smithi193.stdout:6/767: sync 2024-03-23T08:23:30.958 INFO:tasks.workunit.client.1.smithi193.stdout:7/879: sync 2024-03-23T08:23:30.958 INFO:tasks.workunit.client.1.smithi193.stdout:4/933: dwrite d11/d1a/d33/d42/db2/f132 [0,4194304] 0 2024-03-23T08:23:30.959 INFO:tasks.workunit.client.1.smithi193.stdout:0/908: symlink db/dc6/l13f 0 2024-03-23T08:23:30.963 INFO:tasks.workunit.client.1.smithi193.stdout:7/880: readlink d6/d9/d1f/d3e/d65/lf6 0 2024-03-23T08:23:30.964 INFO:tasks.workunit.client.1.smithi193.stdout:9/976: read d2/d7/db/d17/d33/f63 [2079683,62973] 0 2024-03-23T08:23:30.965 INFO:tasks.workunit.client.1.smithi193.stdout:1/959: symlink da/dd/df/d7e/d86/d95/l13d 0 2024-03-23T08:23:30.966 INFO:tasks.workunit.client.1.smithi193.stdout:9/977: chown d2/d7/db/d17/d33/d70/de1 4492343 1 2024-03-23T08:23:30.966 INFO:tasks.workunit.client.1.smithi193.stdout:1/960: stat da/dd/df/d52/d82/db0/lc2 0 2024-03-23T08:23:30.967 INFO:tasks.workunit.client.1.smithi193.stdout:0/909: write db/d11/d14/d21/dd2/f125 [252837,121380] 0 2024-03-23T08:23:30.968 INFO:tasks.workunit.client.1.smithi193.stdout:6/768: creat d0/d27/d81/df/d11/d36/d61/d12/ff2 x:0 0 0 2024-03-23T08:23:30.969 INFO:tasks.workunit.client.1.smithi193.stdout:7/881: mkdir d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/d112/d132 0 2024-03-23T08:23:30.972 INFO:tasks.workunit.client.1.smithi193.stdout:6/769: truncate d0/d27/d81/da1/da9/fe9 406422 0 2024-03-23T08:23:30.974 INFO:tasks.workunit.client.1.smithi193.stdout:7/882: symlink d6/d9/d1f/d23/d33/d38/d66/d113/l133 0 2024-03-23T08:23:30.975 INFO:tasks.workunit.client.1.smithi193.stdout:6/770: chown d0/d27/d81/df/d11/d36/f6e 427 1 2024-03-23T08:23:30.976 INFO:tasks.workunit.client.1.smithi193.stdout:9/978: dwrite d2/d7/db/d1b/fef [0,4194304] 0 2024-03-23T08:23:30.977 INFO:tasks.workunit.client.1.smithi193.stdout:7/883: write d6/d9/d1f/d23/d33/d38/d66/fa3 [849971,62014] 0 2024-03-23T08:23:30.981 INFO:tasks.workunit.client.1.smithi193.stdout:4/934: write d11/d1a/d33/d42/db2/dbe/d50/f134 [72643,17190] 0 2024-03-23T08:23:30.982 INFO:tasks.workunit.client.1.smithi193.stdout:6/771: getdents d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/db1/de4 0 2024-03-23T08:23:30.984 INFO:tasks.workunit.client.1.smithi193.stdout:0/910: write db/d11/d14/d51/f99 [306159,16788] 0 2024-03-23T08:23:30.985 INFO:tasks.workunit.client.1.smithi193.stdout:6/772: fsync d0/d27/d81/df/d11/d36/d61/d12/ff2 0 2024-03-23T08:23:30.988 INFO:tasks.workunit.client.1.smithi193.stdout:7/884: dwrite d6/d9/d1f/daf/fe1 [0,4194304] 0 2024-03-23T08:23:30.988 INFO:tasks.workunit.client.1.smithi193.stdout:0/911: mknod db/d11/d14/d24/d2a/d4b/d105/c140 0 2024-03-23T08:23:30.992 INFO:tasks.workunit.client.1.smithi193.stdout:6/773: mkdir d0/d27/d81/df/d11/d36/d61/d3a/df3 0 2024-03-23T08:23:30.993 INFO:tasks.workunit.client.1.smithi193.stdout:7/885: chown d6/d9/d1f/d23/d33/d3c/d41/d47/lf4 11743675 1 2024-03-23T08:23:30.997 INFO:tasks.workunit.client.1.smithi193.stdout:7/886: mknod d6/d9/d4e/d81/dca/c134 0 2024-03-23T08:23:30.997 INFO:tasks.workunit.client.1.smithi193.stdout:4/935: dwrite d11/d2d/da8/d109/d10e/f11b [0,4194304] 0 2024-03-23T08:23:30.998 INFO:tasks.workunit.client.1.smithi193.stdout:0/912: chown db/d11/d14/d3b/c117 737095 1 2024-03-23T08:23:31.000 INFO:tasks.workunit.client.1.smithi193.stdout:7/887: chown d6/d9/d1f/d23/d33/d38/d3b 0 1 2024-03-23T08:23:31.000 INFO:tasks.workunit.client.1.smithi193.stdout:4/936: write d11/d2d/d3a/f76 [6830584,50314] 0 2024-03-23T08:23:31.003 INFO:tasks.workunit.client.1.smithi193.stdout:7/888: chown d6/d9/d1f/d5f/f8a 3 1 2024-03-23T08:23:31.006 INFO:tasks.workunit.client.1.smithi193.stdout:6/774: dwrite d0/d27/d81/df/d11/d36/fa3 [0,4194304] 0 2024-03-23T08:23:31.007 INFO:tasks.workunit.client.1.smithi193.stdout:7/889: chown d6/d9/d1f/d23/d33/d38/d66/d113/l133 39829 1 2024-03-23T08:23:31.010 INFO:tasks.workunit.client.1.smithi193.stdout:7/890: stat d6/d9/d1f/daf/fef 0 2024-03-23T08:23:31.012 INFO:tasks.workunit.client.1.smithi193.stdout:0/913: dwrite db/d11/d14/d21/dd2/f125 [0,4194304] 0 2024-03-23T08:23:31.013 INFO:tasks.workunit.client.1.smithi193.stdout:7/891: dread - d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/d112/f11f zero size 2024-03-23T08:23:31.015 INFO:tasks.workunit.client.1.smithi193.stdout:0/914: dread - db/d11/d14/d24/d2a/de3/fe0 zero size 2024-03-23T08:23:31.017 INFO:tasks.workunit.client.1.smithi193.stdout:7/892: truncate d6/d9/d1f/d3e/fab 115238 0 2024-03-23T08:23:31.020 INFO:tasks.workunit.client.1.smithi193.stdout:0/915: write db/d11/d14/d21/dd2/fe1 [2436730,7907] 0 2024-03-23T08:23:31.022 INFO:tasks.workunit.client.1.smithi193.stdout:0/916: mknod db/c141 0 2024-03-23T08:23:31.026 INFO:tasks.workunit.client.1.smithi193.stdout:0/917: rename db/d11/d14/d24/d2a/d4d/l112 to db/d11/d14/d21/d33/l142 0 2024-03-23T08:23:31.028 INFO:tasks.workunit.client.1.smithi193.stdout:0/918: mkdir db/d11/d14/d24/d47/d143 0 2024-03-23T08:23:31.029 INFO:tasks.workunit.client.1.smithi193.stdout:0/919: mkdir db/d11/d14/d24/d2a/de3/d129/d144 0 2024-03-23T08:23:31.030 INFO:tasks.workunit.client.1.smithi193.stdout:4/937: write d11/d2d/f4b [2465292,65598] 0 2024-03-23T08:23:31.032 INFO:tasks.workunit.client.1.smithi193.stdout:4/938: fsync d11/d1a/d33/dce/fd7 0 2024-03-23T08:23:31.032 INFO:tasks.workunit.client.1.smithi193.stdout:0/920: unlink db/d11/d14/c73 0 2024-03-23T08:23:31.034 INFO:tasks.workunit.client.1.smithi193.stdout:7/893: write d6/d9/d1f/f2a [3125072,90864] 0 2024-03-23T08:23:31.036 INFO:tasks.workunit.client.1.smithi193.stdout:0/921: write db/d11/d14/d51/f119 [1478614,6007] 0 2024-03-23T08:23:31.036 INFO:tasks.workunit.client.1.smithi193.stdout:7/894: chown d6/d9/d4e/d81 1195 1 2024-03-23T08:23:31.041 INFO:tasks.workunit.client.1.smithi193.stdout:6/775: dwrite d0/d27/d81/d87/fa8 [0,4194304] 0 2024-03-23T08:23:31.041 INFO:tasks.workunit.client.1.smithi193.stdout:0/922: write db/d11/d14/d24/f25 [4221801,12848] 0 2024-03-23T08:23:31.041 INFO:tasks.workunit.client.1.smithi193.stdout:4/939: truncate d11/d1a/f72 762901 0 2024-03-23T08:23:31.045 INFO:tasks.workunit.client.1.smithi193.stdout:6/776: unlink d0/l4 0 2024-03-23T08:23:31.045 INFO:tasks.workunit.client.1.smithi193.stdout:7/895: mkdir d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/df3/da1/de0/d135 0 2024-03-23T08:23:31.048 INFO:tasks.workunit.client.1.smithi193.stdout:0/923: unlink db/d11/d14/d21/d33/d69/d9b/dc1/f124 0 2024-03-23T08:23:31.049 INFO:tasks.workunit.client.1.smithi193.stdout:5/932: sync 2024-03-23T08:23:31.049 INFO:tasks.workunit.client.1.smithi193.stdout:8/923: sync 2024-03-23T08:23:31.049 INFO:tasks.workunit.client.1.smithi193.stdout:2/944: sync 2024-03-23T08:23:31.049 INFO:tasks.workunit.client.1.smithi193.stdout:3/770: sync 2024-03-23T08:23:31.050 INFO:tasks.workunit.client.1.smithi193.stdout:8/924: stat df/f33 0 2024-03-23T08:23:31.050 INFO:tasks.workunit.client.1.smithi193.stdout:6/777: truncate d0/d27/d81/df/d11/d3b/d6c/fbe 731049 0 2024-03-23T08:23:31.053 INFO:tasks.workunit.client.1.smithi193.stdout:0/924: unlink db/d11/d14/d3b/d71/ca5 0 2024-03-23T08:23:31.053 INFO:tasks.workunit.client.1.smithi193.stdout:2/945: dread - d0/d1b/d14/d56/d7c/fff zero size 2024-03-23T08:23:31.054 INFO:tasks.workunit.client.1.smithi193.stdout:4/940: dread d11/d1a/d33/d42/db2/f132 [0,4194304] 0 2024-03-23T08:23:31.054 INFO:tasks.workunit.client.1.smithi193.stdout:3/771: unlink d1/de/d13/d21/d27/d7c/f87 0 2024-03-23T08:23:31.055 INFO:tasks.workunit.client.1.smithi193.stdout:7/896: truncate d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/df3/fa0 4399991 0 2024-03-23T08:23:31.056 INFO:tasks.workunit.client.1.smithi193.stdout:8/925: creat df/d10/f132 x:0 0 0 2024-03-23T08:23:31.058 INFO:tasks.workunit.client.1.smithi193.stdout:0/925: chown db/d11/d14/d24/f5f 2210 1 2024-03-23T08:23:31.058 INFO:tasks.workunit.client.1.smithi193.stdout:4/941: unlink d11/d95/d107/f11d 0 2024-03-23T08:23:31.060 INFO:tasks.workunit.client.1.smithi193.stdout:3/772: dread - d1/de/d13/d21/d27/d7c/fdb zero size 2024-03-23T08:23:31.060 INFO:tasks.workunit.client.1.smithi193.stdout:5/933: link d0/d1c/d7a/db1/d72/d73/d45/f8a d0/d5a/de9/f13a 0 2024-03-23T08:23:31.061 INFO:tasks.workunit.client.1.smithi193.stdout:0/926: dread - db/d11/d14/d51/ff5 zero size 2024-03-23T08:23:31.062 INFO:tasks.workunit.client.1.smithi193.stdout:2/946: creat d0/d1b/d14/d56/da3/db0/f104 x:0 0 0 2024-03-23T08:23:31.062 INFO:tasks.workunit.client.1.smithi193.stdout:8/926: mknod df/d10/c133 0 2024-03-23T08:23:31.064 INFO:tasks.workunit.client.1.smithi193.stdout:3/773: symlink d1/de/d13/l108 0 2024-03-23T08:23:31.064 INFO:tasks.workunit.client.1.smithi193.stdout:6/778: dwrite d0/d27/d81/df/d11/f30 [0,4194304] 0 2024-03-23T08:23:31.066 INFO:tasks.workunit.client.1.smithi193.stdout:2/947: write d0/d28/f8c [233867,74505] 0 2024-03-23T08:23:31.068 INFO:tasks.workunit.client.1.smithi193.stdout:0/927: chown db/d11/d14/d24/d47/dc7/c104 1104925 1 2024-03-23T08:23:31.069 INFO:tasks.workunit.client.1.smithi193.stdout:7/897: rename d6/d9/d1f/d4d/d6b/lc3 to d6/d9/d1f/d4d/d7c/d8e/l136 0 2024-03-23T08:23:31.069 INFO:tasks.workunit.client.1.smithi193.stdout:4/942: link d11/d2d/f68 d11/d27/d85/de1/de6/f139 0 2024-03-23T08:23:31.074 INFO:tasks.workunit.client.1.smithi193.stdout:0/928: fdatasync db/d11/d14/d24/d2a/de3/f10b 0 2024-03-23T08:23:31.076 INFO:tasks.workunit.client.1.smithi193.stdout:5/934: dread d0/d1c/d7a/db1/d72/d73/d45/fa0 [0,4194304] 0 2024-03-23T08:23:31.077 INFO:tasks.workunit.client.1.smithi193.stdout:4/943: mknod d11/d2d/d58/d8b/c13a 0 2024-03-23T08:23:31.078 INFO:tasks.workunit.client.1.smithi193.stdout:0/929: chown db/d11/d14/d3b/d71 4 1 2024-03-23T08:23:31.081 INFO:tasks.workunit.client.1.smithi193.stdout:6/779: link d0/d27/d81/fc5 d0/d27/d81/da1/da9/ff4 0 2024-03-23T08:23:31.083 INFO:tasks.workunit.client.1.smithi193.stdout:5/935: readlink d0/d1f/d93/de1/le4 0 2024-03-23T08:23:31.085 INFO:tasks.workunit.client.1.smithi193.stdout:0/930: symlink db/d11/d14/d24/d47/d59/dd4/d13c/l145 0 2024-03-23T08:23:31.086 INFO:tasks.workunit.client.1.smithi193.stdout:5/936: truncate d0/d1c/d3b/dbc/d112/d8e/fa5 1182255 0 2024-03-23T08:23:31.088 INFO:tasks.workunit.client.1.smithi193.stdout:6/780: write d0/d27/d81/da1/da9/ff4 [1765527,119107] 0 2024-03-23T08:23:31.090 INFO:tasks.workunit.client.1.smithi193.stdout:5/937: symlink d0/d1c/d7a/db1/d72/l13b 0 2024-03-23T08:23:31.090 INFO:tasks.workunit.client.1.smithi193.stdout:1/961: sync 2024-03-23T08:23:31.092 INFO:tasks.workunit.client.1.smithi193.stdout:2/948: dread d0/d1b/d14/d56/da3/fb4 [0,4194304] 0 2024-03-23T08:23:31.093 INFO:tasks.workunit.client.1.smithi193.stdout:1/962: chown da/dd/faf 6219 1 2024-03-23T08:23:31.094 INFO:tasks.workunit.client.1.smithi193.stdout:3/774: getdents d1/de/d13/d21/d27 0 2024-03-23T08:23:31.095 INFO:tasks.workunit.client.1.smithi193.stdout:8/927: dread df/d10/d3e/d54/d10d/f5b [0,4194304] 0 2024-03-23T08:23:31.095 INFO:tasks.workunit.client.1.smithi193.stdout:5/938: mkdir d0/d5a/dbf/d106/d13c 0 2024-03-23T08:23:31.123 INFO:tasks.workunit.client.1.smithi193.stdout:9/979: sync 2024-03-23T08:23:31.147 INFO:tasks.workunit.client.1.smithi193.stdout:6/781: link d0/d27/d81/da1/da9/fe9 d0/d27/d81/df/d11/ff5 0 2024-03-23T08:23:31.148 INFO:tasks.workunit.client.1.smithi193.stdout:8/928: dread df/d4b/d4d/d9c/fa6 [0,4194304] 0 2024-03-23T08:23:31.149 INFO:tasks.workunit.client.1.smithi193.stdout:4/944: dread d11/d1a/d33/d44/f54 [0,4194304] 0 2024-03-23T08:23:31.150 INFO:tasks.workunit.client.1.smithi193.stdout:1/963: link da/dd/df/d52/d82/db0/c139 da/dd/df/d7e/d7f/df6/c13e 0 2024-03-23T08:23:31.152 INFO:tasks.workunit.client.1.smithi193.stdout:7/898: dread d6/d9/d1f/d4d/d6b/fd5 [0,4194304] 0 2024-03-23T08:23:31.152 INFO:tasks.workunit.client.1.smithi193.stdout:9/980: write d2/d7/db/d1b/d43/d71/db4/dec/f10e [1528801,58725] 0 2024-03-23T08:23:31.153 INFO:tasks.workunit.client.1.smithi193.stdout:5/939: mkdir d0/d1c/d7a/db1/d72/d99/d13d 0 2024-03-23T08:23:31.155 INFO:tasks.workunit.client.1.smithi193.stdout:4/945: readlink d11/d2d/d58/d80/d9d/l126 0 2024-03-23T08:23:31.156 INFO:tasks.workunit.client.1.smithi193.stdout:5/940: fsync d0/d1f/fc4 0 2024-03-23T08:23:31.159 INFO:tasks.workunit.client.1.smithi193.stdout:6/782: creat d0/d27/d81/df/d11/d3b/d51/dd6/ff6 x:0 0 0 2024-03-23T08:23:31.160 INFO:tasks.workunit.client.1.smithi193.stdout:7/899: fdatasync d6/d9/d1f/d23/d33/d3c/d41/f51 0 2024-03-23T08:23:31.161 INFO:tasks.workunit.client.1.smithi193.stdout:1/964: rename da/d13/le1 to da/dd/df/d7e/d7f/dde/l13f 0 2024-03-23T08:23:31.163 INFO:tasks.workunit.client.1.smithi193.stdout:0/931: dread db/d11/d14/d24/f7d [0,4194304] 0 2024-03-23T08:23:31.163 INFO:tasks.workunit.client.1.smithi193.stdout:5/941: mknod d0/d1c/d7a/c13e 0 2024-03-23T08:23:31.164 INFO:tasks.workunit.client.1.smithi193.stdout:8/929: dwrite df/f18 [0,4194304] 0 2024-03-23T08:23:31.166 INFO:tasks.workunit.client.1.smithi193.stdout:3/775: dread d1/de/d13/d21/d25/d2b/f4d [0,4194304] 0 2024-03-23T08:23:31.168 INFO:tasks.workunit.client.1.smithi193.stdout:1/965: write da/dd/df/d52/d82/fc7 [1105309,112047] 0 2024-03-23T08:23:31.173 INFO:tasks.workunit.client.1.smithi193.stdout:9/981: link d2/d7/db/d17/cbd d2/d7/db/d17/d4b/d8d/c144 0 2024-03-23T08:23:31.173 INFO:tasks.workunit.client.1.smithi193.stdout:4/946: dwrite d11/d8e/dac/ffa [0,4194304] 0 2024-03-23T08:23:31.174 INFO:tasks.workunit.client.1.smithi193.stdout:5/942: fdatasync d0/d1c/d7a/db1/d72/f11d 0 2024-03-23T08:23:31.177 INFO:tasks.workunit.client.1.smithi193.stdout:7/900: creat d6/d9/d1f/d23/d33/d38/d66/d113/f137 x:0 0 0 2024-03-23T08:23:31.178 INFO:tasks.workunit.client.1.smithi193.stdout:4/947: stat d11/d1a/d33/d42/db2/dbe/d50/dfb 0 2024-03-23T08:23:31.182 INFO:tasks.workunit.client.1.smithi193.stdout:3/776: fdatasync d1/de/d13/d21/d27/f1c 0 2024-03-23T08:23:31.184 INFO:tasks.workunit.client.1.smithi193.stdout:8/930: mkdir df/d4b/d4d/d2a/d10b/d134 0 2024-03-23T08:23:31.184 INFO:tasks.workunit.client.1.smithi193.stdout:1/966: mknod da/d13/d3f/d12b/c140 0 2024-03-23T08:23:31.185 INFO:tasks.workunit.client.1.smithi193.stdout:4/948: write d11/d2d/d58/f86 [3477140,81760] 0 2024-03-23T08:23:31.186 INFO:tasks.workunit.client.1.smithi193.stdout:5/943: rename d0/d1c/d3b/dbc/d112/d8e/fa5 to d0/d1c/d3b/d4b/df9/f13f 0 2024-03-23T08:23:31.186 INFO:tasks.workunit.client.1.smithi193.stdout:0/932: dwrite db/d11/d14/d24/d2a/fbe [0,4194304] 0 2024-03-23T08:23:31.189 INFO:tasks.workunit.client.1.smithi193.stdout:7/901: rmdir d6/d9/d1f/d23/d33/d38/d3b/db7 39 2024-03-23T08:23:31.194 INFO:tasks.workunit.client.1.smithi193.stdout:3/777: dread - d1/de/d13/d21/d59/fde zero size 2024-03-23T08:23:31.195 INFO:tasks.workunit.client.1.smithi193.stdout:8/931: creat df/d4b/d4d/d2a/d10b/f135 x:0 0 0 2024-03-23T08:23:31.196 INFO:tasks.workunit.client.1.smithi193.stdout:5/944: creat d0/d5a/de9/f140 x:0 0 0 2024-03-23T08:23:31.197 INFO:tasks.workunit.client.1.smithi193.stdout:9/982: dwrite d2/d7/db/d1b/d5c/ff5 [0,4194304] 0 2024-03-23T08:23:31.197 INFO:tasks.workunit.client.1.smithi193.stdout:4/949: symlink d11/d2d/d58/d80/db9/l13b 0 2024-03-23T08:23:31.198 INFO:tasks.workunit.client.1.smithi193.stdout:6/783: dwrite d0/d27/d81/df/d11/d3b/f76 [0,4194304] 0 2024-03-23T08:23:31.202 INFO:tasks.workunit.client.1.smithi193.stdout:1/967: rename da/dd/df/d7e/d7f to da/dd/df/d7c/d141 0 2024-03-23T08:23:31.203 INFO:tasks.workunit.client.1.smithi193.stdout:7/902: readlink d6/d9/d1f/d23/d33/d38/d46/d82/lb3 0 2024-03-23T08:23:31.204 INFO:tasks.workunit.client.1.smithi193.stdout:3/778: write d1/de/d13/d21/ff4 [974379,54646] 0 2024-03-23T08:23:31.209 INFO:tasks.workunit.client.1.smithi193.stdout:5/945: creat d0/d1f/d93/de1/f141 x:0 0 0 2024-03-23T08:23:31.209 INFO:tasks.workunit.client.1.smithi193.stdout:0/933: rename db/d11/d14/d24/d2a/c4e to db/d11/d14/d24/d2a/d4b/c146 0 2024-03-23T08:23:31.210 INFO:tasks.workunit.client.1.smithi193.stdout:4/950: mknod d11/d1a/d33/d42/c13c 0 2024-03-23T08:23:31.210 INFO:tasks.workunit.client.1.smithi193.stdout:7/903: stat d6/d9/d1f/d23/d33/d3c/d41/d47/fb2 0 2024-03-23T08:23:31.210 INFO:tasks.workunit.client.1.smithi193.stdout:1/968: mknod da/dd/df/ddc/c142 0 2024-03-23T08:23:31.210 INFO:tasks.workunit.client.1.smithi193.stdout:8/932: write df/f48 [550242,87517] 0 2024-03-23T08:23:31.211 INFO:tasks.workunit.client.1.smithi193.stdout:6/784: mknod d0/d27/d81/da1/da9/cf7 0 2024-03-23T08:23:31.212 INFO:tasks.workunit.client.1.smithi193.stdout:4/951: creat d11/d1a/d33/d7f/de0/f13d x:0 0 0 2024-03-23T08:23:31.213 INFO:tasks.workunit.client.1.smithi193.stdout:7/904: unlink d6/d9/d1f/d23/ld7 0 2024-03-23T08:23:31.215 INFO:tasks.workunit.client.1.smithi193.stdout:6/785: write d0/d27/d81/df/d11/d36/d61/d3a/fb4 [2239977,13083] 0 2024-03-23T08:23:31.216 INFO:tasks.workunit.client.1.smithi193.stdout:8/933: symlink df/d10/d3e/d54/d10d/l136 0 2024-03-23T08:23:31.216 INFO:tasks.workunit.client.1.smithi193.stdout:4/952: mkdir d11/d27/d85/de1/d13e 0 2024-03-23T08:23:31.217 INFO:tasks.workunit.client.1.smithi193.stdout:7/905: write d6/d9/d1f/d23/d33/d3c/f4b [2615681,114348] 0 2024-03-23T08:23:31.218 INFO:tasks.workunit.client.1.smithi193.stdout:3/779: getdents d1/de/d13/d21/d59/dc6 0 2024-03-23T08:23:31.220 INFO:tasks.workunit.client.1.smithi193.stdout:1/969: rmdir da/dd/df/d7c/d141/df6/d13c 0 2024-03-23T08:23:31.222 INFO:tasks.workunit.client.1.smithi193.stdout:6/786: write d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f7e [1915706,80802] 0 2024-03-23T08:23:31.223 INFO:tasks.workunit.client.1.smithi193.stdout:7/906: fsync d6/d9/d1f/d5f/ffb 0 2024-03-23T08:23:31.224 INFO:tasks.workunit.client.1.smithi193.stdout:4/953: creat d11/d12f/f13f x:0 0 0 2024-03-23T08:23:31.226 INFO:tasks.workunit.client.1.smithi193.stdout:8/934: truncate df/d10/f29 3979180 0 2024-03-23T08:23:31.226 INFO:tasks.workunit.client.1.smithi193.stdout:6/787: fdatasync d0/d27/d81/da1/fb5 0 2024-03-23T08:23:31.228 INFO:tasks.workunit.client.1.smithi193.stdout:1/970: truncate da/dd/df/d52/d82/f105 997391 0 2024-03-23T08:23:31.230 INFO:tasks.workunit.client.1.smithi193.stdout:7/907: mknod d6/d9/d1f/daf/c138 0 2024-03-23T08:23:31.231 INFO:tasks.workunit.client.1.smithi193.stdout:9/983: dread d2/d7/db/d1b/d43/d71/db4/dec/d3c/f117 [0,4194304] 0 2024-03-23T08:23:31.231 INFO:tasks.workunit.client.1.smithi193.stdout:4/954: chown d11/d2d/d3a/f5e 6699 1 2024-03-23T08:23:31.232 INFO:tasks.workunit.client.1.smithi193.stdout:3/780: creat d1/de/d13/d21/d25/d2b/d3c/d4a/de4/d105/deb/f109 x:0 0 0 2024-03-23T08:23:31.234 INFO:tasks.workunit.client.1.smithi193.stdout:4/955: write d11/d27/d85/f8a [940271,65961] 0 2024-03-23T08:23:31.234 INFO:tasks.workunit.client.1.smithi193.stdout:9/984: write d2/d7/db/f60 [320439,121657] 0 2024-03-23T08:23:31.236 INFO:tasks.workunit.client.1.smithi193.stdout:0/934: dread db/d11/d14/d21/fd0 [0,4194304] 0 2024-03-23T08:23:31.238 INFO:tasks.workunit.client.1.smithi193.stdout:6/788: unlink d0/d27/d81/df/d11/d3b/d51/d9c/f8a 0 2024-03-23T08:23:31.238 INFO:tasks.workunit.client.1.smithi193.stdout:9/985: write d2/d7/db/d32/f10d [3229663,104068] 0 2024-03-23T08:23:31.244 INFO:tasks.workunit.client.1.smithi193.stdout:3/781: fdatasync d1/de/d13/d21/d50/fc8 0 2024-03-23T08:23:31.245 INFO:tasks.workunit.client.1.smithi193.stdout:7/908: truncate d6/d9/d1f/d23/f45 3293264 0 2024-03-23T08:23:31.248 INFO:tasks.workunit.client.1.smithi193.stdout:4/956: link d11/d2d/d58/d8b/dbc/de9/f11a d11/d12f/f140 0 2024-03-23T08:23:31.249 INFO:tasks.workunit.client.1.smithi193.stdout:8/935: dread df/d4b/d4d/f69 [0,4194304] 0 2024-03-23T08:23:31.249 INFO:tasks.workunit.client.1.smithi193.stdout:6/789: mkdir d0/d27/d81/df/d11/d36/d61/d12/d21/df8 0 2024-03-23T08:23:31.249 INFO:tasks.workunit.client.1.smithi193.stdout:7/909: read - d6/d9/d1f/d23/d33/d38/d3b/d6c/de7/f128 zero size 2024-03-23T08:23:31.252 INFO:tasks.workunit.client.1.smithi193.stdout:3/782: chown d1/de/d13/d21/d59/d60/fb7 2430592 1 2024-03-23T08:23:31.252 INFO:tasks.workunit.client.1.smithi193.stdout:6/790: write d0/d27/d81/da1/db8/fdf [859608,43279] 0 2024-03-23T08:23:31.252 INFO:tasks.workunit.client.1.smithi193.stdout:8/936: chown df/d4b/d4d/d2a/l74 24 1 2024-03-23T08:23:31.254 INFO:tasks.workunit.client.1.smithi193.stdout:9/986: rename d2/d7/db/d17/d4b/d59/d75 to d2/d7/db/d17/d33/d70/de1/d145 0 2024-03-23T08:23:31.254 INFO:tasks.workunit.client.1.smithi193.stdout:4/957: mknod d11/d12f/c141 0 2024-03-23T08:23:31.256 INFO:tasks.workunit.client.1.smithi193.stdout:7/910: creat d6/d9/d4e/d81/f139 x:0 0 0 2024-03-23T08:23:31.257 INFO:tasks.workunit.client.1.smithi193.stdout:8/937: truncate df/d63/d84/dc5/f11b 8825 0 2024-03-23T08:23:31.257 INFO:tasks.workunit.client.1.smithi193.stdout:9/987: read d2/d7/db/d1b/f81 [1029558,52437] 0 2024-03-23T08:23:31.258 INFO:tasks.workunit.client.1.smithi193.stdout:9/988: fdatasync d2/d7/db/d1b/fef 0 2024-03-23T08:23:31.259 INFO:tasks.workunit.client.1.smithi193.stdout:3/783: rmdir d1/de/d13/d21/d59/dc3 39 2024-03-23T08:23:31.262 INFO:tasks.workunit.client.1.smithi193.stdout:2/949: sync 2024-03-23T08:23:31.262 INFO:tasks.workunit.client.1.smithi193.stdout:7/911: rmdir d6/d9/d1f/d23/d33/d38/d66 39 2024-03-23T08:23:31.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/791: creat d0/d27/d81/df/d11/d36/ff9 x:0 0 0 2024-03-23T08:23:31.266 INFO:tasks.workunit.client.1.smithi193.stdout:2/950: chown d0/d1b/d14/d66/d50/d92/da9 3 1 2024-03-23T08:23:31.267 INFO:tasks.workunit.client.1.smithi193.stdout:4/958: getdents d11/d2d/d58/d80 0 2024-03-23T08:23:31.269 INFO:tasks.workunit.client.1.smithi193.stdout:1/971: write da/d13/f7d [772498,96942] 0 2024-03-23T08:23:31.271 INFO:tasks.workunit.client.1.smithi193.stdout:8/938: dwrite df/d63/d84/dc5/f129 [0,4194304] 0 2024-03-23T08:23:31.274 INFO:tasks.workunit.client.1.smithi193.stdout:6/792: read d0/d27/d81/df/d11/d3b/d51/daa/fca [310376,3535] 0 2024-03-23T08:23:31.274 INFO:tasks.workunit.client.1.smithi193.stdout:4/959: write d11/d1a/d33/dce/f102 [245103,79495] 0 2024-03-23T08:23:31.276 INFO:tasks.workunit.client.1.smithi193.stdout:1/972: dread - da/dd/df/d52/f13a zero size 2024-03-23T08:23:31.277 INFO:tasks.workunit.client.1.smithi193.stdout:9/989: dwrite d2/d7/db/d7a/d95/da9/f13e [0,4194304] 0 2024-03-23T08:23:31.277 INFO:tasks.workunit.client.1.smithi193.stdout:0/935: truncate db/d11/d14/d24/f39 3935583 0 2024-03-23T08:23:31.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/939: mkdir df/d63/d76/d137 0 2024-03-23T08:23:31.281 INFO:tasks.workunit.client.1.smithi193.stdout:2/951: symlink d0/d28/l105 0 2024-03-23T08:23:31.283 INFO:tasks.workunit.client.1.smithi193.stdout:7/912: dwrite d6/d9/d1f/d4d/d6b/fd5 [4194304,4194304] 0 2024-03-23T08:23:31.284 INFO:tasks.workunit.client.1.smithi193.stdout:9/990: fsync d2/d7/db/d7a/d95/fa3 0 2024-03-23T08:23:31.287 INFO:tasks.workunit.client.1.smithi193.stdout:0/936: creat db/d11/d14/d24/d2a/de3/f147 x:0 0 0 2024-03-23T08:23:31.288 INFO:tasks.workunit.client.1.smithi193.stdout:2/952: symlink d0/d28/d9f/l106 0 2024-03-23T08:23:31.289 INFO:tasks.workunit.client.1.smithi193.stdout:6/793: mkdir d0/d27/d81/df/d11/d3b/d51/db2/dfa 0 2024-03-23T08:23:31.292 INFO:tasks.workunit.client.1.smithi193.stdout:0/937: read db/d11/d14/d21/d33/f68 [1579987,114498] 0 2024-03-23T08:23:31.292 INFO:tasks.workunit.client.1.smithi193.stdout:4/960: dwrite d11/d2d/d58/d8b/ff7 [0,4194304] 0 2024-03-23T08:23:31.293 INFO:tasks.workunit.client.1.smithi193.stdout:7/913: mknod d6/d9/d1f/de8/c13a 0 2024-03-23T08:23:31.293 INFO:tasks.workunit.client.1.smithi193.stdout:3/784: dread d1/de/f10 [4194304,4194304] 0 2024-03-23T08:23:31.295 INFO:tasks.workunit.client.1.smithi193.stdout:9/991: truncate d2/d7/f6f 2092533 0 2024-03-23T08:23:31.295 INFO:tasks.workunit.client.1.smithi193.stdout:4/961: chown d11/d1a/d33/d42/db2/dbe/d50/f134 1914489 1 2024-03-23T08:23:31.296 INFO:tasks.workunit.client.1.smithi193.stdout:9/992: stat d2/d7/db/d32/f10d 0 2024-03-23T08:23:31.297 INFO:tasks.workunit.client.1.smithi193.stdout:4/962: stat d11/d1a/d33/d42/db2/dbe/d50/lad 0 2024-03-23T08:23:31.298 INFO:tasks.workunit.client.1.smithi193.stdout:9/993: chown d2/d7/f139 774 1 2024-03-23T08:23:31.300 INFO:tasks.workunit.client.1.smithi193.stdout:6/794: write d0/d27/d81/df/d11/d36/d61/f1d [2386198,100446] 0 2024-03-23T08:23:31.300 INFO:tasks.workunit.client.1.smithi193.stdout:1/973: write da/d13/d3f/f87 [1503197,43346] 0 2024-03-23T08:23:31.302 INFO:tasks.workunit.client.1.smithi193.stdout:3/785: symlink d1/de/d28/l10a 0 2024-03-23T08:23:31.303 INFO:tasks.workunit.client.1.smithi193.stdout:6/795: fdatasync d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/fcc 0 2024-03-23T08:23:31.304 INFO:tasks.workunit.client.1.smithi193.stdout:7/914: unlink d6/f32 0 2024-03-23T08:23:31.304 INFO:tasks.workunit.client.1.smithi193.stdout:9/994: fsync d2/d7/db/d1b/f30 0 2024-03-23T08:23:31.306 INFO:tasks.workunit.client.1.smithi193.stdout:9/995: readlink d2/d7/db/d1b/d43/d71/db4/dec/d5f/lbf 0 2024-03-23T08:23:31.307 INFO:tasks.workunit.client.1.smithi193.stdout:2/953: creat d0/d28/f107 x:0 0 0 2024-03-23T08:23:31.308 INFO:tasks.workunit.client.1.smithi193.stdout:1/974: mkdir da/d13/d24/dac/dea/d76/d77/de7/d143 0 2024-03-23T08:23:31.308 INFO:tasks.workunit.client.1.smithi193.stdout:2/954: fsync d0/d1b/d14/d66/d50/f91 0 2024-03-23T08:23:31.310 INFO:tasks.workunit.client.1.smithi193.stdout:3/786: creat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/f10b x:0 0 0 2024-03-23T08:23:31.311 INFO:tasks.workunit.client.1.smithi193.stdout:3/787: write d1/de/d13/d21/d25/d2b/d3c/d4a/f93 [3587901,62801] 0 2024-03-23T08:23:31.315 INFO:tasks.workunit.client.1.smithi193.stdout:1/975: symlink da/dd/df/d7c/d141/db2/ddb/l144 0 2024-03-23T08:23:31.317 INFO:tasks.workunit.client.1.smithi193.stdout:2/955: mkdir d0/d1b/d14/d56/d7c/ddc/d108 0 2024-03-23T08:23:31.317 INFO:tasks.workunit.client.1.smithi193.stdout:1/976: write da/dd/df/d7c/fe5 [2543072,108859] 0 2024-03-23T08:23:31.321 INFO:tasks.workunit.client.1.smithi193.stdout:4/963: truncate d11/d95/f9a 2062100 0 2024-03-23T08:23:31.322 INFO:tasks.workunit.client.1.smithi193.stdout:3/788: chown d1/de/d13/d21/d50/d6f/d6e/cd7 95440 1 2024-03-23T08:23:31.325 INFO:tasks.workunit.client.1.smithi193.stdout:1/977: mkdir da/d13/d3f/d12b/d145 0 2024-03-23T08:23:31.326 INFO:tasks.workunit.client.1.smithi193.stdout:1/978: chown da/l33 1658681 1 2024-03-23T08:23:31.327 INFO:tasks.workunit.client.1.smithi193.stdout:1/979: truncate da/d13/d24/f101 535136 0 2024-03-23T08:23:31.327 INFO:tasks.workunit.client.1.smithi193.stdout:6/796: rmdir d0/d27/d81/df/d11/d3b/d51/daa/deb 0 2024-03-23T08:23:31.328 INFO:tasks.workunit.client.1.smithi193.stdout:6/797: write d0/d27/d81/d87/fa8 [3576793,93521] 0 2024-03-23T08:23:31.329 INFO:tasks.workunit.client.1.smithi193.stdout:7/915: creat d6/d9/d1f/d23/d33/d38/d3b/f13b x:0 0 0 2024-03-23T08:23:31.332 INFO:tasks.workunit.client.1.smithi193.stdout:0/938: getdents db/d11/d14/d21/d33/d69/dde 0 2024-03-23T08:23:31.333 INFO:tasks.workunit.client.1.smithi193.stdout:9/996: getdents d2/d7/db/d7a 0 2024-03-23T08:23:31.334 INFO:tasks.workunit.client.1.smithi193.stdout:9/997: readlink d2/d7/db/d32/lc3 0 2024-03-23T08:23:31.334 INFO:tasks.workunit.client.1.smithi193.stdout:4/964: mkdir d11/d95/d142 0 2024-03-23T08:23:31.336 INFO:tasks.workunit.client.1.smithi193.stdout:3/789: write d1/de/d13/d96/d99/fee [3399894,82154] 0 2024-03-23T08:23:31.336 INFO:tasks.workunit.client.1.smithi193.stdout:2/956: mkdir d0/d1b/d14/da1/dd5/d109 0 2024-03-23T08:23:31.340 INFO:tasks.workunit.client.1.smithi193.stdout:1/980: creat da/dd/df/d2e/db4/f146 x:0 0 0 2024-03-23T08:23:31.341 INFO:tasks.workunit.client.1.smithi193.stdout:6/798: creat d0/d27/d81/da1/ffb x:0 0 0 2024-03-23T08:23:31.343 INFO:tasks.workunit.client.1.smithi193.stdout:4/965: dwrite d11/d1a/d33/d42/db2/dbe/d50/f134 [8388608,4194304] 0 2024-03-23T08:23:31.343 INFO:tasks.workunit.client.1.smithi193.stdout:0/939: creat db/d11/d14/d21/d33/f148 x:0 0 0 2024-03-23T08:23:31.343 INFO:tasks.workunit.client.1.smithi193.stdout:9/998: rmdir d2/d7/db/d12d 39 2024-03-23T08:23:31.345 INFO:tasks.workunit.client.1.smithi193.stdout:3/790: mknod d1/de/d78/c10c 0 2024-03-23T08:23:31.345 INFO:tasks.workunit.client.1.smithi193.stdout:4/966: fsync d11/d1a/d33/d44/f4a 0 2024-03-23T08:23:31.346 INFO:tasks.workunit.client.1.smithi193.stdout:0/940: chown db/d11/d14/d21/c9e 332 1 2024-03-23T08:23:31.347 INFO:tasks.workunit.client.1.smithi193.stdout:2/957: creat d0/d1b/d14/d56/d7c/f10a x:0 0 0 2024-03-23T08:23:31.347 INFO:tasks.workunit.client.1.smithi193.stdout:7/916: mknod d6/d9/d1f/d23/d33/d129/c13c 0 2024-03-23T08:23:31.348 INFO:tasks.workunit.client.1.smithi193.stdout:1/981: mknod da/dd/df/d2e/db4/c147 0 2024-03-23T08:23:31.348 INFO:tasks.workunit.client.1.smithi193.stdout:6/799: mkdir d0/d27/d81/df/d11/d56/dfc 0 2024-03-23T08:23:31.350 INFO:tasks.workunit.client.1.smithi193.stdout:9/999: fdatasync d2/d7/db/d1b/d5c/f94 0 2024-03-23T08:23:31.352 INFO:tasks.workunit.client.1.smithi193.stdout:6/800: dread - d0/d27/d81/df/d11/d3b/d51/dd6/ff6 zero size 2024-03-23T08:23:31.353 INFO:tasks.workunit.client.1.smithi193.stdout:1/982: write da/d13/d3f/d12e/f138 [156767,92493] 0 2024-03-23T08:23:31.353 INFO:tasks.workunit.client.1.smithi193.stdout:0/941: dread db/d11/d14/d21/d33/d69/d9b/fce [0,4194304] 0 2024-03-23T08:23:31.354 INFO:tasks.workunit.client.1.smithi193.stdout:4/967: link d11/d27/c117 d11/d2d/da8/c143 0 2024-03-23T08:23:31.356 INFO:tasks.workunit.client.1.smithi193.stdout:6/801: write d0/d27/d81/da1/fe8 [379747,125186] 0 2024-03-23T08:23:31.357 INFO:tasks.workunit.client.1.smithi193.stdout:7/917: rename d6/d9/d1f/d23/d33/d38/d66/d113 to d6/d9/d1f/d23/d33/d3c/d41/d47/d13d 0 2024-03-23T08:23:31.358 INFO:tasks.workunit.client.1.smithi193.stdout:0/942: write db/d11/d14/d21/d33/d48/d85/f101 [745003,9310] 0 2024-03-23T08:23:31.359 INFO:tasks.workunit.client.1.smithi193.stdout:4/968: dread - d11/d2d/d58/d8b/dbc/d84/fe5 zero size 2024-03-23T08:23:31.363 INFO:tasks.workunit.client.1.smithi193.stdout:0/943: creat db/d11/d14/d3b/d71/f149 x:0 0 0 2024-03-23T08:23:31.366 INFO:tasks.workunit.client.1.smithi193.stdout:7/918: creat d6/d9/d1f/d23/d33/d38/d46/d82/d12b/f13e x:0 0 0 2024-03-23T08:23:31.367 INFO:tasks.workunit.client.1.smithi193.stdout:1/983: dwrite da/dd/df/d52/d82/fc7 [0,4194304] 0 2024-03-23T08:23:31.367 INFO:tasks.workunit.client.1.smithi193.stdout:5/946: sync 2024-03-23T08:23:31.372 INFO:tasks.workunit.client.1.smithi193.stdout:0/944: chown db/d11/d14/d24/d2a/de3/l100 118 1 2024-03-23T08:23:31.376 INFO:tasks.workunit.client.1.smithi193.stdout:7/919: mknod d6/d9/d1f/d4d/d7c/c13f 0 2024-03-23T08:23:31.377 INFO:tasks.workunit.client.1.smithi193.stdout:4/969: write d11/d2d/d58/d80/db9/fdf [4466350,44785] 0 2024-03-23T08:23:31.377 INFO:tasks.workunit.client.1.smithi193.stdout:6/802: dwrite d0/d27/d81/fa6 [4194304,4194304] 0 2024-03-23T08:23:31.378 INFO:tasks.workunit.client.1.smithi193.stdout:3/791: dread d1/de/d13/d21/d25/d34/f47 [0,4194304] 0 2024-03-23T08:23:31.380 INFO:tasks.workunit.client.1.smithi193.stdout:7/920: chown d6/d9/d1f/d4d/d7c/c108 277 1 2024-03-23T08:23:31.380 INFO:tasks.workunit.client.1.smithi193.stdout:4/970: dread d11/d1a/d33/d42/d52/f87 [0,4194304] 0 2024-03-23T08:23:31.382 INFO:tasks.workunit.client.1.smithi193.stdout:0/945: symlink db/d11/d14/d24/d47/l14a 0 2024-03-23T08:23:31.382 INFO:tasks.workunit.client.1.smithi193.stdout:7/921: chown d6/d9/d1f/f94 11 1 2024-03-23T08:23:31.383 INFO:tasks.workunit.client.1.smithi193.stdout:3/792: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d85/f10d x:0 0 0 2024-03-23T08:23:31.383 INFO:tasks.workunit.client.1.smithi193.stdout:1/984: link da/dd/l69 da/dd/df/d7c/d141/l148 0 2024-03-23T08:23:31.385 INFO:tasks.workunit.client.1.smithi193.stdout:0/946: mkdir db/d11/d14/d24/d47/d59/d8c/d14b 0 2024-03-23T08:23:31.386 INFO:tasks.workunit.client.1.smithi193.stdout:7/922: mknod d6/d9/d1f/d4d/d7c/d92/c140 0 2024-03-23T08:23:31.386 INFO:tasks.workunit.client.1.smithi193.stdout:1/985: rmdir da/dd/df/d2e/dd5 39 2024-03-23T08:23:31.387 INFO:tasks.workunit.client.1.smithi193.stdout:4/971: mknod d11/d1a/d33/d42/db2/c144 0 2024-03-23T08:23:31.388 INFO:tasks.workunit.client.1.smithi193.stdout:3/793: dread d1/f2f [0,4194304] 0 2024-03-23T08:23:31.389 INFO:tasks.workunit.client.1.smithi193.stdout:0/947: fdatasync db/d11/d14/d24/d2a/d4d/d93/fcc 0 2024-03-23T08:23:31.389 INFO:tasks.workunit.client.1.smithi193.stdout:1/986: chown da/d13/d24/dac/dea/d76 685648182 1 2024-03-23T08:23:31.392 INFO:tasks.workunit.client.1.smithi193.stdout:7/923: write d6/d9/d1f/d23/d33/d3c/d41/d47/d13d/f137 [959526,93261] 0 2024-03-23T08:23:31.396 INFO:tasks.workunit.client.1.smithi193.stdout:6/803: truncate d0/d27/d81/da1/db8/fdf 530320 0 2024-03-23T08:23:31.396 INFO:tasks.workunit.client.1.smithi193.stdout:5/947: dwrite d0/d1c/d3b/dbc/d112/d8e/fab [0,4194304] 0 2024-03-23T08:23:31.398 INFO:tasks.workunit.client.1.smithi193.stdout:0/948: mkdir db/d11/d14/d24/d47/d14c 0 2024-03-23T08:23:31.399 INFO:tasks.workunit.client.1.smithi193.stdout:3/794: rmdir d1/de/d13/d21/d25/d2b/d3c/d4a/d85/db9/dfc 39 2024-03-23T08:23:31.402 INFO:tasks.workunit.client.1.smithi193.stdout:7/924: dread d6/d9/fa5 [0,4194304] 0 2024-03-23T08:23:31.402 INFO:tasks.workunit.client.1.smithi193.stdout:6/804: readlink d0/d27/d81/df/d11/d36/d61/l22 0 2024-03-23T08:23:31.403 INFO:tasks.workunit.client.1.smithi193.stdout:5/948: symlink d0/d1c/d3b/d81/df2/d97/l142 0 2024-03-23T08:23:31.405 INFO:tasks.workunit.client.1.smithi193.stdout:0/949: read db/d11/d14/d3b/d71/fc2 [969625,19024] 0 2024-03-23T08:23:31.406 INFO:tasks.workunit.client.1.smithi193.stdout:3/795: symlink d1/de/d78/l10e 0 2024-03-23T08:23:31.407 INFO:tasks.workunit.client.1.smithi193.stdout:6/805: chown d0/d27/d81/df/d11/d3b 6082156 1 2024-03-23T08:23:31.407 INFO:tasks.workunit.client.1.smithi193.stdout:4/972: truncate d11/d1a/d33/d42/f59 1163664 0 2024-03-23T08:23:31.408 INFO:tasks.workunit.client.1.smithi193.stdout:1/987: dwrite da/d13/d24/dac/dea/d76/ffe [0,4194304] 0 2024-03-23T08:23:31.409 INFO:tasks.workunit.client.1.smithi193.stdout:5/949: creat d0/d1c/d7a/f143 x:0 0 0 2024-03-23T08:23:31.412 INFO:tasks.workunit.client.1.smithi193.stdout:0/950: write db/d11/d14/f134 [652047,120424] 0 2024-03-23T08:23:31.412 INFO:tasks.workunit.client.1.smithi193.stdout:5/950: fsync d0/f15 0 2024-03-23T08:23:31.414 INFO:tasks.workunit.client.1.smithi193.stdout:3/796: chown d1/de/d28/le3 124513 1 2024-03-23T08:23:31.415 INFO:tasks.workunit.client.1.smithi193.stdout:6/806: fdatasync d0/d27/d81/df/d11/d36/d61/fcb 0 2024-03-23T08:23:31.418 INFO:tasks.workunit.client.1.smithi193.stdout:4/973: truncate d11/d1a/d33/d42/db2/dbe/d50/f5b 9481338 0 2024-03-23T08:23:31.422 INFO:tasks.workunit.client.1.smithi193.stdout:6/807: mkdir d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/dfd 0 2024-03-23T08:23:31.424 INFO:tasks.workunit.client.1.smithi193.stdout:5/951: symlink d0/d1c/d7a/db1/d72/d73/d57/d138/l144 0 2024-03-23T08:23:31.426 INFO:tasks.workunit.client.1.smithi193.stdout:0/951: write db/d11/d14/d24/d47/dc7/f12f [2206088,86144] 0 2024-03-23T08:23:31.430 INFO:tasks.workunit.client.1.smithi193.stdout:6/808: fdatasync d0/d27/d81/df/d11/d3b/f49 0 2024-03-23T08:23:31.431 INFO:tasks.workunit.client.1.smithi193.stdout:1/988: dwrite da/dd/f64 [4194304,4194304] 0 2024-03-23T08:23:31.431 INFO:tasks.workunit.client.1.smithi193.stdout:5/952: chown d0/d1c/d7a/db1/d72/l134 490253 1 2024-03-23T08:23:31.432 INFO:tasks.workunit.client.1.smithi193.stdout:3/797: dwrite d1/de/f11 [0,4194304] 0 2024-03-23T08:23:31.434 INFO:tasks.workunit.client.1.smithi193.stdout:4/974: dwrite d11/d2d/d3a/f76 [0,4194304] 0 2024-03-23T08:23:31.438 INFO:tasks.workunit.client.1.smithi193.stdout:0/952: write db/d11/d14/d21/d33/d48/d85/fa7 [675256,12885] 0 2024-03-23T08:23:31.438 INFO:tasks.workunit.client.1.smithi193.stdout:3/798: write d1/de/d78/fc7 [289096,85690] 0 2024-03-23T08:23:31.440 INFO:tasks.workunit.client.1.smithi193.stdout:1/989: creat da/dd/df/d7e/d86/d95/dc4/f149 x:0 0 0 2024-03-23T08:23:31.441 INFO:tasks.workunit.client.1.smithi193.stdout:4/975: chown d11/d2d/d3a/f137 19 1 2024-03-23T08:23:31.443 INFO:tasks.workunit.client.1.smithi193.stdout:4/976: stat d11/l7b 0 2024-03-23T08:23:31.444 INFO:tasks.workunit.client.1.smithi193.stdout:6/809: mknod d0/d27/d81/df/d11/d56/cfe 0 2024-03-23T08:23:31.445 INFO:tasks.workunit.client.1.smithi193.stdout:0/953: stat db/d11/d14/d21/dd2/leb 0 2024-03-23T08:23:31.452 INFO:tasks.workunit.client.1.smithi193.stdout:6/810: creat d0/d27/d81/df/d11/d3b/d51/dd6/fff x:0 0 0 2024-03-23T08:23:31.454 INFO:tasks.workunit.client.1.smithi193.stdout:5/953: dwrite d0/d1c/d7a/db1/d72/d99/dd8/f108 [0,4194304] 0 2024-03-23T08:23:31.463 INFO:tasks.workunit.client.1.smithi193.stdout:3/799: dwrite d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/fa0 [0,4194304] 0 2024-03-23T08:23:31.464 INFO:tasks.workunit.client.1.smithi193.stdout:5/954: readlink d0/d1c/d7a/db1/d72/d73/d57/le8 0 2024-03-23T08:23:31.465 INFO:tasks.workunit.client.1.smithi193.stdout:1/990: dwrite da/f4e [0,4194304] 0 2024-03-23T08:23:31.468 INFO:tasks.workunit.client.1.smithi193.stdout:3/800: unlink d1/f84 0 2024-03-23T08:23:31.472 INFO:tasks.workunit.client.1.smithi193.stdout:0/954: dwrite db/d11/d14/d24/f97 [0,4194304] 0 2024-03-23T08:23:31.476 INFO:tasks.workunit.client.1.smithi193.stdout:4/977: dread d11/d2d/d58/d8b/dbc/f2f [0,4194304] 0 2024-03-23T08:23:31.477 INFO:tasks.workunit.client.1.smithi193.stdout:3/801: readlink d1/de/d13/d21/d50/d44/lfd 0 2024-03-23T08:23:31.478 INFO:tasks.workunit.client.1.smithi193.stdout:5/955: rename d0/f12 to d0/d1c/d3b/d4b/dff/f145 0 2024-03-23T08:23:31.478 INFO:tasks.workunit.client.1.smithi193.stdout:1/991: rename da/dd/df to da/dd/df/d7c/d141/d14a 22 2024-03-23T08:23:31.479 INFO:tasks.workunit.client.1.smithi193.stdout:6/811: dwrite d0/d27/ff1 [0,4194304] 0 2024-03-23T08:23:31.480 INFO:tasks.workunit.client.1.smithi193.stdout:4/978: mkdir d11/d8d/d145 0 2024-03-23T08:23:31.482 INFO:tasks.workunit.client.1.smithi193.stdout:0/955: creat db/d11/d14/d24/d47/d14c/f14d x:0 0 0 2024-03-23T08:23:31.483 INFO:tasks.workunit.client.1.smithi193.stdout:4/979: dread d11/d2d/d58/d8b/dbc/de9/f11a [0,4194304] 0 2024-03-23T08:23:31.485 INFO:tasks.workunit.client.1.smithi193.stdout:8/940: sync 2024-03-23T08:23:31.486 INFO:tasks.workunit.client.1.smithi193.stdout:6/812: fdatasync d0/d27/d81/df/d11/d36/fba 0 2024-03-23T08:23:31.491 INFO:tasks.workunit.client.1.smithi193.stdout:5/956: creat d0/d1c/d7a/db1/f146 x:0 0 0 2024-03-23T08:23:31.492 INFO:tasks.workunit.client.1.smithi193.stdout:1/992: write da/dd/f14 [314883,76343] 0 2024-03-23T08:23:31.492 INFO:tasks.workunit.client.1.smithi193.stdout:0/956: symlink db/d11/d14/d21/l14e 0 2024-03-23T08:23:31.493 INFO:tasks.workunit.client.1.smithi193.stdout:3/802: write d1/de/d13/f88 [1253248,104488] 0 2024-03-23T08:23:31.497 INFO:tasks.workunit.client.1.smithi193.stdout:8/941: link df/d10/cc4 df/d10/d3e/d54/d10d/c138 0 2024-03-23T08:23:31.499 INFO:tasks.workunit.client.1.smithi193.stdout:5/957: creat d0/d1c/d7a/d86/f147 x:0 0 0 2024-03-23T08:23:31.500 INFO:tasks.workunit.client.1.smithi193.stdout:0/957: stat db/d11/d14/d24/d2a/d96/fb9 0 2024-03-23T08:23:31.500 INFO:tasks.workunit.client.1.smithi193.stdout:1/993: write da/d13/f94 [978363,72693] 0 2024-03-23T08:23:31.503 INFO:tasks.workunit.client.1.smithi193.stdout:6/813: dwrite d0/d27/d81/da1/fe8 [0,4194304] 0 2024-03-23T08:23:31.503 INFO:tasks.workunit.client.1.smithi193.stdout:8/942: chown df/d63/d76/cd6 2 1 2024-03-23T08:23:31.507 INFO:tasks.workunit.client.1.smithi193.stdout:1/994: symlink da/dd/df/ddc/dfd/l14b 0 2024-03-23T08:23:31.507 INFO:tasks.workunit.client.1.smithi193.stdout:3/803: dwrite d1/de/d78/fa9 [0,4194304] 0 2024-03-23T08:23:31.508 INFO:tasks.workunit.client.1.smithi193.stdout:8/943: creat df/dcc/f139 x:0 0 0 2024-03-23T08:23:31.508 INFO:tasks.workunit.client.1.smithi193.stdout:1/995: fdatasync da/d13/f94 0 2024-03-23T08:23:31.509 INFO:tasks.workunit.client.1.smithi193.stdout:0/958: rename db/d11/d14/d21/f49 to db/d11/d14/d3b/d71/f14f 0 2024-03-23T08:23:31.510 INFO:tasks.workunit.client.1.smithi193.stdout:6/814: write d0/d27/d81/df/d11/d36/f41 [1052378,105780] 0 2024-03-23T08:23:31.511 INFO:tasks.workunit.client.1.smithi193.stdout:1/996: chown da/d13/d24/dac/de4 18281642 1 2024-03-23T08:23:31.515 INFO:tasks.workunit.client.1.smithi193.stdout:6/815: unlink d0/d27/d81/dce/cd5 0 2024-03-23T08:23:31.515 INFO:tasks.workunit.client.1.smithi193.stdout:1/997: symlink da/dd/df/d7c/l14c 0 2024-03-23T08:23:31.516 INFO:tasks.workunit.client.1.smithi193.stdout:0/959: write db/d11/d14/d24/d47/d59/d8c/fb7 [1126039,32965] 0 2024-03-23T08:23:31.519 INFO:tasks.workunit.client.1.smithi193.stdout:3/804: dwrite d1/de/d13/d96/d99/f30 [0,4194304] 0 2024-03-23T08:23:31.520 INFO:tasks.workunit.client.1.smithi193.stdout:1/998: creat da/dd/df/ddc/dfd/f14d x:0 0 0 2024-03-23T08:23:31.525 INFO:tasks.workunit.client.1.smithi193.stdout:1/999: dread da/dd/f39 [0,4194304] 0 2024-03-23T08:23:31.525 INFO:tasks.workunit.client.1.smithi193.stdout:0/960: mkdir db/d11/d14/d24/d2a/d4d/d93/d150 0 2024-03-23T08:23:31.527 INFO:tasks.workunit.client.1.smithi193.stdout:8/944: write df/d10/d3e/f5f [671611,123934] 0 2024-03-23T08:23:31.529 INFO:tasks.workunit.client.1.smithi193.stdout:0/961: readlink db/d11/d14/d24/d2a/d96/l12a 0 2024-03-23T08:23:31.532 INFO:tasks.workunit.client.1.smithi193.stdout:8/945: creat df/d10/d3e/d54/dd2/f13a x:0 0 0 2024-03-23T08:23:31.535 INFO:tasks.workunit.client.1.smithi193.stdout:3/805: dwrite d1/de/d13/d21/d59/fde [0,4194304] 0 2024-03-23T08:23:31.537 INFO:tasks.workunit.client.1.smithi193.stdout:8/946: creat df/d10/d67/d7f/f13b x:0 0 0 2024-03-23T08:23:31.540 INFO:tasks.workunit.client.1.smithi193.stdout:0/962: link db/d11/d14/d24/d2a/d4b/d105/c116 db/d11/d14/d24/d47/d59/d8c/d14b/c151 0 2024-03-23T08:23:31.541 INFO:tasks.workunit.client.1.smithi193.stdout:8/947: readlink df/d10/d67/d7f/lab 0 2024-03-23T08:23:31.541 INFO:tasks.workunit.client.1.smithi193.stdout:6/816: dread d0/d27/d81/fa6 [0,4194304] 0 2024-03-23T08:23:31.545 INFO:tasks.workunit.client.1.smithi193.stdout:0/963: creat db/d11/d14/d21/d33/d69/f152 x:0 0 0 2024-03-23T08:23:31.547 INFO:tasks.workunit.client.1.smithi193.stdout:6/817: rename d0/d27/d81/df/d11/d36/la2 to d0/d27/d81/df/dae/l100 0 2024-03-23T08:23:31.550 INFO:tasks.workunit.client.1.smithi193.stdout:6/818: dread d0/d27/d81/df/d11/d3b/d51/daa/dda/fac [0,4194304] 0 2024-03-23T08:23:31.555 INFO:tasks.workunit.client.1.smithi193.stdout:3/806: dread d1/de/d13/d21/d59/d60/f6b [4194304,4194304] 0 2024-03-23T08:23:31.557 INFO:tasks.workunit.client.1.smithi193.stdout:8/948: dwrite df/dc6/d8b/f9e [0,4194304] 0 2024-03-23T08:23:31.558 INFO:tasks.workunit.client.1.smithi193.stdout:3/807: write d1/de/d13/d21/f81 [2354729,64159] 0 2024-03-23T08:23:31.563 INFO:tasks.workunit.client.1.smithi193.stdout:6/819: dread d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f7e [0,4194304] 0 2024-03-23T08:23:31.565 INFO:tasks.workunit.client.1.smithi193.stdout:8/949: rename df/d4b/d4d/d2a/l58 to df/d4b/d4d/d2a/d10b/d134/l13c 0 2024-03-23T08:23:31.569 INFO:tasks.workunit.client.1.smithi193.stdout:0/964: dread db/d11/d14/d24/d47/d59/fd7 [0,4194304] 0 2024-03-23T08:23:31.572 INFO:tasks.workunit.client.1.smithi193.stdout:0/965: write db/d52/f11f [1264351,115630] 0 2024-03-23T08:23:31.575 INFO:tasks.workunit.client.1.smithi193.stdout:3/808: dwrite d1/de/d13/d21/d25/f56 [0,4194304] 0 2024-03-23T08:23:31.575 INFO:tasks.workunit.client.1.smithi193.stdout:8/950: dwrite df/d10/d67/d7f/f115 [0,4194304] 0 2024-03-23T08:23:31.585 INFO:tasks.workunit.client.1.smithi193.stdout:8/951: write df/d4b/fe4 [958932,70316] 0 2024-03-23T08:23:31.585 INFO:tasks.workunit.client.1.smithi193.stdout:3/809: write d1/de/d13/d96/d99/fee [1888638,22238] 0 2024-03-23T08:23:31.585 INFO:tasks.workunit.client.1.smithi193.stdout:3/810: read d1/de/d13/d21/d25/d2b/f4d [2696011,68580] 0 2024-03-23T08:23:31.586 INFO:tasks.workunit.client.1.smithi193.stdout:3/811: write d1/de/fd3 [806407,65974] 0 2024-03-23T08:23:31.589 INFO:tasks.workunit.client.1.smithi193.stdout:6/820: dwrite d0/d27/d81/da1/db8/fdf [0,4194304] 0 2024-03-23T08:23:31.595 INFO:tasks.workunit.client.1.smithi193.stdout:3/812: mknod d1/de/d13/d21/d25/d2b/d3c/d4a/d85/c10f 0 2024-03-23T08:23:31.596 INFO:tasks.workunit.client.1.smithi193.stdout:6/821: rename d0/d27/d81/df/d11/d3b/d51/daa/dda/la4 to d0/d27/d81/df/d11/d3b/d51/daa/l101 0 2024-03-23T08:23:31.599 INFO:tasks.workunit.client.1.smithi193.stdout:6/822: mkdir d0/d27/d81/df/d11/d36/d61/d7c/d102 0 2024-03-23T08:23:31.601 INFO:tasks.workunit.client.1.smithi193.stdout:2/958: sync 2024-03-23T08:23:31.601 INFO:tasks.workunit.client.1.smithi193.stdout:6/823: creat d0/d27/d81/df/d11/d36/d61/d3a/f103 x:0 0 0 2024-03-23T08:23:31.602 INFO:tasks.workunit.client.1.smithi193.stdout:3/813: link d1/de/d13/d21/d25/d34/c77 d1/de/d13/d21/d25/d2b/d3c/d4a/dae/dd9/c110 0 2024-03-23T08:23:31.603 INFO:tasks.workunit.client.1.smithi193.stdout:6/824: fdatasync d0/d27/d81/df/d11/d36/fba 0 2024-03-23T08:23:31.604 INFO:tasks.workunit.client.1.smithi193.stdout:2/959: mkdir d0/d1b/d14/da1/dd5/d72/d10b 0 2024-03-23T08:23:31.605 INFO:tasks.workunit.client.1.smithi193.stdout:3/814: creat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/f111 x:0 0 0 2024-03-23T08:23:31.606 INFO:tasks.workunit.client.1.smithi193.stdout:6/825: symlink d0/d27/d81/df/d11/d36/d61/d75/l104 0 2024-03-23T08:23:31.607 INFO:tasks.workunit.client.1.smithi193.stdout:3/815: chown d1/de/d13/d21/d25/d34/f8f 10962725 1 2024-03-23T08:23:31.610 INFO:tasks.workunit.client.1.smithi193.stdout:8/952: dread df/d10/d3e/f5f [0,4194304] 0 2024-03-23T08:23:31.610 INFO:tasks.workunit.client.1.smithi193.stdout:6/826: readlink d0/d27/d81/df/d11/d36/d61/d3a/d8f/lc1 0 2024-03-23T08:23:31.614 INFO:tasks.workunit.client.1.smithi193.stdout:3/816: link d1/de/d13/d21/d25/d2b/d3c/f6a d1/de/d13/d21/d50/d6f/de8/f112 0 2024-03-23T08:23:31.615 INFO:tasks.workunit.client.1.smithi193.stdout:0/966: dwrite db/d11/d14/d21/d33/f68 [0,4194304] 0 2024-03-23T08:23:31.617 INFO:tasks.workunit.client.1.smithi193.stdout:6/827: rename d0/d27/d81/df/d11/d36/d61/d7c/c5f to d0/d27/d81/df/d11/d56/dfc/c105 0 2024-03-23T08:23:31.619 INFO:tasks.workunit.client.1.smithi193.stdout:7/925: sync 2024-03-23T08:23:31.619 INFO:tasks.workunit.client.1.smithi193.stdout:4/980: sync 2024-03-23T08:23:31.619 INFO:tasks.workunit.client.1.smithi193.stdout:5/958: sync 2024-03-23T08:23:31.622 INFO:tasks.workunit.client.1.smithi193.stdout:7/926: read - d6/d9/d1f/d5f/ffb zero size 2024-03-23T08:23:31.622 INFO:tasks.workunit.client.1.smithi193.stdout:2/960: dread d0/d1b/d14/d66/f8e [4194304,4194304] 0 2024-03-23T08:23:31.623 INFO:tasks.workunit.client.1.smithi193.stdout:8/953: link df/dcc/f12a df/d10/d3e/f13d 0 2024-03-23T08:23:31.625 INFO:tasks.workunit.client.1.smithi193.stdout:5/959: symlink d0/d1c/d3b/d4b/df9/l148 0 2024-03-23T08:23:31.626 INFO:tasks.workunit.client.1.smithi193.stdout:0/967: symlink db/d11/d14/d21/d33/d69/d9b/l153 0 2024-03-23T08:23:31.627 INFO:tasks.workunit.client.1.smithi193.stdout:0/968: write db/d11/d14/d21/dd2/f125 [3857907,76089] 0 2024-03-23T08:23:31.627 INFO:tasks.workunit.client.1.smithi193.stdout:4/981: fdatasync d11/d1a/d33/d42/d52/f6c 0 2024-03-23T08:23:31.630 INFO:tasks.workunit.client.1.smithi193.stdout:8/954: mkdir df/d63/d84/da8/d13e 0 2024-03-23T08:23:31.631 INFO:tasks.workunit.client.1.smithi193.stdout:2/961: mknod d0/d1b/d14/d66/d4e/dce/c10c 0 2024-03-23T08:23:31.633 INFO:tasks.workunit.client.1.smithi193.stdout:2/962: readlink d0/d28/l2c 0 2024-03-23T08:23:31.633 INFO:tasks.workunit.client.1.smithi193.stdout:5/960: chown d0/l3f 277878850 1 2024-03-23T08:23:31.634 INFO:tasks.workunit.client.1.smithi193.stdout:3/817: dwrite d1/de/d13/d21/d59/d60/fb7 [0,4194304] 0 2024-03-23T08:23:31.639 INFO:tasks.workunit.client.1.smithi193.stdout:6/828: write d0/d27/d81/df/d11/d36/d61/d3a/f6b [2994184,121356] 0 2024-03-23T08:23:31.639 INFO:tasks.workunit.client.1.smithi193.stdout:4/982: symlink d11/d1a/l146 0 2024-03-23T08:23:31.643 INFO:tasks.workunit.client.1.smithi193.stdout:5/961: creat d0/d1c/d7a/db1/d72/d73/d45/d6e/d91/dfd/f149 x:0 0 0 2024-03-23T08:23:31.644 INFO:tasks.workunit.client.1.smithi193.stdout:8/955: creat df/dc6/d8b/f13f x:0 0 0 2024-03-23T08:23:31.644 INFO:tasks.workunit.client.1.smithi193.stdout:6/829: creat d0/d27/d81/df/d11/d36/d61/d12/f106 x:0 0 0 2024-03-23T08:23:31.647 INFO:tasks.workunit.client.1.smithi193.stdout:2/963: creat d0/d1b/d14/da1/dd5/db7/f10d x:0 0 0 2024-03-23T08:23:31.649 INFO:tasks.workunit.client.1.smithi193.stdout:7/927: dread d6/d9/d1f/d4d/d7c/d92/fde [4194304,4194304] 0 2024-03-23T08:23:31.650 INFO:tasks.workunit.client.1.smithi193.stdout:3/818: getdents d1 0 2024-03-23T08:23:31.653 INFO:tasks.workunit.client.1.smithi193.stdout:6/830: write d0/d27/d81/da1/ffb [606525,177] 0 2024-03-23T08:23:31.655 INFO:tasks.workunit.client.1.smithi193.stdout:0/969: dwrite db/d11/d14/d21/d33/d69/fab [0,4194304] 0 2024-03-23T08:23:31.655 INFO:tasks.workunit.client.1.smithi193.stdout:4/983: dwrite d11/d2d/da8/d109/d10e/db8/f2b [4194304,4194304] 0 2024-03-23T08:23:31.656 INFO:tasks.workunit.client.1.smithi193.stdout:2/964: write d0/d1b/d14/d56/fdd [203161,124386] 0 2024-03-23T08:23:31.666 INFO:tasks.workunit.client.1.smithi193.stdout:8/956: link df/d10/d83/dcb/f11a df/d10/d67/f140 0 2024-03-23T08:23:31.666 INFO:tasks.workunit.client.1.smithi193.stdout:5/962: dread d0/d1c/d7a/db1/f8b [0,4194304] 0 2024-03-23T08:23:31.666 INFO:tasks.workunit.client.1.smithi193.stdout:2/965: stat d0/f7 0 2024-03-23T08:23:31.666 INFO:tasks.workunit.client.1.smithi193.stdout:5/963: creat d0/d1c/d3b/dbc/f14a x:0 0 0 2024-03-23T08:23:31.667 INFO:tasks.workunit.client.1.smithi193.stdout:0/970: fdatasync db/d11/d14/d21/d33/d69/d9b/df2/ff6 0 2024-03-23T08:23:31.669 INFO:tasks.workunit.client.1.smithi193.stdout:3/819: link d1/de/d13/d21/d50/d6f/d6e/f72 d1/de/d13/d21/d25/d2b/d3c/d4a/de4/dd1/f113 0 2024-03-23T08:23:31.670 INFO:tasks.workunit.client.1.smithi193.stdout:0/971: fsync db/d11/d14/d24/d2a/d4d/d93/fcc 0 2024-03-23T08:23:31.671 INFO:tasks.workunit.client.1.smithi193.stdout:4/984: rename d11/d2d/d58/d80 to d11/d1a/d33/d42/db2/dbe/d147 0 2024-03-23T08:23:31.673 INFO:tasks.workunit.client.1.smithi193.stdout:3/820: symlink d1/de/d13/d96/l114 0 2024-03-23T08:23:31.675 INFO:tasks.workunit.client.1.smithi193.stdout:8/957: rename df/d4b/d4d/d9c/fe2 to df/d10/d67/d7f/d103/f141 0 2024-03-23T08:23:31.675 INFO:tasks.workunit.client.1.smithi193.stdout:0/972: fdatasync db/f1e 0 2024-03-23T08:23:31.676 INFO:tasks.workunit.client.1.smithi193.stdout:2/966: dwrite d0/d1b/d14/f41 [0,4194304] 0 2024-03-23T08:23:31.677 INFO:tasks.workunit.client.1.smithi193.stdout:4/985: chown d11/d1a/d33/d42/db2/dbe/d147/d9d/f114 19595 1 2024-03-23T08:23:31.679 INFO:tasks.workunit.client.1.smithi193.stdout:8/958: chown df/d10/d3e/d54/le9 2 1 2024-03-23T08:23:31.680 INFO:tasks.workunit.client.1.smithi193.stdout:5/964: dwrite d0/d1c/d3b/dbc/d112/d8e/fa7 [0,4194304] 0 2024-03-23T08:23:31.681 INFO:tasks.workunit.client.1.smithi193.stdout:0/973: chown db/d11/d14/d24/d47/d59/d8c/c11c 490208791 1 2024-03-23T08:23:31.687 INFO:tasks.workunit.client.1.smithi193.stdout:5/965: write d0/d1c/d3b/dbc/d112/d8e/f131 [759914,91868] 0 2024-03-23T08:23:31.691 INFO:tasks.workunit.client.1.smithi193.stdout:4/986: rename d11/d8d to d11/d27/d85/d148 0 2024-03-23T08:23:31.691 INFO:tasks.workunit.client.1.smithi193.stdout:3/821: dwrite d1/de/d13/d21/d59/d60/fd2 [4194304,4194304] 0 2024-03-23T08:23:31.694 INFO:tasks.workunit.client.1.smithi193.stdout:0/974: unlink db/d11/d14/d21/d33/d69/cf8 0 2024-03-23T08:23:31.694 INFO:tasks.workunit.client.1.smithi193.stdout:4/987: read - d11/d1a/f12d zero size 2024-03-23T08:23:31.695 INFO:tasks.workunit.client.1.smithi193.stdout:8/959: dwrite df/d4b/d4d/f22 [0,4194304] 0 2024-03-23T08:23:31.696 INFO:tasks.workunit.client.1.smithi193.stdout:0/975: stat db/d52/cb0 0 2024-03-23T08:23:31.698 INFO:tasks.workunit.client.1.smithi193.stdout:2/967: getdents d0/d1b/d14/d66 0 2024-03-23T08:23:31.699 INFO:tasks.workunit.client.1.smithi193.stdout:4/988: chown d11/d1a/d33/d42/f4d 12353 1 2024-03-23T08:23:31.701 INFO:tasks.workunit.client.1.smithi193.stdout:0/976: readlink db/d11/d14/d21/d33/l3c 0 2024-03-23T08:23:31.701 INFO:tasks.workunit.client.1.smithi193.stdout:3/822: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/l115 0 2024-03-23T08:23:31.702 INFO:tasks.workunit.client.1.smithi193.stdout:6/831: dwrite d0/d27/d81/df/d11/d3b/d6c/f90 [0,4194304] 0 2024-03-23T08:23:31.703 INFO:tasks.workunit.client.1.smithi193.stdout:4/989: read d11/d2d/d58/d8b/dbc/f2f [9765,14462] 0 2024-03-23T08:23:31.704 INFO:tasks.workunit.client.1.smithi193.stdout:8/960: symlink df/l142 0 2024-03-23T08:23:31.708 INFO:tasks.workunit.client.1.smithi193.stdout:3/823: creat d1/de/f116 x:0 0 0 2024-03-23T08:23:31.709 INFO:tasks.workunit.client.1.smithi193.stdout:0/977: write db/d11/d14/d24/d47/d59/f72 [836922,127857] 0 2024-03-23T08:23:31.711 INFO:tasks.workunit.client.1.smithi193.stdout:8/961: rmdir df/d63/d76 39 2024-03-23T08:23:31.713 INFO:tasks.workunit.client.1.smithi193.stdout:3/824: symlink d1/de/d28/l117 0 2024-03-23T08:23:31.714 INFO:tasks.workunit.client.1.smithi193.stdout:0/978: mknod db/d11/d14/d24/d2a/de3/c154 0 2024-03-23T08:23:31.716 INFO:tasks.workunit.client.1.smithi193.stdout:3/825: write d1/fbd [5546924,23788] 0 2024-03-23T08:23:31.719 INFO:tasks.workunit.client.1.smithi193.stdout:2/968: dwrite d0/d1b/d14/d66/d50/d92/fc8 [0,4194304] 0 2024-03-23T08:23:31.719 INFO:tasks.workunit.client.1.smithi193.stdout:0/979: rmdir db/d11/d14/d24/dc9 39 2024-03-23T08:23:31.726 INFO:tasks.workunit.client.1.smithi193.stdout:4/990: dwrite d11/d1a/d33/d42/db2/dbe/d147/d9d/f114 [0,4194304] 0 2024-03-23T08:23:31.726 INFO:tasks.workunit.client.1.smithi193.stdout:0/980: readlink db/d11/d14/d21/d33/l142 0 2024-03-23T08:23:31.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:31 smithi070 ceph-mon[39699]: pgmap v25: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 83 MiB/s rd, 196 MiB/s wr, 387 op/s 2024-03-23T08:23:31.733 INFO:tasks.workunit.client.1.smithi193.stdout:2/969: creat d0/d1b/d14/da1/dd5/d109/f10e x:0 0 0 2024-03-23T08:23:31.733 INFO:tasks.workunit.client.1.smithi193.stdout:3/826: unlink d1/de/d13/d21/d25/d2b/d3c/d4a/dae/dd9/c110 0 2024-03-23T08:23:31.733 INFO:tasks.workunit.client.1.smithi193.stdout:4/991: mknod d11/d1a/d33/d42/db2/c149 0 2024-03-23T08:23:31.733 INFO:tasks.workunit.client.1.smithi193.stdout:8/962: truncate df/d10/f17 2415919 0 2024-03-23T08:23:31.734 INFO:tasks.workunit.client.1.smithi193.stdout:4/992: chown d11/d2d/da8 1507 1 2024-03-23T08:23:31.734 INFO:tasks.workunit.client.1.smithi193.stdout:0/981: rename db/d11/d14/d24/d47/fac to db/d11/d14/d24/d2a/de3/ddb/f155 0 2024-03-23T08:23:31.734 INFO:tasks.workunit.client.1.smithi193.stdout:8/963: stat df/f18 0 2024-03-23T08:23:31.735 INFO:tasks.workunit.client.1.smithi193.stdout:2/970: fsync d0/d1b/d3b/f7a 0 2024-03-23T08:23:31.736 INFO:tasks.workunit.client.1.smithi193.stdout:4/993: write d11/d2d/da8/f130 [3346783,89707] 0 2024-03-23T08:23:31.738 INFO:tasks.workunit.client.1.smithi193.stdout:5/966: dread d0/d1f/d93/fa3 [0,4194304] 0 2024-03-23T08:23:31.739 INFO:tasks.workunit.client.1.smithi193.stdout:2/971: creat d0/d1b/d14/d66/d4e/dce/f10f x:0 0 0 2024-03-23T08:23:31.739 INFO:tasks.workunit.client.1.smithi193.stdout:0/982: getdents db/d11/d14/d24/d2a/d96/d11d 0 2024-03-23T08:23:31.741 INFO:tasks.workunit.client.1.smithi193.stdout:5/967: mkdir d0/d1f/d93/dcd/d14b 0 2024-03-23T08:23:31.743 INFO:tasks.workunit.client.1.smithi193.stdout:5/968: chown d0/d1c/d7a/db1/d72/d73/d57/l7b 11934 1 2024-03-23T08:23:31.743 INFO:tasks.workunit.client.1.smithi193.stdout:4/994: rename d11/d1a/d33/d42/c4e to d11/d2d/d58/d8b/dbc/c14a 0 2024-03-23T08:23:31.746 INFO:tasks.workunit.client.1.smithi193.stdout:5/969: rmdir d0/d1c/df7 39 2024-03-23T08:23:31.746 INFO:tasks.workunit.client.1.smithi193.stdout:4/995: creat d11/d2d/da8/d109/f14b x:0 0 0 2024-03-23T08:23:31.747 INFO:tasks.workunit.client.1.smithi193.stdout:4/996: fdatasync d11/d2d/da8/f130 0 2024-03-23T08:23:31.750 INFO:tasks.workunit.client.1.smithi193.stdout:4/997: chown d11/d1a/d33/d7f/cae 491642 1 2024-03-23T08:23:31.752 INFO:tasks.workunit.client.1.smithi193.stdout:4/998: readlink d11/d2d/d58/d8b/dbc/lc2 0 2024-03-23T08:23:31.755 INFO:tasks.workunit.client.1.smithi193.stdout:5/970: dwrite d0/d5a/dbf/f101 [0,4194304] 0 2024-03-23T08:23:31.756 INFO:tasks.workunit.client.1.smithi193.stdout:2/972: dread d0/d1b/d3b/f7a [0,4194304] 0 2024-03-23T08:23:31.762 INFO:tasks.workunit.client.1.smithi193.stdout:2/973: creat d0/d1b/d14/d66/f110 x:0 0 0 2024-03-23T08:23:31.762 INFO:tasks.workunit.client.1.smithi193.stdout:4/999: symlink d11/d1a/d33/d42/db2/dbe/l14c 0 2024-03-23T08:23:31.763 INFO:tasks.workunit.client.1.smithi193.stdout:3/827: dread d1/de/d13/d21/d25/f7e [0,4194304] 0 2024-03-23T08:23:31.765 INFO:tasks.workunit.client.1.smithi193.stdout:2/974: chown d0/d1b/d14/da1/dd5/db7/lbd 870654 1 2024-03-23T08:23:31.767 INFO:tasks.workunit.client.1.smithi193.stdout:5/971: dwrite d0/d1c/d3b/d4b/f10e [0,4194304] 0 2024-03-23T08:23:31.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/983: dread db/d11/d14/d21/d33/d48/f54 [0,4194304] 0 2024-03-23T08:23:31.773 INFO:tasks.workunit.client.1.smithi193.stdout:5/972: write d0/d1c/d3b/d4b/dff/f145 [3217427,108959] 0 2024-03-23T08:23:31.775 INFO:tasks.workunit.client.1.smithi193.stdout:5/973: mkdir d0/d1f/d93/de1/d14c 0 2024-03-23T08:23:31.777 INFO:tasks.workunit.client.1.smithi193.stdout:0/984: dwrite db/d11/d14/d24/f97 [0,4194304] 0 2024-03-23T08:23:31.781 INFO:tasks.workunit.client.1.smithi193.stdout:2/975: truncate d0/d1b/d14/d56/fdd 124241 0 2024-03-23T08:23:31.788 INFO:tasks.workunit.client.1.smithi193.stdout:2/976: rename d0/d1b/d77 to d0/d1b/d77/d111 22 2024-03-23T08:23:31.789 INFO:tasks.workunit.client.1.smithi193.stdout:5/974: dwrite d0/d5a/dbf/f111 [0,4194304] 0 2024-03-23T08:23:31.792 INFO:tasks.workunit.client.1.smithi193.stdout:7/928: sync 2024-03-23T08:23:31.794 INFO:tasks.workunit.client.1.smithi193.stdout:2/977: creat d0/d1b/f112 x:0 0 0 2024-03-23T08:23:31.798 INFO:tasks.workunit.client.1.smithi193.stdout:2/978: mkdir d0/d28/d9f/d113 0 2024-03-23T08:23:31.798 INFO:tasks.workunit.client.1.smithi193.stdout:5/975: dwrite d0/d1c/d3b/dbc/d112/d8e/fa7 [4194304,4194304] 0 2024-03-23T08:23:31.799 INFO:tasks.workunit.client.1.smithi193.stdout:5/976: fsync d0/d1c/d3b/dbc/d112/f87 0 2024-03-23T08:23:31.800 INFO:tasks.workunit.client.1.smithi193.stdout:7/929: rename d6/d9/d1f/f94 to d6/d9/d1f/d23/d33/d38/d66/dac/dfa/f141 0 2024-03-23T08:23:31.804 INFO:tasks.workunit.client.1.smithi193.stdout:7/930: write d6/d9/d1f/daf/fef [771028,88111] 0 2024-03-23T08:23:31.806 INFO:tasks.workunit.client.1.smithi193.stdout:2/979: getdents d0/d1b/d14/d56/d7c/d7e 0 2024-03-23T08:23:31.807 INFO:tasks.workunit.client.1.smithi193.stdout:7/931: rmdir d6/d9/d1f/de8 39 2024-03-23T08:23:31.807 INFO:tasks.workunit.client.1.smithi193.stdout:0/985: dread db/d11/d14/d21/d33/d69/d9b/df2/ff6 [0,4194304] 0 2024-03-23T08:23:31.809 INFO:tasks.workunit.client.1.smithi193.stdout:5/977: link d0/d1c/d7a/db1/d72/d73/f6c d0/d1f/d93/de1/d14c/f14d 0 2024-03-23T08:23:31.810 INFO:tasks.workunit.client.1.smithi193.stdout:2/980: readlink d0/d1b/l6 0 2024-03-23T08:23:31.810 INFO:tasks.workunit.client.1.smithi193.stdout:0/986: write db/d11/d14/d21/d33/d69/fab [2522962,103626] 0 2024-03-23T08:23:31.811 INFO:tasks.workunit.client.1.smithi193.stdout:7/932: mkdir d6/d9/d1f/d23/d33/d142 0 2024-03-23T08:23:31.812 INFO:tasks.workunit.client.1.smithi193.stdout:2/981: read - d0/d28/fd0 zero size 2024-03-23T08:23:31.813 INFO:tasks.workunit.client.1.smithi193.stdout:2/982: dread - d0/d28/f107 zero size 2024-03-23T08:23:31.815 INFO:tasks.workunit.client.1.smithi193.stdout:0/987: symlink db/d11/l156 0 2024-03-23T08:23:31.816 INFO:tasks.workunit.client.1.smithi193.stdout:0/988: stat db/d11/d14/d24/cae 0 2024-03-23T08:23:31.820 INFO:tasks.workunit.client.1.smithi193.stdout:5/978: truncate d0/d1c/d7a/db1/d72/d73/f6c 2666722 0 2024-03-23T08:23:31.820 INFO:tasks.workunit.client.1.smithi193.stdout:7/933: dwrite d6/d9/f6f [0,4194304] 0 2024-03-23T08:23:31.824 INFO:tasks.workunit.client.1.smithi193.stdout:0/989: rename db/d11/d14/d21/d33/d69/cb1 to db/d11/d14/d3b/d71/c157 0 2024-03-23T08:23:31.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:31 smithi193 ceph-mon[48370]: pgmap v25: 65 pgs: 65 active+clean; 1.8 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 83 MiB/s rd, 196 MiB/s wr, 387 op/s 2024-03-23T08:23:31.828 INFO:tasks.workunit.client.1.smithi193.stdout:7/934: mkdir d6/d9/d1f/d23/d33/d38/d3b/db7/d143 0 2024-03-23T08:23:31.835 INFO:tasks.workunit.client.1.smithi193.stdout:5/979: rename d0/d1f/d93/de1/d14c/f14d to d0/d1f/d93/de1/f14e 0 2024-03-23T08:23:31.836 INFO:tasks.workunit.client.1.smithi193.stdout:5/980: chown d0/d5a/dbf/f101 169942373 1 2024-03-23T08:23:31.839 INFO:tasks.workunit.client.1.smithi193.stdout:7/935: dwrite d6/d9/d1f/d23/d33/d38/d66/dac/dfa/f141 [0,4194304] 0 2024-03-23T08:23:31.841 INFO:tasks.workunit.client.1.smithi193.stdout:5/981: mknod d0/d1c/d7a/d86/c14f 0 2024-03-23T08:23:31.842 INFO:tasks.workunit.client.1.smithi193.stdout:7/936: mknod d6/d9/d1f/d23/d33/d38/d3b/d6c/de7/c144 0 2024-03-23T08:23:31.842 INFO:tasks.workunit.client.1.smithi193.stdout:0/990: dread db/d11/d14/d24/f7d [0,4194304] 0 2024-03-23T08:23:31.845 INFO:tasks.workunit.client.1.smithi193.stdout:7/937: truncate d6/d9/f15 1395638 0 2024-03-23T08:23:31.846 INFO:tasks.workunit.client.1.smithi193.stdout:5/982: creat d0/d1c/f150 x:0 0 0 2024-03-23T08:23:31.849 INFO:tasks.workunit.client.1.smithi193.stdout:5/983: symlink d0/d1c/d3b/d4b/dff/l151 0 2024-03-23T08:23:31.850 INFO:tasks.workunit.client.1.smithi193.stdout:0/991: getdents db/d11/d14/d24/d2a 0 2024-03-23T08:23:31.851 INFO:tasks.workunit.client.1.smithi193.stdout:7/938: dread d6/d9/d1f/d4d/d7c/d8e/fbf [0,4194304] 0 2024-03-23T08:23:31.852 INFO:tasks.workunit.client.1.smithi193.stdout:0/992: write db/d52/db8/dcf/f109 [1316212,45685] 0 2024-03-23T08:23:31.852 INFO:tasks.workunit.client.1.smithi193.stdout:5/984: mknod d0/d5a/dbf/d106/d13c/c152 0 2024-03-23T08:23:31.854 INFO:tasks.workunit.client.1.smithi193.stdout:0/993: readlink db/d11/d14/l1b 0 2024-03-23T08:23:31.854 INFO:tasks.workunit.client.1.smithi193.stdout:5/985: creat d0/d1c/d7a/f153 x:0 0 0 2024-03-23T08:23:31.856 INFO:tasks.workunit.client.1.smithi193.stdout:0/994: stat db/d11/d14/d24/f39 0 2024-03-23T08:23:31.856 INFO:tasks.workunit.client.1.smithi193.stdout:0/995: chown db/d52/db8 0 1 2024-03-23T08:23:31.856 INFO:tasks.workunit.client.1.smithi193.stdout:7/939: creat d6/d9/d1f/d23/d33/d38/d3b/d6c/de7/d117/f145 x:0 0 0 2024-03-23T08:23:31.857 INFO:tasks.workunit.client.1.smithi193.stdout:5/986: mkdir d0/d1f/d93/dcd/d154 0 2024-03-23T08:23:31.859 INFO:tasks.workunit.client.1.smithi193.stdout:7/940: truncate d6/d9/d1f/d23/d33/d38/d3b/d6c/f8d 1383908 0 2024-03-23T08:23:31.860 INFO:tasks.workunit.client.1.smithi193.stdout:0/996: creat db/d11/d14/d24/d2a/d4b/d105/f158 x:0 0 0 2024-03-23T08:23:31.861 INFO:tasks.workunit.client.1.smithi193.stdout:7/941: write d6/d9/fd [2233275,77714] 0 2024-03-23T08:23:31.863 INFO:tasks.workunit.client.1.smithi193.stdout:0/997: creat db/d11/d14/d3b/f159 x:0 0 0 2024-03-23T08:23:31.869 INFO:tasks.workunit.client.1.smithi193.stdout:5/987: write d0/d1c/d7a/f8f [1082858,84536] 0 2024-03-23T08:23:31.870 INFO:tasks.workunit.client.1.smithi193.stdout:7/942: dwrite d6/d9/d4e/fae [4194304,4194304] 0 2024-03-23T08:23:31.872 INFO:tasks.workunit.client.1.smithi193.stdout:0/998: truncate db/d11/d14/d21/f41 424515 0 2024-03-23T08:23:31.876 INFO:tasks.workunit.client.1.smithi193.stdout:7/943: creat d6/d9/d1f/d23/d33/d3c/d41/d47/f146 x:0 0 0 2024-03-23T08:23:31.876 INFO:tasks.workunit.client.1.smithi193.stdout:5/988: mknod d0/d1c/d7a/db1/d72/d73/c155 0 2024-03-23T08:23:31.878 INFO:tasks.workunit.client.1.smithi193.stdout:5/989: chown d0/d1c/d3b/d81/df2 54273984 1 2024-03-23T08:23:31.880 INFO:tasks.workunit.client.1.smithi193.stdout:0/999: creat db/d11/d14/d21/d33/d69/d9b/f15a x:0 0 0 2024-03-23T08:23:31.880 INFO:tasks.workunit.client.1.smithi193.stdout:5/990: mknod d0/d1c/d3b/d4b/dff/c156 0 2024-03-23T08:23:31.881 INFO:tasks.workunit.client.1.smithi193.stdout:5/991: stat d0/d1c/d7a/db1/d72/d73/c110 0 2024-03-23T08:23:31.883 INFO:tasks.workunit.client.1.smithi193.stdout:5/992: fdatasync d0/d1c/d3b/d4b/d55/f80 0 2024-03-23T08:23:31.887 INFO:tasks.workunit.client.1.smithi193.stdout:5/993: write d0/d1c/d3b/d81/f82 [297055,87919] 0 2024-03-23T08:23:31.889 INFO:tasks.workunit.client.1.smithi193.stdout:6/832: sync 2024-03-23T08:23:31.891 INFO:tasks.workunit.client.1.smithi193.stdout:5/994: rename d0/d1c/d3b/d4b/d55/c114 to d0/d1c/d3b/d130/c157 0 2024-03-23T08:23:31.894 INFO:tasks.workunit.client.1.smithi193.stdout:7/944: dread d6/d9/fd [0,4194304] 0 2024-03-23T08:23:31.894 INFO:tasks.workunit.client.1.smithi193.stdout:6/833: write d0/d27/d81/da1/da9/fe9 [1164696,40673] 0 2024-03-23T08:23:31.897 INFO:tasks.workunit.client.1.smithi193.stdout:6/834: creat d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f107 x:0 0 0 2024-03-23T08:23:31.898 INFO:tasks.workunit.client.1.smithi193.stdout:7/945: mknod d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/df3/da1/de0/d135/c147 0 2024-03-23T08:23:31.907 INFO:tasks.workunit.client.1.smithi193.stdout:6/835: write d0/d27/d81/df/d11/d3b/d51/d9c/f96 [1191209,114804] 0 2024-03-23T08:23:31.912 INFO:tasks.workunit.client.1.smithi193.stdout:5/995: dread d0/d1f/d93/de1/f105 [4194304,4194304] 0 2024-03-23T08:23:31.913 INFO:tasks.workunit.client.1.smithi193.stdout:6/836: dwrite d0/d27/d81/df/d11/d36/d61/d3a/fb4 [0,4194304] 0 2024-03-23T08:23:31.919 INFO:tasks.workunit.client.1.smithi193.stdout:6/837: dwrite d0/d27/d81/df/d11/d3b/f76 [0,4194304] 0 2024-03-23T08:23:31.920 INFO:tasks.workunit.client.1.smithi193.stdout:6/838: truncate d0/fc6 1063272 0 2024-03-23T08:23:31.953 INFO:tasks.workunit.client.1.smithi193.stdout:3/828: sync 2024-03-23T08:23:31.954 INFO:tasks.workunit.client.1.smithi193.stdout:8/964: sync 2024-03-23T08:23:31.954 INFO:tasks.workunit.client.1.smithi193.stdout:8/965: fdatasync df/d10/d3e/d54/d10d/f9d 0 2024-03-23T08:23:31.954 INFO:tasks.workunit.client.1.smithi193.stdout:3/829: write d1/de/d13/d96/d99/fee [2249435,52254] 0 2024-03-23T08:23:32.012 INFO:tasks.workunit.client.1.smithi193.stdout:2/983: sync 2024-03-23T08:23:32.043 INFO:tasks.workunit.client.1.smithi193.stdout:7/946: mknod d6/d9/d1f/d3e/d65/d70/c148 0 2024-03-23T08:23:32.044 INFO:tasks.workunit.client.1.smithi193.stdout:5/996: mkdir d0/d1f/d93/dcd/d158 0 2024-03-23T08:23:32.045 INFO:tasks.workunit.client.1.smithi193.stdout:7/947: dread d6/d9/d1f/f50 [0,4194304] 0 2024-03-23T08:23:32.046 INFO:tasks.workunit.client.1.smithi193.stdout:6/839: rmdir d0/d27/d81/df/d11/d36/d61/d3a 39 2024-03-23T08:23:32.048 INFO:tasks.workunit.client.1.smithi193.stdout:8/966: symlink df/d4b/d4d/d2a/l143 0 2024-03-23T08:23:32.050 INFO:tasks.workunit.client.1.smithi193.stdout:5/997: mkdir d0/d5a/dbf/d159 0 2024-03-23T08:23:32.056 INFO:tasks.workunit.client.1.smithi193.stdout:2/984: symlink d0/l114 0 2024-03-23T08:23:32.057 INFO:tasks.workunit.client.1.smithi193.stdout:5/998: creat d0/d1f/f15a x:0 0 0 2024-03-23T08:23:32.057 INFO:tasks.workunit.client.1.smithi193.stdout:7/948: dwrite d6/d9/d1f/d3e/d65/f75 [0,4194304] 0 2024-03-23T08:23:32.062 INFO:tasks.workunit.client.1.smithi193.stdout:5/999: readlink d0/d1c/d3b/d4b/d55/l129 0 2024-03-23T08:23:32.062 INFO:tasks.workunit.client.1.smithi193.stdout:6/840: symlink d0/d27/d81/df/d11/d36/d61/d3a/df3/l108 0 2024-03-23T08:23:32.062 INFO:tasks.workunit.client.1.smithi193.stdout:8/967: creat df/d10/f144 x:0 0 0 2024-03-23T08:23:32.067 INFO:tasks.workunit.client.1.smithi193.stdout:7/949: dwrite d6/d9/d1f/d23/d33/d38/d66/fa3 [0,4194304] 0 2024-03-23T08:23:32.068 INFO:tasks.workunit.client.1.smithi193.stdout:7/950: write d6/d9/f3d [364373,76631] 0 2024-03-23T08:23:32.068 INFO:tasks.workunit.client.1.smithi193.stdout:2/985: mknod d0/d1b/d14/d66/d50/c115 0 2024-03-23T08:23:32.069 INFO:tasks.workunit.client.1.smithi193.stdout:6/841: getdents d0/d27/d81/df/d11/d36/d61/d7c/d69/de1 0 2024-03-23T08:23:32.073 INFO:tasks.workunit.client.1.smithi193.stdout:8/968: getdents df/d63 0 2024-03-23T08:23:32.075 INFO:tasks.workunit.client.1.smithi193.stdout:8/969: write df/d10/f2f [3717474,753] 0 2024-03-23T08:23:32.075 INFO:tasks.workunit.client.1.smithi193.stdout:6/842: chown d0/d27/d81/df/d11/d36/d61/d7c/d69/c94 149 1 2024-03-23T08:23:32.078 INFO:tasks.workunit.client.1.smithi193.stdout:2/986: creat d0/d1b/d14/d66/d50/d8f/f116 x:0 0 0 2024-03-23T08:23:32.078 INFO:tasks.workunit.client.1.smithi193.stdout:6/843: fsync d0/d27/d81/df/d11/d36/d61/d3a/f45 0 2024-03-23T08:23:32.078 INFO:tasks.workunit.client.1.smithi193.stdout:8/970: mkdir df/d63/d84/df0/d145 0 2024-03-23T08:23:32.081 INFO:tasks.workunit.client.1.smithi193.stdout:6/844: unlink d0/d27/d81/da1/ffb 0 2024-03-23T08:23:32.084 INFO:tasks.workunit.client.1.smithi193.stdout:6/845: write d0/d27/d81/da1/fd8 [293478,74329] 0 2024-03-23T08:23:32.084 INFO:tasks.workunit.client.1.smithi193.stdout:8/971: write df/d10/d3e/d54/d10d/f9d [5159347,55646] 0 2024-03-23T08:23:32.090 INFO:tasks.workunit.client.1.smithi193.stdout:2/987: dwrite d0/d1b/d19/f80 [0,4194304] 0 2024-03-23T08:23:32.097 INFO:tasks.workunit.client.1.smithi193.stdout:3/830: dwrite d1/de/d13/d21/d25/d2b/d3c/d4a/de4/dd1/f113 [4194304,4194304] 0 2024-03-23T08:23:32.099 INFO:tasks.workunit.client.1.smithi193.stdout:8/972: dwrite df/d4b/d4d/d9c/fa6 [0,4194304] 0 2024-03-23T08:23:32.099 INFO:tasks.workunit.client.1.smithi193.stdout:2/988: truncate d0/d1b/d14/d66/d50/d8f/f116 487382 0 2024-03-23T08:23:32.108 INFO:tasks.workunit.client.1.smithi193.stdout:6/846: link d0/d27/d81/fe0 d0/d27/d81/df/d11/d3b/d51/db2/dfa/f109 0 2024-03-23T08:23:32.111 INFO:tasks.workunit.client.1.smithi193.stdout:3/831: creat d1/de/f118 x:0 0 0 2024-03-23T08:23:32.111 INFO:tasks.workunit.client.1.smithi193.stdout:6/847: dread - d0/d27/d81/df/d11/d3b/d6c/f88 zero size 2024-03-23T08:23:32.111 INFO:tasks.workunit.client.1.smithi193.stdout:6/848: fdatasync d0/d27/d81/df/d11/d56/f8d 0 2024-03-23T08:23:32.111 INFO:tasks.workunit.client.1.smithi193.stdout:8/973: link df/d10/d3e/d54/dd2/f1b df/d10/d3e/d54/d10d/d101/f146 0 2024-03-23T08:23:32.111 INFO:tasks.workunit.client.1.smithi193.stdout:7/951: dread d6/d9/d1f/d3e/d42/f60 [0,4194304] 0 2024-03-23T08:23:32.113 INFO:tasks.workunit.client.1.smithi193.stdout:3/832: dread d1/de/d78/fc7 [0,4194304] 0 2024-03-23T08:23:32.114 INFO:tasks.workunit.client.1.smithi193.stdout:7/952: write d6/d9/d1f/d23/d33/d38/d3b/f48 [1057401,30022] 0 2024-03-23T08:23:32.117 INFO:tasks.workunit.client.1.smithi193.stdout:3/833: mkdir d1/de/d13/d21/d25/d2b/d3c/d119 0 2024-03-23T08:23:32.118 INFO:tasks.workunit.client.1.smithi193.stdout:6/849: getdents d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/db1 0 2024-03-23T08:23:32.119 INFO:tasks.workunit.client.1.smithi193.stdout:7/953: unlink d6/d9/d1f/d23/d33/d38/d66/dac/cea 0 2024-03-23T08:23:32.120 INFO:tasks.workunit.client.1.smithi193.stdout:6/850: rmdir d0/d27/d81/df/d11/d36/d61/d7c/d69 39 2024-03-23T08:23:32.120 INFO:tasks.workunit.client.1.smithi193.stdout:3/834: write d1/de/d13/d21/d25/f7e [1271201,110656] 0 2024-03-23T08:23:32.122 INFO:tasks.workunit.client.1.smithi193.stdout:6/851: write d0/d27/d81/df/d11/d36/d61/d3a/f103 [632423,49784] 0 2024-03-23T08:23:32.126 INFO:tasks.workunit.client.1.smithi193.stdout:6/852: chown d0/d27/d81/df/d11/d36/d61/d12/ff2 4747625 1 2024-03-23T08:23:32.127 INFO:tasks.workunit.client.1.smithi193.stdout:3/835: link d1/de/d13/f5b d1/de/d13/d21/d25/d34/da8/f11a 0 2024-03-23T08:23:32.131 INFO:tasks.workunit.client.1.smithi193.stdout:3/836: mkdir d1/de/d13/d21/d50/d11b 0 2024-03-23T08:23:32.132 INFO:tasks.workunit.client.1.smithi193.stdout:6/853: rename d0/d27/d81/df/l28 to d0/d27/d81/df/d11/d36/d61/d7c/l10a 0 2024-03-23T08:23:32.134 INFO:tasks.workunit.client.1.smithi193.stdout:3/837: rename d1/de/d13/d21/f82 to d1/de/d13/d21/d25/d2b/d3c/d119/f11c 0 2024-03-23T08:23:32.135 INFO:tasks.workunit.client.1.smithi193.stdout:8/974: dread df/feb [0,4194304] 0 2024-03-23T08:23:32.135 INFO:tasks.workunit.client.1.smithi193.stdout:7/954: dread d6/d9/d1f/d23/d33/d38/d46/d82/fed [0,4194304] 0 2024-03-23T08:23:32.136 INFO:tasks.workunit.client.1.smithi193.stdout:3/838: write d1/de/d28/fec [480696,98550] 0 2024-03-23T08:23:32.137 INFO:tasks.workunit.client.1.smithi193.stdout:6/854: creat d0/d27/d81/df/d11/d36/f10b x:0 0 0 2024-03-23T08:23:32.139 INFO:tasks.workunit.client.1.smithi193.stdout:7/955: rmdir d6/d9/d1f/d23/d33/d38/d46/d82/d12b 39 2024-03-23T08:23:32.140 INFO:tasks.workunit.client.1.smithi193.stdout:6/855: creat d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f10c x:0 0 0 2024-03-23T08:23:32.142 INFO:tasks.workunit.client.1.smithi193.stdout:7/956: truncate d6/d9/d1f/d3e/d65/f75 4885610 0 2024-03-23T08:23:32.142 INFO:tasks.workunit.client.1.smithi193.stdout:3/839: getdents d1 0 2024-03-23T08:23:32.144 INFO:tasks.workunit.client.1.smithi193.stdout:8/975: getdents df/d10/d83/dcb 0 2024-03-23T08:23:32.147 INFO:tasks.workunit.client.1.smithi193.stdout:7/957: rmdir d6/d9/d1f/d23/d33/d3c/df0 0 2024-03-23T08:23:32.148 INFO:tasks.workunit.client.1.smithi193.stdout:6/856: dwrite d0/d27/d81/df/d11/f30 [0,4194304] 0 2024-03-23T08:23:32.150 INFO:tasks.workunit.client.1.smithi193.stdout:7/958: dread - d6/d9/d1f/d3e/d65/fe9 zero size 2024-03-23T08:23:32.150 INFO:tasks.workunit.client.1.smithi193.stdout:8/976: dread df/d10/d3e/d54/f85 [0,4194304] 0 2024-03-23T08:23:32.153 INFO:tasks.workunit.client.1.smithi193.stdout:7/959: write d6/d9/d1f/d23/d33/f107 [259055,37419] 0 2024-03-23T08:23:32.156 INFO:tasks.workunit.client.1.smithi193.stdout:6/857: rename d0/d27/d81/df/d11/ca7 to d0/d27/d81/df/c10d 0 2024-03-23T08:23:32.159 INFO:tasks.workunit.client.1.smithi193.stdout:7/960: truncate d6/d9/d1f/d3e/d42/f71 2286541 0 2024-03-23T08:23:32.160 INFO:tasks.workunit.client.1.smithi193.stdout:8/977: getdents df/d4b/d4d/d9c 0 2024-03-23T08:23:32.167 INFO:tasks.workunit.client.1.smithi193.stdout:3/840: dread d1/de/d13/d96/d99/fee [0,4194304] 0 2024-03-23T08:23:32.167 INFO:tasks.workunit.client.1.smithi193.stdout:6/858: dwrite d0/d27/d81/df/d11/d36/d61/d12/f106 [0,4194304] 0 2024-03-23T08:23:32.171 INFO:tasks.workunit.client.1.smithi193.stdout:8/978: dwrite df/d10/d67/fa9 [0,4194304] 0 2024-03-23T08:23:32.177 INFO:tasks.workunit.client.1.smithi193.stdout:8/979: creat df/d4b/d4d/d9c/f147 x:0 0 0 2024-03-23T08:23:32.179 INFO:tasks.workunit.client.1.smithi193.stdout:3/841: getdents d1/de/d13/d21 0 2024-03-23T08:23:32.190 INFO:tasks.workunit.client.1.smithi193.stdout:6/859: dread d0/d27/d81/df/d11/d3b/d51/db2/dfa/f109 [0,4194304] 0 2024-03-23T08:23:32.192 INFO:tasks.workunit.client.1.smithi193.stdout:6/860: unlink d0/d27/d81/fa6 0 2024-03-23T08:23:32.194 INFO:tasks.workunit.client.1.smithi193.stdout:6/861: write d0/d27/d81/f4e [2033693,12487] 0 2024-03-23T08:23:32.197 INFO:tasks.workunit.client.1.smithi193.stdout:8/980: dread df/d4b/f66 [0,4194304] 0 2024-03-23T08:23:32.200 INFO:tasks.workunit.client.1.smithi193.stdout:8/981: creat df/d4b/f148 x:0 0 0 2024-03-23T08:23:32.201 INFO:tasks.workunit.client.1.smithi193.stdout:6/862: dwrite d0/d27/d81/df/d11/d3b/d6c/fbe [0,4194304] 0 2024-03-23T08:23:32.203 INFO:tasks.workunit.client.1.smithi193.stdout:8/982: fdatasync df/d63/d84/fdd 0 2024-03-23T08:23:32.204 INFO:tasks.workunit.client.1.smithi193.stdout:8/983: truncate df/d10/f132 964706 0 2024-03-23T08:23:32.205 INFO:tasks.workunit.client.1.smithi193.stdout:6/863: mknod d0/d27/d81/df/d11/d36/d61/d7c/d69/c10e 0 2024-03-23T08:23:32.213 INFO:tasks.workunit.client.1.smithi193.stdout:8/984: dwrite df/d63/d84/da8/ff8 [0,4194304] 0 2024-03-23T08:23:32.214 INFO:tasks.workunit.client.1.smithi193.stdout:6/864: dwrite d0/d27/d81/df/d11/d36/d61/d12/ff2 [0,4194304] 0 2024-03-23T08:23:32.216 INFO:tasks.workunit.client.1.smithi193.stdout:6/865: write d0/d27/d81/df/d11/d36/d61/f6d [1177147,111303] 0 2024-03-23T08:23:32.218 INFO:tasks.workunit.client.1.smithi193.stdout:8/985: dread df/d10/d83/dcb/f12e [0,4194304] 0 2024-03-23T08:23:32.219 INFO:tasks.workunit.client.1.smithi193.stdout:8/986: chown df/d4b/d4d/f45 2007007 1 2024-03-23T08:23:32.229 INFO:tasks.workunit.client.1.smithi193.stdout:6/866: truncate d0/d27/d81/df/d11/d36/d61/f6d 2575041 0 2024-03-23T08:23:32.230 INFO:tasks.workunit.client.1.smithi193.stdout:8/987: truncate df/d10/d67/d7f/fc2 4110065 0 2024-03-23T08:23:32.233 INFO:tasks.workunit.client.1.smithi193.stdout:8/988: stat df/d4b/d4d/c31 0 2024-03-23T08:23:32.250 INFO:tasks.workunit.client.1.smithi193.stdout:6/867: rmdir d0/d27/d81/df/d11/d36 39 2024-03-23T08:23:32.255 INFO:tasks.workunit.client.1.smithi193.stdout:8/989: truncate df/f18 970642 0 2024-03-23T08:23:32.258 INFO:tasks.workunit.client.1.smithi193.stdout:6/868: dwrite d0/d27/d81/df/d11/d36/d61/d3a/fb4 [0,4194304] 0 2024-03-23T08:23:32.258 INFO:tasks.workunit.client.1.smithi193.stdout:8/990: mkdir df/d63/d84/da8/d13e/d149 0 2024-03-23T08:23:32.264 INFO:tasks.workunit.client.1.smithi193.stdout:8/991: rename df/d10/d3e/lf9 to df/d63/d84/dc5/l14a 0 2024-03-23T08:23:32.265 INFO:tasks.workunit.client.1.smithi193.stdout:6/869: dwrite d0/d27/d81/df/d11/d36/f41 [0,4194304] 0 2024-03-23T08:23:32.268 INFO:tasks.workunit.client.1.smithi193.stdout:8/992: chown df/d10/lf5 1 1 2024-03-23T08:23:32.269 INFO:tasks.workunit.client.1.smithi193.stdout:6/870: symlink d0/d27/d81/df/d11/d36/d61/d7c/d69/de1/l10f 0 2024-03-23T08:23:32.270 INFO:tasks.workunit.client.1.smithi193.stdout:6/871: chown d0/d27/d81/df/d11/d3b/d6c/fc9 211 1 2024-03-23T08:23:32.271 INFO:tasks.workunit.client.1.smithi193.stdout:8/993: creat df/d10/f14b x:0 0 0 2024-03-23T08:23:32.274 INFO:tasks.workunit.client.1.smithi193.stdout:8/994: readlink df/d4b/d4d/d2a/d10b/d134/l13c 0 2024-03-23T08:23:32.275 INFO:tasks.workunit.client.1.smithi193.stdout:6/872: fsync d0/d27/d81/f86 0 2024-03-23T08:23:32.277 INFO:tasks.workunit.client.1.smithi193.stdout:8/995: unlink df/d63/d84/df0/l127 0 2024-03-23T08:23:32.280 INFO:tasks.workunit.client.1.smithi193.stdout:6/873: unlink d0/d27/d81/df/d11/d36/d61/d7c/d69/c10e 0 2024-03-23T08:23:32.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/996: fsync df/d4b/f71 0 2024-03-23T08:23:32.282 INFO:tasks.workunit.client.1.smithi193.stdout:8/997: write df/d10/d3e/d54/dd2/f8c [3918705,63202] 0 2024-03-23T08:23:32.284 INFO:tasks.workunit.client.1.smithi193.stdout:6/874: fsync d0/d27/d81/df/d11/d36/d61/f48 0 2024-03-23T08:23:32.285 INFO:tasks.workunit.client.1.smithi193.stdout:8/998: truncate df/d4b/d4d/d9c/fb8 653148 0 2024-03-23T08:23:32.285 INFO:tasks.workunit.client.1.smithi193.stdout:8/999: dread - df/d10/d67/d7f/f13b zero size 2024-03-23T08:23:32.286 INFO:tasks.workunit.client.1.smithi193.stdout:6/875: creat d0/d27/d81/df/d11/d3b/d51/dd6/f110 x:0 0 0 2024-03-23T08:23:32.307 INFO:tasks.workunit.client.1.smithi193.stdout:2/989: sync 2024-03-23T08:23:32.309 INFO:tasks.workunit.client.1.smithi193.stdout:2/990: mkdir d0/d1b/d14/d56/da3/d117 0 2024-03-23T08:23:32.311 INFO:tasks.workunit.client.1.smithi193.stdout:2/991: read d0/d1b/d14/d66/d50/d92/da9/dba/fc7 [2222988,54994] 0 2024-03-23T08:23:32.312 INFO:tasks.workunit.client.1.smithi193.stdout:2/992: fsync d0/d1b/d77/d9b/fbc 0 2024-03-23T08:23:32.315 INFO:tasks.workunit.client.1.smithi193.stdout:2/993: mknod d0/d1b/d14/d66/d4e/dce/c118 0 2024-03-23T08:23:32.319 INFO:tasks.workunit.client.1.smithi193.stdout:2/994: link d0/d1b/d19/c23 d0/dcf/c119 0 2024-03-23T08:23:32.321 INFO:tasks.workunit.client.1.smithi193.stdout:2/995: rename d0/d1b/d19/fe9 to d0/d1b/d14/d66/d4e/dce/f11a 0 2024-03-23T08:23:32.322 INFO:tasks.workunit.client.1.smithi193.stdout:2/996: readlink d0/lc1 0 2024-03-23T08:23:32.323 INFO:tasks.workunit.client.1.smithi193.stdout:2/997: creat d0/d1b/d14/da1/dd5/d72/d10b/f11b x:0 0 0 2024-03-23T08:23:32.324 INFO:tasks.workunit.client.1.smithi193.stdout:2/998: chown d0/d1b/d14/d25 1831117 1 2024-03-23T08:23:32.326 INFO:tasks.workunit.client.1.smithi193.stdout:2/999: mkdir d0/d1b/d14/da1/dd5/db7/dc0/d11c 0 2024-03-23T08:23:32.402 INFO:tasks.workunit.client.1.smithi193.stdout:7/961: sync 2024-03-23T08:23:32.404 INFO:tasks.workunit.client.1.smithi193.stdout:7/962: mkdir d6/d9/d1f/d23/d33/d38/d3b/db7/d143/d149 0 2024-03-23T08:23:32.410 INFO:tasks.workunit.client.1.smithi193.stdout:7/963: dwrite d6/d9/d1f/d3e/f5e [0,4194304] 0 2024-03-23T08:23:32.451 INFO:tasks.workunit.client.1.smithi193.stdout:3/842: sync 2024-03-23T08:23:32.451 INFO:tasks.workunit.client.1.smithi193.stdout:6/876: sync 2024-03-23T08:23:32.516 INFO:tasks.workunit.client.1.smithi193.stdout:7/964: truncate d6/d9/d1f/d3e/d42/f59 2098369 0 2024-03-23T08:23:32.520 INFO:tasks.workunit.client.1.smithi193.stdout:7/965: getdents d6/d9/d1f/d23/d33/d38/d66/dac 0 2024-03-23T08:23:32.522 INFO:tasks.workunit.client.1.smithi193.stdout:7/966: fsync d6/d9/d1f/d3e/d65/d70/ffc 0 2024-03-23T08:23:32.523 INFO:tasks.workunit.client.1.smithi193.stdout:7/967: stat d6/d9/d1f/d23/d69/ca8 0 2024-03-23T08:23:32.526 INFO:tasks.workunit.client.1.smithi193.stdout:7/968: creat d6/d9/d1f/d23/d33/d3c/f14a x:0 0 0 2024-03-23T08:23:32.531 INFO:tasks.workunit.client.1.smithi193.stdout:6/877: unlink d0/d27/d81/df/d11/d36/d61/l22 0 2024-03-23T08:23:32.533 INFO:tasks.workunit.client.1.smithi193.stdout:7/969: link d6/d9/d1f/d23/d33/d38/dc0/le4 d6/d9/d1f/d23/d33/d3c/d41/l14b 0 2024-03-23T08:23:32.535 INFO:tasks.workunit.client.1.smithi193.stdout:3/843: mkdir d1/de/d13/d11d 0 2024-03-23T08:23:32.549 INFO:tasks.workunit.client.1.smithi193.stdout:7/970: truncate d6/d9/f31 545220 0 2024-03-23T08:23:32.554 INFO:tasks.workunit.client.1.smithi193.stdout:3/844: dwrite d1/de/d13/d21/d25/d2b/f4d [0,4194304] 0 2024-03-23T08:23:32.555 INFO:tasks.workunit.client.1.smithi193.stdout:7/971: getdents d6/d9/d1f/d23/d33/d38/d3b 0 2024-03-23T08:23:32.556 INFO:tasks.workunit.client.1.smithi193.stdout:3/845: write d1/de/d13/d21/d25/d2b/f4d [4895708,124091] 0 2024-03-23T08:23:32.559 INFO:tasks.workunit.client.1.smithi193.stdout:3/846: truncate d1/de/d13/d21/d50/d6f/fa7 4406406 0 2024-03-23T08:23:32.565 INFO:tasks.workunit.client.1.smithi193.stdout:7/972: dread d6/d9/d1f/d23/d33/d38/d3b/d6c/f8d [0,4194304] 0 2024-03-23T08:23:32.566 INFO:tasks.workunit.client.1.smithi193.stdout:3/847: dwrite d1/de/d28/fad [0,4194304] 0 2024-03-23T08:23:32.567 INFO:tasks.workunit.client.1.smithi193.stdout:7/973: write d6/d9/d1f/d23/d33/d38/d3b/f13b [464744,55014] 0 2024-03-23T08:23:32.570 INFO:tasks.workunit.client.1.smithi193.stdout:3/848: symlink d1/de/d13/d21/l11e 0 2024-03-23T08:23:32.573 INFO:tasks.workunit.client.1.smithi193.stdout:3/849: unlink d1/de/d28/fad 0 2024-03-23T08:23:32.577 INFO:tasks.workunit.client.1.smithi193.stdout:7/974: dwrite d6/d9/d1f/d23/d33/d3c/d41/d47/f146 [0,4194304] 0 2024-03-23T08:23:32.581 INFO:tasks.workunit.client.1.smithi193.stdout:7/975: unlink d6/d9/d1f/d23/f45 0 2024-03-23T08:23:32.588 INFO:tasks.workunit.client.1.smithi193.stdout:7/976: dwrite d6/d9/d1f/d23/d33/d38/ff9 [0,4194304] 0 2024-03-23T08:23:32.589 INFO:tasks.workunit.client.1.smithi193.stdout:3/850: symlink d1/de/d13/d21/d50/d6f/l11f 0 2024-03-23T08:23:32.590 INFO:tasks.workunit.client.1.smithi193.stdout:7/977: chown d6/d9/d1f/d23/d33/d129/c13c 978490542 1 2024-03-23T08:23:32.591 INFO:tasks.workunit.client.1.smithi193.stdout:3/851: mknod d1/de/d28/c120 0 2024-03-23T08:23:32.592 INFO:tasks.workunit.client.1.smithi193.stdout:7/978: mkdir d6/d9/d1f/d23/d69/d14c 0 2024-03-23T08:23:32.596 INFO:tasks.workunit.client.1.smithi193.stdout:3/852: creat d1/de/d13/f121 x:0 0 0 2024-03-23T08:23:32.597 INFO:tasks.workunit.client.1.smithi193.stdout:3/853: write d1/f20 [323274,64206] 0 2024-03-23T08:23:32.601 INFO:tasks.workunit.client.1.smithi193.stdout:3/854: creat d1/de/d13/d21/d25/d2b/d3c/d4a/dae/dd9/f122 x:0 0 0 2024-03-23T08:23:32.604 INFO:tasks.workunit.client.1.smithi193.stdout:7/979: dwrite d6/d9/d1f/d23/d33/d38/f44 [0,4194304] 0 2024-03-23T08:23:32.609 INFO:tasks.workunit.client.1.smithi193.stdout:3/855: rename d1/de/d13/d21/d25/d34/c8b to d1/de/d13/d21/d50/d11b/c123 0 2024-03-23T08:23:32.619 INFO:tasks.workunit.client.1.smithi193.stdout:3/856: dread d1/de/d13/d21/d25/d34/f47 [0,4194304] 0 2024-03-23T08:23:32.633 INFO:tasks.workunit.client.1.smithi193.stdout:3/857: dread d1/de/d13/d96/d99/f64 [0,4194304] 0 2024-03-23T08:23:32.634 INFO:tasks.workunit.client.1.smithi193.stdout:3/858: chown d1/de/d13/d21/d25/d2b/d3c/l43 1 1 2024-03-23T08:23:32.635 INFO:tasks.workunit.client.1.smithi193.stdout:3/859: truncate d1/de/f16 37338 0 2024-03-23T08:23:32.641 INFO:tasks.workunit.client.1.smithi193.stdout:3/860: dwrite d1/de/d13/f88 [0,4194304] 0 2024-03-23T08:23:32.685 INFO:tasks.workunit.client.1.smithi193.stdout:6/878: sync 2024-03-23T08:23:32.686 INFO:tasks.workunit.client.1.smithi193.stdout:6/879: chown d0/d27/d81/da1/fb5 38421512 1 2024-03-23T08:23:32.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:32 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:23:32.760 INFO:tasks.workunit.client.1.smithi193.stdout:3/861: sync 2024-03-23T08:23:32.760 INFO:tasks.workunit.client.1.smithi193.stdout:7/980: sync 2024-03-23T08:23:32.762 INFO:tasks.workunit.client.1.smithi193.stdout:3/862: write d1/de/d13/d21/d25/d34/f8f [959073,74857] 0 2024-03-23T08:23:32.763 INFO:tasks.workunit.client.1.smithi193.stdout:3/863: chown d1/de/d13/d96/d99/fee 8060254 1 2024-03-23T08:23:32.764 INFO:tasks.workunit.client.1.smithi193.stdout:3/864: chown d1/de/d13/d21/d25/d2b/d3c/d4a/dae/df5 60307 1 2024-03-23T08:23:32.764 INFO:tasks.workunit.client.1.smithi193.stdout:7/981: getdents d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/d112 0 2024-03-23T08:23:32.767 INFO:tasks.workunit.client.1.smithi193.stdout:7/982: creat d6/d9/d1f/d23/d33/d3c/d41/d47/d13d/f14d x:0 0 0 2024-03-23T08:23:32.774 INFO:tasks.workunit.client.1.smithi193.stdout:3/865: dwrite d1/de/f71 [4194304,4194304] 0 2024-03-23T08:23:32.774 INFO:tasks.workunit.client.1.smithi193.stdout:3/866: fdatasync d1/de/d13/f121 0 2024-03-23T08:23:32.787 INFO:tasks.workunit.client.1.smithi193.stdout:6/880: sync 2024-03-23T08:23:32.788 INFO:tasks.workunit.client.1.smithi193.stdout:6/881: stat d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/db1 0 2024-03-23T08:23:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:32 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:23:32.851 INFO:tasks.workunit.client.1.smithi193.stdout:7/983: rmdir d6/d9/d1f/d23/d33/d142 0 2024-03-23T08:23:32.852 INFO:tasks.workunit.client.1.smithi193.stdout:7/984: write d6/d9/d1f/d4d/d6b/f11a [502248,100549] 0 2024-03-23T08:23:32.856 INFO:tasks.workunit.client.1.smithi193.stdout:3/867: rename d1/de/d13/l108 to d1/de/d13/d21/d59/dc6/l124 0 2024-03-23T08:23:32.858 INFO:tasks.workunit.client.1.smithi193.stdout:7/985: fdatasync d6/d9/d1f/d3e/d65/d7d/f9f 0 2024-03-23T08:23:32.859 INFO:tasks.workunit.client.1.smithi193.stdout:3/868: mkdir d1/de/d13/d96/d125 0 2024-03-23T08:23:32.860 INFO:tasks.workunit.client.1.smithi193.stdout:6/882: getdents d0/d27/d81/df/d11/d36/d61/d7c 0 2024-03-23T08:23:32.865 INFO:tasks.workunit.client.1.smithi193.stdout:6/883: getdents d0/d27/d81/df/d11/d3b/df0 0 2024-03-23T08:23:32.867 INFO:tasks.workunit.client.1.smithi193.stdout:6/884: creat d0/d27/d81/df/d11/d56/f111 x:0 0 0 2024-03-23T08:23:32.872 INFO:tasks.workunit.client.1.smithi193.stdout:7/986: dwrite d6/d9/d1f/d23/d33/d38/dc0/f110 [4194304,4194304] 0 2024-03-23T08:23:32.875 INFO:tasks.workunit.client.1.smithi193.stdout:3/869: dread d1/de/d13/d96/d99/f30 [0,4194304] 0 2024-03-23T08:23:32.878 INFO:tasks.workunit.client.1.smithi193.stdout:7/987: read d6/d9/d1f/d23/d33/d3c/d41/d47/fcc [46047,31764] 0 2024-03-23T08:23:32.882 INFO:tasks.workunit.client.1.smithi193.stdout:6/885: dwrite d0/d27/d81/df/d11/d36/f6e [0,4194304] 0 2024-03-23T08:23:32.882 INFO:tasks.workunit.client.1.smithi193.stdout:7/988: rmdir d6/d9/d1f/d23/d33/d38/d46 39 2024-03-23T08:23:32.882 INFO:tasks.workunit.client.1.smithi193.stdout:6/886: dread - d0/d27/d81/df/d11/d3b/fbc zero size 2024-03-23T08:23:32.886 INFO:tasks.workunit.client.1.smithi193.stdout:3/870: dwrite d1/de/d13/d21/d25/d2b/d3c/d119/f11c [0,4194304] 0 2024-03-23T08:23:32.888 INFO:tasks.workunit.client.1.smithi193.stdout:6/887: unlink d0/d27/d81/df/d11/ld9 0 2024-03-23T08:23:32.889 INFO:tasks.workunit.client.1.smithi193.stdout:7/989: unlink d6/d9/d1f/d4d/d7c/cf8 0 2024-03-23T08:23:32.890 INFO:tasks.workunit.client.1.smithi193.stdout:3/871: mknod d1/de/d28/c126 0 2024-03-23T08:23:32.891 INFO:tasks.workunit.client.1.smithi193.stdout:6/888: mkdir d0/d27/d81/df/dae/d112 0 2024-03-23T08:23:32.897 INFO:tasks.workunit.client.1.smithi193.stdout:3/872: creat d1/de/d13/d21/d25/d2b/f127 x:0 0 0 2024-03-23T08:23:32.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/889: rename d0/d27/d81/da1/lad to d0/d27/d81/df/d11/d3b/d51/l113 0 2024-03-23T08:23:32.903 INFO:tasks.workunit.client.1.smithi193.stdout:7/990: dwrite d6/d9/f15 [0,4194304] 0 2024-03-23T08:23:32.917 INFO:tasks.workunit.client.1.smithi193.stdout:6/890: dwrite d0/d27/d81/da1/fb5 [0,4194304] 0 2024-03-23T08:23:32.919 INFO:tasks.workunit.client.1.smithi193.stdout:3/873: truncate d1/de/f10 7597098 0 2024-03-23T08:23:32.925 INFO:tasks.workunit.client.1.smithi193.stdout:6/891: dread d0/d27/d81/df/d11/d36/d61/d12/ff2 [0,4194304] 0 2024-03-23T08:23:32.951 INFO:tasks.workunit.client.1.smithi193.stdout:7/991: creat d6/d9/d1f/d23/d33/d3c/d41/f14e x:0 0 0 2024-03-23T08:23:32.953 INFO:tasks.workunit.client.1.smithi193.stdout:7/992: dread - d6/d9/d1f/d23/d33/d3c/d41/d47/dc7/d112/f11f zero size 2024-03-23T08:23:32.958 INFO:tasks.workunit.client.1.smithi193.stdout:6/892: symlink d0/d27/d81/df/d11/d36/d61/d3a/d8f/de2/l114 0 2024-03-23T08:23:32.960 INFO:tasks.workunit.client.1.smithi193.stdout:7/993: creat d6/d9/d1f/d23/d33/d3c/d120/f14f x:0 0 0 2024-03-23T08:23:32.965 INFO:tasks.workunit.client.1.smithi193.stdout:7/994: dread d6/d9/d1f/d5f/f8a [0,4194304] 0 2024-03-23T08:23:32.973 INFO:tasks.workunit.client.1.smithi193.stdout:3/874: truncate d1/de/d13/d21/d25/d2b/d3c/d4a/f93 8171243 0 2024-03-23T08:23:32.974 INFO:tasks.workunit.client.1.smithi193.stdout:6/893: write d0/d27/d81/df/d11/d56/f5c [472013,119459] 0 2024-03-23T08:23:32.975 INFO:tasks.workunit.client.1.smithi193.stdout:7/995: truncate d6/d9/f15 2107135 0 2024-03-23T08:23:32.975 INFO:tasks.workunit.client.1.smithi193.stdout:3/875: write d1/de/d13/d21/d25/d2b/d3c/d4a/dae/f10b [525796,57434] 0 2024-03-23T08:23:32.977 INFO:tasks.workunit.client.1.smithi193.stdout:6/894: readlink d0/d27/d81/df/d11/d3b/d51/daa/l101 0 2024-03-23T08:23:32.978 INFO:tasks.workunit.client.1.smithi193.stdout:6/895: fsync d0/d27/d81/f4e 0 2024-03-23T08:23:32.979 INFO:tasks.workunit.client.1.smithi193.stdout:6/896: readlink d0/l3 0 2024-03-23T08:23:32.981 INFO:tasks.workunit.client.1.smithi193.stdout:6/897: write d0/d27/d81/df/d11/d3b/d6c/f90 [557506,120650] 0 2024-03-23T08:23:32.983 INFO:tasks.workunit.client.1.smithi193.stdout:6/898: truncate d0/d27/d81/df/d11/d36/f10b 716790 0 2024-03-23T08:23:32.985 INFO:tasks.workunit.client.1.smithi193.stdout:6/899: dread - d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f10c zero size 2024-03-23T08:23:32.988 INFO:tasks.workunit.client.1.smithi193.stdout:7/996: dread d6/d9/d1f/d23/d33/d38/d3b/f5a [0,4194304] 0 2024-03-23T08:23:32.990 INFO:tasks.workunit.client.1.smithi193.stdout:7/997: read - d6/d9/d1f/d23/d33/d38/f53 zero size 2024-03-23T08:23:32.992 INFO:tasks.workunit.client.1.smithi193.stdout:7/998: write d6/d9/d1f/d23/d33/d3c/d120/f14f [520411,117883] 0 2024-03-23T08:23:33.002 INFO:tasks.workunit.client.1.smithi193.stdout:7/999: dwrite d6/d9/d1f/d23/d33/d38/d46/f5b [0,4194304] 0 2024-03-23T08:23:33.006 INFO:tasks.workunit.client.1.smithi193.stdout:6/900: dread d0/d27/d81/df/d11/d36/fa3 [0,4194304] 0 2024-03-23T08:23:33.009 INFO:tasks.workunit.client.1.smithi193.stdout:6/901: mknod d0/d27/d81/c115 0 2024-03-23T08:23:33.015 INFO:tasks.workunit.client.1.smithi193.stdout:6/902: dwrite d0/d27/d81/df/d11/d36/ff9 [0,4194304] 0 2024-03-23T08:23:33.018 INFO:tasks.workunit.client.1.smithi193.stdout:6/903: mkdir d0/d27/d81/df/d11/d3b/d51/d9c/d116 0 2024-03-23T08:23:33.025 INFO:tasks.workunit.client.1.smithi193.stdout:6/904: dwrite d0/d27/d81/df/d11/d36/d61/d3a/f45 [4194304,4194304] 0 2024-03-23T08:23:33.033 INFO:tasks.workunit.client.1.smithi193.stdout:6/905: dwrite d0/d27/d81/df/d11/d3b/d51/d9c/f96 [0,4194304] 0 2024-03-23T08:23:33.040 INFO:tasks.workunit.client.1.smithi193.stdout:6/906: dwrite d0/d27/d81/fc5 [0,4194304] 0 2024-03-23T08:23:33.048 INFO:tasks.workunit.client.1.smithi193.stdout:6/907: write d0/f8b [5325529,8074] 0 2024-03-23T08:23:33.054 INFO:tasks.workunit.client.1.smithi193.stdout:6/908: dwrite d0/d27/d81/d87/fa8 [0,4194304] 0 2024-03-23T08:23:33.076 INFO:tasks.workunit.client.1.smithi193.stdout:6/909: dwrite d0/d27/d81/df/d11/d36/d61/f48 [0,4194304] 0 2024-03-23T08:23:33.085 INFO:tasks.workunit.client.1.smithi193.stdout:6/910: dread d0/d27/d81/df/d11/d56/f5c [0,4194304] 0 2024-03-23T08:23:33.089 INFO:tasks.workunit.client.1.smithi193.stdout:6/911: link d0/d27/d81/df/d11/d36/d61/d7c/d69/de1/l10f d0/d27/d81/df/d11/l117 0 2024-03-23T08:23:33.090 INFO:tasks.workunit.client.1.smithi193.stdout:6/912: write d0/d27/d81/df/d11/d36/f6e [1784735,106124] 0 2024-03-23T08:23:33.092 INFO:tasks.workunit.client.1.smithi193.stdout:6/913: mkdir d0/d27/d118 0 2024-03-23T08:23:33.094 INFO:tasks.workunit.client.1.smithi193.stdout:6/914: mknod d0/d27/d81/df/d11/d56/dfc/c119 0 2024-03-23T08:23:33.097 INFO:tasks.workunit.client.1.smithi193.stdout:6/915: dread d0/fc6 [0,4194304] 0 2024-03-23T08:23:33.101 INFO:tasks.workunit.client.1.smithi193.stdout:6/916: dread d0/d27/d81/df/d11/d3b/f49 [0,4194304] 0 2024-03-23T08:23:33.103 INFO:tasks.workunit.client.1.smithi193.stdout:6/917: mkdir d0/d27/d81/df/d11/d36/d61/d7c/d69/de1/d11a 0 2024-03-23T08:23:33.105 INFO:tasks.workunit.client.1.smithi193.stdout:6/918: truncate d0/d27/d81/df/d11/d36/f10b 1257230 0 2024-03-23T08:23:33.106 INFO:tasks.workunit.client.1.smithi193.stdout:6/919: creat d0/d27/d81/df/dae/f11b x:0 0 0 2024-03-23T08:23:33.107 INFO:tasks.workunit.client.1.smithi193.stdout:6/920: write d0/d27/d81/df/d11/d36/d61/d7c/fbf [5088221,95764] 0 2024-03-23T08:23:33.110 INFO:tasks.workunit.client.1.smithi193.stdout:6/921: symlink d0/d27/d81/df/d11/d3b/d51/l11c 0 2024-03-23T08:23:33.121 INFO:tasks.workunit.client.1.smithi193.stdout:6/922: dread d0/d27/d81/df/d11/d3b/d6c/f90 [0,4194304] 0 2024-03-23T08:23:33.123 INFO:tasks.workunit.client.1.smithi193.stdout:6/923: symlink d0/d27/d81/df/d11/d36/d61/d75/l11d 0 2024-03-23T08:23:33.125 INFO:tasks.workunit.client.1.smithi193.stdout:6/924: mkdir d0/d27/d81/df/d11/d36/d61/d3a/df3/d11e 0 2024-03-23T08:23:33.127 INFO:tasks.workunit.client.1.smithi193.stdout:6/925: mknod d0/d27/d81/df/d11/d3b/c11f 0 2024-03-23T08:23:33.129 INFO:tasks.workunit.client.1.smithi193.stdout:6/926: getdents d0/d27/d81/df/d11/d36/d61/d7c/d102 0 2024-03-23T08:23:33.131 INFO:tasks.workunit.client.1.smithi193.stdout:6/927: mknod d0/d27/d81/df/d11/d3b/d51/daa/c120 0 2024-03-23T08:23:33.136 INFO:tasks.workunit.client.1.smithi193.stdout:6/928: dwrite d0/d27/d81/df/d11/d36/d61/d12/f35 [0,4194304] 0 2024-03-23T08:23:33.139 INFO:tasks.workunit.client.1.smithi193.stdout:6/929: creat d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/d9d/f121 x:0 0 0 2024-03-23T08:23:33.141 INFO:tasks.workunit.client.1.smithi193.stdout:6/930: chown d0/d27/d81/df/d11/d56/fd1 128 1 2024-03-23T08:23:33.148 INFO:tasks.workunit.client.1.smithi193.stdout:6/931: write d0/d27/d81/df/d11/d3b/d51/daa/fca [727688,119528] 0 2024-03-23T08:23:33.149 INFO:tasks.workunit.client.1.smithi193.stdout:6/932: readlink d0/l23 0 2024-03-23T08:23:33.151 INFO:tasks.workunit.client.1.smithi193.stdout:6/933: rmdir d0/d27/d81/df/d11/d36/d61/d75 39 2024-03-23T08:23:33.153 INFO:tasks.workunit.client.1.smithi193.stdout:3/876: sync 2024-03-23T08:23:33.153 INFO:tasks.workunit.client.1.smithi193.stdout:6/934: creat d0/d27/d81/dce/f122 x:0 0 0 2024-03-23T08:23:33.154 INFO:tasks.workunit.client.1.smithi193.stdout:6/935: chown d0/d27/d81/df/d11/d36/d61/d12 7253 1 2024-03-23T08:23:33.161 INFO:tasks.workunit.client.1.smithi193.stdout:3/877: write d1/de/d13/d21/d25/d2b/f39 [594264,87704] 0 2024-03-23T08:23:33.162 INFO:tasks.workunit.client.1.smithi193.stdout:3/878: chown d1/de/d13/d21/d50/fc8 0 1 2024-03-23T08:23:33.165 INFO:tasks.workunit.client.1.smithi193.stdout:3/879: dread d1/de/d13/d21/d50/d6f/d6e/f72 [4194304,4194304] 0 2024-03-23T08:23:33.168 INFO:tasks.workunit.client.1.smithi193.stdout:3/880: unlink d1/de/d13/d21/d25/d2b/f127 0 2024-03-23T08:23:33.171 INFO:tasks.workunit.client.1.smithi193.stdout:3/881: dread d1/f20 [0,4194304] 0 2024-03-23T08:23:33.173 INFO:tasks.workunit.client.1.smithi193.stdout:3/882: creat d1/de/d13/d96/f128 x:0 0 0 2024-03-23T08:23:33.174 INFO:tasks.workunit.client.1.smithi193.stdout:3/883: truncate d1/de/d28/fc5 5159941 0 2024-03-23T08:23:33.176 INFO:tasks.workunit.client.1.smithi193.stdout:3/884: truncate d1/de/f10 4095099 0 2024-03-23T08:23:33.188 INFO:tasks.workunit.client.1.smithi193.stdout:3/885: dread d1/de/d13/d21/d25/d2b/d3c/d4a/f93 [0,4194304] 0 2024-03-23T08:23:33.191 INFO:tasks.workunit.client.1.smithi193.stdout:3/886: rename d1/de/d28/fec to d1/de/d13/d21/d59/dc6/f129 0 2024-03-23T08:23:33.193 INFO:tasks.workunit.client.1.smithi193.stdout:3/887: write d1/de/d13/d21/f23 [2947005,62490] 0 2024-03-23T08:23:33.196 INFO:tasks.workunit.client.1.smithi193.stdout:3/888: fsync d1/de/d13/d21/d27/f1f 0 2024-03-23T08:23:33.196 INFO:tasks.workunit.client.1.smithi193.stdout:3/889: chown d1/c6 877582 1 2024-03-23T08:23:33.199 INFO:tasks.workunit.client.1.smithi193.stdout:3/890: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/d85/db9/d12a 0 2024-03-23T08:23:33.210 INFO:tasks.workunit.client.1.smithi193.stdout:3/891: dread d1/de/d13/d21/f23 [0,4194304] 0 2024-03-23T08:23:33.212 INFO:tasks.workunit.client.1.smithi193.stdout:3/892: creat d1/de/d13/d21/d59/dc6/d103/f12b x:0 0 0 2024-03-23T08:23:33.216 INFO:tasks.workunit.client.1.smithi193.stdout:3/893: rename d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/f106 to d1/de/d13/d21/d25/d2b/d3c/d4a/dae/dd9/f12c 0 2024-03-23T08:23:33.218 INFO:tasks.workunit.client.1.smithi193.stdout:3/894: truncate d1/de/d13/d21/ff4 2019948 0 2024-03-23T08:23:33.220 INFO:tasks.workunit.client.1.smithi193.stdout:3/895: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/f12d x:0 0 0 2024-03-23T08:23:33.221 INFO:tasks.workunit.client.1.smithi193.stdout:3/896: fdatasync d1/de/d13/d21/d25/d2b/d3c/d4a/ff9 0 2024-03-23T08:23:33.222 INFO:tasks.workunit.client.1.smithi193.stdout:3/897: truncate d1/de/d13/d21/f23 4553798 0 2024-03-23T08:23:33.227 INFO:tasks.workunit.client.1.smithi193.stdout:3/898: dwrite d1/fbd [0,4194304] 0 2024-03-23T08:23:33.246 INFO:tasks.workunit.client.1.smithi193.stdout:3/899: dread d1/de/f70 [0,4194304] 0 2024-03-23T08:23:33.248 INFO:tasks.workunit.client.1.smithi193.stdout:3/900: mknod d1/de/d13/d21/d25/d2b/d3c/d4a/de4/d105/c12e 0 2024-03-23T08:23:33.256 INFO:tasks.workunit.client.1.smithi193.stdout:3/901: getdents d1/de/d13/d21/d59/dc6 0 2024-03-23T08:23:33.257 INFO:tasks.workunit.client.1.smithi193.stdout:3/902: dread - d1/de/d13/d21/d25/d2b/d3c/d4a/de4/d105/deb/f109 zero size 2024-03-23T08:23:33.261 INFO:tasks.workunit.client.1.smithi193.stdout:3/903: creat d1/de/d13/d21/d25/f12f x:0 0 0 2024-03-23T08:23:33.264 INFO:tasks.workunit.client.1.smithi193.stdout:3/904: write d1/de/d13/d21/f81 [4983742,43910] 0 2024-03-23T08:23:33.265 INFO:tasks.workunit.client.1.smithi193.stdout:3/905: fdatasync d1/fc 0 2024-03-23T08:23:33.267 INFO:tasks.workunit.client.1.smithi193.stdout:3/906: truncate d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/fc4 4770240 0 2024-03-23T08:23:33.269 INFO:tasks.workunit.client.1.smithi193.stdout:3/907: truncate d1/de/d13/d21/d25/d2b/d3c/d4a/dae/dd9/f12c 237484 0 2024-03-23T08:23:33.273 INFO:tasks.workunit.client.1.smithi193.stdout:3/908: write d1/de/d13/d21/d25/d2b/f7f [99364,32879] 0 2024-03-23T08:23:33.275 INFO:tasks.workunit.client.1.smithi193.stdout:3/909: symlink d1/de/d13/d21/d25/d2b/d3c/d119/l130 0 2024-03-23T08:23:33.278 INFO:tasks.workunit.client.1.smithi193.stdout:6/936: sync 2024-03-23T08:23:33.280 INFO:tasks.workunit.client.1.smithi193.stdout:3/910: dwrite d1/de/d13/d21/d59/fde [0,4194304] 0 2024-03-23T08:23:33.282 INFO:tasks.workunit.client.1.smithi193.stdout:6/937: rmdir d0/d27/d81/df/d11/d36/d61/d12/d21/d2a 39 2024-03-23T08:23:33.285 INFO:tasks.workunit.client.1.smithi193.stdout:6/938: truncate d0/d27/d81/da1/da9/fe9 1749919 0 2024-03-23T08:23:33.285 INFO:tasks.workunit.client.1.smithi193.stdout:3/911: truncate d1/de/f10 1954261 0 2024-03-23T08:23:33.286 INFO:tasks.workunit.client.1.smithi193.stdout:6/939: fdatasync d0/d27/d81/da1/fd8 0 2024-03-23T08:23:33.288 INFO:tasks.workunit.client.1.smithi193.stdout:3/912: mknod d1/de/d13/d21/d25/d2b/d3c/d4a/de4/d105/c131 0 2024-03-23T08:23:33.289 INFO:tasks.workunit.client.1.smithi193.stdout:6/940: write d0/d27/d81/df/d11/d3b/d6c/f90 [3702474,110529] 0 2024-03-23T08:23:33.290 INFO:tasks.workunit.client.1.smithi193.stdout:6/941: readlink d0/d27/d81/df/d11/d36/d61/d3a/d8f/de2/l114 0 2024-03-23T08:23:33.292 INFO:tasks.workunit.client.1.smithi193.stdout:3/913: read d1/de/d13/d21/d27/f63 [748720,73508] 0 2024-03-23T08:23:33.293 INFO:tasks.workunit.client.1.smithi193.stdout:6/942: dread d0/d27/d81/da1/fd3 [0,4194304] 0 2024-03-23T08:23:33.294 INFO:tasks.workunit.client.1.smithi193.stdout:3/914: chown d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/cb6 4 1 2024-03-23T08:23:33.294 INFO:tasks.workunit.client.1.smithi193.stdout:6/943: chown d0/d27/d81/da1/db8/fdf 12716 1 2024-03-23T08:23:33.295 INFO:tasks.workunit.client.1.smithi193.stdout:3/915: chown d1/de/d13/f88 0 1 2024-03-23T08:23:33.296 INFO:tasks.workunit.client.1.smithi193.stdout:6/944: creat d0/d27/d81/df/d11/d56/dfc/f123 x:0 0 0 2024-03-23T08:23:33.297 INFO:tasks.workunit.client.1.smithi193.stdout:6/945: readlink d0/d27/d81/df/d11/d56/l71 0 2024-03-23T08:23:33.299 INFO:tasks.workunit.client.1.smithi193.stdout:6/946: write d0/d27/d81/df/d11/d36/f6e [4555644,87383] 0 2024-03-23T08:23:33.300 INFO:tasks.workunit.client.1.smithi193.stdout:3/916: dwrite d1/de/f11 [4194304,4194304] 0 2024-03-23T08:23:33.303 INFO:tasks.workunit.client.1.smithi193.stdout:3/917: creat d1/f132 x:0 0 0 2024-03-23T08:23:33.307 INFO:tasks.workunit.client.1.smithi193.stdout:3/918: truncate d1/de/d13/d21/d27/f63 1219705 0 2024-03-23T08:23:33.307 INFO:tasks.workunit.client.1.smithi193.stdout:3/919: chown d1/de/c12 51 1 2024-03-23T08:23:33.311 INFO:tasks.workunit.client.1.smithi193.stdout:3/920: unlink d1/de/d13/d21/d25/d2b/f7f 0 2024-03-23T08:23:33.312 INFO:tasks.workunit.client.1.smithi193.stdout:6/947: write d0/d27/d81/df/f9b [365382,1700] 0 2024-03-23T08:23:33.316 INFO:tasks.workunit.client.1.smithi193.stdout:3/921: unlink d1/de/d13/f5b 0 2024-03-23T08:23:33.317 INFO:tasks.workunit.client.1.smithi193.stdout:6/948: fsync d0/d27/d81/df/d11/d56/f5c 0 2024-03-23T08:23:33.323 INFO:tasks.workunit.client.1.smithi193.stdout:3/922: link d1/de/d13/d21/d59/d60/fb7 d1/de/d13/d21/d27/d7c/f133 0 2024-03-23T08:23:33.331 INFO:tasks.workunit.client.1.smithi193.stdout:6/949: dread d0/d27/d81/df/f4f [0,4194304] 0 2024-03-23T08:23:33.340 INFO:tasks.workunit.client.1.smithi193.stdout:6/950: dread d0/d27/d81/df/d11/d36/d61/f6d [0,4194304] 0 2024-03-23T08:23:33.416 INFO:tasks.workunit.client.1.smithi193.stdout:3/923: sync 2024-03-23T08:23:33.418 INFO:tasks.workunit.client.1.smithi193.stdout:3/924: symlink d1/de/d13/d21/d59/df2/l134 0 2024-03-23T08:23:33.420 INFO:tasks.workunit.client.1.smithi193.stdout:3/925: mkdir d1/de/d13/d21/d59/dc6/d135 0 2024-03-23T08:23:33.429 INFO:tasks.workunit.client.1.smithi193.stdout:3/926: dwrite d1/de/d13/d21/d50/d44/f4f [0,4194304] 0 2024-03-23T08:23:33.438 INFO:tasks.workunit.client.1.smithi193.stdout:3/927: dwrite d1/de/d13/d21/d25/d2b/d3c/d4a/de4/dd1/f113 [4194304,4194304] 0 2024-03-23T08:23:33.441 INFO:tasks.workunit.client.1.smithi193.stdout:3/928: dread d1/de/d13/d21/d25/d2b/d3c/d4a/dae/dd9/f12c [0,4194304] 0 2024-03-23T08:23:33.445 INFO:tasks.workunit.client.1.smithi193.stdout:3/929: dread d1/de/d13/d21/d25/d2b/d3c/d4a/dae/f10b [0,4194304] 0 2024-03-23T08:23:33.447 INFO:tasks.workunit.client.1.smithi193.stdout:3/930: creat d1/de/d28/d104/f136 x:0 0 0 2024-03-23T08:23:33.449 INFO:tasks.workunit.client.1.smithi193.stdout:3/931: dread - d1/de/d13/d21/d25/d2b/d3c/d4a/dae/f111 zero size 2024-03-23T08:23:33.454 INFO:tasks.workunit.client.1.smithi193.stdout:6/951: sync 2024-03-23T08:23:33.457 INFO:tasks.workunit.client.1.smithi193.stdout:6/952: creat d0/d27/d81/df/d11/d36/d61/d7c/f124 x:0 0 0 2024-03-23T08:23:33.459 INFO:tasks.workunit.client.1.smithi193.stdout:6/953: mknod d0/d27/d81/df/d11/d56/c125 0 2024-03-23T08:23:33.461 INFO:tasks.workunit.client.1.smithi193.stdout:6/954: unlink d0/d27/d81/df/d11/f37 0 2024-03-23T08:23:33.463 INFO:tasks.workunit.client.1.smithi193.stdout:3/932: dread d1/de/d13/d21/d50/d44/f79 [0,4194304] 0 2024-03-23T08:23:33.467 INFO:tasks.workunit.client.1.smithi193.stdout:6/955: write d0/d27/d81/df/d11/d3b/d51/daa/fb6 [884756,63212] 0 2024-03-23T08:23:33.471 INFO:tasks.workunit.client.1.smithi193.stdout:3/933: dread d1/de/d13/d21/d59/d60/fd2 [0,4194304] 0 2024-03-23T08:23:33.472 INFO:tasks.workunit.client.1.smithi193.stdout:6/956: chown d0/d27/d81/df/d11/d36/d61/d3a/d8f/le3 534 1 2024-03-23T08:23:33.477 INFO:tasks.workunit.client.1.smithi193.stdout:6/957: creat d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/dfd/f126 x:0 0 0 2024-03-23T08:23:33.478 INFO:tasks.workunit.client.1.smithi193.stdout:6/958: readlink d0/d27/d81/df/d11/d36/d61/d3a/df3/l108 0 2024-03-23T08:23:33.480 INFO:tasks.workunit.client.1.smithi193.stdout:6/959: mkdir d0/d27/d81/dce/d127 0 2024-03-23T08:23:33.482 INFO:tasks.workunit.client.1.smithi193.stdout:6/960: creat d0/d27/d81/df/d11/d3b/d51/db2/f128 x:0 0 0 2024-03-23T08:23:33.484 INFO:tasks.workunit.client.1.smithi193.stdout:6/961: mknod d0/d27/d81/da1/da9/c129 0 2024-03-23T08:23:33.487 INFO:tasks.workunit.client.1.smithi193.stdout:6/962: unlink d0/d27/d81/df/d11/d36/d61/d7c/d69/de1/l10f 0 2024-03-23T08:23:33.492 INFO:tasks.workunit.client.1.smithi193.stdout:6/963: write d0/d27/d81/df/d11/d56/fd1 [5445035,30252] 0 2024-03-23T08:23:33.499 INFO:tasks.workunit.client.1.smithi193.stdout:6/964: truncate d0/d27/d81/df/d11/d36/d61/fcb 625012 0 2024-03-23T08:23:33.501 INFO:tasks.workunit.client.1.smithi193.stdout:6/965: creat d0/d27/d81/df/d11/d3b/d6c/f12a x:0 0 0 2024-03-23T08:23:33.502 INFO:tasks.workunit.client.1.smithi193.stdout:6/966: dread - d0/d27/d81/df/d11/d3b/d6c/fef zero size 2024-03-23T08:23:33.507 INFO:tasks.workunit.client.1.smithi193.stdout:6/967: dwrite d0/d27/d81/df/d11/d36/f10b [0,4194304] 0 2024-03-23T08:23:33.511 INFO:tasks.workunit.client.1.smithi193.stdout:6/968: creat d0/d27/d81/da1/de5/f12b x:0 0 0 2024-03-23T08:23:33.518 INFO:tasks.workunit.client.1.smithi193.stdout:6/969: dwrite d0/d27/d81/df/d11/d3b/d6c/f12a [0,4194304] 0 2024-03-23T08:23:33.528 INFO:tasks.workunit.client.1.smithi193.stdout:3/934: sync 2024-03-23T08:23:33.530 INFO:tasks.workunit.client.1.smithi193.stdout:3/935: fdatasync d1/f20 0 2024-03-23T08:23:33.532 INFO:tasks.workunit.client.1.smithi193.stdout:3/936: mknod d1/de/d13/d11d/c137 0 2024-03-23T08:23:33.532 INFO:tasks.workunit.client.1.smithi193.stdout:6/970: dread d0/d27/d81/df/f9b [0,4194304] 0 2024-03-23T08:23:33.534 INFO:tasks.workunit.client.1.smithi193.stdout:6/971: write d0/d27/d81/df/d11/d3b/d51/dd6/ff6 [429070,85479] 0 2024-03-23T08:23:33.534 INFO:tasks.workunit.client.1.smithi193.stdout:3/937: creat d1/de/d13/d21/d50/d6f/f138 x:0 0 0 2024-03-23T08:23:33.539 INFO:tasks.workunit.client.1.smithi193.stdout:3/938: getdents d1/de/d13/d96/d99 0 2024-03-23T08:23:33.540 INFO:tasks.workunit.client.1.smithi193.stdout:6/972: getdents d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/d9d 0 2024-03-23T08:23:33.542 INFO:tasks.workunit.client.1.smithi193.stdout:3/939: mkdir d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/ddf/d139 0 2024-03-23T08:23:33.545 INFO:tasks.workunit.client.1.smithi193.stdout:3/940: creat d1/de/d13/d11d/f13a x:0 0 0 2024-03-23T08:23:33.547 INFO:tasks.workunit.client.1.smithi193.stdout:6/973: dwrite d0/d27/d81/df/d11/d3b/d51/dd6/fff [0,4194304] 0 2024-03-23T08:23:33.547 INFO:tasks.workunit.client.1.smithi193.stdout:3/941: mkdir d1/d13b 0 2024-03-23T08:23:33.551 INFO:tasks.workunit.client.1.smithi193.stdout:3/942: truncate d1/de/d13/d96/d99/f64 4965727 0 2024-03-23T08:23:33.551 INFO:tasks.workunit.client.1.smithi193.stdout:6/974: getdents d0/d27/d118 0 2024-03-23T08:23:33.551 INFO:tasks.workunit.client.1.smithi193.stdout:3/943: stat d1/de/d13/d21/d25/d2b/d3c 0 2024-03-23T08:23:33.553 INFO:tasks.workunit.client.1.smithi193.stdout:3/944: write d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/dfe/fff [762373,81092] 0 2024-03-23T08:23:33.557 INFO:tasks.workunit.client.1.smithi193.stdout:3/945: unlink d1/de/d13/d21/d50/d11b/c123 0 2024-03-23T08:23:33.559 INFO:tasks.workunit.client.1.smithi193.stdout:3/946: creat d1/de/d13/d21/d50/d6f/d6e/f13c x:0 0 0 2024-03-23T08:23:33.567 INFO:tasks.workunit.client.1.smithi193.stdout:3/947: truncate d1/de/d13/d21/d25/d34/f47 1190106 0 2024-03-23T08:23:33.576 INFO:tasks.workunit.client.1.smithi193.stdout:3/948: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/de4/d105/deb/l13d 0 2024-03-23T08:23:33.584 INFO:tasks.workunit.client.1.smithi193.stdout:3/949: dwrite d1/de/d13/d21/d50/d44/f79 [4194304,4194304] 0 2024-03-23T08:23:33.585 INFO:tasks.workunit.client.1.smithi193.stdout:3/950: chown d1/de/d13/d21/d25/f12f 41061 1 2024-03-23T08:23:33.593 INFO:tasks.workunit.client.1.smithi193.stdout:3/951: write d1/de/f10 [2011858,78889] 0 2024-03-23T08:23:33.595 INFO:tasks.workunit.client.1.smithi193.stdout:3/952: fdatasync d1/de/d13/d21/d50/fc8 0 2024-03-23T08:23:33.600 INFO:tasks.workunit.client.1.smithi193.stdout:3/953: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/dae/dd9/l13e 0 2024-03-23T08:23:33.601 INFO:tasks.workunit.client.1.smithi193.stdout:3/954: write d1/de/d13/d21/d59/d60/f6b [4826775,122728] 0 2024-03-23T08:23:33.606 INFO:tasks.workunit.client.1.smithi193.stdout:3/955: dwrite d1/de/d13/d21/d27/f1f [4194304,4194304] 0 2024-03-23T08:23:33.607 INFO:tasks.workunit.client.1.smithi193.stdout:3/956: readlink d1/l17 0 2024-03-23T08:23:33.613 INFO:tasks.workunit.client.1.smithi193.stdout:6/975: sync 2024-03-23T08:23:33.613 INFO:tasks.workunit.client.1.smithi193.stdout:3/957: truncate d1/de/fd3 639304 0 2024-03-23T08:23:33.617 INFO:tasks.workunit.client.1.smithi193.stdout:6/976: creat d0/d27/d81/df/d11/d3b/d51/daa/dda/f12c x:0 0 0 2024-03-23T08:23:33.617 INFO:tasks.workunit.client.1.smithi193.stdout:3/958: creat d1/de/d13/d21/d50/da2/f13f x:0 0 0 2024-03-23T08:23:33.618 INFO:tasks.workunit.client.1.smithi193.stdout:6/977: chown d0/d27/d81/df/d11/d3b/d51/d9c/d116 0 1 2024-03-23T08:23:33.620 INFO:tasks.workunit.client.1.smithi193.stdout:3/959: stat d1/de/d13/d21/d25/d2b/d3c/d4a/d85/db9/dfc/ca5 0 2024-03-23T08:23:33.620 INFO:tasks.workunit.client.1.smithi193.stdout:3/960: fdatasync d1/de/d13/d21/d59/fde 0 2024-03-23T08:23:33.622 INFO:tasks.workunit.client.1.smithi193.stdout:6/978: link d0/d27/d81/df/dae/f11b d0/d27/d81/df/d11/d36/d61/d3a/d8f/f12d 0 2024-03-23T08:23:33.627 INFO:tasks.workunit.client.1.smithi193.stdout:6/979: getdents d0/d27/d81/d87 0 2024-03-23T08:23:33.629 INFO:tasks.workunit.client.1.smithi193.stdout:3/961: write d1/de/d13/d21/d25/d2b/d3c/f6a [1034617,59766] 0 2024-03-23T08:23:33.631 INFO:tasks.workunit.client.1.smithi193.stdout:6/980: getdents d0/d27/d81/da1/db8 0 2024-03-23T08:23:33.632 INFO:tasks.workunit.client.1.smithi193.stdout:6/981: readlink d0/d27/d81/df/d11/d36/d61/d12/l2b 0 2024-03-23T08:23:33.633 INFO:tasks.workunit.client.1.smithi193.stdout:6/982: readlink d0/d27/d81/df/d11/d3b/d51/daa/l101 0 2024-03-23T08:23:33.634 INFO:tasks.workunit.client.1.smithi193.stdout:3/962: link d1/de/d13/d21/d25/d2b/d3c/d4a/d85/f10d d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/d9f/dfe/f140 0 2024-03-23T08:23:33.635 INFO:tasks.workunit.client.1.smithi193.stdout:6/983: write d0/d27/d81/df/d11/d56/fd1 [405576,73812] 0 2024-03-23T08:23:33.635 INFO:tasks.workunit.client.1.smithi193.stdout:3/963: chown d1/de/f16 1701088 1 2024-03-23T08:23:33.636 INFO:tasks.workunit.client.1.smithi193.stdout:6/984: chown d0/d27/d81/df/d11/d3b/d6c 4136384 1 2024-03-23T08:23:33.639 INFO:tasks.workunit.client.1.smithi193.stdout:6/985: unlink d0/d27/d81/df/d11/d3b/l58 0 2024-03-23T08:23:33.639 INFO:tasks.workunit.client.1.smithi193.stdout:3/964: creat d1/de/d13/d21/d25/d2b/d3c/d4a/d85/db9/d12a/f141 x:0 0 0 2024-03-23T08:23:33.640 INFO:tasks.workunit.client.1.smithi193.stdout:6/986: fdatasync d0/d27/d81/df/d11/d36/d61/d12/d21/d2a/f10c 0 2024-03-23T08:23:33.641 INFO:tasks.workunit.client.1.smithi193.stdout:3/965: readlink d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/l92 0 2024-03-23T08:23:33.642 INFO:tasks.workunit.client.1.smithi193.stdout:6/987: mknod d0/d27/d81/df/d11/d3b/d51/c12e 0 2024-03-23T08:23:33.644 INFO:tasks.workunit.client.1.smithi193.stdout:6/988: write d0/d27/d81/fc5 [1655646,68817] 0 2024-03-23T08:23:33.645 INFO:tasks.workunit.client.1.smithi193.stdout:3/966: dread d1/de/d13/d21/d25/f56 [0,4194304] 0 2024-03-23T08:23:33.647 INFO:tasks.workunit.client.1.smithi193.stdout:3/967: mkdir d1/d142 0 2024-03-23T08:23:33.648 INFO:tasks.workunit.client.1.smithi193.stdout:6/989: rename d0/d27/d81/da1/fb5 to d0/d27/d81/da1/f12f 0 2024-03-23T08:23:33.650 INFO:tasks.workunit.client.1.smithi193.stdout:3/968: dread d1/de/f70 [0,4194304] 0 2024-03-23T08:23:33.659 INFO:tasks.workunit.client.1.smithi193.stdout:3/969: dwrite d1/de/d13/d21/d27/f63 [0,4194304] 0 2024-03-23T08:23:33.661 INFO:tasks.workunit.client.1.smithi193.stdout:3/970: chown d1/de/d13/d21/d27/f1f 376314194 1 2024-03-23T08:23:33.663 INFO:tasks.workunit.client.1.smithi193.stdout:3/971: creat d1/de/d13/d21/d25/d2b/d3c/d119/f143 x:0 0 0 2024-03-23T08:23:33.682 INFO:tasks.workunit.client.1.smithi193.stdout:3/972: dread d1/de/f89 [0,4194304] 0 2024-03-23T08:23:33.683 INFO:tasks.workunit.client.1.smithi193.stdout:3/973: write d1/f2f [1211854,1255] 0 2024-03-23T08:23:33.684 INFO:tasks.workunit.client.1.smithi193.stdout:3/974: stat d1/de/d13/d96/d99/f30 0 2024-03-23T08:23:33.687 INFO:tasks.workunit.client.1.smithi193.stdout:3/975: rename d1/de/d13/d21/d50/d6f/l5f to d1/de/d13/d21/d27/l144 0 2024-03-23T08:23:33.688 INFO:tasks.workunit.client.1.smithi193.stdout:3/976: mknod d1/de/d13/d21/d50/d6f/d6e/c145 0 2024-03-23T08:23:33.698 INFO:tasks.workunit.client.1.smithi193.stdout:3/977: dwrite d1/de/d13/d21/d25/d34/f47 [0,4194304] 0 2024-03-23T08:23:33.704 INFO:tasks.workunit.client.1.smithi193.stdout:3/978: unlink d1/de/d13/d21/d25/d34/da8/l107 0 2024-03-23T08:23:33.705 INFO:tasks.workunit.client.1.smithi193.stdout:6/990: sync 2024-03-23T08:23:33.707 INFO:tasks.workunit.client.1.smithi193.stdout:6/991: mknod d0/d27/d81/df/d11/d36/d61/d7c/d69/c130 0 2024-03-23T08:23:33.710 INFO:tasks.workunit.client.1.smithi193.stdout:3/979: rename d1/de/d13/d21/d25/d2b/f4d to d1/de/d13/d96/d99/f146 0 2024-03-23T08:23:33.711 INFO:tasks.workunit.client.1.smithi193.stdout:3/980: write d1/fc [795814,92724] 0 2024-03-23T08:23:33.712 INFO:tasks.workunit.client.1.smithi193.stdout:6/992: mkdir d0/d27/d81/df/d11/d36/d61/d3a/df3/d11e/d131 0 2024-03-23T08:23:33.713 INFO:tasks.workunit.client.1.smithi193.stdout:3/981: symlink d1/de/d13/d96/l147 0 2024-03-23T08:23:33.714 INFO:tasks.workunit.client.1.smithi193.stdout:3/982: fdatasync d1/de/f118 0 2024-03-23T08:23:33.718 INFO:tasks.workunit.client.1.smithi193.stdout:3/983: dread d1/de/d13/d21/d25/d2b/d3c/d119/f11c [0,4194304] 0 2024-03-23T08:23:33.721 INFO:tasks.workunit.client.1.smithi193.stdout:3/984: fsync d1/de/d13/d21/d25/d34/da8/fba 0 2024-03-23T08:23:33.723 INFO:tasks.workunit.client.1.smithi193.stdout:3/985: mknod d1/de/d28/c148 0 2024-03-23T08:23:33.724 INFO:tasks.workunit.client.1.smithi193.stdout:3/986: write d1/de/d13/d96/d99/f55 [11993,91405] 0 2024-03-23T08:23:33.726 INFO:tasks.workunit.client.1.smithi193.stdout:3/987: unlink d1/de/d13/d96/d99/l48 0 2024-03-23T08:23:33.727 INFO:tasks.workunit.client.1.smithi193.stdout:3/988: read d1/de/d13/d21/f23 [1170883,116308] 0 2024-03-23T08:23:33.728 INFO:tasks.workunit.client.1.smithi193.stdout:3/989: chown d1/de/d13/d21/d25/d2b/d3c/d4a/d85/fd8 29240669 1 2024-03-23T08:23:33.730 INFO:tasks.workunit.client.1.smithi193.stdout:3/990: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/d85/db9/d12a/l149 0 2024-03-23T08:23:33.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:33 smithi070 ceph-mon[39699]: pgmap v26: 65 pgs: 65 active+clean; 2.0 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 98 MiB/s rd, 236 MiB/s wr, 444 op/s 2024-03-23T08:23:33.733 INFO:tasks.workunit.client.1.smithi193.stdout:3/991: creat d1/de/d13/d21/d59/dc6/f14a x:0 0 0 2024-03-23T08:23:33.736 INFO:tasks.workunit.client.1.smithi193.stdout:3/992: rename d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/ddf to d1/d13b/d14b 0 2024-03-23T08:23:33.739 INFO:tasks.workunit.client.1.smithi193.stdout:3/993: getdents d1/de/d13/d21/d50/d11b 0 2024-03-23T08:23:33.739 INFO:tasks.workunit.client.1.smithi193.stdout:3/994: fdatasync d1/f132 0 2024-03-23T08:23:33.741 INFO:tasks.workunit.client.1.smithi193.stdout:3/995: creat d1/de/d13/d21/d50/d6f/f14c x:0 0 0 2024-03-23T08:23:33.743 INFO:tasks.workunit.client.1.smithi193.stdout:3/996: symlink d1/de/d13/d21/d25/d2b/d3c/d4a/d5a/l14d 0 2024-03-23T08:23:33.751 INFO:tasks.workunit.client.1.smithi193.stdout:3/997: dwrite d1/de/f16 [0,4194304] 0 2024-03-23T08:23:33.754 INFO:tasks.workunit.client.1.smithi193.stdout:3/998: fsync d1/de/d13/d21/d25/d34/da8/f11a 0 2024-03-23T08:23:33.760 INFO:tasks.workunit.client.1.smithi193.stdout:3/999: dwrite d1/de/d13/d21/d25/d34/f8f [0,4194304] 0 2024-03-23T08:23:33.771 INFO:tasks.workunit.client.1.smithi193.stdout:6/993: sync 2024-03-23T08:23:33.773 INFO:tasks.workunit.client.1.smithi193.stdout:6/994: dread - d0/d27/d81/df/d11/d56/fed zero size 2024-03-23T08:23:33.779 INFO:tasks.workunit.client.1.smithi193.stdout:6/995: dwrite d0/d27/d81/df/d11/d3b/d6c/fbe [0,4194304] 0 2024-03-23T08:23:33.782 INFO:tasks.workunit.client.1.smithi193.stdout:6/996: creat d0/d27/d81/df/d11/d3b/d6c/f132 x:0 0 0 2024-03-23T08:23:33.783 INFO:tasks.workunit.client.1.smithi193.stdout:6/997: rename d0/d27/d81/df/d11/d3b/d51/d9c/d116 to d0/d27/d81/df/d11/d3b/d51/d9c/d116/d133 22 2024-03-23T08:23:33.786 INFO:tasks.workunit.client.1.smithi193.stdout:6/998: symlink d0/d27/d81/df/d11/d3b/d51/dd6/l134 0 2024-03-23T08:23:33.788 INFO:tasks.workunit.client.1.smithi193.stdout:6/999: rmdir d0/d27/d81/df/d11/d36/d61/d7c/d69/de1 39 2024-03-23T08:23:33.792 INFO:tasks.workunit.client.1.smithi193.stderr:+ rm -rf -- ./tmp.pjPqDdIjg0 2024-03-23T08:23:33.824 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:33 smithi193 ceph-mon[48370]: pgmap v26: 65 pgs: 65 active+clean; 2.0 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 98 MiB/s rd, 236 MiB/s wr, 444 op/s 2024-03-23T08:23:35.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:35 smithi070 ceph-mon[39699]: pgmap v27: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 109 MiB/s rd, 253 MiB/s wr, 472 op/s 2024-03-23T08:23:35.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:35 smithi193 ceph-mon[48370]: pgmap v27: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 109 MiB/s rd, 253 MiB/s wr, 472 op/s 2024-03-23T08:23:37.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:37 smithi070 ceph-mon[39699]: pgmap v28: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 87 MiB/s rd, 196 MiB/s wr, 398 op/s 2024-03-23T08:23:37.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:37 smithi193 ceph-mon[48370]: pgmap v28: 65 pgs: 65 active+clean; 2.3 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 87 MiB/s rd, 196 MiB/s wr, 398 op/s 2024-03-23T08:23:39.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:39 smithi070 ceph-mon[39699]: pgmap v29: 65 pgs: 65 active+clean; 2.1 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 93 MiB/s rd, 209 MiB/s wr, 469 op/s 2024-03-23T08:23:39.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:39 smithi193 ceph-mon[48370]: pgmap v29: 65 pgs: 65 active+clean; 2.1 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 93 MiB/s rd, 209 MiB/s wr, 469 op/s 2024-03-23T08:23:41.526 INFO:tasks.workunit.client.0.smithi070.stdout:8/16: sync 2024-03-23T08:23:41.529 INFO:tasks.workunit.client.0.smithi070.stdout:8/17: mknod d5/c9 0 2024-03-23T08:23:41.532 INFO:tasks.workunit.client.0.smithi070.stdout:8/18: mkdir d5/da 0 2024-03-23T08:23:41.536 INFO:tasks.workunit.client.0.smithi070.stdout:8/19: unlink l2 0 2024-03-23T08:23:41.548 INFO:tasks.workunit.client.0.smithi070.stdout:8/20: dwrite d5/f7 [0,4194304] 0 2024-03-23T08:23:41.551 INFO:tasks.workunit.client.0.smithi070.stdout:8/21: dread d5/f7 [0,4194304] 0 2024-03-23T08:23:41.561 INFO:tasks.workunit.client.0.smithi070.stdout:8/22: write f4 [71286,42628] 0 2024-03-23T08:23:41.569 INFO:tasks.workunit.client.0.smithi070.stdout:8/23: dwrite f4 [0,4194304] 0 2024-03-23T08:23:41.571 INFO:tasks.workunit.client.0.smithi070.stdout:8/24: creat d5/d8/fb x:0 0 0 2024-03-23T08:23:41.575 INFO:tasks.workunit.client.0.smithi070.stdout:8/25: rmdir d5/da 0 2024-03-23T08:23:41.577 INFO:tasks.workunit.client.0.smithi070.stdout:8/26: write d5/d8/fb [347271,15832] 0 2024-03-23T08:23:41.577 INFO:tasks.workunit.client.0.smithi070.stdout:8/27: read d5/f7 [1829798,44108] 0 2024-03-23T08:23:41.695 INFO:tasks.workunit.client.0.smithi070.stdout:3/92: sync 2024-03-23T08:23:41.695 INFO:tasks.workunit.client.0.smithi070.stdout:9/20: sync 2024-03-23T08:23:41.695 INFO:tasks.workunit.client.0.smithi070.stdout:1/61: sync 2024-03-23T08:23:41.695 INFO:tasks.workunit.client.0.smithi070.stdout:5/27: sync 2024-03-23T08:23:41.695 INFO:tasks.workunit.client.0.smithi070.stdout:4/14: sync 2024-03-23T08:23:41.695 INFO:tasks.workunit.client.0.smithi070.stdout:7/6: sync 2024-03-23T08:23:41.696 INFO:tasks.workunit.client.0.smithi070.stdout:0/37: sync 2024-03-23T08:23:41.696 INFO:tasks.workunit.client.0.smithi070.stdout:6/240: sync 2024-03-23T08:23:41.696 INFO:tasks.workunit.client.0.smithi070.stdout:2/16: sync 2024-03-23T08:23:41.698 INFO:tasks.workunit.client.0.smithi070.stdout:9/21: chown l1 322 1 2024-03-23T08:23:41.700 INFO:tasks.workunit.client.0.smithi070.stdout:6/241: dread d0/d23/d28/d35/f46 [0,4194304] 0 2024-03-23T08:23:41.701 INFO:tasks.workunit.client.0.smithi070.stdout:7/7: symlink l3 0 2024-03-23T08:23:41.701 INFO:tasks.workunit.client.0.smithi070.stdout:4/15: write f0 [2851737,65842] 0 2024-03-23T08:23:41.702 INFO:tasks.workunit.client.0.smithi070.stdout:5/28: symlink l5 0 2024-03-23T08:23:41.704 INFO:tasks.workunit.client.0.smithi070.stdout:0/38: rename l9 to ld 0 2024-03-23T08:23:41.706 INFO:tasks.workunit.client.0.smithi070.stdout:5/29: dread f4 [0,4194304] 0 2024-03-23T08:23:41.706 INFO:tasks.workunit.client.0.smithi070.stdout:5/30: chown f1 1632 1 2024-03-23T08:23:41.709 INFO:tasks.workunit.client.0.smithi070.stdout:1/62: symlink d8/db/l18 0 2024-03-23T08:23:41.712 INFO:tasks.workunit.client.0.smithi070.stdout:9/22: creat f6 x:0 0 0 2024-03-23T08:23:41.713 INFO:tasks.workunit.client.0.smithi070.stdout:9/23: dread - f6 zero size 2024-03-23T08:23:41.713 INFO:tasks.workunit.client.0.smithi070.stdout:9/24: rmdir - no directory 2024-03-23T08:23:41.717 INFO:tasks.workunit.client.0.smithi070.stdout:7/8: rename c2 to c4 0 2024-03-23T08:23:41.721 INFO:tasks.workunit.client.0.smithi070.stdout:0/39: chown ld 2154 1 2024-03-23T08:23:41.721 INFO:tasks.workunit.client.0.smithi070.stdout:2/17: dwrite d1/f4 [0,4194304] 0 2024-03-23T08:23:41.723 INFO:tasks.workunit.client.0.smithi070.stdout:2/18: chown d1/l2 7131761 1 2024-03-23T08:23:41.723 INFO:tasks.workunit.client.0.smithi070.stdout:3/93: dwrite db/f25 [0,4194304] 0 2024-03-23T08:23:41.724 INFO:tasks.workunit.client.0.smithi070.stdout:3/94: chown db/df 1 1 2024-03-23T08:23:41.728 INFO:tasks.workunit.client.0.smithi070.stdout:5/31: write f4 [4508139,68547] 0 2024-03-23T08:23:41.731 INFO:tasks.workunit.client.0.smithi070.stdout:1/63: truncate d8/db/f16 445062 0 2024-03-23T08:23:41.731 INFO:tasks.workunit.client.0.smithi070.stdout:1/64: readlink d8/ld 0 2024-03-23T08:23:41.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:41 smithi070 ceph-mon[39699]: pgmap v30: 65 pgs: 65 active+clean; 2.1 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 48 MiB/s rd, 110 MiB/s wr, 269 op/s 2024-03-23T08:23:41.736 INFO:tasks.workunit.client.0.smithi070.stdout:4/16: mknod d3/c4 0 2024-03-23T08:23:41.736 INFO:tasks.workunit.client.0.smithi070.stdout:7/9: dread f1 [0,4194304] 0 2024-03-23T08:23:41.736 INFO:tasks.workunit.client.0.smithi070.stdout:4/17: stat d3/c4 0 2024-03-23T08:23:41.739 INFO:tasks.workunit.client.0.smithi070.stdout:5/32: truncate f1 2593584 0 2024-03-23T08:23:41.742 INFO:tasks.workunit.client.0.smithi070.stdout:9/25: link f6 f7 0 2024-03-23T08:23:41.743 INFO:tasks.workunit.client.0.smithi070.stdout:6/242: creat d0/d23/f55 x:0 0 0 2024-03-23T08:23:41.746 INFO:tasks.workunit.client.0.smithi070.stdout:9/26: creat f8 x:0 0 0 2024-03-23T08:23:41.747 INFO:tasks.workunit.client.0.smithi070.stdout:6/243: symlink d0/d23/d28/d35/d3a/d3c/l56 0 2024-03-23T08:23:41.748 INFO:tasks.workunit.client.0.smithi070.stdout:3/95: creat db/f2a x:0 0 0 2024-03-23T08:23:41.748 INFO:tasks.workunit.client.0.smithi070.stdout:3/96: chown db/l10 319603664 1 2024-03-23T08:23:41.749 INFO:tasks.workunit.client.0.smithi070.stdout:9/27: mkdir d9 0 2024-03-23T08:23:41.751 INFO:tasks.workunit.client.0.smithi070.stdout:6/244: creat d0/d23/d28/d35/f57 x:0 0 0 2024-03-23T08:23:41.752 INFO:tasks.workunit.client.0.smithi070.stdout:3/97: write db/f23 [702968,129867] 0 2024-03-23T08:23:41.752 INFO:tasks.workunit.client.0.smithi070.stdout:2/19: write d1/f4 [5171302,61772] 0 2024-03-23T08:23:41.754 INFO:tasks.workunit.client.0.smithi070.stdout:9/28: write f7 [468330,52770] 0 2024-03-23T08:23:41.754 INFO:tasks.workunit.client.0.smithi070.stdout:9/29: dread - f8 zero size 2024-03-23T08:23:41.754 INFO:tasks.workunit.client.0.smithi070.stdout:9/30: chown l5 14947671 1 2024-03-23T08:23:41.756 INFO:tasks.workunit.client.0.smithi070.stdout:2/20: rmdir d1 39 2024-03-23T08:23:41.759 INFO:tasks.workunit.client.0.smithi070.stdout:3/98: mknod db/df/d15/c2b 0 2024-03-23T08:23:41.767 INFO:tasks.workunit.client.0.smithi070.stdout:9/31: getdents d9 0 2024-03-23T08:23:41.768 INFO:tasks.workunit.client.0.smithi070.stdout:9/32: fdatasync f7 0 2024-03-23T08:23:41.768 INFO:tasks.workunit.client.0.smithi070.stdout:2/21: creat d1/f5 x:0 0 0 2024-03-23T08:23:41.770 INFO:tasks.workunit.client.0.smithi070.stdout:1/65: dwrite d8/f9 [0,4194304] 0 2024-03-23T08:23:41.770 INFO:tasks.workunit.client.0.smithi070.stdout:2/22: stat d1/f5 0 2024-03-23T08:23:41.771 INFO:tasks.workunit.client.0.smithi070.stdout:9/33: chown l1 0 1 2024-03-23T08:23:41.771 INFO:tasks.workunit.client.0.smithi070.stdout:6/245: write d0/f20 [1034179,33200] 0 2024-03-23T08:23:41.776 INFO:tasks.workunit.client.0.smithi070.stdout:2/23: write d1/f5 [200034,114492] 0 2024-03-23T08:23:41.776 INFO:tasks.workunit.client.0.smithi070.stdout:7/10: dwrite f1 [0,4194304] 0 2024-03-23T08:23:41.777 INFO:tasks.workunit.client.0.smithi070.stdout:1/66: dread - d8/db/f15 zero size 2024-03-23T08:23:41.783 INFO:tasks.workunit.client.0.smithi070.stdout:6/246: rename d0/d23/d28/d35/f57 to d0/d3/d2c/f58 0 2024-03-23T08:23:41.784 INFO:tasks.workunit.client.0.smithi070.stdout:2/24: dread d1/f5 [0,4194304] 0 2024-03-23T08:23:41.785 INFO:tasks.workunit.client.0.smithi070.stdout:1/67: readlink d8/db/l18 0 2024-03-23T08:23:41.786 INFO:tasks.workunit.client.0.smithi070.stdout:7/11: mknod c5 0 2024-03-23T08:23:41.789 INFO:tasks.workunit.client.0.smithi070.stdout:9/34: getdents d9 0 2024-03-23T08:23:41.790 INFO:tasks.workunit.client.0.smithi070.stdout:2/25: unlink d1/l3 0 2024-03-23T08:23:41.791 INFO:tasks.workunit.client.0.smithi070.stdout:9/35: fsync f8 0 2024-03-23T08:23:41.792 INFO:tasks.workunit.client.0.smithi070.stdout:7/12: symlink l6 0 2024-03-23T08:23:41.792 INFO:tasks.workunit.client.0.smithi070.stdout:5/33: dwrite f1 [0,4194304] 0 2024-03-23T08:23:41.794 INFO:tasks.workunit.client.0.smithi070.stdout:2/26: write d1/f4 [4223576,11866] 0 2024-03-23T08:23:41.796 INFO:tasks.workunit.client.0.smithi070.stdout:6/247: truncate d0/d3/d2c/f34 126054 0 2024-03-23T08:23:41.798 INFO:tasks.workunit.client.0.smithi070.stdout:3/99: dwrite db/df/d15/f1e [0,4194304] 0 2024-03-23T08:23:41.798 INFO:tasks.workunit.client.0.smithi070.stdout:5/34: write f1 [1371598,65641] 0 2024-03-23T08:23:41.803 INFO:tasks.workunit.client.0.smithi070.stdout:1/68: link d8/db/l17 d8/l19 0 2024-03-23T08:23:41.803 INFO:tasks.workunit.client.0.smithi070.stdout:5/35: chown l2 25 1 2024-03-23T08:23:41.804 INFO:tasks.workunit.client.0.smithi070.stdout:5/36: write f4 [422459,113093] 0 2024-03-23T08:23:41.805 INFO:tasks.workunit.client.0.smithi070.stdout:9/36: creat d9/fa x:0 0 0 2024-03-23T08:23:41.806 INFO:tasks.workunit.client.0.smithi070.stdout:7/13: dread f1 [0,4194304] 0 2024-03-23T08:23:41.807 INFO:tasks.workunit.client.0.smithi070.stdout:2/27: truncate f0 363976 0 2024-03-23T08:23:41.811 INFO:tasks.workunit.client.0.smithi070.stdout:3/100: rename db/df/f13 to db/df/d15/d22/f2c 0 2024-03-23T08:23:41.812 INFO:tasks.workunit.client.0.smithi070.stdout:3/101: chown db/df/d15/d22 477979 1 2024-03-23T08:23:41.812 INFO:tasks.workunit.client.0.smithi070.stdout:2/28: dread d1/f4 [4194304,4194304] 0 2024-03-23T08:23:41.813 INFO:tasks.workunit.client.0.smithi070.stdout:5/37: dwrite f4 [0,4194304] 0 2024-03-23T08:23:41.814 INFO:tasks.workunit.client.0.smithi070.stdout:5/38: chown l3 4 1 2024-03-23T08:23:41.816 INFO:tasks.workunit.client.0.smithi070.stdout:9/37: unlink d9/fa 0 2024-03-23T08:23:41.821 INFO:tasks.workunit.client.0.smithi070.stdout:3/102: fdatasync db/df/d15/f1e 0 2024-03-23T08:23:41.824 INFO:tasks.workunit.client.0.smithi070.stdout:7/14: rename l3 to l7 0 2024-03-23T08:23:41.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:41 smithi193 ceph-mon[48370]: pgmap v30: 65 pgs: 65 active+clean; 2.1 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 48 MiB/s rd, 110 MiB/s wr, 269 op/s 2024-03-23T08:23:41.825 INFO:tasks.workunit.client.0.smithi070.stdout:2/29: mkdir d1/d6 0 2024-03-23T08:23:41.829 INFO:tasks.workunit.client.0.smithi070.stdout:7/15: dread f1 [0,4194304] 0 2024-03-23T08:23:41.829 INFO:tasks.workunit.client.0.smithi070.stdout:6/248: write d0/d23/d28/d35/d3a/d3c/f51 [669568,84289] 0 2024-03-23T08:23:41.831 INFO:tasks.workunit.client.0.smithi070.stdout:5/39: mknod c6 0 2024-03-23T08:23:41.835 INFO:tasks.workunit.client.0.smithi070.stdout:2/30: truncate d1/f5 1345230 0 2024-03-23T08:23:41.837 INFO:tasks.workunit.client.0.smithi070.stdout:6/249: truncate d0/d23/f24 6690204 0 2024-03-23T08:23:41.838 INFO:tasks.workunit.client.0.smithi070.stdout:7/16: creat f8 x:0 0 0 2024-03-23T08:23:41.841 INFO:tasks.workunit.client.0.smithi070.stdout:2/31: mkdir d1/d7 0 2024-03-23T08:23:41.844 INFO:tasks.workunit.client.0.smithi070.stdout:3/103: write db/df/f19 [4170480,73035] 0 2024-03-23T08:23:41.845 INFO:tasks.workunit.client.0.smithi070.stdout:6/250: mknod d0/d23/d28/d35/d3a/d3c/c59 0 2024-03-23T08:23:41.846 INFO:tasks.workunit.client.0.smithi070.stdout:7/17: symlink l9 0 2024-03-23T08:23:41.848 INFO:tasks.workunit.client.0.smithi070.stdout:9/38: rmdir d9 0 2024-03-23T08:23:41.849 INFO:tasks.workunit.client.0.smithi070.stdout:3/104: fdatasync f6 0 2024-03-23T08:23:41.851 INFO:tasks.workunit.client.0.smithi070.stdout:7/18: write f1 [3488547,55810] 0 2024-03-23T08:23:41.853 INFO:tasks.workunit.client.0.smithi070.stdout:9/39: mkdir db 0 2024-03-23T08:23:41.856 INFO:tasks.workunit.client.0.smithi070.stdout:9/40: dread f6 [0,4194304] 0 2024-03-23T08:23:41.857 INFO:tasks.workunit.client.0.smithi070.stdout:3/105: mknod db/df/d15/c2d 0 2024-03-23T08:23:41.859 INFO:tasks.workunit.client.0.smithi070.stdout:7/19: mknod ca 0 2024-03-23T08:23:41.865 INFO:tasks.workunit.client.0.smithi070.stdout:7/20: dwrite f1 [0,4194304] 0 2024-03-23T08:23:41.866 INFO:tasks.workunit.client.0.smithi070.stdout:9/41: symlink db/lc 0 2024-03-23T08:23:41.872 INFO:tasks.workunit.client.0.smithi070.stdout:3/106: link db/df/d15/d22/f2c db/df/f2e 0 2024-03-23T08:23:41.872 INFO:tasks.workunit.client.0.smithi070.stdout:7/21: creat fb x:0 0 0 2024-03-23T08:23:41.874 INFO:tasks.workunit.client.0.smithi070.stdout:6/251: dwrite d0/f50 [0,4194304] 0 2024-03-23T08:23:41.878 INFO:tasks.workunit.client.0.smithi070.stdout:9/42: dread f4 [0,4194304] 0 2024-03-23T08:23:41.879 INFO:tasks.workunit.client.0.smithi070.stdout:6/252: read - d0/d23/d28/d35/d3a/d3c/f53 zero size 2024-03-23T08:23:41.881 INFO:tasks.workunit.client.0.smithi070.stdout:9/43: mknod db/cd 0 2024-03-23T08:23:41.884 INFO:tasks.workunit.client.0.smithi070.stdout:7/22: dwrite f1 [4194304,4194304] 0 2024-03-23T08:23:41.886 INFO:tasks.workunit.client.0.smithi070.stdout:9/44: creat db/fe x:0 0 0 2024-03-23T08:23:41.886 INFO:tasks.workunit.client.0.smithi070.stdout:9/45: readlink l1 0 2024-03-23T08:23:41.887 INFO:tasks.workunit.client.0.smithi070.stdout:9/46: dread f7 [0,4194304] 0 2024-03-23T08:23:41.892 INFO:tasks.workunit.client.0.smithi070.stdout:7/23: link l6 lc 0 2024-03-23T08:23:41.893 INFO:tasks.workunit.client.0.smithi070.stdout:7/24: dread - f8 zero size 2024-03-23T08:23:41.894 INFO:tasks.workunit.client.0.smithi070.stdout:7/25: stat c5 0 2024-03-23T08:23:41.894 INFO:tasks.workunit.client.0.smithi070.stdout:6/253: link d0/d23/d1c/c47 d0/d23/d1c/d2f/c5a 0 2024-03-23T08:23:41.894 INFO:tasks.workunit.client.0.smithi070.stdout:7/26: truncate f8 933345 0 2024-03-23T08:23:41.900 INFO:tasks.workunit.client.0.smithi070.stdout:6/254: truncate d0/d23/d28/d35/f46 771362 0 2024-03-23T08:23:41.900 INFO:tasks.workunit.client.0.smithi070.stdout:8/28: sync 2024-03-23T08:23:41.902 INFO:tasks.workunit.client.0.smithi070.stdout:8/29: write d5/d8/fb [1134016,19951] 0 2024-03-23T08:23:41.904 INFO:tasks.workunit.client.0.smithi070.stdout:7/27: dwrite f8 [0,4194304] 0 2024-03-23T08:23:41.906 INFO:tasks.workunit.client.0.smithi070.stdout:7/28: creat fd x:0 0 0 2024-03-23T08:23:41.909 INFO:tasks.workunit.client.0.smithi070.stdout:7/29: truncate fd 161324 0 2024-03-23T08:23:41.910 INFO:tasks.workunit.client.0.smithi070.stdout:7/30: symlink le 0 2024-03-23T08:23:41.912 INFO:tasks.workunit.client.0.smithi070.stdout:7/31: symlink lf 0 2024-03-23T08:23:41.914 INFO:tasks.workunit.client.0.smithi070.stdout:7/32: write f8 [4055327,52802] 0 2024-03-23T08:23:41.915 INFO:tasks.workunit.client.0.smithi070.stdout:7/33: read f8 [3484656,126298] 0 2024-03-23T08:23:41.918 INFO:tasks.workunit.client.0.smithi070.stdout:7/34: dread fd [0,4194304] 0 2024-03-23T08:23:41.925 INFO:tasks.workunit.client.0.smithi070.stdout:7/35: unlink fb 0 2024-03-23T08:23:41.927 INFO:tasks.workunit.client.0.smithi070.stdout:7/36: mknod c10 0 2024-03-23T08:23:41.932 INFO:tasks.workunit.client.0.smithi070.stdout:7/37: dread f1 [4194304,4194304] 0 2024-03-23T08:23:41.933 INFO:tasks.workunit.client.0.smithi070.stdout:7/38: stat le 0 2024-03-23T08:23:41.935 INFO:tasks.workunit.client.0.smithi070.stdout:7/39: write f1 [6196326,57625] 0 2024-03-23T08:23:41.942 INFO:tasks.workunit.client.0.smithi070.stdout:7/40: dwrite f8 [0,4194304] 0 2024-03-23T08:23:41.947 INFO:tasks.workunit.client.0.smithi070.stdout:7/41: dread f1 [4194304,4194304] 0 2024-03-23T08:23:41.953 INFO:tasks.workunit.client.0.smithi070.stdout:7/42: link l6 l11 0 2024-03-23T08:23:41.956 INFO:tasks.workunit.client.0.smithi070.stdout:7/43: fdatasync fd 0 2024-03-23T08:23:41.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.957+0000 7f836198b700 1 -- 172.21.15.70:0/129127457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fb440 msgr2=0x7f835c0fb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:41.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.957+0000 7f836198b700 1 --2- 172.21.15.70:0/129127457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fb440 0x7f835c0fb850 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8344004a50 tx=0x7f834400af40 comp rx=0 tx=0).stop 2024-03-23T08:23:41.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.957+0000 7f836198b700 1 -- 172.21.15.70:0/129127457 shutdown_connections 2024-03-23T08:23:41.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.957+0000 7f836198b700 1 --2- 172.21.15.70:0/129127457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0fc6f0 0x7f835c0fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:41.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.957+0000 7f836198b700 1 --2- 172.21.15.70:0/129127457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fb440 0x7f835c0fb850 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:41.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.957+0000 7f836198b700 1 -- 172.21.15.70:0/129127457 >> 172.21.15.70:0/129127457 conn(0x7f835c0f6ba0 msgr2=0x7f835c0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.957+0000 7f836198b700 1 -- 172.21.15.70:0/129127457 shutdown_connections 2024-03-23T08:23:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.957+0000 7f836198b700 1 -- 172.21.15.70:0/129127457 wait complete. 2024-03-23T08:23:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.958+0000 7f836198b700 1 Processor -- start 2024-03-23T08:23:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.958+0000 7f836198b700 1 -- start start 2024-03-23T08:23:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.958+0000 7f836198b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0fb440 0x7f835c10ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.958+0000 7f836198b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fc6f0 0x7f835c10b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.958+0000 7f836198b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c111510 con 0x7f835c0fc6f0 2024-03-23T08:23:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.958+0000 7f836198b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c111680 con 0x7f835c0fb440 2024-03-23T08:23:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.958+0000 7f835a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fc6f0 0x7f835c10b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.958+0000 7f835a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fc6f0 0x7f835c10b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55792/0 (socket says 172.21.15.70:55792) 2024-03-23T08:23:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.958+0000 7f835a7fc700 1 -- 172.21.15.70:0/174391455 learned_addr learned my addr 172.21.15.70:0/174391455 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f835affd700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0fb440 0x7f835c10ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f835affd700 1 -- 172.21.15.70:0/174391455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fc6f0 msgr2=0x7f835c10b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f835affd700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fc6f0 0x7f835c10b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f835affd700 1 -- 172.21.15.70:0/174391455 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344005040 con 0x7f835c0fb440 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f835a7fc700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fc6f0 0x7f835c10b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f835affd700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0fb440 0x7f835c10ce60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8344000c00 tx=0x7f834400af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f83537fe700 1 -- 172.21.15.70:0/174391455 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834400f030 con 0x7f835c0fb440 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f83537fe700 1 -- 172.21.15.70:0/174391455 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f834400a3b0 con 0x7f835c0fb440 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f83537fe700 1 -- 172.21.15.70:0/174391455 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834401d6c0 con 0x7f835c0fb440 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c10b870 con 0x7f835c0fb440 2024-03-23T08:23:41.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.959+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c10be30 con 0x7f835c0fb440 2024-03-23T08:23:41.961 INFO:tasks.workunit.client.0.smithi070.stdout:7/44: dread f8 [0,4194304] 0 2024-03-23T08:23:41.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.960+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835c05f5f0 con 0x7f835c0fb440 2024-03-23T08:23:41.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.962+0000 7f83537fe700 1 -- 172.21.15.70:0/174391455 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f8344008050 con 0x7f835c0fb440 2024-03-23T08:23:41.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.962+0000 7f83537fe700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f834806aec0 0x7f834806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:41.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.962+0000 7f83537fe700 1 -- 172.21.15.70:0/174391455 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f834408afb0 con 0x7f835c0fb440 2024-03-23T08:23:41.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.962+0000 7f835a7fc700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f834806aec0 0x7f834806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:41.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.963+0000 7f835a7fc700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f834806aec0 0x7f834806d370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f834c0041d0 tx=0x7f834c00b040 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:41.964 INFO:tasks.workunit.client.0.smithi070.stdout:7/45: creat f12 x:0 0 0 2024-03-23T08:23:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:41.967+0000 7f83537fe700 1 -- 172.21.15.70:0/174391455 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f83440b8560 con 0x7f835c0fb440 2024-03-23T08:23:41.967 INFO:tasks.workunit.client.0.smithi070.stdout:7/46: readlink lc 0 2024-03-23T08:23:41.983 INFO:tasks.workunit.client.0.smithi070.stdout:7/47: unlink c4 0 2024-03-23T08:23:41.983 INFO:tasks.workunit.client.0.smithi070.stdout:7/48: read - f12 zero size 2024-03-23T08:23:41.986 INFO:tasks.workunit.client.0.smithi070.stdout:7/49: unlink lf 0 2024-03-23T08:23:41.992 INFO:tasks.workunit.client.0.smithi070.stdout:7/50: dwrite fd [0,4194304] 0 2024-03-23T08:23:41.995 INFO:tasks.workunit.client.0.smithi070.stdout:7/51: mknod c13 0 2024-03-23T08:23:42.006 INFO:tasks.workunit.client.0.smithi070.stdout:7/52: dwrite f12 [0,4194304] 0 2024-03-23T08:23:42.202 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.202+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f835c1036d0 con 0x7f834806aec0 2024-03-23T08:23:42.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.203+0000 7f83537fe700 1 -- 172.21.15.70:0/174391455 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f835c1036d0 con 0x7f834806aec0 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f834806aec0 msgr2=0x7f834806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f834806aec0 0x7f834806d370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f834c0041d0 tx=0x7f834c00b040 comp rx=0 tx=0).stop 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0fb440 msgr2=0x7f835c10ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0fb440 0x7f835c10ce60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8344000c00 tx=0x7f834400af40 comp rx=0 tx=0).stop 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 shutdown_connections 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f834806aec0 0x7f834806d370 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0fb440 0x7f835c10ce60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 --2- 172.21.15.70:0/174391455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f835c0fc6f0 0x7f835c10b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 >> 172.21.15.70:0/174391455 conn(0x7f835c0f6ba0 msgr2=0x7f835c0f82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:42.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 shutdown_connections 2024-03-23T08:23:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.206+0000 7f836198b700 1 -- 172.21.15.70:0/174391455 wait complete. 2024-03-23T08:23:42.216 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:23:42.245 INFO:tasks.workunit.client.0.smithi070.stdout:1/69: sync 2024-03-23T08:23:42.247 INFO:tasks.workunit.client.0.smithi070.stdout:1/70: rmdir d8/db 39 2024-03-23T08:23:42.251 INFO:tasks.workunit.client.0.smithi070.stdout:1/71: fsync d8/db/f12 0 2024-03-23T08:23:42.251 INFO:tasks.workunit.client.0.smithi070.stdout:1/72: chown d8 1002 1 2024-03-23T08:23:42.255 INFO:tasks.workunit.client.0.smithi070.stdout:1/73: mkdir d8/d1a 0 2024-03-23T08:23:42.257 INFO:tasks.workunit.client.0.smithi070.stdout:1/74: mkdir d8/d1b 0 2024-03-23T08:23:42.261 INFO:tasks.workunit.client.0.smithi070.stdout:1/75: creat d8/f1c x:0 0 0 2024-03-23T08:23:42.265 INFO:tasks.workunit.client.0.smithi070.stdout:1/76: getdents d8/d11 0 2024-03-23T08:23:42.267 INFO:tasks.workunit.client.0.smithi070.stdout:1/77: mknod d8/d1a/c1d 0 2024-03-23T08:23:42.274 INFO:tasks.workunit.client.0.smithi070.stdout:1/78: write d8/fe [4746857,63531] 0 2024-03-23T08:23:42.276 INFO:tasks.workunit.client.0.smithi070.stdout:1/79: creat d8/d1a/f1e x:0 0 0 2024-03-23T08:23:42.277 INFO:tasks.workunit.client.0.smithi070.stdout:1/80: chown d8/fe 7661 1 2024-03-23T08:23:42.287 INFO:tasks.workunit.client.0.smithi070.stdout:1/81: creat d8/db/f1f x:0 0 0 2024-03-23T08:23:42.288 INFO:tasks.workunit.client.0.smithi070.stdout:1/82: chown d8/d11 1751632 1 2024-03-23T08:23:42.289 INFO:tasks.workunit.client.0.smithi070.stdout:1/83: chown c4 400482272 1 2024-03-23T08:23:42.292 INFO:tasks.workunit.client.0.smithi070.stdout:1/84: fdatasync f3 0 2024-03-23T08:23:42.297 INFO:tasks.workunit.client.0.smithi070.stdout:1/85: mknod d8/db/c20 0 2024-03-23T08:23:42.300 INFO:tasks.workunit.client.0.smithi070.stdout:1/86: rename d8/db/f12 to d8/db/f21 0 2024-03-23T08:23:42.304 INFO:tasks.workunit.client.0.smithi070.stdout:1/87: link d8/db/f1f d8/d1b/f22 0 2024-03-23T08:23:42.306 INFO:tasks.workunit.client.0.smithi070.stdout:1/88: creat d8/d1a/f23 x:0 0 0 2024-03-23T08:23:42.310 INFO:tasks.workunit.client.0.smithi070.stdout:1/89: creat d8/d1b/f24 x:0 0 0 2024-03-23T08:23:42.313 INFO:tasks.workunit.client.0.smithi070.stdout:1/90: rename d8/fe to d8/d1b/f25 0 2024-03-23T08:23:42.321 INFO:tasks.workunit.client.0.smithi070.stdout:1/91: fdatasync d8/db/f21 0 2024-03-23T08:23:42.325 INFO:tasks.workunit.client.0.smithi070.stdout:1/92: mkdir d8/d26 0 2024-03-23T08:23:42.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.326+0000 7f817a674700 1 -- 172.21.15.70:0/684972756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 msgr2=0x7f8174102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:42.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.326+0000 7f817a674700 1 --2- 172.21.15.70:0/684972756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 0x7f8174102390 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8168004940 tx=0x7f8168009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:42.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.326+0000 7f817a674700 1 -- 172.21.15.70:0/684972756 shutdown_connections 2024-03-23T08:23:42.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.326+0000 7f817a674700 1 --2- 172.21.15.70:0/684972756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 0x7f8174102390 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.326+0000 7f817a674700 1 --2- 172.21.15.70:0/684972756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8174066ef0 0x7f81740ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.326+0000 7f817a674700 1 -- 172.21.15.70:0/684972756 >> 172.21.15.70:0/684972756 conn(0x7f81740f7480 msgr2=0x7f81740f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:42.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.326+0000 7f817a674700 1 -- 172.21.15.70:0/684972756 shutdown_connections 2024-03-23T08:23:42.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.326+0000 7f817a674700 1 -- 172.21.15.70:0/684972756 wait complete. 2024-03-23T08:23:42.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f817a674700 1 Processor -- start 2024-03-23T08:23:42.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f817a674700 1 -- start start 2024-03-23T08:23:42.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f817a674700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8174066ef0 0x7f817410d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:42.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f817a674700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 0x7f817410dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:42.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f817a674700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817410e320 con 0x7f81740fffb0 2024-03-23T08:23:42.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f817a674700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817410e490 con 0x7f8174066ef0 2024-03-23T08:23:42.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f81737fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 0x7f817410dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:42.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f81737fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 0x7f817410dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55804/0 (socket says 172.21.15.70:55804) 2024-03-23T08:23:42.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f81737fe700 1 -- 172.21.15.70:0/1986876134 learned_addr learned my addr 172.21.15.70:0/1986876134 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:42.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.327+0000 7f8173fff700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8174066ef0 0x7f817410d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:42.328 INFO:tasks.workunit.client.0.smithi070.stdout:1/93: creat d8/db/f27 x:0 0 0 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f81737fe700 1 -- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8174066ef0 msgr2=0x7f817410d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f81737fe700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8174066ef0 0x7f817410d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f81737fe700 1 -- 172.21.15.70:0/1986876134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8168005040 con 0x7f81740fffb0 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f8173fff700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8174066ef0 0x7f817410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f81737fe700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 0x7f817410dd20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8168007d40 tx=0x7f8168009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f8170ff9700 1 -- 172.21.15.70:0/1986876134 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f816800e030 con 0x7f81740fffb0 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f8170ff9700 1 -- 172.21.15.70:0/1986876134 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f816801c400 con 0x7f81740fffb0 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8174112e90 con 0x7f81740fffb0 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f8170ff9700 1 -- 172.21.15.70:0/1986876134 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f816800b650 con 0x7f81740fffb0 2024-03-23T08:23:42.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.328+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8174113420 con 0x7f81740fffb0 2024-03-23T08:23:42.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.329+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817404c710 con 0x7f81740fffb0 2024-03-23T08:23:42.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.330+0000 7f8170ff9700 1 -- 172.21.15.70:0/1986876134 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f816801c570 con 0x7f81740fffb0 2024-03-23T08:23:42.332 INFO:tasks.workunit.client.0.smithi070.stdout:1/94: link f3 d8/d1a/f28 0 2024-03-23T08:23:42.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.330+0000 7f8170ff9700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f815c06aed0 0x7f815c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:42.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.331+0000 7f8170ff9700 1 -- 172.21.15.70:0/1986876134 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f816808ad10 con 0x7f81740fffb0 2024-03-23T08:23:42.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.331+0000 7f8173fff700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f815c06aed0 0x7f815c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:42.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.331+0000 7f8173fff700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f815c06aed0 0x7f815c06d380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8164009b80 tx=0x7f816400b040 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:42.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.335+0000 7f8170ff9700 1 -- 172.21.15.70:0/1986876134 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f8168058d10 con 0x7f81740fffb0 2024-03-23T08:23:42.340 INFO:tasks.workunit.client.0.smithi070.stdout:1/95: dwrite d8/f9 [4194304,4194304] 0 2024-03-23T08:23:42.375 INFO:tasks.workunit.client.0.smithi070.stdout:1/96: write d8/db/f16 [474340,5798] 0 2024-03-23T08:23:42.378 INFO:tasks.workunit.client.0.smithi070.stdout:1/97: creat d8/d11/f29 x:0 0 0 2024-03-23T08:23:42.386 INFO:tasks.workunit.client.0.smithi070.stdout:1/98: dwrite d8/d1b/f24 [0,4194304] 0 2024-03-23T08:23:42.393 INFO:tasks.workunit.client.0.smithi070.stdout:1/99: dread d8/db/f21 [0,4194304] 0 2024-03-23T08:23:42.394 INFO:tasks.workunit.client.0.smithi070.stdout:1/100: dread - d8/f1c zero size 2024-03-23T08:23:42.397 INFO:tasks.workunit.client.0.smithi070.stdout:5/40: sync 2024-03-23T08:23:42.397 INFO:tasks.workunit.client.0.smithi070.stdout:9/47: sync 2024-03-23T08:23:42.397 INFO:tasks.workunit.client.0.smithi070.stdout:9/48: fdatasync f7 0 2024-03-23T08:23:42.398 INFO:tasks.workunit.client.0.smithi070.stdout:1/101: symlink d8/db/l2a 0 2024-03-23T08:23:42.398 INFO:tasks.workunit.client.0.smithi070.stdout:5/41: write f4 [1981763,1141] 0 2024-03-23T08:23:42.400 INFO:tasks.workunit.client.0.smithi070.stdout:9/49: symlink db/lf 0 2024-03-23T08:23:42.404 INFO:tasks.workunit.client.0.smithi070.stdout:5/42: creat f7 x:0 0 0 2024-03-23T08:23:42.404 INFO:tasks.workunit.client.0.smithi070.stdout:9/50: symlink db/l10 0 2024-03-23T08:23:42.406 INFO:tasks.workunit.client.0.smithi070.stdout:1/102: dread d8/f9 [0,4194304] 0 2024-03-23T08:23:42.406 INFO:tasks.workunit.client.0.smithi070.stdout:9/51: symlink db/l11 0 2024-03-23T08:23:42.407 INFO:tasks.workunit.client.0.smithi070.stdout:9/52: readlink db/l10 0 2024-03-23T08:23:42.407 INFO:tasks.workunit.client.0.smithi070.stdout:1/103: read f3 [2567463,85098] 0 2024-03-23T08:23:42.436 INFO:tasks.workunit.client.0.smithi070.stdout:9/53: mkdir db/d12 0 2024-03-23T08:23:42.442 INFO:tasks.workunit.client.0.smithi070.stdout:1/104: fsync d8/db/f21 0 2024-03-23T08:23:42.444 INFO:tasks.workunit.client.0.smithi070.stdout:1/105: truncate d8/db/f1f 477788 0 2024-03-23T08:23:42.451 INFO:tasks.workunit.client.0.smithi070.stdout:1/106: unlink d8/l19 0 2024-03-23T08:23:42.460 INFO:tasks.workunit.client.0.smithi070.stdout:9/54: rmdir db 39 2024-03-23T08:23:42.474 INFO:tasks.workunit.client.0.smithi070.stdout:9/55: link db/l10 db/l13 0 2024-03-23T08:23:42.477 INFO:tasks.workunit.client.0.smithi070.stdout:9/56: readlink db/lc 0 2024-03-23T08:23:42.479 INFO:tasks.workunit.client.0.smithi070.stdout:1/107: write d8/db/f15 [410346,54360] 0 2024-03-23T08:23:42.491 INFO:tasks.workunit.client.0.smithi070.stdout:9/57: dwrite f6 [0,4194304] 0 2024-03-23T08:23:42.493 INFO:tasks.workunit.client.0.smithi070.stdout:9/58: chown db/l11 76098323 1 2024-03-23T08:23:42.496 INFO:tasks.workunit.client.0.smithi070.stdout:1/108: write f3 [2779277,128063] 0 2024-03-23T08:23:42.499 INFO:tasks.workunit.client.0.smithi070.stdout:9/59: unlink l3 0 2024-03-23T08:23:42.499 INFO:tasks.workunit.client.0.smithi070.stdout:1/109: chown d8/db 118578744 1 2024-03-23T08:23:42.503 INFO:tasks.workunit.client.0.smithi070.stdout:9/60: dread - f8 zero size 2024-03-23T08:23:42.507 INFO:tasks.workunit.client.0.smithi070.stdout:1/110: mknod d8/d1a/c2b 0 2024-03-23T08:23:42.514 INFO:tasks.workunit.client.0.smithi070.stdout:9/61: mkdir db/d12/d14 0 2024-03-23T08:23:42.516 INFO:tasks.workunit.client.0.smithi070.stdout:9/62: mknod db/c15 0 2024-03-23T08:23:42.519 INFO:tasks.workunit.client.0.smithi070.stdout:9/63: rmdir db/d12 39 2024-03-23T08:23:42.546 INFO:tasks.workunit.client.0.smithi070.stdout:9/64: dwrite f8 [0,4194304] 0 2024-03-23T08:23:42.548 INFO:tasks.workunit.client.0.smithi070.stdout:1/111: dwrite d8/db/f1f [0,4194304] 0 2024-03-23T08:23:42.550 INFO:tasks.workunit.client.0.smithi070.stdout:1/112: fdatasync d8/d1a/f23 0 2024-03-23T08:23:42.557 INFO:tasks.workunit.client.0.smithi070.stdout:9/65: mknod db/d12/c16 0 2024-03-23T08:23:42.562 INFO:tasks.workunit.client.0.smithi070.stdout:1/113: readlink d8/db/l17 0 2024-03-23T08:23:42.562 INFO:tasks.workunit.client.0.smithi070.stdout:1/114: fsync d8/d1a/f1e 0 2024-03-23T08:23:42.562 INFO:tasks.workunit.client.0.smithi070.stdout:9/66: dread f6 [0,4194304] 0 2024-03-23T08:23:42.564 INFO:tasks.workunit.client.0.smithi070.stdout:9/67: creat db/d12/f17 x:0 0 0 2024-03-23T08:23:42.567 INFO:tasks.workunit.client.0.smithi070.stdout:9/68: unlink f7 0 2024-03-23T08:23:42.572 INFO:tasks.workunit.client.0.smithi070.stdout:9/69: mkdir db/d18 0 2024-03-23T08:23:42.572 INFO:tasks.workunit.client.0.smithi070.stdout:1/115: dwrite d8/db/f27 [0,4194304] 0 2024-03-23T08:23:42.576 INFO:tasks.workunit.client.0.smithi070.stdout:1/116: mkdir d8/db/d2c 0 2024-03-23T08:23:42.576 INFO:tasks.workunit.client.0.smithi070.stdout:0/40: sync 2024-03-23T08:23:42.576 INFO:tasks.workunit.client.0.smithi070.stdout:4/18: sync 2024-03-23T08:23:42.576 INFO:tasks.workunit.client.0.smithi070.stdout:3/107: sync 2024-03-23T08:23:42.577 INFO:tasks.workunit.client.0.smithi070.stdout:9/70: dread f4 [0,4194304] 0 2024-03-23T08:23:42.577 INFO:tasks.workunit.client.0.smithi070.stdout:3/108: fdatasync db/df/f19 0 2024-03-23T08:23:42.578 INFO:tasks.workunit.client.0.smithi070.stdout:0/41: symlink le 0 2024-03-23T08:23:42.579 INFO:tasks.workunit.client.0.smithi070.stdout:0/42: readlink l0 0 2024-03-23T08:23:42.579 INFO:tasks.workunit.client.0.smithi070.stdout:9/71: write f4 [3730125,15475] 0 2024-03-23T08:23:42.581 INFO:tasks.workunit.client.0.smithi070.stdout:4/19: mkdir d3/d5 0 2024-03-23T08:23:42.582 INFO:tasks.workunit.client.0.smithi070.stdout:4/20: fsync f0 0 2024-03-23T08:23:42.586 INFO:tasks.workunit.client.0.smithi070.stdout:9/72: creat db/d18/f19 x:0 0 0 2024-03-23T08:23:42.587 INFO:tasks.workunit.client.0.smithi070.stdout:4/21: dread f0 [0,4194304] 0 2024-03-23T08:23:42.588 INFO:tasks.workunit.client.0.smithi070.stdout:9/73: stat f6 0 2024-03-23T08:23:42.589 INFO:tasks.workunit.client.0.smithi070.stdout:1/117: fsync d8/db/f27 0 2024-03-23T08:23:42.589 INFO:tasks.workunit.client.0.smithi070.stdout:9/74: chown l0 0 1 2024-03-23T08:23:42.591 INFO:tasks.workunit.client.0.smithi070.stdout:4/22: unlink d3/c4 0 2024-03-23T08:23:42.592 INFO:tasks.workunit.client.0.smithi070.stdout:9/75: unlink db/fe 0 2024-03-23T08:23:42.594 INFO:tasks.workunit.client.0.smithi070.stdout:4/23: mkdir d3/d6 0 2024-03-23T08:23:42.595 INFO:tasks.workunit.client.0.smithi070.stdout:1/118: dread d8/db/f21 [0,4194304] 0 2024-03-23T08:23:42.595 INFO:tasks.workunit.client.0.smithi070.stdout:9/76: creat db/d12/f1a x:0 0 0 2024-03-23T08:23:42.597 INFO:tasks.workunit.client.0.smithi070.stdout:1/119: fdatasync d8/db/f27 0 2024-03-23T08:23:42.604 INFO:tasks.workunit.client.0.smithi070.stdout:1/120: symlink d8/d26/l2d 0 2024-03-23T08:23:42.606 INFO:tasks.workunit.client.0.smithi070.stdout:9/77: creat db/f1b x:0 0 0 2024-03-23T08:23:42.607 INFO:tasks.workunit.client.0.smithi070.stdout:4/24: dread f1 [0,4194304] 0 2024-03-23T08:23:42.607 INFO:tasks.workunit.client.0.smithi070.stdout:3/109: dwrite db/f17 [4194304,4194304] 0 2024-03-23T08:23:42.608 INFO:tasks.workunit.client.0.smithi070.stdout:1/121: chown d8/db/f15 432300 1 2024-03-23T08:23:42.610 INFO:tasks.workunit.client.0.smithi070.stdout:9/78: chown db/d18/f19 17093416 1 2024-03-23T08:23:42.611 INFO:tasks.workunit.client.0.smithi070.stdout:0/43: dwrite f8 [0,4194304] 0 2024-03-23T08:23:42.613 INFO:tasks.workunit.client.0.smithi070.stdout:3/110: symlink db/df/l2f 0 2024-03-23T08:23:42.613 INFO:tasks.workunit.client.0.smithi070.stdout:3/111: readlink db/l10 0 2024-03-23T08:23:42.619 INFO:tasks.workunit.client.0.smithi070.stdout:4/25: rename f0 to d3/d6/f7 0 2024-03-23T08:23:42.622 INFO:tasks.workunit.client.0.smithi070.stdout:3/112: dwrite db/df/f19 [0,4194304] 0 2024-03-23T08:23:42.622 INFO:tasks.workunit.client.0.smithi070.stdout:1/122: mknod d8/db/c2e 0 2024-03-23T08:23:42.624 INFO:tasks.workunit.client.0.smithi070.stdout:0/44: dread - fb zero size 2024-03-23T08:23:42.626 INFO:tasks.workunit.client.0.smithi070.stdout:9/79: rename l0 to db/d18/l1c 0 2024-03-23T08:23:42.628 INFO:tasks.workunit.client.0.smithi070.stdout:1/123: rmdir d8/d1b 39 2024-03-23T08:23:42.630 INFO:tasks.workunit.client.0.smithi070.stdout:3/113: unlink db/df/f19 0 2024-03-23T08:23:42.631 INFO:tasks.workunit.client.0.smithi070.stdout:0/45: fsync fb 0 2024-03-23T08:23:42.636 INFO:tasks.workunit.client.0.smithi070.stdout:9/80: readlink db/d18/l1c 0 2024-03-23T08:23:42.636 INFO:tasks.workunit.client.0.smithi070.stdout:9/81: chown db/lc 621257121 1 2024-03-23T08:23:42.639 INFO:tasks.workunit.client.0.smithi070.stdout:4/26: write f1 [4519935,111721] 0 2024-03-23T08:23:42.640 INFO:tasks.workunit.client.0.smithi070.stdout:3/114: rmdir db/df/d15 39 2024-03-23T08:23:42.642 INFO:tasks.workunit.client.0.smithi070.stdout:0/46: truncate f2 2789336 0 2024-03-23T08:23:42.643 INFO:tasks.workunit.client.0.smithi070.stdout:9/82: rmdir db/d12 39 2024-03-23T08:23:42.644 INFO:tasks.workunit.client.0.smithi070.stdout:4/27: creat d3/d6/f8 x:0 0 0 2024-03-23T08:23:42.645 INFO:tasks.workunit.client.0.smithi070.stdout:4/28: chown d3 1 1 2024-03-23T08:23:42.646 INFO:tasks.workunit.client.0.smithi070.stdout:4/29: write f1 [4528129,45739] 0 2024-03-23T08:23:42.647 INFO:tasks.workunit.client.0.smithi070.stdout:6/255: sync 2024-03-23T08:23:42.647 INFO:tasks.workunit.client.0.smithi070.stdout:2/32: sync 2024-03-23T08:23:42.648 INFO:tasks.workunit.client.0.smithi070.stdout:8/30: sync 2024-03-23T08:23:42.650 INFO:tasks.workunit.client.0.smithi070.stdout:6/256: chown d0/d23/d28/d35/d3a/d3c 27136892 1 2024-03-23T08:23:42.650 INFO:tasks.workunit.client.0.smithi070.stdout:1/124: creat d8/f2f x:0 0 0 2024-03-23T08:23:42.654 INFO:tasks.workunit.client.0.smithi070.stdout:9/83: write db/d12/f1a [768127,43666] 0 2024-03-23T08:23:42.656 INFO:tasks.workunit.client.0.smithi070.stdout:3/115: write f6 [1728606,85804] 0 2024-03-23T08:23:42.659 INFO:tasks.workunit.client.0.smithi070.stdout:4/30: dwrite f1 [0,4194304] 0 2024-03-23T08:23:42.664 INFO:tasks.workunit.client.0.smithi070.stdout:6/257: truncate d0/d3/f4 3015842 0 2024-03-23T08:23:42.666 INFO:tasks.workunit.client.0.smithi070.stdout:9/84: creat db/d12/f1d x:0 0 0 2024-03-23T08:23:42.667 INFO:tasks.workunit.client.0.smithi070.stdout:2/33: rename d1/f5 to d1/d7/f8 0 2024-03-23T08:23:42.669 INFO:tasks.workunit.client.0.smithi070.stdout:3/116: unlink db/f25 0 2024-03-23T08:23:42.669 INFO:tasks.workunit.client.0.smithi070.stdout:6/258: dread d0/d23/f10 [0,4194304] 0 2024-03-23T08:23:42.672 INFO:tasks.workunit.client.0.smithi070.stdout:9/85: dread f4 [0,4194304] 0 2024-03-23T08:23:42.672 INFO:tasks.workunit.client.0.smithi070.stdout:9/86: chown db/d12/d14 848030 1 2024-03-23T08:23:42.672 INFO:tasks.workunit.client.0.smithi070.stdout:9/87: stat f8 0 2024-03-23T08:23:42.673 INFO:tasks.workunit.client.0.smithi070.stdout:1/125: mknod d8/db/d2c/c30 0 2024-03-23T08:23:42.674 INFO:tasks.workunit.client.0.smithi070.stdout:4/31: mkdir d3/d6/d9 0 2024-03-23T08:23:42.674 INFO:tasks.workunit.client.0.smithi070.stdout:1/126: write d8/db/f15 [1468310,47115] 0 2024-03-23T08:23:42.675 INFO:tasks.workunit.client.0.smithi070.stdout:4/32: chown f2 924874533 1 2024-03-23T08:23:42.676 INFO:tasks.workunit.client.0.smithi070.stdout:4/33: rename d3 to d3/d5/da 22 2024-03-23T08:23:42.678 INFO:tasks.workunit.client.0.smithi070.stdout:9/88: dread f8 [0,4194304] 0 2024-03-23T08:23:42.678 INFO:tasks.workunit.client.0.smithi070.stdout:4/34: chown d3/d6/f7 64980633 1 2024-03-23T08:23:42.678 INFO:tasks.workunit.client.0.smithi070.stdout:8/31: link d5/d8/fb d5/d8/fc 0 2024-03-23T08:23:42.680 INFO:tasks.workunit.client.0.smithi070.stdout:8/32: truncate d5/d8/fb 2105169 0 2024-03-23T08:23:42.682 INFO:tasks.workunit.client.0.smithi070.stdout:4/35: dread f1 [0,4194304] 0 2024-03-23T08:23:42.687 INFO:tasks.workunit.client.0.smithi070.stdout:2/34: dread f0 [0,4194304] 0 2024-03-23T08:23:42.688 INFO:tasks.workunit.client.0.smithi070.stdout:6/259: dread d0/d23/f17 [0,4194304] 0 2024-03-23T08:23:42.689 INFO:tasks.workunit.client.0.smithi070.stdout:6/260: chown d0/d3/c15 316 1 2024-03-23T08:23:42.690 INFO:tasks.workunit.client.0.smithi070.stdout:3/117: symlink db/df/d1a/l30 0 2024-03-23T08:23:42.694 INFO:tasks.workunit.client.0.smithi070.stdout:1/127: chown d8/d1b/f22 7708603 1 2024-03-23T08:23:42.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.694+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8174113610 con 0x7f815c06aed0 2024-03-23T08:23:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.695+0000 7f8170ff9700 1 -- 172.21.15.70:0/1986876134 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f8174113610 con 0x7f815c06aed0 2024-03-23T08:23:42.698 INFO:tasks.workunit.client.0.smithi070.stdout:1/128: dread d8/d1a/f28 [0,4194304] 0 2024-03-23T08:23:42.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f815c06aed0 msgr2=0x7f815c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:42.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f815c06aed0 0x7f815c06d380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8164009b80 tx=0x7f816400b040 comp rx=0 tx=0).stop 2024-03-23T08:23:42.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 msgr2=0x7f817410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:42.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 0x7f817410dd20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8168007d40 tx=0x7f8168009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:42.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 shutdown_connections 2024-03-23T08:23:42.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f815c06aed0 0x7f815c06d380 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8164009b80 tx=0x7f816400b040 comp rx=0 tx=0).stop 2024-03-23T08:23:42.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8174066ef0 0x7f817410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 --2- 172.21.15.70:0/1986876134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81740fffb0 0x7f817410dd20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 >> 172.21.15.70:0/1986876134 conn(0x7f81740f7480 msgr2=0x7f81740f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:42.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 shutdown_connections 2024-03-23T08:23:42.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.698+0000 7f817a674700 1 -- 172.21.15.70:0/1986876134 wait complete. 2024-03-23T08:23:42.700 INFO:tasks.workunit.client.0.smithi070.stdout:1/129: chown d8/d1a/f28 173054901 1 2024-03-23T08:23:42.700 INFO:tasks.workunit.client.0.smithi070.stdout:4/36: fdatasync d3/d6/f7 0 2024-03-23T08:23:42.702 INFO:tasks.workunit.client.0.smithi070.stdout:2/35: mknod d1/d7/c9 0 2024-03-23T08:23:42.702 INFO:tasks.workunit.client.0.smithi070.stdout:0/47: write f2 [833042,123107] 0 2024-03-23T08:23:42.703 INFO:tasks.workunit.client.0.smithi070.stdout:5/43: sync 2024-03-23T08:23:42.703 INFO:tasks.workunit.client.0.smithi070.stdout:7/53: sync 2024-03-23T08:23:42.705 INFO:tasks.workunit.client.0.smithi070.stdout:6/261: fsync d0/d23/f2a 0 2024-03-23T08:23:42.707 INFO:tasks.workunit.client.0.smithi070.stdout:6/262: stat d0/d23/d1c/d2f/f4c 0 2024-03-23T08:23:42.712 INFO:tasks.workunit.client.0.smithi070.stdout:9/89: unlink db/l13 0 2024-03-23T08:23:42.714 INFO:tasks.workunit.client.0.smithi070.stdout:0/48: dwrite f2 [0,4194304] 0 2024-03-23T08:23:42.718 INFO:tasks.workunit.client.0.smithi070.stdout:8/33: dwrite d5/d8/fb [0,4194304] 0 2024-03-23T08:23:42.718 INFO:tasks.workunit.client.0.smithi070.stdout:8/34: chown d5/f7 907 1 2024-03-23T08:23:42.720 INFO:tasks.workunit.client.0.smithi070.stdout:4/37: creat d3/fb x:0 0 0 2024-03-23T08:23:42.720 INFO:tasks.workunit.client.0.smithi070.stdout:4/38: readlink - no filename 2024-03-23T08:23:42.725 INFO:tasks.workunit.client.0.smithi070.stdout:7/54: chown l11 800899 1 2024-03-23T08:23:42.725 INFO:tasks.workunit.client.0.smithi070.stdout:5/44: creat f8 x:0 0 0 2024-03-23T08:23:42.726 INFO:tasks.workunit.client.0.smithi070.stdout:3/118: unlink db/df/d15/f1e 0 2024-03-23T08:23:42.728 INFO:tasks.workunit.client.0.smithi070.stdout:6/263: write d0/d3/d2c/f58 [120068,73936] 0 2024-03-23T08:23:42.731 INFO:tasks.workunit.client.0.smithi070.stdout:2/36: dread d1/d7/f8 [0,4194304] 0 2024-03-23T08:23:42.731 INFO:tasks.workunit.client.0.smithi070.stdout:1/130: dwrite d8/d1b/f25 [0,4194304] 0 2024-03-23T08:23:42.734 INFO:tasks.workunit.client.0.smithi070.stdout:7/55: dread fd [0,4194304] 0 2024-03-23T08:23:42.735 INFO:tasks.workunit.client.0.smithi070.stdout:8/35: symlink d5/ld 0 2024-03-23T08:23:42.737 INFO:tasks.workunit.client.0.smithi070.stdout:8/36: write d5/d8/fc [1484059,125095] 0 2024-03-23T08:23:42.738 INFO:tasks.workunit.client.0.smithi070.stdout:1/131: dread d8/db/f1f [0,4194304] 0 2024-03-23T08:23:42.740 INFO:tasks.workunit.client.0.smithi070.stdout:6/264: dread - d0/d23/d4d/f54 zero size 2024-03-23T08:23:42.740 INFO:tasks.workunit.client.0.smithi070.stdout:4/39: dread f1 [0,4194304] 0 2024-03-23T08:23:42.746 INFO:tasks.workunit.client.0.smithi070.stdout:8/37: write d5/f7 [1388544,13484] 0 2024-03-23T08:23:42.748 INFO:tasks.workunit.client.0.smithi070.stdout:3/119: mknod db/c31 0 2024-03-23T08:23:42.751 INFO:tasks.workunit.client.0.smithi070.stdout:7/56: dwrite f1 [0,4194304] 0 2024-03-23T08:23:42.755 INFO:tasks.workunit.client.0.smithi070.stdout:4/40: rename d3/fb to d3/fc 0 2024-03-23T08:23:42.756 INFO:tasks.workunit.client.0.smithi070.stdout:6/265: mkdir d0/d23/d28/d35/d3a/d3c/d5b 0 2024-03-23T08:23:42.756 INFO:tasks.workunit.client.0.smithi070.stdout:5/45: link c6 c9 0 2024-03-23T08:23:42.759 INFO:tasks.workunit.client.0.smithi070.stdout:8/38: creat d5/fe x:0 0 0 2024-03-23T08:23:42.760 INFO:tasks.workunit.client.0.smithi070.stdout:9/90: getdents db/d12 0 2024-03-23T08:23:42.761 INFO:tasks.workunit.client.0.smithi070.stdout:3/120: creat db/df/d15/d22/f32 x:0 0 0 2024-03-23T08:23:42.762 INFO:tasks.workunit.client.0.smithi070.stdout:0/49: write f3 [3744111,75032] 0 2024-03-23T08:23:42.764 INFO:tasks.workunit.client.0.smithi070.stdout:7/57: mkdir d14 0 2024-03-23T08:23:42.766 INFO:tasks.workunit.client.0.smithi070.stdout:4/41: creat d3/fd x:0 0 0 2024-03-23T08:23:42.770 INFO:tasks.workunit.client.0.smithi070.stdout:2/37: write d1/d7/f8 [1576110,27177] 0 2024-03-23T08:23:42.772 INFO:tasks.workunit.client.0.smithi070.stdout:5/46: dwrite f4 [4194304,4194304] 0 2024-03-23T08:23:42.782 INFO:tasks.workunit.client.0.smithi070.stdout:0/50: dwrite f3 [4194304,4194304] 0 2024-03-23T08:23:42.785 INFO:tasks.workunit.client.0.smithi070.stdout:3/121: rmdir db/df 39 2024-03-23T08:23:42.787 INFO:tasks.workunit.client.0.smithi070.stdout:4/42: dwrite d3/d6/f7 [0,4194304] 0 2024-03-23T08:23:42.790 INFO:tasks.workunit.client.0.smithi070.stdout:6/266: write d0/d23/d28/d35/d3a/d3c/f53 [404395,128845] 0 2024-03-23T08:23:42.792 INFO:tasks.workunit.client.0.smithi070.stdout:5/47: dwrite f7 [0,4194304] 0 2024-03-23T08:23:42.794 INFO:tasks.workunit.client.0.smithi070.stdout:9/91: creat db/f1e x:0 0 0 2024-03-23T08:23:42.797 INFO:tasks.workunit.client.0.smithi070.stdout:9/92: chown l1 40171044 1 2024-03-23T08:23:42.801 INFO:tasks.workunit.client.0.smithi070.stdout:8/39: truncate d5/d8/fc 3859818 0 2024-03-23T08:23:42.803 INFO:tasks.workunit.client.0.smithi070.stdout:7/58: rmdir d14 0 2024-03-23T08:23:42.803 INFO:tasks.workunit.client.0.smithi070.stdout:7/59: rmdir - no directory 2024-03-23T08:23:42.805 INFO:tasks.workunit.client.0.smithi070.stdout:4/43: dwrite d3/fd [0,4194304] 0 2024-03-23T08:23:42.809 INFO:tasks.workunit.client.0.smithi070.stdout:4/44: write d3/fd [1376311,30392] 0 2024-03-23T08:23:42.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.851+0000 7fa0881e1700 1 -- 172.21.15.70:0/1380106923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fdfe0 msgr2=0x7fa0800fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:42.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.851+0000 7fa0881e1700 1 --2- 172.21.15.70:0/1380106923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fdfe0 0x7fa0800fe450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa070004940 tx=0x7fa070009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:42.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.851+0000 7fa0881e1700 1 -- 172.21.15.70:0/1380106923 shutdown_connections 2024-03-23T08:23:42.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.851+0000 7fa0881e1700 1 --2- 172.21.15.70:0/1380106923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fdfe0 0x7fa0800fe450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.851+0000 7fa0881e1700 1 --2- 172.21.15.70:0/1380106923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0800fce70 0x7fa0800fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.851+0000 7fa0881e1700 1 -- 172.21.15.70:0/1380106923 >> 172.21.15.70:0/1380106923 conn(0x7fa0800f8550 msgr2=0x7fa0800fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.851+0000 7fa0881e1700 1 -- 172.21.15.70:0/1380106923 shutdown_connections 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.851+0000 7fa0881e1700 1 -- 172.21.15.70:0/1380106923 wait complete. 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa0881e1700 1 Processor -- start 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa0881e1700 1 -- start start 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa0881e1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fce70 0x7fa08010d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa0881e1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0800fdfe0 0x7fa08010dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa0881e1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08010e3e0 con 0x7fa0800fce70 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa0881e1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa080112d80 con 0x7fa0800fdfe0 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa085f7d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fce70 0x7fa08010d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa085f7d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fce70 0x7fa08010d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55818/0 (socket says 172.21.15.70:55818) 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa085f7d700 1 -- 172.21.15.70:0/1896080058 learned_addr learned my addr 172.21.15.70:0/1896080058 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.852+0000 7fa08577c700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0800fdfe0 0x7fa08010dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:42.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.853+0000 7fa085f7d700 1 -- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0800fdfe0 msgr2=0x7fa08010dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:42.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.853+0000 7fa085f7d700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0800fdfe0 0x7fa08010dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:42.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.853+0000 7fa085f7d700 1 -- 172.21.15.70:0/1896080058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa070005040 con 0x7fa0800fce70 2024-03-23T08:23:42.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.853+0000 7fa085f7d700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fce70 0x7fa08010d810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00b790 tx=0x7fa07c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:42.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.853+0000 7fa0767fc700 1 -- 172.21.15.70:0/1896080058 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa07c00f6f0 con 0x7fa0800fce70 2024-03-23T08:23:42.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.853+0000 7fa0767fc700 1 -- 172.21.15.70:0/1896080058 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa07c00fd00 con 0x7fa0800fce70 2024-03-23T08:23:42.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.853+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa080112fe0 con 0x7fa0800fce70 2024-03-23T08:23:42.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.853+0000 7fa0767fc700 1 -- 172.21.15.70:0/1896080058 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa07c0103a0 con 0x7fa0800fce70 2024-03-23T08:23:42.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.853+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0801134c0 con 0x7fa0800fce70 2024-03-23T08:23:42.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.854+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa08004c710 con 0x7fa0800fce70 2024-03-23T08:23:42.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.854+0000 7fa0767fc700 1 -- 172.21.15.70:0/1896080058 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fa07c00f850 con 0x7fa0800fce70 2024-03-23T08:23:42.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.855+0000 7fa0767fc700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fa06c06aed0 0x7fa06c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:42.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.855+0000 7fa0767fc700 1 -- 172.21.15.70:0/1896080058 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa07c0894a0 con 0x7fa0800fce70 2024-03-23T08:23:42.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.855+0000 7fa08577c700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fa06c06aed0 0x7fa06c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:42.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.855+0000 7fa08577c700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fa06c06aed0 0x7fa06c06d380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa07000b3b0 tx=0x7fa070009e30 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:42.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:42.859+0000 7fa0767fc700 1 -- 172.21.15.70:0/1896080058 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fa07c057550 con 0x7fa0800fce70 2024-03-23T08:23:42.870 INFO:tasks.workunit.client.0.smithi070.stdout:9/93: fdatasync f8 0 2024-03-23T08:23:42.872 INFO:tasks.workunit.client.0.smithi070.stdout:4/45: chown f1 11577 1 2024-03-23T08:23:42.872 INFO:tasks.workunit.client.0.smithi070.stdout:6/267: chown d0/d23/d1c/l4b 0 1 2024-03-23T08:23:42.873 INFO:tasks.workunit.client.0.smithi070.stdout:3/122: dwrite db/f21 [0,4194304] 0 2024-03-23T08:23:42.875 INFO:tasks.workunit.client.0.smithi070.stdout:0/51: fsync f3 0 2024-03-23T08:23:42.880 INFO:tasks.workunit.client.0.smithi070.stdout:2/38: link d1/l2 d1/la 0 2024-03-23T08:23:42.882 INFO:tasks.workunit.client.0.smithi070.stdout:9/94: dread db/d12/f1a [0,4194304] 0 2024-03-23T08:23:42.888 INFO:tasks.workunit.client.0.smithi070.stdout:7/60: truncate f12 2209567 0 2024-03-23T08:23:42.892 INFO:tasks.workunit.client.0.smithi070.stdout:2/39: chown d1/la 1 1 2024-03-23T08:23:42.893 INFO:tasks.workunit.client.0.smithi070.stdout:5/48: link c6 ca 0 2024-03-23T08:23:42.893 INFO:tasks.workunit.client.0.smithi070.stdout:5/49: chown l5 62 1 2024-03-23T08:23:42.895 INFO:tasks.workunit.client.0.smithi070.stdout:6/268: unlink d0/f25 0 2024-03-23T08:23:42.896 INFO:tasks.workunit.client.0.smithi070.stdout:3/123: creat db/d20/f33 x:0 0 0 2024-03-23T08:23:42.898 INFO:tasks.workunit.client.0.smithi070.stdout:0/52: write fb [1035174,84051] 0 2024-03-23T08:23:42.900 INFO:tasks.workunit.client.0.smithi070.stdout:8/40: getdents d5/d8 0 2024-03-23T08:23:42.902 INFO:tasks.workunit.client.0.smithi070.stdout:0/53: dread f8 [0,4194304] 0 2024-03-23T08:23:42.904 INFO:tasks.workunit.client.0.smithi070.stdout:5/50: creat fb x:0 0 0 2024-03-23T08:23:42.905 INFO:tasks.workunit.client.0.smithi070.stdout:4/46: truncate d3/fd 847485 0 2024-03-23T08:23:42.909 INFO:tasks.workunit.client.0.smithi070.stdout:5/51: dread f4 [0,4194304] 0 2024-03-23T08:23:42.909 INFO:tasks.workunit.client.0.smithi070.stdout:0/54: dread fb [0,4194304] 0 2024-03-23T08:23:42.912 INFO:tasks.workunit.client.0.smithi070.stdout:8/41: symlink d5/lf 0 2024-03-23T08:23:42.913 INFO:tasks.workunit.client.0.smithi070.stdout:8/42: write d5/f7 [358820,108042] 0 2024-03-23T08:23:42.914 INFO:tasks.workunit.client.0.smithi070.stdout:2/40: symlink d1/d6/lb 0 2024-03-23T08:23:42.916 INFO:tasks.workunit.client.0.smithi070.stdout:9/95: creat db/f1f x:0 0 0 2024-03-23T08:23:42.919 INFO:tasks.workunit.client.0.smithi070.stdout:6/269: unlink d0/d23/f24 0 2024-03-23T08:23:42.922 INFO:tasks.workunit.client.0.smithi070.stdout:3/124: mknod db/c34 0 2024-03-23T08:23:42.924 INFO:tasks.workunit.client.0.smithi070.stdout:5/52: mkdir dc 0 2024-03-23T08:23:42.925 INFO:tasks.workunit.client.0.smithi070.stdout:7/61: link ca c15 0 2024-03-23T08:23:42.928 INFO:tasks.workunit.client.0.smithi070.stdout:2/41: creat d1/fc x:0 0 0 2024-03-23T08:23:42.929 INFO:tasks.workunit.client.0.smithi070.stdout:8/43: creat d5/d8/f10 x:0 0 0 2024-03-23T08:23:42.931 INFO:tasks.workunit.client.0.smithi070.stdout:9/96: mknod db/d18/c20 0 2024-03-23T08:23:42.931 INFO:tasks.workunit.client.0.smithi070.stdout:9/97: chown f8 245505 1 2024-03-23T08:23:42.932 INFO:tasks.workunit.client.0.smithi070.stdout:9/98: chown db/f1b 1359343111 1 2024-03-23T08:23:42.932 INFO:tasks.workunit.client.0.smithi070.stdout:9/99: chown db/f1f 176576 1 2024-03-23T08:23:42.933 INFO:tasks.workunit.client.0.smithi070.stdout:6/270: stat d0/c2 0 2024-03-23T08:23:42.933 INFO:tasks.workunit.client.0.smithi070.stdout:4/47: mknod d3/d5/ce 0 2024-03-23T08:23:42.936 INFO:tasks.workunit.client.0.smithi070.stdout:3/125: mknod db/d14/c35 0 2024-03-23T08:23:42.936 INFO:tasks.workunit.client.0.smithi070.stdout:3/126: fdatasync f6 0 2024-03-23T08:23:42.936 INFO:tasks.workunit.client.0.smithi070.stdout:3/127: read db/f21 [1188094,111891] 0 2024-03-23T08:23:42.938 INFO:tasks.workunit.client.0.smithi070.stdout:7/62: write f8 [3602109,61462] 0 2024-03-23T08:23:42.940 INFO:tasks.workunit.client.0.smithi070.stdout:2/42: rename d1/f4 to d1/fd 0 2024-03-23T08:23:42.949 INFO:tasks.workunit.client.0.smithi070.stdout:5/53: unlink f4 0 2024-03-23T08:23:42.956 INFO:tasks.workunit.client.0.smithi070.stdout:5/54: dwrite fb [0,4194304] 0 2024-03-23T08:23:42.956 INFO:tasks.workunit.client.0.smithi070.stdout:5/55: chown dc 133053170 1 2024-03-23T08:23:42.960 INFO:tasks.workunit.client.0.smithi070.stdout:9/100: truncate f8 581475 0 2024-03-23T08:23:42.961 INFO:tasks.workunit.client.0.smithi070.stdout:9/101: dread - db/f1b zero size 2024-03-23T08:23:42.961 INFO:tasks.workunit.client.0.smithi070.stdout:0/55: truncate f8 3038426 0 2024-03-23T08:23:42.962 INFO:tasks.workunit.client.0.smithi070.stdout:0/56: chown la 60 1 2024-03-23T08:23:42.965 INFO:tasks.workunit.client.0.smithi070.stdout:0/57: dread f3 [4194304,4194304] 0 2024-03-23T08:23:42.967 INFO:tasks.workunit.client.0.smithi070.stdout:0/58: chown lc 427388 1 2024-03-23T08:23:42.972 INFO:tasks.workunit.client.0.smithi070.stdout:6/271: dwrite d0/d23/d1c/f2e [0,4194304] 0 2024-03-23T08:23:42.978 INFO:tasks.workunit.client.0.smithi070.stdout:8/44: write d5/d8/fb [4405420,16330] 0 2024-03-23T08:23:42.978 INFO:tasks.workunit.client.0.smithi070.stdout:8/45: dread - d5/fe zero size 2024-03-23T08:23:42.981 INFO:tasks.workunit.client.0.smithi070.stdout:8/46: write d5/fe [9168,127151] 0 2024-03-23T08:23:42.984 INFO:tasks.workunit.client.0.smithi070.stdout:7/63: truncate f1 6569788 0 2024-03-23T08:23:42.984 INFO:tasks.workunit.client.0.smithi070.stdout:3/128: creat db/f36 x:0 0 0 2024-03-23T08:23:42.988 INFO:tasks.workunit.client.0.smithi070.stdout:6/272: dread d0/d3/d2c/f34 [0,4194304] 0 2024-03-23T08:23:42.989 INFO:tasks.workunit.client.0.smithi070.stdout:5/56: creat dc/fd x:0 0 0 2024-03-23T08:23:42.989 INFO:tasks.workunit.client.0.smithi070.stdout:5/57: rename dc to dc/de 22 2024-03-23T08:23:42.990 INFO:tasks.workunit.client.0.smithi070.stdout:5/58: readlink l5 0 2024-03-23T08:23:42.990 INFO:tasks.workunit.client.0.smithi070.stdout:5/59: fsync f1 0 2024-03-23T08:23:42.994 INFO:tasks.workunit.client.0.smithi070.stdout:2/43: getdents d1/d6 0 2024-03-23T08:23:42.999 INFO:tasks.workunit.client.0.smithi070.stdout:5/60: unlink fb 0 2024-03-23T08:23:43.003 INFO:tasks.workunit.client.0.smithi070.stdout:2/44: creat d1/fe x:0 0 0 2024-03-23T08:23:43.006 INFO:tasks.workunit.client.0.smithi070.stdout:5/61: mknod dc/cf 0 2024-03-23T08:23:43.011 INFO:tasks.workunit.client.0.smithi070.stdout:0/59: getdents . 0 2024-03-23T08:23:43.011 INFO:tasks.workunit.client.0.smithi070.stdout:7/64: dwrite f12 [0,4194304] 0 2024-03-23T08:23:43.012 INFO:tasks.workunit.client.0.smithi070.stdout:7/65: fsync f8 0 2024-03-23T08:23:43.013 INFO:tasks.workunit.client.0.smithi070.stdout:2/45: mkdir d1/d7/df 0 2024-03-23T08:23:43.015 INFO:tasks.workunit.client.0.smithi070.stdout:8/47: getdents d5 0 2024-03-23T08:23:43.016 INFO:tasks.workunit.client.0.smithi070.stdout:5/62: symlink dc/l10 0 2024-03-23T08:23:43.019 INFO:tasks.workunit.client.0.smithi070.stdout:7/66: creat f16 x:0 0 0 2024-03-23T08:23:43.020 INFO:tasks.workunit.client.0.smithi070.stdout:7/67: readlink l6 0 2024-03-23T08:23:43.021 INFO:tasks.workunit.client.0.smithi070.stdout:2/46: unlink d1/l2 0 2024-03-23T08:23:43.021 INFO:tasks.workunit.client.0.smithi070.stdout:6/273: link d0/d23/d1c/d38/c3f d0/c5c 0 2024-03-23T08:23:43.023 INFO:tasks.workunit.client.0.smithi070.stdout:8/48: creat d5/d8/f11 x:0 0 0 2024-03-23T08:23:43.023 INFO:tasks.workunit.client.0.smithi070.stdout:3/129: getdents db/df/d15 0 2024-03-23T08:23:43.025 INFO:tasks.workunit.client.0.smithi070.stdout:5/63: unlink dc/l10 0 2024-03-23T08:23:43.028 INFO:tasks.workunit.client.0.smithi070.stdout:2/47: mkdir d1/d6/d10 0 2024-03-23T08:23:43.028 INFO:tasks.workunit.client.0.smithi070.stdout:2/48: chown d1/fc 15846952 1 2024-03-23T08:23:43.028 INFO:tasks.workunit.client.0.smithi070.stdout:8/49: mkdir d5/d8/d12 0 2024-03-23T08:23:43.029 INFO:tasks.workunit.client.0.smithi070.stdout:3/130: rmdir db/df 39 2024-03-23T08:23:43.030 INFO:tasks.workunit.client.0.smithi070.stdout:5/64: symlink dc/l11 0 2024-03-23T08:23:43.031 INFO:tasks.workunit.client.0.smithi070.stdout:7/68: truncate f1 4407067 0 2024-03-23T08:23:43.033 INFO:tasks.workunit.client.0.smithi070.stdout:2/49: dread d1/d7/f8 [0,4194304] 0 2024-03-23T08:23:43.034 INFO:tasks.workunit.client.0.smithi070.stdout:2/50: truncate d1/fc 505767 0 2024-03-23T08:23:43.036 INFO:tasks.workunit.client.0.smithi070.stdout:3/131: chown db/df/d15/c2d 16 1 2024-03-23T08:23:43.036 INFO:tasks.workunit.client.0.smithi070.stdout:5/65: dread f7 [0,4194304] 0 2024-03-23T08:23:43.038 INFO:tasks.workunit.client.0.smithi070.stdout:7/69: symlink l17 0 2024-03-23T08:23:43.042 INFO:tasks.workunit.client.0.smithi070.stdout:0/60: truncate f2 465965 0 2024-03-23T08:23:43.043 INFO:tasks.workunit.client.0.smithi070.stdout:2/51: dread d1/d7/f8 [0,4194304] 0 2024-03-23T08:23:43.046 INFO:tasks.workunit.client.0.smithi070.stdout:0/61: dread f3 [4194304,4194304] 0 2024-03-23T08:23:43.048 INFO:tasks.workunit.client.0.smithi070.stdout:3/132: creat db/df/d1a/f37 x:0 0 0 2024-03-23T08:23:43.052 INFO:tasks.workunit.client.0.smithi070.stdout:8/50: rmdir d5/d8/d12 0 2024-03-23T08:23:43.058 INFO:tasks.workunit.client.0.smithi070.stdout:5/66: unlink f7 0 2024-03-23T08:23:43.060 INFO:tasks.workunit.client.0.smithi070.stdout:0/62: creat ff x:0 0 0 2024-03-23T08:23:43.064 INFO:tasks.workunit.client.0.smithi070.stdout:3/133: mknod db/df/d1a/c38 0 2024-03-23T08:23:43.065 INFO:tasks.workunit.client.0.smithi070.stdout:3/134: write db/f2a [330876,130430] 0 2024-03-23T08:23:43.068 INFO:tasks.workunit.client.0.smithi070.stdout:8/51: creat d5/d8/f13 x:0 0 0 2024-03-23T08:23:43.070 INFO:tasks.workunit.client.0.smithi070.stdout:5/67: symlink dc/l12 0 2024-03-23T08:23:43.071 INFO:tasks.workunit.client.0.smithi070.stdout:8/52: write d5/d8/f13 [682777,85878] 0 2024-03-23T08:23:43.072 INFO:tasks.workunit.client.0.smithi070.stdout:0/63: rename la to l10 0 2024-03-23T08:23:43.073 INFO:tasks.workunit.client.0.smithi070.stdout:7/70: link c15 c18 0 2024-03-23T08:23:43.076 INFO:tasks.workunit.client.0.smithi070.stdout:3/135: dwrite db/df/d1a/f37 [0,4194304] 0 2024-03-23T08:23:43.080 INFO:tasks.workunit.client.0.smithi070.stdout:5/68: mknod dc/c13 0 2024-03-23T08:23:43.081 INFO:tasks.workunit.client.0.smithi070.stdout:0/64: dread f8 [0,4194304] 0 2024-03-23T08:23:43.081 INFO:tasks.workunit.client.0.smithi070.stdout:0/65: readlink le 0 2024-03-23T08:23:43.081 INFO:tasks.workunit.client.0.smithi070.stdout:8/53: symlink d5/l14 0 2024-03-23T08:23:43.082 INFO:tasks.workunit.client.0.smithi070.stdout:8/54: read d5/fe [114841,60153] 0 2024-03-23T08:23:43.083 INFO:tasks.workunit.client.0.smithi070.stdout:7/71: mknod c19 0 2024-03-23T08:23:43.085 INFO:tasks.workunit.client.0.smithi070.stdout:2/52: mkdir d1/d7/d11 0 2024-03-23T08:23:43.089 INFO:tasks.workunit.client.0.smithi070.stdout:7/72: rename l17 to l1a 0 2024-03-23T08:23:43.093 INFO:tasks.workunit.client.0.smithi070.stdout:8/55: unlink c3 0 2024-03-23T08:23:43.101 INFO:tasks.workunit.client.0.smithi070.stdout:0/66: write f8 [1081685,59573] 0 2024-03-23T08:23:43.101 INFO:tasks.workunit.client.0.smithi070.stdout:2/53: creat d1/d7/d11/f12 x:0 0 0 2024-03-23T08:23:43.103 INFO:tasks.workunit.client.0.smithi070.stdout:3/136: getdents db/d14 0 2024-03-23T08:23:43.105 INFO:tasks.workunit.client.0.smithi070.stdout:2/54: chown d1/la 4662 1 2024-03-23T08:23:43.105 INFO:tasks.workunit.client.0.smithi070.stdout:8/56: link d5/lf d5/l15 0 2024-03-23T08:23:43.108 INFO:tasks.workunit.client.0.smithi070.stdout:3/137: truncate f6 2366638 0 2024-03-23T08:23:43.109 INFO:tasks.workunit.client.0.smithi070.stdout:3/138: chown db/df/d1a 27 1 2024-03-23T08:23:43.109 INFO:tasks.workunit.client.0.smithi070.stdout:8/57: mkdir d5/d8/d16 0 2024-03-23T08:23:43.110 INFO:tasks.workunit.client.0.smithi070.stdout:3/139: read - db/d20/f33 zero size 2024-03-23T08:23:43.112 INFO:tasks.workunit.client.0.smithi070.stdout:2/55: symlink d1/d7/df/l13 0 2024-03-23T08:23:43.113 INFO:tasks.workunit.client.0.smithi070.stdout:3/140: dread db/d14/f1b [0,4194304] 0 2024-03-23T08:23:43.116 INFO:tasks.workunit.client.0.smithi070.stdout:8/58: dwrite d5/d8/f13 [0,4194304] 0 2024-03-23T08:23:43.120 INFO:tasks.workunit.client.0.smithi070.stdout:2/56: dread - d1/fe zero size 2024-03-23T08:23:43.126 INFO:tasks.workunit.client.0.smithi070.stdout:8/59: rename d5/d8/fc to d5/d8/f17 0 2024-03-23T08:23:43.128 INFO:tasks.workunit.client.0.smithi070.stdout:8/60: chown d5/l14 14027 1 2024-03-23T08:23:43.128 INFO:tasks.workunit.client.0.smithi070.stdout:0/67: dwrite fb [0,4194304] 0 2024-03-23T08:23:43.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.168+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa080102310 con 0x7fa06c06aed0 2024-03-23T08:23:43.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.178+0000 7fa0767fc700 1 -- 172.21.15.70:0/1896080058 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fa080102310 con 0x7fa06c06aed0 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (7m) 41s ago 8m 26.3M - 0.25.0 c8568f914cd2 700168bef09e 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (8m) 41s ago 8m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (7m) 50s ago 7m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (7m) 41s ago 8m 92.4M - 9.4.7 954c08fa6188 87fc95a6ce8f 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (2m) 41s ago 2m 107M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (3m) 41s ago 3m 11.2M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (2m) 50s ago 2m 69.7M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (2m) 50s ago 2m 12.9M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:9283 running (9m) 41s ago 9m 178M - 17.2.7-726-g9c4e0c34 837124da804f 85d0a92e59bc 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283 running (66s) 50s ago 7m 481M - 18.2.2-954-g82316b04 b24e1baba637 d2827fb230ee 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (9m) 41s ago 10m 51.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (7m) 50s ago 7m 39.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 starting - - - - 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (7m) 50s ago 7m 18.6M - 1.5.0 0da6a335fe13 4197d838b05a 2024-03-23T08:23:43.179 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (6m) 41s ago 6m 94.5M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:23:43.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (6m) 41s ago 6m 102M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:23:43.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (5m) 41s ago 5m 89.0M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:23:43.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (5m) 50s ago 5m 111M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:23:43.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (4m) 50s ago 4m 95.2M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:23:43.180 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (4m) 50s ago 4m 94.1M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:23:43.180 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (6m) 41s ago 8m 58.2M - 2.43.0 a07b618ecd1d e842f03864f0 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.180+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fa06c06aed0 msgr2=0x7fa06c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.180+0000 7fa0881e1700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fa06c06aed0 0x7fa06c06d380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa07000b3b0 tx=0x7fa070009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.180+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fce70 msgr2=0x7fa08010d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.180+0000 7fa0881e1700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fce70 0x7fa08010d810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00b790 tx=0x7fa07c009250 comp rx=0 tx=0).stop 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.180+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 shutdown_connections 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.180+0000 7fa0881e1700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fa06c06aed0 0x7fa06c06d380 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.180+0000 7fa0881e1700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0800fce70 0x7fa08010d810 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.180+0000 7fa0881e1700 1 --2- 172.21.15.70:0/1896080058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0800fdfe0 0x7fa08010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.180+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 >> 172.21.15.70:0/1896080058 conn(0x7fa0800f8550 msgr2=0x7fa0800f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.181+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 shutdown_connections 2024-03-23T08:23:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.181+0000 7fa0881e1700 1 -- 172.21.15.70:0/1896080058 wait complete. 2024-03-23T08:23:43.191 INFO:tasks.workunit.client.0.smithi070.stdout:2/57: unlink d1/d6/lb 0 2024-03-23T08:23:43.192 INFO:tasks.workunit.client.0.smithi070.stdout:2/58: fdatasync d1/fe 0 2024-03-23T08:23:43.194 INFO:tasks.workunit.client.0.smithi070.stdout:8/61: dread d5/fe [0,4194304] 0 2024-03-23T08:23:43.199 INFO:tasks.workunit.client.0.smithi070.stdout:8/62: dread d5/d8/f13 [0,4194304] 0 2024-03-23T08:23:43.203 INFO:tasks.workunit.client.0.smithi070.stdout:0/68: truncate f8 2190384 0 2024-03-23T08:23:43.208 INFO:tasks.workunit.client.0.smithi070.stdout:3/141: dwrite f6 [0,4194304] 0 2024-03-23T08:23:43.208 INFO:tasks.workunit.client.0.smithi070.stdout:2/59: dwrite d1/fc [0,4194304] 0 2024-03-23T08:23:43.209 INFO:tasks.workunit.client.0.smithi070.stdout:2/60: rename d1/d7/df to d1/d7/df/d14 22 2024-03-23T08:23:43.211 INFO:tasks.workunit.client.0.smithi070.stdout:3/142: chown db/df/d15 1168628 1 2024-03-23T08:23:43.215 INFO:tasks.workunit.client.0.smithi070.stdout:3/143: dread db/df/f2e [0,4194304] 0 2024-03-23T08:23:43.217 INFO:tasks.workunit.client.0.smithi070.stdout:0/69: truncate f3 6140101 0 2024-03-23T08:23:43.220 INFO:tasks.workunit.client.0.smithi070.stdout:2/61: symlink d1/d6/l15 0 2024-03-23T08:23:43.221 INFO:tasks.workunit.client.0.smithi070.stdout:8/63: dwrite d5/d8/f17 [0,4194304] 0 2024-03-23T08:23:43.221 INFO:tasks.workunit.client.0.smithi070.stdout:2/62: readlink d1/la 0 2024-03-23T08:23:43.226 INFO:tasks.workunit.client.0.smithi070.stdout:0/70: creat f11 x:0 0 0 2024-03-23T08:23:43.227 INFO:tasks.workunit.client.0.smithi070.stdout:0/71: write fb [1404097,22772] 0 2024-03-23T08:23:43.228 INFO:tasks.workunit.client.0.smithi070.stdout:3/144: rename db/df/l1f to db/df/d15/l39 0 2024-03-23T08:23:43.228 INFO:tasks.workunit.client.0.smithi070.stdout:3/145: chown db/d20 2 1 2024-03-23T08:23:43.229 INFO:tasks.workunit.client.0.smithi070.stdout:8/64: creat d5/d8/f18 x:0 0 0 2024-03-23T08:23:43.230 INFO:tasks.workunit.client.0.smithi070.stdout:8/65: write d5/d8/f10 [921948,49316] 0 2024-03-23T08:23:43.230 INFO:tasks.workunit.client.0.smithi070.stdout:3/146: truncate db/d20/f33 789811 0 2024-03-23T08:23:43.231 INFO:tasks.workunit.client.0.smithi070.stdout:8/66: chown d5/d8/f13 4 1 2024-03-23T08:23:43.231 INFO:tasks.workunit.client.0.smithi070.stdout:3/147: stat db/df 0 2024-03-23T08:23:43.232 INFO:tasks.workunit.client.0.smithi070.stdout:0/72: symlink l12 0 2024-03-23T08:23:43.234 INFO:tasks.workunit.client.0.smithi070.stdout:3/148: symlink db/d20/l3a 0 2024-03-23T08:23:43.235 INFO:tasks.workunit.client.0.smithi070.stdout:8/67: rename f4 to d5/d8/f19 0 2024-03-23T08:23:43.238 INFO:tasks.workunit.client.0.smithi070.stdout:3/149: symlink db/df/d15/d22/l3b 0 2024-03-23T08:23:43.239 INFO:tasks.workunit.client.0.smithi070.stdout:8/68: mkdir d5/d8/d16/d1a 0 2024-03-23T08:23:43.240 INFO:tasks.workunit.client.0.smithi070.stdout:3/150: dread db/d20/f33 [0,4194304] 0 2024-03-23T08:23:43.245 INFO:tasks.workunit.client.0.smithi070.stdout:8/69: link d5/d8/f17 d5/d8/d16/d1a/f1b 0 2024-03-23T08:23:43.246 INFO:tasks.workunit.client.0.smithi070.stdout:8/70: stat d5/d8/f11 0 2024-03-23T08:23:43.246 INFO:tasks.workunit.client.0.smithi070.stdout:3/151: dwrite db/df/d1a/f37 [0,4194304] 0 2024-03-23T08:23:43.249 INFO:tasks.workunit.client.0.smithi070.stdout:3/152: creat db/df/d1a/f3c x:0 0 0 2024-03-23T08:23:43.251 INFO:tasks.workunit.client.0.smithi070.stdout:8/71: link d5/l14 d5/d8/l1c 0 2024-03-23T08:23:43.254 INFO:tasks.workunit.client.0.smithi070.stdout:8/72: dread d5/d8/fb [0,4194304] 0 2024-03-23T08:23:43.255 INFO:tasks.workunit.client.0.smithi070.stdout:8/73: write d5/d8/f11 [506210,76973] 0 2024-03-23T08:23:43.258 INFO:tasks.workunit.client.0.smithi070.stdout:8/74: creat d5/d8/f1d x:0 0 0 2024-03-23T08:23:43.261 INFO:tasks.workunit.client.0.smithi070.stdout:8/75: creat d5/d8/f1e x:0 0 0 2024-03-23T08:23:43.264 INFO:tasks.workunit.client.0.smithi070.stdout:8/76: symlink d5/d8/d16/d1a/l1f 0 2024-03-23T08:23:43.265 INFO:tasks.workunit.client.0.smithi070.stdout:0/73: dread f3 [0,4194304] 0 2024-03-23T08:23:43.268 INFO:tasks.workunit.client.0.smithi070.stdout:0/74: fsync f3 0 2024-03-23T08:23:43.272 INFO:tasks.workunit.client.0.smithi070.stdout:8/77: link d5/l14 d5/d8/d16/l20 0 2024-03-23T08:23:43.274 INFO:tasks.workunit.client.0.smithi070.stdout:0/75: rename f2 to f13 0 2024-03-23T08:23:43.279 INFO:tasks.workunit.client.0.smithi070.stdout:0/76: rename f8 to f14 0 2024-03-23T08:23:43.280 INFO:tasks.workunit.client.0.smithi070.stdout:8/78: dwrite d5/d8/f1d [0,4194304] 0 2024-03-23T08:23:43.286 INFO:tasks.workunit.client.0.smithi070.stdout:8/79: creat d5/d8/f21 x:0 0 0 2024-03-23T08:23:43.287 INFO:tasks.workunit.client.0.smithi070.stdout:8/80: readlink d5/d8/d16/d1a/l1f 0 2024-03-23T08:23:43.288 INFO:tasks.workunit.client.0.smithi070.stdout:8/81: write d5/d8/f10 [1161521,86456] 0 2024-03-23T08:23:43.289 INFO:tasks.workunit.client.0.smithi070.stdout:8/82: chown d5/d8/d16/d1a/l1f 20 1 2024-03-23T08:23:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 -- 172.21.15.70:0/2320896793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 msgr2=0x7f9934102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 --2- 172.21.15.70:0/2320896793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 0x7f9934102340 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9920004940 tx=0x7f9920009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 -- 172.21.15.70:0/2320896793 shutdown_connections 2024-03-23T08:23:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 --2- 172.21.15.70:0/2320896793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 0x7f9934102340 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 --2- 172.21.15.70:0/2320896793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9934066280 0x7f99340ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 -- 172.21.15.70:0/2320896793 >> 172.21.15.70:0/2320896793 conn(0x7f99340f7430 msgr2=0x7f99340f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 -- 172.21.15.70:0/2320896793 shutdown_connections 2024-03-23T08:23:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 -- 172.21.15.70:0/2320896793 wait complete. 2024-03-23T08:23:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 Processor -- start 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.320+0000 7f993b1b4700 1 -- start start 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f993b1b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9934066280 0x7f993410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f993b1b4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 0x7f993410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f993b1b4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993410e4c0 con 0x7f99340fff60 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f993b1b4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993410ed10 con 0x7f9934066280 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f9933fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 0x7f993410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f9933fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 0x7f993410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55824/0 (socket says 172.21.15.70:55824) 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f9933fff700 1 -- 172.21.15.70:0/4286382734 learned_addr learned my addr 172.21.15.70:0/4286382734 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f9938f50700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9934066280 0x7f993410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f9933fff700 1 -- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9934066280 msgr2=0x7f993410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f9933fff700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9934066280 0x7f993410d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f9933fff700 1 -- 172.21.15.70:0/4286382734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9920005040 con 0x7f99340fff60 2024-03-23T08:23:43.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f9933fff700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 0x7f993410dec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9920009380 tx=0x7f9920009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:43.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.321+0000 7f99317fa700 1 -- 172.21.15.70:0/4286382734 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f992000e040 con 0x7f99340fff60 2024-03-23T08:23:43.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.322+0000 7f99317fa700 1 -- 172.21.15.70:0/4286382734 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f992000bd20 con 0x7f99340fff60 2024-03-23T08:23:43.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.322+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993410ef90 con 0x7f99340fff60 2024-03-23T08:23:43.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.322+0000 7f99317fa700 1 -- 172.21.15.70:0/4286382734 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f992001c630 con 0x7f99340fff60 2024-03-23T08:23:43.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.322+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993410f4d0 con 0x7f99340fff60 2024-03-23T08:23:43.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.323+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993405f5f0 con 0x7f99340fff60 2024-03-23T08:23:43.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.323+0000 7f99317fa700 1 -- 172.21.15.70:0/4286382734 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f9920013420 con 0x7f99340fff60 2024-03-23T08:23:43.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.323+0000 7f99317fa700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f992406aec0 0x7f992406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:43.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.323+0000 7f99317fa700 1 -- 172.21.15.70:0/4286382734 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f992008a3a0 con 0x7f99340fff60 2024-03-23T08:23:43.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.324+0000 7f9938f50700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f992406aec0 0x7f992406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:43.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.324+0000 7f9938f50700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f992406aec0 0x7f992406d370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9934065fc0 tx=0x7f992800b040 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:43.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.328+0000 7f99317fa700 1 -- 172.21.15.70:0/4286382734 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f9920058460 con 0x7f99340fff60 2024-03-23T08:23:43.448 INFO:tasks.workunit.client.0.smithi070.stdout:4/48: sync 2024-03-23T08:23:43.448 INFO:tasks.workunit.client.0.smithi070.stdout:1/132: sync 2024-03-23T08:23:43.449 INFO:tasks.workunit.client.0.smithi070.stdout:1/133: chown d8/db/c20 15749378 1 2024-03-23T08:23:43.478 INFO:tasks.workunit.client.0.smithi070.stdout:0/77: link c7 c15 0 2024-03-23T08:23:43.487 INFO:tasks.workunit.client.0.smithi070.stdout:0/78: unlink l12 0 2024-03-23T08:23:43.487 INFO:tasks.workunit.client.0.smithi070.stdout:1/134: dread d8/d1a/f28 [0,4194304] 0 2024-03-23T08:23:43.487 INFO:tasks.workunit.client.0.smithi070.stdout:0/79: readlink l0 0 2024-03-23T08:23:43.488 INFO:tasks.workunit.client.0.smithi070.stdout:1/135: write d8/d11/f29 [369046,72120] 0 2024-03-23T08:23:43.489 INFO:tasks.workunit.client.0.smithi070.stdout:1/136: chown d8/d1a/f1e 11783597 1 2024-03-23T08:23:43.492 INFO:tasks.workunit.client.0.smithi070.stdout:4/49: creat d3/d6/d9/ff x:0 0 0 2024-03-23T08:23:43.493 INFO:tasks.workunit.client.0.smithi070.stdout:0/80: mkdir d16 0 2024-03-23T08:23:43.493 INFO:tasks.workunit.client.0.smithi070.stdout:0/81: truncate f11 874175 0 2024-03-23T08:23:43.497 INFO:tasks.workunit.client.0.smithi070.stdout:1/137: mknod d8/db/c31 0 2024-03-23T08:23:43.499 INFO:tasks.workunit.client.0.smithi070.stdout:0/82: dwrite ff [0,4194304] 0 2024-03-23T08:23:43.505 INFO:tasks.workunit.client.0.smithi070.stdout:4/50: dwrite d3/fc [0,4194304] 0 2024-03-23T08:23:43.506 INFO:tasks.workunit.client.0.smithi070.stdout:1/138: creat d8/db/f32 x:0 0 0 2024-03-23T08:23:43.509 INFO:tasks.workunit.client.0.smithi070.stdout:0/83: mknod d16/c17 0 2024-03-23T08:23:43.511 INFO:tasks.workunit.client.0.smithi070.stdout:1/139: creat d8/d1b/f33 x:0 0 0 2024-03-23T08:23:43.513 INFO:tasks.workunit.client.0.smithi070.stdout:1/140: fdatasync d8/d1b/f24 0 2024-03-23T08:23:43.516 INFO:tasks.workunit.client.0.smithi070.stdout:1/141: mknod d8/d1b/c34 0 2024-03-23T08:23:43.517 INFO:tasks.workunit.client.0.smithi070.stdout:1/142: readlink d8/db/l17 0 2024-03-23T08:23:43.519 INFO:tasks.workunit.client.0.smithi070.stdout:8/83: dwrite d5/fe [0,4194304] 0 2024-03-23T08:23:43.520 INFO:tasks.workunit.client.0.smithi070.stdout:1/143: fdatasync d8/db/f21 0 2024-03-23T08:23:43.524 INFO:tasks.workunit.client.0.smithi070.stdout:1/144: symlink d8/d1b/l35 0 2024-03-23T08:23:43.525 INFO:tasks.workunit.client.0.smithi070.stdout:1/145: chown d8/d1a 66429783 1 2024-03-23T08:23:43.527 INFO:tasks.workunit.client.0.smithi070.stdout:1/146: chown d8/d11/c14 21545 1 2024-03-23T08:23:43.529 INFO:tasks.workunit.client.0.smithi070.stdout:8/84: dwrite d5/f7 [0,4194304] 0 2024-03-23T08:23:43.533 INFO:tasks.workunit.client.0.smithi070.stdout:0/84: fdatasync ff 0 2024-03-23T08:23:43.535 INFO:tasks.workunit.client.0.smithi070.stdout:1/147: dread d8/f9 [0,4194304] 0 2024-03-23T08:23:43.538 INFO:tasks.workunit.client.0.smithi070.stdout:0/85: creat d16/f18 x:0 0 0 2024-03-23T08:23:43.538 INFO:tasks.workunit.client.0.smithi070.stdout:8/85: creat d5/d8/d16/f22 x:0 0 0 2024-03-23T08:23:43.542 INFO:tasks.workunit.client.0.smithi070.stdout:8/86: write d5/d8/f18 [360201,52824] 0 2024-03-23T08:23:43.542 INFO:tasks.workunit.client.0.smithi070.stdout:0/86: symlink d16/l19 0 2024-03-23T08:23:43.543 INFO:tasks.workunit.client.0.smithi070.stdout:8/87: dread - d5/d8/d16/f22 zero size 2024-03-23T08:23:43.544 INFO:tasks.workunit.client.0.smithi070.stdout:8/88: chown d5 3 1 2024-03-23T08:23:43.549 INFO:tasks.workunit.client.0.smithi070.stdout:0/87: creat d16/f1a x:0 0 0 2024-03-23T08:23:43.550 INFO:tasks.workunit.client.0.smithi070.stdout:8/89: creat d5/f23 x:0 0 0 2024-03-23T08:23:43.551 INFO:tasks.workunit.client.0.smithi070.stdout:1/148: link d8/db/l2a d8/l36 0 2024-03-23T08:23:43.551 INFO:tasks.workunit.client.0.smithi070.stdout:1/149: readlink d8/d11/l13 0 2024-03-23T08:23:43.551 INFO:tasks.workunit.client.0.smithi070.stdout:8/90: write d5/d8/f18 [96521,106968] 0 2024-03-23T08:23:43.552 INFO:tasks.workunit.client.0.smithi070.stdout:0/88: mknod d16/c1b 0 2024-03-23T08:23:43.553 INFO:tasks.workunit.client.0.smithi070.stdout:0/89: read ff [2932051,9078] 0 2024-03-23T08:23:43.554 INFO:tasks.workunit.client.0.smithi070.stdout:8/91: read d5/f7 [2426719,65700] 0 2024-03-23T08:23:43.559 INFO:tasks.workunit.client.0.smithi070.stdout:8/92: creat d5/d8/d16/f24 x:0 0 0 2024-03-23T08:23:43.560 INFO:tasks.workunit.client.0.smithi070.stdout:8/93: read d5/d8/f19 [123031,37424] 0 2024-03-23T08:23:43.565 INFO:tasks.workunit.client.0.smithi070.stdout:0/90: link d16/f18 d16/f1c 0 2024-03-23T08:23:43.567 INFO:tasks.workunit.client.0.smithi070.stdout:1/150: link d8/d1a/f1e d8/f37 0 2024-03-23T08:23:43.568 INFO:tasks.workunit.client.0.smithi070.stdout:8/94: dread d5/d8/f17 [0,4194304] 0 2024-03-23T08:23:43.571 INFO:tasks.workunit.client.0.smithi070.stdout:1/151: creat d8/d26/f38 x:0 0 0 2024-03-23T08:23:43.573 INFO:tasks.workunit.client.0.smithi070.stdout:1/152: write d8/db/f27 [30924,69560] 0 2024-03-23T08:23:43.575 INFO:tasks.workunit.client.0.smithi070.stdout:8/95: creat d5/d8/f25 x:0 0 0 2024-03-23T08:23:43.579 INFO:tasks.workunit.client.0.smithi070.stdout:1/153: fdatasync d8/db/f21 0 2024-03-23T08:23:43.579 INFO:tasks.workunit.client.0.smithi070.stdout:8/96: write d5/d8/f18 [247344,72084] 0 2024-03-23T08:23:43.585 INFO:tasks.workunit.client.0.smithi070.stdout:8/97: write d5/d8/d16/f22 [175543,91408] 0 2024-03-23T08:23:43.592 INFO:tasks.workunit.client.0.smithi070.stdout:8/98: unlink d5/d8/d16/d1a/f1b 0 2024-03-23T08:23:43.592 INFO:tasks.workunit.client.0.smithi070.stdout:8/99: chown d5/d8/d16/d1a 29418205 1 2024-03-23T08:23:43.593 INFO:tasks.workunit.client.0.smithi070.stdout:8/100: write d5/d8/f1e [728653,31684] 0 2024-03-23T08:23:43.595 INFO:tasks.workunit.client.0.smithi070.stdout:9/102: sync 2024-03-23T08:23:43.596 INFO:tasks.workunit.client.0.smithi070.stdout:1/154: link d8/d1a/f1e d8/d1a/f39 0 2024-03-23T08:23:43.619 INFO:tasks.workunit.client.0.smithi070.stdout:9/103: creat db/f21 x:0 0 0 2024-03-23T08:23:43.622 INFO:tasks.workunit.client.0.smithi070.stdout:9/104: fsync db/d12/f1a 0 2024-03-23T08:23:43.624 INFO:tasks.workunit.client.0.smithi070.stdout:9/105: mkdir db/d12/d22 0 2024-03-23T08:23:43.628 INFO:tasks.workunit.client.0.smithi070.stdout:9/106: creat db/d12/d14/f23 x:0 0 0 2024-03-23T08:23:43.629 INFO:tasks.workunit.client.0.smithi070.stdout:9/107: write db/d12/d14/f23 [702484,4191] 0 2024-03-23T08:23:43.629 INFO:tasks.workunit.client.0.smithi070.stdout:1/155: rmdir d8/db 39 2024-03-23T08:23:43.629 INFO:tasks.workunit.client.0.smithi070.stdout:9/108: readlink db/l11 0 2024-03-23T08:23:43.631 INFO:tasks.workunit.client.0.smithi070.stdout:9/109: write db/f1b [20420,25382] 0 2024-03-23T08:23:43.632 INFO:tasks.workunit.client.0.smithi070.stdout:1/156: symlink d8/d1b/l3a 0 2024-03-23T08:23:43.633 INFO:tasks.workunit.client.0.smithi070.stdout:9/110: creat db/d18/f24 x:0 0 0 2024-03-23T08:23:43.635 INFO:tasks.workunit.client.0.smithi070.stdout:1/157: dread d8/d11/f29 [0,4194304] 0 2024-03-23T08:23:43.637 INFO:tasks.workunit.client.0.smithi070.stdout:1/158: creat d8/d11/f3b x:0 0 0 2024-03-23T08:23:43.644 INFO:tasks.workunit.client.0.smithi070.stdout:0/91: write f14 [2893476,48969] 0 2024-03-23T08:23:43.646 INFO:tasks.workunit.client.0.smithi070.stdout:1/159: dwrite d8/db/f15 [0,4194304] 0 2024-03-23T08:23:43.647 INFO:tasks.workunit.client.0.smithi070.stdout:1/160: dread - d8/f37 zero size 2024-03-23T08:23:43.648 INFO:tasks.workunit.client.0.smithi070.stdout:8/101: truncate d5/fe 2818665 0 2024-03-23T08:23:43.650 INFO:tasks.workunit.client.0.smithi070.stdout:8/102: dread - d5/d8/d16/f24 zero size 2024-03-23T08:23:43.650 INFO:tasks.workunit.client.0.smithi070.stdout:0/92: dread ff [0,4194304] 0 2024-03-23T08:23:43.653 INFO:tasks.workunit.client.0.smithi070.stdout:0/93: creat d16/f1d x:0 0 0 2024-03-23T08:23:43.658 INFO:tasks.workunit.client.0.smithi070.stdout:0/94: mknod d16/c1e 0 2024-03-23T08:23:43.659 INFO:tasks.workunit.client.0.smithi070.stdout:8/103: dwrite d5/d8/d16/f22 [0,4194304] 0 2024-03-23T08:23:43.662 INFO:tasks.workunit.client.0.smithi070.stdout:8/104: rename d5/d8/d16/d1a to d5/d8/d26 0 2024-03-23T08:23:43.663 INFO:tasks.workunit.client.0.smithi070.stdout:0/95: dread f3 [0,4194304] 0 2024-03-23T08:23:43.665 INFO:tasks.workunit.client.0.smithi070.stdout:8/105: creat d5/d8/d26/f27 x:0 0 0 2024-03-23T08:23:43.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.693+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f993404bed0 con 0x7f99340fff60 2024-03-23T08:23:43.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.693+0000 7f99317fa700 1 -- 172.21.15.70:0/4286382734 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+848 (secure 0 0 0) 0x7f9920057ff0 con 0x7f99340fff60 2024-03-23T08:23:43.693 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:23:43.693 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:23:43.694 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:23:43.695 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-23T08:23:43.695 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:23:43.695 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:23:43.695 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:23:43.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.695+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f992406aec0 msgr2=0x7f992406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.695+0000 7f993b1b4700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f992406aec0 0x7f992406d370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9934065fc0 tx=0x7f992800b040 comp rx=0 tx=0).stop 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.695+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 msgr2=0x7f993410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.695+0000 7f993b1b4700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 0x7f993410dec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9920009380 tx=0x7f9920009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.696+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 shutdown_connections 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.696+0000 7f993b1b4700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f992406aec0 0x7f992406d370 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.696+0000 7f993b1b4700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9934066280 0x7f993410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.696+0000 7f993b1b4700 1 --2- 172.21.15.70:0/4286382734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340fff60 0x7f993410dec0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.696+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 >> 172.21.15.70:0/4286382734 conn(0x7f99340f7430 msgr2=0x7f99340f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.696+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 shutdown_connections 2024-03-23T08:23:43.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.696+0000 7f993b1b4700 1 -- 172.21.15.70:0/4286382734 wait complete. 2024-03-23T08:23:43.704 INFO:tasks.workunit.client.0.smithi070.stdout:1/161: truncate d8/db/f27 1357930 0 2024-03-23T08:23:43.709 INFO:tasks.workunit.client.0.smithi070.stdout:0/96: write d16/f18 [914494,120514] 0 2024-03-23T08:23:43.711 INFO:tasks.workunit.client.0.smithi070.stdout:0/97: creat d16/f1f x:0 0 0 2024-03-23T08:23:43.713 INFO:tasks.workunit.client.0.smithi070.stdout:1/162: rmdir d8 39 2024-03-23T08:23:43.718 INFO:tasks.workunit.client.0.smithi070.stdout:0/98: dwrite f14 [0,4194304] 0 2024-03-23T08:23:43.720 INFO:tasks.workunit.client.0.smithi070.stdout:1/163: creat d8/db/d2c/f3c x:0 0 0 2024-03-23T08:23:43.723 INFO:tasks.workunit.client.0.smithi070.stdout:0/99: mkdir d16/d20 0 2024-03-23T08:23:43.724 INFO:tasks.workunit.client.0.smithi070.stdout:1/164: write d8/db/d2c/f3c [86974,13571] 0 2024-03-23T08:23:43.728 INFO:tasks.workunit.client.0.smithi070.stdout:1/165: creat d8/d11/f3d x:0 0 0 2024-03-23T08:23:43.732 INFO:tasks.workunit.client.0.smithi070.stdout:1/166: dread d8/d11/f29 [0,4194304] 0 2024-03-23T08:23:43.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:43 smithi070 ceph-mon[39699]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:43 smithi070 ceph-mon[39699]: pgmap v31: 65 pgs: 65 active+clean; 1.6 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 48 MiB/s rd, 113 MiB/s wr, 328 op/s 2024-03-23T08:23:43.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:43 smithi070 ceph-mon[39699]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:43.733 INFO:tasks.workunit.client.0.smithi070.stdout:0/100: dwrite f14 [0,4194304] 0 2024-03-23T08:23:43.740 INFO:tasks.workunit.client.0.smithi070.stdout:0/101: symlink d16/d20/l21 0 2024-03-23T08:23:43.740 INFO:tasks.workunit.client.0.smithi070.stdout:0/102: readlink l0 0 2024-03-23T08:23:43.740 INFO:tasks.workunit.client.0.smithi070.stdout:0/103: readlink le 0 2024-03-23T08:23:43.751 INFO:tasks.workunit.client.0.smithi070.stdout:0/104: rename d16/d20 to d16/d22 0 2024-03-23T08:23:43.751 INFO:tasks.workunit.client.0.smithi070.stdout:0/105: fsync d16/f1c 0 2024-03-23T08:23:43.754 INFO:tasks.workunit.client.0.smithi070.stdout:1/167: dwrite d8/d1a/f28 [4194304,4194304] 0 2024-03-23T08:23:43.755 INFO:tasks.workunit.client.0.smithi070.stdout:0/106: creat d16/d22/f23 x:0 0 0 2024-03-23T08:23:43.756 INFO:tasks.workunit.client.0.smithi070.stdout:0/107: fsync d16/f1d 0 2024-03-23T08:23:43.758 INFO:tasks.workunit.client.0.smithi070.stdout:1/168: creat d8/db/f3e x:0 0 0 2024-03-23T08:23:43.760 INFO:tasks.workunit.client.0.smithi070.stdout:0/108: creat d16/d22/f24 x:0 0 0 2024-03-23T08:23:43.761 INFO:tasks.workunit.client.0.smithi070.stdout:1/169: mkdir d8/db/d2c/d3f 0 2024-03-23T08:23:43.764 INFO:tasks.workunit.client.0.smithi070.stdout:1/170: mkdir d8/db/d40 0 2024-03-23T08:23:43.766 INFO:tasks.workunit.client.0.smithi070.stdout:0/109: link d16/f1a d16/d22/f25 0 2024-03-23T08:23:43.767 INFO:tasks.workunit.client.0.smithi070.stdout:0/110: write d16/f18 [1479572,113184] 0 2024-03-23T08:23:43.767 INFO:tasks.workunit.client.0.smithi070.stdout:0/111: chown f14 10818467 1 2024-03-23T08:23:43.768 INFO:tasks.workunit.client.0.smithi070.stdout:0/112: write d16/f1c [1635016,66325] 0 2024-03-23T08:23:43.769 INFO:tasks.workunit.client.0.smithi070.stdout:0/113: write ff [1282621,88646] 0 2024-03-23T08:23:43.771 INFO:tasks.workunit.client.0.smithi070.stdout:0/114: read f14 [1913125,111401] 0 2024-03-23T08:23:43.775 INFO:tasks.workunit.client.0.smithi070.stdout:0/115: mkdir d16/d26 0 2024-03-23T08:23:43.779 INFO:tasks.workunit.client.0.smithi070.stdout:0/116: dread ff [0,4194304] 0 2024-03-23T08:23:43.779 INFO:tasks.workunit.client.0.smithi070.stdout:0/117: fsync d16/f1c 0 2024-03-23T08:23:43.782 INFO:tasks.workunit.client.0.smithi070.stdout:0/118: write ff [1710428,5014] 0 2024-03-23T08:23:43.783 INFO:tasks.workunit.client.0.smithi070.stdout:1/171: write d8/db/f21 [3886229,6286] 0 2024-03-23T08:23:43.785 INFO:tasks.workunit.client.0.smithi070.stdout:0/119: mkdir d16/d26/d27 0 2024-03-23T08:23:43.786 INFO:tasks.workunit.client.0.smithi070.stdout:0/120: chown fb 1265246 1 2024-03-23T08:23:43.786 INFO:tasks.workunit.client.0.smithi070.stdout:1/172: mkdir d8/d26/d41 0 2024-03-23T08:23:43.789 INFO:tasks.workunit.client.0.smithi070.stdout:0/121: symlink d16/l28 0 2024-03-23T08:23:43.790 INFO:tasks.workunit.client.0.smithi070.stdout:1/173: dread d8/d1a/f28 [4194304,4194304] 0 2024-03-23T08:23:43.792 INFO:tasks.workunit.client.0.smithi070.stdout:0/122: creat d16/d22/f29 x:0 0 0 2024-03-23T08:23:43.794 INFO:tasks.workunit.client.0.smithi070.stdout:1/174: mkdir d8/d42 0 2024-03-23T08:23:43.795 INFO:tasks.workunit.client.0.smithi070.stdout:0/123: creat d16/d22/f2a x:0 0 0 2024-03-23T08:23:43.797 INFO:tasks.workunit.client.0.smithi070.stdout:1/175: mkdir d8/db/d40/d43 0 2024-03-23T08:23:43.798 INFO:tasks.workunit.client.0.smithi070.stdout:0/124: creat d16/d22/f2b x:0 0 0 2024-03-23T08:23:43.800 INFO:tasks.workunit.client.0.smithi070.stdout:1/176: mkdir d8/db/d40/d43/d44 0 2024-03-23T08:23:43.801 INFO:tasks.workunit.client.0.smithi070.stdout:0/125: creat d16/d22/f2c x:0 0 0 2024-03-23T08:23:43.806 INFO:tasks.workunit.client.0.smithi070.stdout:1/177: rename d8/d11/f3d to d8/db/f45 0 2024-03-23T08:23:43.806 INFO:tasks.workunit.client.0.smithi070.stdout:1/178: chown d8/d1a/f1e 5 1 2024-03-23T08:23:43.808 INFO:tasks.workunit.client.0.smithi070.stdout:1/179: truncate d8/db/f16 873031 0 2024-03-23T08:23:43.808 INFO:tasks.workunit.client.0.smithi070.stdout:0/126: link lc d16/l2d 0 2024-03-23T08:23:43.808 INFO:tasks.workunit.client.0.smithi070.stdout:1/180: dread - d8/db/f3e zero size 2024-03-23T08:23:43.809 INFO:tasks.workunit.client.0.smithi070.stdout:0/127: dread - d16/f1f zero size 2024-03-23T08:23:43.813 INFO:tasks.workunit.client.0.smithi070.stdout:6/274: sync 2024-03-23T08:23:43.819 INFO:tasks.workunit.client.0.smithi070.stdout:0/128: dwrite d16/f18 [0,4194304] 0 2024-03-23T08:23:43.819 INFO:tasks.workunit.client.0.smithi070.stdout:1/181: dwrite d8/d11/f29 [0,4194304] 0 2024-03-23T08:23:43.823 INFO:tasks.workunit.client.0.smithi070.stdout:0/129: write d16/f1c [111665,7816] 0 2024-03-23T08:23:43.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:43 smithi193 ceph-mon[48370]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:43.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:43 smithi193 ceph-mon[48370]: pgmap v31: 65 pgs: 65 active+clean; 1.6 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 48 MiB/s rd, 113 MiB/s wr, 328 op/s 2024-03-23T08:23:43.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:43 smithi193 ceph-mon[48370]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:43.825 INFO:tasks.workunit.client.0.smithi070.stdout:1/182: truncate d8/d1b/f33 122218 0 2024-03-23T08:23:43.825 INFO:tasks.workunit.client.0.smithi070.stdout:0/130: write d16/f1d [294828,73411] 0 2024-03-23T08:23:43.825 INFO:tasks.workunit.client.0.smithi070.stdout:1/183: fsync d8/db/f3e 0 2024-03-23T08:23:43.828 INFO:tasks.workunit.client.0.smithi070.stdout:0/131: symlink d16/l2e 0 2024-03-23T08:23:43.831 INFO:tasks.workunit.client.0.smithi070.stdout:1/184: fsync d8/d1a/f1e 0 2024-03-23T08:23:43.834 INFO:tasks.workunit.client.0.smithi070.stdout:0/132: readlink lc 0 2024-03-23T08:23:43.836 INFO:tasks.workunit.client.0.smithi070.stdout:1/185: dread d8/d1a/f28 [4194304,4194304] 0 2024-03-23T08:23:43.837 INFO:tasks.workunit.client.0.smithi070.stdout:6/275: link d0/d23/d1c/d2f/f4c d0/d23/d1c/f5d 0 2024-03-23T08:23:43.840 INFO:tasks.workunit.client.0.smithi070.stdout:1/186: mkdir d8/d11/d46 0 2024-03-23T08:23:43.840 INFO:tasks.workunit.client.0.smithi070.stdout:1/187: stat d8/d11/c14 0 2024-03-23T08:23:43.842 INFO:tasks.workunit.client.0.smithi070.stdout:1/188: truncate d8/d1a/f23 46398 0 2024-03-23T08:23:43.844 INFO:tasks.workunit.client.0.smithi070.stdout:6/276: creat d0/d23/d28/d35/d3a/d3c/d5b/f5e x:0 0 0 2024-03-23T08:23:43.850 INFO:tasks.workunit.client.0.smithi070.stdout:0/133: getdents d16/d26 0 2024-03-23T08:23:43.851 INFO:tasks.workunit.client.0.smithi070.stdout:0/134: fsync d16/f1f 0 2024-03-23T08:23:43.852 INFO:tasks.workunit.client.0.smithi070.stdout:0/135: write d16/d22/f23 [528957,44180] 0 2024-03-23T08:23:43.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.852+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3702465730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e5806f9d0 msgr2=0x7f8e5806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:43.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.852+0000 7f8e5ed1d700 1 --2- 172.21.15.70:0/3702465730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e5806f9d0 0x7f8e5806e460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c004880 tx=0x7f8e4c009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:43.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.853+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3702465730 shutdown_connections 2024-03-23T08:23:43.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.853+0000 7f8e5ed1d700 1 --2- 172.21.15.70:0/3702465730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e5806e9a0 0x7f8e5806ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.853+0000 7f8e5ed1d700 1 --2- 172.21.15.70:0/3702465730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e5806f9d0 0x7f8e5806e460 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.853+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3702465730 >> 172.21.15.70:0/3702465730 conn(0x7f8e58069de0 msgr2=0x7f8e5806c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:43.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.853+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3702465730 shutdown_connections 2024-03-23T08:23:43.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.853+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3702465730 wait complete. 2024-03-23T08:23:43.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.853+0000 7f8e5ed1d700 1 Processor -- start 2024-03-23T08:23:43.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.853+0000 7f8e5ed1d700 1 -- start start 2024-03-23T08:23:43.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.854+0000 7f8e5ed1d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e5806e9a0 0x7f8e5811c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:43.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.854+0000 7f8e5ed1d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e5806f9d0 0x7f8e5811caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:43.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.854+0000 7f8e5ed1d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5811d0f0 con 0x7f8e5806f9d0 2024-03-23T08:23:43.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.854+0000 7f8e5ed1d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5811d260 con 0x7f8e5806e9a0 2024-03-23T08:23:43.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.854+0000 7f8e5cab9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e5806e9a0 0x7f8e5811c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:43.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e57fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e5806f9d0 0x7f8e5811caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:43.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e57fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e5806f9d0 0x7f8e5811caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55842/0 (socket says 172.21.15.70:55842) 2024-03-23T08:23:43.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e57fff700 1 -- 172.21.15.70:0/3314601595 learned_addr learned my addr 172.21.15.70:0/3314601595 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:23:43.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e5cab9700 1 -- 172.21.15.70:0/3314601595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e5806f9d0 msgr2=0x7f8e5811caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:43.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e5cab9700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e5806f9d0 0x7f8e5811caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:43.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e5cab9700 1 -- 172.21.15.70:0/3314601595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c005040 con 0x7f8e5806e9a0 2024-03-23T08:23:43.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e5cab9700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e5806e9a0 0x7f8e5811c5b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0137a0 tx=0x7f8e4c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:43.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e557fa700 1 -- 172.21.15.70:0/3314601595 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4c00e030 con 0x7f8e5806e9a0 2024-03-23T08:23:43.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e58121c60 con 0x7f8e5806e9a0 2024-03-23T08:23:43.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e557fa700 1 -- 172.21.15.70:0/3314601595 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e4c01c400 con 0x7f8e5806e9a0 2024-03-23T08:23:43.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.855+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e581221a0 con 0x7f8e5806e9a0 2024-03-23T08:23:43.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.856+0000 7f8e557fa700 1 -- 172.21.15.70:0/3314601595 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4c012620 con 0x7f8e5806e9a0 2024-03-23T08:23:43.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.857+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c0052e0 con 0x7f8e5806e9a0 2024-03-23T08:23:43.857 INFO:tasks.workunit.client.0.smithi070.stdout:0/136: creat d16/d22/f2f x:0 0 0 2024-03-23T08:23:43.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.857+0000 7f8e557fa700 1 -- 172.21.15.70:0/3314601595 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f8e4c013a20 con 0x7f8e5806e9a0 2024-03-23T08:23:43.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.858+0000 7f8e557fa700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8e4806afa0 0x7f8e4806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:23:43.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.858+0000 7f8e557fa700 1 -- 172.21.15.70:0/3314601595 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8e4c08ad90 con 0x7f8e5806e9a0 2024-03-23T08:23:43.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.858+0000 7f8e57fff700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8e4806afa0 0x7f8e4806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:23:43.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.858+0000 7f8e57fff700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8e4806afa0 0x7f8e4806d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009b80 tx=0x7f8e4400b040 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:23:43.862 INFO:tasks.workunit.client.0.smithi070.stdout:0/137: creat d16/f30 x:0 0 0 2024-03-23T08:23:43.862 INFO:tasks.workunit.client.0.smithi070.stdout:6/277: write d0/d23/d28/f4a [84110,125631] 0 2024-03-23T08:23:43.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:43.863+0000 7f8e557fa700 1 -- 172.21.15.70:0/3314601595 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f8e4c058d90 con 0x7f8e5806e9a0 2024-03-23T08:23:43.864 INFO:tasks.workunit.client.0.smithi070.stdout:0/138: write d16/d22/f2c [67648,34413] 0 2024-03-23T08:23:43.868 INFO:tasks.workunit.client.0.smithi070.stdout:6/278: read d0/d23/d28/f36 [2929254,46497] 0 2024-03-23T08:23:43.871 INFO:tasks.workunit.client.0.smithi070.stdout:1/189: dwrite d8/db/f27 [0,4194304] 0 2024-03-23T08:23:43.874 INFO:tasks.workunit.client.0.smithi070.stdout:0/139: dread f3 [0,4194304] 0 2024-03-23T08:23:43.875 INFO:tasks.workunit.client.0.smithi070.stdout:6/279: symlink d0/d23/d1c/l5f 0 2024-03-23T08:23:43.876 INFO:tasks.workunit.client.0.smithi070.stdout:1/190: creat d8/d1a/f47 x:0 0 0 2024-03-23T08:23:43.878 INFO:tasks.workunit.client.0.smithi070.stdout:0/140: symlink d16/l31 0 2024-03-23T08:23:43.880 INFO:tasks.workunit.client.0.smithi070.stdout:6/280: dread d0/d23/f2a [0,4194304] 0 2024-03-23T08:23:43.881 INFO:tasks.workunit.client.0.smithi070.stdout:6/281: readlink d0/d3/ld 0 2024-03-23T08:23:43.882 INFO:tasks.workunit.client.0.smithi070.stdout:0/141: mkdir d16/d32 0 2024-03-23T08:23:43.882 INFO:tasks.workunit.client.0.smithi070.stdout:1/191: mkdir d8/db/d2c/d3f/d48 0 2024-03-23T08:23:43.885 INFO:tasks.workunit.client.0.smithi070.stdout:1/192: write d8/d11/f29 [4300273,116303] 0 2024-03-23T08:23:43.886 INFO:tasks.workunit.client.0.smithi070.stdout:1/193: chown d8/d11/f3b 5905007 1 2024-03-23T08:23:43.886 INFO:tasks.workunit.client.0.smithi070.stdout:0/142: rename l5 to d16/d22/l33 0 2024-03-23T08:23:43.887 INFO:tasks.workunit.client.0.smithi070.stdout:1/194: readlink d8/d11/l13 0 2024-03-23T08:23:43.888 INFO:tasks.workunit.client.0.smithi070.stdout:0/143: creat d16/f34 x:0 0 0 2024-03-23T08:23:43.892 INFO:tasks.workunit.client.0.smithi070.stdout:0/144: chown d16/l2d 920294345 1 2024-03-23T08:23:43.894 INFO:tasks.workunit.client.0.smithi070.stdout:1/195: dread d8/db/f1f [0,4194304] 0 2024-03-23T08:23:43.895 INFO:tasks.workunit.client.0.smithi070.stdout:1/196: stat d8/db/d40/d43 0 2024-03-23T08:23:43.896 INFO:tasks.workunit.client.0.smithi070.stdout:0/145: creat d16/d26/d27/f35 x:0 0 0 2024-03-23T08:23:43.896 INFO:tasks.workunit.client.0.smithi070.stdout:1/197: chown d8/d1b/f24 15 1 2024-03-23T08:23:43.897 INFO:tasks.workunit.client.0.smithi070.stdout:1/198: write d8/db/f16 [339307,37534] 0 2024-03-23T08:23:43.899 INFO:tasks.workunit.client.0.smithi070.stdout:0/146: truncate d16/d22/f2a 103887 0 2024-03-23T08:23:43.900 INFO:tasks.workunit.client.0.smithi070.stdout:1/199: chown d8/db/d2c 1949968 1 2024-03-23T08:23:43.902 INFO:tasks.workunit.client.0.smithi070.stdout:0/147: creat d16/d26/f36 x:0 0 0 2024-03-23T08:23:43.904 INFO:tasks.workunit.client.0.smithi070.stdout:6/282: dwrite d0/d23/d28/d35/f46 [0,4194304] 0 2024-03-23T08:23:43.904 INFO:tasks.workunit.client.0.smithi070.stdout:0/148: dread - d16/d22/f29 zero size 2024-03-23T08:23:43.906 INFO:tasks.workunit.client.0.smithi070.stdout:6/283: fdatasync d0/d23/d28/f4a 0 2024-03-23T08:23:43.906 INFO:tasks.workunit.client.0.smithi070.stdout:0/149: read d16/d22/f23 [358263,4661] 0 2024-03-23T08:23:43.907 INFO:tasks.workunit.client.0.smithi070.stdout:6/284: chown d0/d23/d28/d35/c4f 6 1 2024-03-23T08:23:43.911 INFO:tasks.workunit.client.0.smithi070.stdout:0/150: truncate d16/d22/f2c 550503 0 2024-03-23T08:23:43.913 INFO:tasks.workunit.client.0.smithi070.stdout:0/151: chown d16/f34 19947 1 2024-03-23T08:23:43.916 INFO:tasks.workunit.client.0.smithi070.stdout:1/200: dwrite d8/db/f45 [0,4194304] 0 2024-03-23T08:23:43.917 INFO:tasks.workunit.client.0.smithi070.stdout:0/152: rename d16/d22/f2b to d16/d26/f37 0 2024-03-23T08:23:43.919 INFO:tasks.workunit.client.0.smithi070.stdout:0/153: readlink d16/l31 0 2024-03-23T08:23:43.920 INFO:tasks.workunit.client.0.smithi070.stdout:0/154: chown d16/f1f 904 1 2024-03-23T08:23:43.924 INFO:tasks.workunit.client.0.smithi070.stdout:0/155: dread d16/d22/f2c [0,4194304] 0 2024-03-23T08:23:43.924 INFO:tasks.workunit.client.0.smithi070.stdout:0/156: readlink d16/l2e 0 2024-03-23T08:23:43.925 INFO:tasks.workunit.client.0.smithi070.stdout:0/157: fsync d16/f18 0 2024-03-23T08:23:43.929 INFO:tasks.workunit.client.0.smithi070.stdout:0/158: dread f3 [0,4194304] 0 2024-03-23T08:23:43.937 INFO:tasks.workunit.client.0.smithi070.stdout:1/201: dwrite d8/d1b/f22 [0,4194304] 0 2024-03-23T08:23:43.938 INFO:tasks.workunit.client.0.smithi070.stdout:1/202: fsync d8/d11/f3b 0 2024-03-23T08:23:43.940 INFO:tasks.workunit.client.0.smithi070.stdout:0/159: dwrite d16/d26/d27/f35 [0,4194304] 0 2024-03-23T08:23:43.940 INFO:tasks.workunit.client.0.smithi070.stdout:1/203: chown d8/db/d2c/d3f/d48 31242 1 2024-03-23T08:23:43.942 INFO:tasks.workunit.client.0.smithi070.stdout:0/160: fdatasync f14 0 2024-03-23T08:23:43.944 INFO:tasks.workunit.client.0.smithi070.stdout:1/204: unlink d8/d1b/f24 0 2024-03-23T08:23:43.944 INFO:tasks.workunit.client.0.smithi070.stdout:0/161: readlink d16/l19 0 2024-03-23T08:23:43.945 INFO:tasks.workunit.client.0.smithi070.stdout:1/205: write d8/db/f3e [15034,59225] 0 2024-03-23T08:23:43.946 INFO:tasks.workunit.client.0.smithi070.stdout:0/162: symlink d16/d22/l38 0 2024-03-23T08:23:43.951 INFO:tasks.workunit.client.0.smithi070.stdout:1/206: dread d8/d1b/f25 [0,4194304] 0 2024-03-23T08:23:43.952 INFO:tasks.workunit.client.0.smithi070.stdout:1/207: chown d8/db/f3e 32875418 1 2024-03-23T08:23:43.953 INFO:tasks.workunit.client.0.smithi070.stdout:1/208: write d8/d1b/f33 [607724,28780] 0 2024-03-23T08:23:43.956 INFO:tasks.workunit.client.0.smithi070.stdout:1/209: mkdir d8/d26/d49 0 2024-03-23T08:23:43.956 INFO:tasks.workunit.client.0.smithi070.stdout:0/163: link d16/f1a d16/d22/f39 0 2024-03-23T08:23:43.957 INFO:tasks.workunit.client.0.smithi070.stdout:1/210: chown d8/db/d2c 19 1 2024-03-23T08:23:43.957 INFO:tasks.workunit.client.0.smithi070.stdout:0/164: fsync d16/f1c 0 2024-03-23T08:23:43.963 INFO:tasks.workunit.client.0.smithi070.stdout:1/211: fdatasync d8/d1b/f22 0 2024-03-23T08:23:43.963 INFO:tasks.workunit.client.0.smithi070.stdout:0/165: dwrite d16/f18 [0,4194304] 0 2024-03-23T08:23:43.964 INFO:tasks.workunit.client.0.smithi070.stdout:0/166: stat d16/f1d 0 2024-03-23T08:23:43.964 INFO:tasks.workunit.client.0.smithi070.stdout:1/212: write d8/d1b/f22 [179068,68257] 0 2024-03-23T08:23:43.965 INFO:tasks.workunit.client.0.smithi070.stdout:1/213: read - d8/d1a/f1e zero size 2024-03-23T08:23:43.966 INFO:tasks.workunit.client.0.smithi070.stdout:1/214: fsync d8/d1a/f47 0 2024-03-23T08:23:43.971 INFO:tasks.workunit.client.0.smithi070.stdout:0/167: link d16/d22/l38 d16/l3a 0 2024-03-23T08:23:43.973 INFO:tasks.workunit.client.0.smithi070.stdout:0/168: write d16/f1f [167414,122825] 0 2024-03-23T08:23:43.974 INFO:tasks.workunit.client.0.smithi070.stdout:0/169: fdatasync fb 0 2024-03-23T08:23:43.974 INFO:tasks.workunit.client.0.smithi070.stdout:1/215: symlink d8/d26/d41/l4a 0 2024-03-23T08:23:43.974 INFO:tasks.workunit.client.0.smithi070.stdout:0/170: stat d16/l31 0 2024-03-23T08:23:43.982 INFO:tasks.workunit.client.0.smithi070.stdout:0/171: rename d16/d22/f2a to d16/f3b 0 2024-03-23T08:23:43.983 INFO:tasks.workunit.client.0.smithi070.stdout:1/216: link d8/f2f d8/d1b/f4b 0 2024-03-23T08:23:43.986 INFO:tasks.workunit.client.0.smithi070.stdout:1/217: creat d8/db/d40/f4c x:0 0 0 2024-03-23T08:23:43.987 INFO:tasks.workunit.client.0.smithi070.stdout:1/218: write d8/db/f21 [1260711,35210] 0 2024-03-23T08:23:43.995 INFO:tasks.workunit.client.0.smithi070.stdout:0/172: getdents d16 0 2024-03-23T08:23:43.997 INFO:tasks.workunit.client.0.smithi070.stdout:0/173: link d16/c1b d16/d22/c3c 0 2024-03-23T08:23:43.999 INFO:tasks.workunit.client.0.smithi070.stdout:0/174: chown c7 124414420 1 2024-03-23T08:23:44.000 INFO:tasks.workunit.client.0.smithi070.stdout:0/175: rmdir d16 39 2024-03-23T08:23:44.003 INFO:tasks.workunit.client.0.smithi070.stdout:0/176: mkdir d16/d26/d27/d3d 0 2024-03-23T08:23:44.005 INFO:tasks.workunit.client.0.smithi070.stdout:0/177: creat d16/f3e x:0 0 0 2024-03-23T08:23:44.011 INFO:tasks.workunit.client.0.smithi070.stdout:0/178: dwrite f11 [0,4194304] 0 2024-03-23T08:23:44.018 INFO:tasks.workunit.client.0.smithi070.stdout:5/69: sync 2024-03-23T08:23:44.019 INFO:tasks.workunit.client.0.smithi070.stdout:7/73: sync 2024-03-23T08:23:44.023 INFO:tasks.workunit.client.0.smithi070.stdout:0/179: getdents d16/d22 0 2024-03-23T08:23:44.024 INFO:tasks.workunit.client.0.smithi070.stdout:5/70: creat dc/f14 x:0 0 0 2024-03-23T08:23:44.025 INFO:tasks.workunit.client.0.smithi070.stdout:5/71: read f1 [1441301,74355] 0 2024-03-23T08:23:44.026 INFO:tasks.workunit.client.0.smithi070.stdout:5/72: write f1 [3053239,62885] 0 2024-03-23T08:23:44.027 INFO:tasks.workunit.client.0.smithi070.stdout:0/180: dread d16/f1d [0,4194304] 0 2024-03-23T08:23:44.029 INFO:tasks.workunit.client.0.smithi070.stdout:0/181: dread - d16/d22/f29 zero size 2024-03-23T08:23:44.030 INFO:tasks.workunit.client.0.smithi070.stdout:5/73: rename ca to dc/c15 0 2024-03-23T08:23:44.031 INFO:tasks.workunit.client.0.smithi070.stdout:0/182: creat d16/f3f x:0 0 0 2024-03-23T08:23:44.034 INFO:tasks.workunit.client.0.smithi070.stdout:5/74: creat dc/f16 x:0 0 0 2024-03-23T08:23:44.039 INFO:tasks.workunit.client.0.smithi070.stdout:0/183: dwrite d16/d22/f24 [0,4194304] 0 2024-03-23T08:23:44.048 INFO:tasks.workunit.client.0.smithi070.stdout:0/184: dwrite d16/f1f [0,4194304] 0 2024-03-23T08:23:44.049 INFO:tasks.workunit.client.0.smithi070.stdout:0/185: write d16/f3e [345512,127714] 0 2024-03-23T08:23:44.050 INFO:tasks.workunit.client.0.smithi070.stdout:0/186: fsync d16/f3f 0 2024-03-23T08:23:44.052 INFO:tasks.workunit.client.0.smithi070.stdout:0/187: mknod d16/d22/c40 0 2024-03-23T08:23:44.054 INFO:tasks.workunit.client.0.smithi070.stdout:0/188: mknod d16/d26/c41 0 2024-03-23T08:23:44.064 INFO:tasks.workunit.client.0.smithi070.stdout:0/189: write d16/d22/f39 [589360,42787] 0 2024-03-23T08:23:44.065 INFO:tasks.workunit.client.0.smithi070.stdout:0/190: read d16/d22/f24 [2049355,38891] 0 2024-03-23T08:23:44.068 INFO:tasks.workunit.client.0.smithi070.stdout:0/191: mknod d16/d22/c42 0 2024-03-23T08:23:44.069 INFO:tasks.workunit.client.0.smithi070.stdout:0/192: dread - d16/f34 zero size 2024-03-23T08:23:44.072 INFO:tasks.workunit.client.0.smithi070.stdout:0/193: rmdir d16/d32 0 2024-03-23T08:23:44.073 INFO:tasks.workunit.client.0.smithi070.stdout:0/194: mkdir d16/d22/d43 0 2024-03-23T08:23:44.074 INFO:tasks.workunit.client.0.smithi070.stdout:0/195: chown d16/d22/l38 100652817 1 2024-03-23T08:23:44.076 INFO:tasks.workunit.client.0.smithi070.stdout:0/196: mknod d16/d22/c44 0 2024-03-23T08:23:44.086 INFO:tasks.workunit.client.0.smithi070.stdout:0/197: dwrite d16/d22/f24 [0,4194304] 0 2024-03-23T08:23:44.088 INFO:tasks.workunit.client.0.smithi070.stdout:0/198: write d16/f1c [1378385,76099] 0 2024-03-23T08:23:44.092 INFO:tasks.workunit.client.0.smithi070.stdout:0/199: truncate d16/d22/f23 437199 0 2024-03-23T08:23:44.099 INFO:tasks.workunit.client.0.smithi070.stdout:0/200: dwrite d16/d22/f2f [0,4194304] 0 2024-03-23T08:23:44.105 INFO:tasks.workunit.client.0.smithi070.stdout:0/201: rename d16/d22/f25 to d16/d26/d27/f45 0 2024-03-23T08:23:44.114 INFO:tasks.workunit.client.0.smithi070.stdout:0/202: write d16/f1d [8197,83549] 0 2024-03-23T08:23:44.116 INFO:tasks.workunit.client.0.smithi070.stdout:0/203: creat d16/f46 x:0 0 0 2024-03-23T08:23:44.117 INFO:tasks.workunit.client.0.smithi070.stdout:0/204: chown d16/d26/d27/f35 319109 1 2024-03-23T08:23:44.118 INFO:tasks.workunit.client.0.smithi070.stdout:0/205: fdatasync d16/f3f 0 2024-03-23T08:23:44.119 INFO:tasks.workunit.client.0.smithi070.stdout:0/206: truncate d16/d22/f39 1524699 0 2024-03-23T08:23:44.123 INFO:tasks.workunit.client.0.smithi070.stdout:0/207: dread d16/f1f [0,4194304] 0 2024-03-23T08:23:44.123 INFO:tasks.workunit.client.0.smithi070.stdout:0/208: chown d16/d26/f37 14076 1 2024-03-23T08:23:44.124 INFO:tasks.workunit.client.0.smithi070.stdout:0/209: fdatasync d16/f3e 0 2024-03-23T08:23:44.129 INFO:tasks.workunit.client.0.smithi070.stdout:0/210: mkdir d16/d22/d43/d47 0 2024-03-23T08:23:44.130 INFO:tasks.workunit.client.0.smithi070.stdout:0/211: mknod d16/d22/c48 0 2024-03-23T08:23:44.135 INFO:tasks.workunit.client.0.smithi070.stdout:0/212: getdents d16/d26 0 2024-03-23T08:23:44.135 INFO:tasks.workunit.client.0.smithi070.stdout:0/213: chown d16/l31 49 1 2024-03-23T08:23:44.135 INFO:tasks.workunit.client.0.smithi070.stdout:0/214: readlink ld 0 2024-03-23T08:23:44.136 INFO:tasks.workunit.client.0.smithi070.stdout:0/215: fsync fb 0 2024-03-23T08:23:44.139 INFO:tasks.workunit.client.0.smithi070.stdout:0/216: truncate d16/d22/f2c 1574167 0 2024-03-23T08:23:44.140 INFO:tasks.workunit.client.0.smithi070.stdout:0/217: chown d16/d22/c3c 190342491 1 2024-03-23T08:23:44.141 INFO:tasks.workunit.client.0.smithi070.stdout:0/218: fsync d16/d22/f24 0 2024-03-23T08:23:44.141 INFO:tasks.workunit.client.0.smithi070.stdout:0/219: readlink d16/l2d 0 2024-03-23T08:23:44.142 INFO:tasks.workunit.client.0.smithi070.stdout:0/220: dread - d16/f46 zero size 2024-03-23T08:23:44.153 INFO:tasks.workunit.client.0.smithi070.stdout:0/221: dwrite f14 [4194304,4194304] 0 2024-03-23T08:23:44.158 INFO:tasks.workunit.client.0.smithi070.stdout:0/222: rename ff to d16/d26/d27/d3d/f49 0 2024-03-23T08:23:44.159 INFO:tasks.workunit.client.0.smithi070.stdout:0/223: chown d16/d22/c3c 3905494 1 2024-03-23T08:23:44.162 INFO:tasks.workunit.client.0.smithi070.stdout:0/224: mkdir d16/d26/d4a 0 2024-03-23T08:23:44.163 INFO:tasks.workunit.client.0.smithi070.stdout:0/225: write d16/d26/d27/f35 [3351528,70403] 0 2024-03-23T08:23:44.167 INFO:tasks.workunit.client.0.smithi070.stdout:0/226: rename d16/f3f to d16/f4b 0 2024-03-23T08:23:44.168 INFO:tasks.workunit.client.0.smithi070.stdout:0/227: read - d16/d26/f36 zero size 2024-03-23T08:23:44.171 INFO:tasks.workunit.client.0.smithi070.stdout:0/228: mkdir d16/d26/d27/d4c 0 2024-03-23T08:23:44.171 INFO:tasks.workunit.client.0.smithi070.stdout:0/229: stat d16/f1a 0 2024-03-23T08:23:44.174 INFO:tasks.workunit.client.0.smithi070.stdout:0/230: mkdir d16/d26/d27/d3d/d4d 0 2024-03-23T08:23:44.177 INFO:tasks.workunit.client.0.smithi070.stdout:0/231: symlink d16/d26/l4e 0 2024-03-23T08:23:44.179 INFO:tasks.workunit.client.0.smithi070.stdout:0/232: mknod d16/d22/c4f 0 2024-03-23T08:23:44.184 INFO:tasks.workunit.client.0.smithi070.stdout:0/233: link ld d16/d26/d4a/l50 0 2024-03-23T08:23:44.185 INFO:tasks.workunit.client.0.smithi070.stdout:0/234: creat d16/f51 x:0 0 0 2024-03-23T08:23:44.187 INFO:tasks.workunit.client.0.smithi070.stdout:0/235: creat d16/d22/d43/f52 x:0 0 0 2024-03-23T08:23:44.188 INFO:tasks.workunit.client.0.smithi070.stdout:0/236: mknod d16/d22/c53 0 2024-03-23T08:23:44.190 INFO:tasks.workunit.client.0.smithi070.stdout:0/237: fdatasync d16/f4b 0 2024-03-23T08:23:44.194 INFO:tasks.workunit.client.0.smithi070.stdout:0/238: dread d16/d22/f39 [0,4194304] 0 2024-03-23T08:23:44.195 INFO:tasks.workunit.client.0.smithi070.stdout:0/239: dread - d16/f30 zero size 2024-03-23T08:23:44.195 INFO:tasks.workunit.client.0.smithi070.stdout:0/240: chown d16/d22/d43/f52 242 1 2024-03-23T08:23:44.197 INFO:tasks.workunit.client.0.smithi070.stdout:0/241: symlink d16/d26/l54 0 2024-03-23T08:23:44.198 INFO:tasks.workunit.client.0.smithi070.stdout:0/242: fdatasync d16/d26/f37 0 2024-03-23T08:23:44.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.199+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e3c000bf0 con 0x7f8e4806afa0 2024-03-23T08:23:44.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.200+0000 7f8e557fa700 1 -- 172.21.15.70:0/3314601595 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f8e3c000bf0 con 0x7f8e4806afa0 2024-03-23T08:23:44.200 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:23:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:23:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:23:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-23T08:23:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-03-23T08:23:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "1/2 daemons upgraded", 2024-03-23T08:23:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-03-23T08:23:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:23:44.201 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:23:44.201 INFO:tasks.workunit.client.0.smithi070.stdout:0/243: mknod d16/d22/d43/c55 0 2024-03-23T08:23:44.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8e4806afa0 msgr2=0x7f8e4806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:44.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8e4806afa0 0x7f8e4806d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009b80 tx=0x7f8e4400b040 comp rx=0 tx=0).stop 2024-03-23T08:23:44.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e5806e9a0 msgr2=0x7f8e5811c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:23:44.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e5806e9a0 0x7f8e5811c5b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0137a0 tx=0x7f8e4c009e30 comp rx=0 tx=0).stop 2024-03-23T08:23:44.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 shutdown_connections 2024-03-23T08:23:44.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8e4806afa0 0x7f8e4806d450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:44.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e5806e9a0 0x7f8e5811c5b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:44.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 --2- 172.21.15.70:0/3314601595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e5806f9d0 0x7f8e5811caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:23:44.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 >> 172.21.15.70:0/3314601595 conn(0x7f8e58069de0 msgr2=0x7f8e5806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:23:44.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 shutdown_connections 2024-03-23T08:23:44.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:23:44.203+0000 7f8e5ed1d700 1 -- 172.21.15.70:0/3314601595 wait complete. 2024-03-23T08:23:44.208 INFO:tasks.workunit.client.0.smithi070.stdout:0/244: dwrite d16/f46 [0,4194304] 0 2024-03-23T08:23:44.212 INFO:tasks.workunit.client.0.smithi070.stdout:0/245: creat d16/d22/d43/f56 x:0 0 0 2024-03-23T08:23:44.216 INFO:tasks.workunit.client.0.smithi070.stdout:0/246: mknod d16/d26/c57 0 2024-03-23T08:23:44.217 INFO:tasks.workunit.client.0.smithi070.stdout:0/247: truncate d16/f4b 928516 0 2024-03-23T08:23:44.223 INFO:tasks.workunit.client.0.smithi070.stdout:0/248: truncate fb 4082331 0 2024-03-23T08:23:44.224 INFO:tasks.workunit.client.0.smithi070.stdout:0/249: mknod d16/d22/d43/c58 0 2024-03-23T08:23:44.236 INFO:tasks.workunit.client.0.smithi070.stdout:0/250: dwrite f13 [0,4194304] 0 2024-03-23T08:23:44.239 INFO:tasks.workunit.client.0.smithi070.stdout:0/251: mknod d16/d26/d27/c59 0 2024-03-23T08:23:44.244 INFO:tasks.workunit.client.0.smithi070.stdout:0/252: getdents d16/d26/d4a 0 2024-03-23T08:23:44.245 INFO:tasks.workunit.client.0.smithi070.stdout:0/253: fdatasync d16/d26/d27/d3d/f49 0 2024-03-23T08:23:44.247 INFO:tasks.workunit.client.0.smithi070.stdout:0/254: getdents d16/d26/d27/d3d/d4d 0 2024-03-23T08:23:44.254 INFO:tasks.workunit.client.0.smithi070.stdout:0/255: dwrite d16/f1a [0,4194304] 0 2024-03-23T08:23:44.294 INFO:tasks.workunit.client.0.smithi070.stdout:3/153: sync 2024-03-23T08:23:44.294 INFO:tasks.workunit.client.0.smithi070.stdout:2/63: sync 2024-03-23T08:23:44.294 INFO:tasks.workunit.client.0.smithi070.stdout:4/51: sync 2024-03-23T08:23:44.294 INFO:tasks.workunit.client.0.smithi070.stdout:9/111: sync 2024-03-23T08:23:44.296 INFO:tasks.workunit.client.0.smithi070.stdout:2/64: readlink d1/d6/l15 0 2024-03-23T08:23:44.296 INFO:tasks.workunit.client.0.smithi070.stdout:4/52: write d3/d6/d9/ff [4917,79715] 0 2024-03-23T08:23:44.297 INFO:tasks.workunit.client.0.smithi070.stdout:9/112: unlink db/l11 0 2024-03-23T08:23:44.298 INFO:tasks.workunit.client.0.smithi070.stdout:3/154: dread db/df/d15/d22/f2c [0,4194304] 0 2024-03-23T08:23:44.299 INFO:tasks.workunit.client.0.smithi070.stdout:2/65: truncate d1/d7/d11/f12 1032186 0 2024-03-23T08:23:44.300 INFO:tasks.workunit.client.0.smithi070.stdout:4/53: mknod d3/c10 0 2024-03-23T08:23:44.300 INFO:tasks.workunit.client.0.smithi070.stdout:9/113: read db/f1b [7846,31951] 0 2024-03-23T08:23:44.300 INFO:tasks.workunit.client.0.smithi070.stdout:0/256: dread d16/d26/d27/f45 [0,4194304] 0 2024-03-23T08:23:44.303 INFO:tasks.workunit.client.0.smithi070.stdout:3/155: mkdir db/df/d15/d3d 0 2024-03-23T08:23:44.305 INFO:tasks.workunit.client.0.smithi070.stdout:2/66: rmdir d1 39 2024-03-23T08:23:44.306 INFO:tasks.workunit.client.0.smithi070.stdout:4/54: mkdir d3/d6/d11 0 2024-03-23T08:23:44.308 INFO:tasks.workunit.client.0.smithi070.stdout:9/114: rename db/d18 to db/d12/d22/d25 0 2024-03-23T08:23:44.309 INFO:tasks.workunit.client.0.smithi070.stdout:9/115: stat l1 0 2024-03-23T08:23:44.311 INFO:tasks.workunit.client.0.smithi070.stdout:0/257: creat d16/d22/d43/f5a x:0 0 0 2024-03-23T08:23:44.313 INFO:tasks.workunit.client.0.smithi070.stdout:3/156: dwrite db/df/d1a/f3c [0,4194304] 0 2024-03-23T08:23:44.313 INFO:tasks.workunit.client.0.smithi070.stdout:0/258: write d16/f34 [956704,93314] 0 2024-03-23T08:23:44.314 INFO:tasks.workunit.client.0.smithi070.stdout:4/55: symlink d3/l12 0 2024-03-23T08:23:44.314 INFO:tasks.workunit.client.0.smithi070.stdout:3/157: chown db/df/d1a 2 1 2024-03-23T08:23:44.322 INFO:tasks.workunit.client.0.smithi070.stdout:0/259: creat d16/f5b x:0 0 0 2024-03-23T08:23:44.322 INFO:tasks.workunit.client.0.smithi070.stdout:9/116: chown db/l10 10 1 2024-03-23T08:23:44.323 INFO:tasks.workunit.client.0.smithi070.stdout:4/56: dwrite d3/d6/f7 [0,4194304] 0 2024-03-23T08:23:44.325 INFO:tasks.workunit.client.0.smithi070.stdout:3/158: creat db/df/d15/f3e x:0 0 0 2024-03-23T08:23:44.328 INFO:tasks.workunit.client.0.smithi070.stdout:9/117: mkdir db/d12/d22/d25/d26 0 2024-03-23T08:23:44.332 INFO:tasks.workunit.client.0.smithi070.stdout:3/159: creat db/d14/f3f x:0 0 0 2024-03-23T08:23:44.332 INFO:tasks.workunit.client.0.smithi070.stdout:0/260: symlink d16/d22/d43/d47/l5c 0 2024-03-23T08:23:44.333 INFO:tasks.workunit.client.0.smithi070.stdout:3/160: write db/df/d1a/f37 [3454683,95817] 0 2024-03-23T08:23:44.334 INFO:tasks.workunit.client.0.smithi070.stdout:4/57: creat d3/d6/d11/f13 x:0 0 0 2024-03-23T08:23:44.335 INFO:tasks.workunit.client.0.smithi070.stdout:4/58: readlink d3/l12 0 2024-03-23T08:23:44.336 INFO:tasks.workunit.client.0.smithi070.stdout:0/261: creat d16/d26/f5d x:0 0 0 2024-03-23T08:23:44.337 INFO:tasks.workunit.client.0.smithi070.stdout:4/59: chown d3/d6/d11/f13 1208 1 2024-03-23T08:23:44.338 INFO:tasks.workunit.client.0.smithi070.stdout:0/262: chown d16/c17 10305 1 2024-03-23T08:23:44.339 INFO:tasks.workunit.client.0.smithi070.stdout:3/161: symlink db/df/d15/d3d/l40 0 2024-03-23T08:23:44.341 INFO:tasks.workunit.client.0.smithi070.stdout:9/118: dwrite f6 [4194304,4194304] 0 2024-03-23T08:23:44.341 INFO:tasks.workunit.client.0.smithi070.stdout:4/60: mknod d3/d6/d9/c14 0 2024-03-23T08:23:44.345 INFO:tasks.workunit.client.0.smithi070.stdout:3/162: fsync db/df/f2e 0 2024-03-23T08:23:44.347 INFO:tasks.workunit.client.0.smithi070.stdout:3/163: fdatasync db/f2a 0 2024-03-23T08:23:44.349 INFO:tasks.workunit.client.0.smithi070.stdout:3/164: readlink db/l10 0 2024-03-23T08:23:44.349 INFO:tasks.workunit.client.0.smithi070.stdout:4/61: symlink d3/d6/d9/l15 0 2024-03-23T08:23:44.351 INFO:tasks.workunit.client.0.smithi070.stdout:0/263: dwrite d16/f3e [0,4194304] 0 2024-03-23T08:23:44.351 INFO:tasks.workunit.client.0.smithi070.stdout:4/62: chown f2 594863769 1 2024-03-23T08:23:44.352 INFO:tasks.workunit.client.0.smithi070.stdout:9/119: creat db/d12/d22/f27 x:0 0 0 2024-03-23T08:23:44.352 INFO:tasks.workunit.client.0.smithi070.stdout:4/63: chown d3/d5 161842219 1 2024-03-23T08:23:44.352 INFO:tasks.workunit.client.0.smithi070.stdout:0/264: chown d16/d22/l21 9427 1 2024-03-23T08:23:44.353 INFO:tasks.workunit.client.0.smithi070.stdout:0/265: stat d16/d22/c53 0 2024-03-23T08:23:44.354 INFO:tasks.workunit.client.0.smithi070.stdout:0/266: chown d16/f30 13 1 2024-03-23T08:23:44.359 INFO:tasks.workunit.client.0.smithi070.stdout:9/120: rmdir db/d12/d22/d25 39 2024-03-23T08:23:44.359 INFO:tasks.workunit.client.0.smithi070.stdout:4/64: dread d3/fd [0,4194304] 0 2024-03-23T08:23:44.360 INFO:tasks.workunit.client.0.smithi070.stdout:4/65: readlink d3/d6/d9/l15 0 2024-03-23T08:23:44.363 INFO:tasks.workunit.client.0.smithi070.stdout:0/267: rename c15 to d16/d22/c5e 0 2024-03-23T08:23:44.369 INFO:tasks.workunit.client.0.smithi070.stdout:3/165: link db/d20/f33 db/df/d15/f41 0 2024-03-23T08:23:44.369 INFO:tasks.workunit.client.0.smithi070.stdout:9/121: fsync f6 0 2024-03-23T08:23:44.371 INFO:tasks.workunit.client.0.smithi070.stdout:9/122: readlink db/lf 0 2024-03-23T08:23:44.371 INFO:tasks.workunit.client.0.smithi070.stdout:4/66: dwrite d3/d6/d9/ff [0,4194304] 0 2024-03-23T08:23:44.373 INFO:tasks.workunit.client.0.smithi070.stdout:9/123: dread - db/f1f zero size 2024-03-23T08:23:44.376 INFO:tasks.workunit.client.0.smithi070.stdout:0/268: dwrite d16/d22/f29 [0,4194304] 0 2024-03-23T08:23:44.378 INFO:tasks.workunit.client.0.smithi070.stdout:3/166: creat db/df/d15/d22/f42 x:0 0 0 2024-03-23T08:23:44.379 INFO:tasks.workunit.client.0.smithi070.stdout:0/269: mkdir d16/d5f 0 2024-03-23T08:23:44.382 INFO:tasks.workunit.client.0.smithi070.stdout:0/270: fdatasync d16/f1f 0 2024-03-23T08:23:44.386 INFO:tasks.workunit.client.0.smithi070.stdout:0/271: stat d16/d22/l21 0 2024-03-23T08:23:44.388 INFO:tasks.workunit.client.0.smithi070.stdout:9/124: dwrite db/d12/d22/d25/f24 [0,4194304] 0 2024-03-23T08:23:44.396 INFO:tasks.workunit.client.0.smithi070.stdout:9/125: dread db/d12/d22/d25/f24 [0,4194304] 0 2024-03-23T08:23:44.397 INFO:tasks.workunit.client.0.smithi070.stdout:3/167: write db/df/d15/d22/f2c [881853,63337] 0 2024-03-23T08:23:44.399 INFO:tasks.workunit.client.0.smithi070.stdout:9/126: chown db/d12/f1a 20 1 2024-03-23T08:23:44.401 INFO:tasks.workunit.client.0.smithi070.stdout:0/272: dwrite d16/d22/d43/f5a [0,4194304] 0 2024-03-23T08:23:44.404 INFO:tasks.workunit.client.0.smithi070.stdout:9/127: chown f8 464066814 1 2024-03-23T08:23:44.405 INFO:tasks.workunit.client.0.smithi070.stdout:9/128: stat db/d12/c16 0 2024-03-23T08:23:44.406 INFO:tasks.workunit.client.0.smithi070.stdout:3/168: mkdir db/d14/d43 0 2024-03-23T08:23:44.408 INFO:tasks.workunit.client.0.smithi070.stdout:9/129: truncate db/f1b 552169 0 2024-03-23T08:23:44.409 INFO:tasks.workunit.client.0.smithi070.stdout:0/273: mkdir d16/d26/d27/d4c/d60 0 2024-03-23T08:23:44.410 INFO:tasks.workunit.client.0.smithi070.stdout:3/169: creat db/df/d1a/f44 x:0 0 0 2024-03-23T08:23:44.410 INFO:tasks.workunit.client.0.smithi070.stdout:0/274: chown f14 159511 1 2024-03-23T08:23:44.412 INFO:tasks.workunit.client.0.smithi070.stdout:0/275: dread d16/f1d [0,4194304] 0 2024-03-23T08:23:44.413 INFO:tasks.workunit.client.0.smithi070.stdout:9/130: rename db/l10 to db/d12/l28 0 2024-03-23T08:23:44.415 INFO:tasks.workunit.client.0.smithi070.stdout:9/131: write db/d12/d22/d25/f19 [467001,2754] 0 2024-03-23T08:23:44.418 INFO:tasks.workunit.client.0.smithi070.stdout:0/276: symlink d16/d26/d27/d4c/d60/l61 0 2024-03-23T08:23:44.418 INFO:tasks.workunit.client.0.smithi070.stdout:3/170: dwrite db/df/d1a/f37 [0,4194304] 0 2024-03-23T08:23:44.420 INFO:tasks.workunit.client.0.smithi070.stdout:9/132: mkdir db/d12/d29 0 2024-03-23T08:23:44.423 INFO:tasks.workunit.client.0.smithi070.stdout:3/171: creat db/d20/f45 x:0 0 0 2024-03-23T08:23:44.430 INFO:tasks.workunit.client.0.smithi070.stdout:9/133: creat db/d12/d22/f2a x:0 0 0 2024-03-23T08:23:44.435 INFO:tasks.workunit.client.0.smithi070.stdout:3/172: getdents db/df/d15/d22 0 2024-03-23T08:23:44.435 INFO:tasks.workunit.client.0.smithi070.stdout:3/173: chown db/c31 2024465857 1 2024-03-23T08:23:44.438 INFO:tasks.workunit.client.0.smithi070.stdout:0/277: rmdir d16 39 2024-03-23T08:23:44.442 INFO:tasks.workunit.client.0.smithi070.stdout:3/174: dwrite f6 [0,4194304] 0 2024-03-23T08:23:44.442 INFO:tasks.workunit.client.0.smithi070.stdout:0/278: rmdir d16/d26/d27/d4c/d60 39 2024-03-23T08:23:44.442 INFO:tasks.workunit.client.0.smithi070.stdout:3/175: stat db/d14 0 2024-03-23T08:23:44.444 INFO:tasks.workunit.client.0.smithi070.stdout:3/176: dread db/d14/f1b [0,4194304] 0 2024-03-23T08:23:44.446 INFO:tasks.workunit.client.0.smithi070.stdout:0/279: chown l10 1 1 2024-03-23T08:23:44.446 INFO:tasks.workunit.client.0.smithi070.stdout:9/134: rmdir db/d12/d22 39 2024-03-23T08:23:44.446 INFO:tasks.workunit.client.0.smithi070.stdout:0/280: read d16/d22/f29 [2876702,90556] 0 2024-03-23T08:23:44.448 INFO:tasks.workunit.client.0.smithi070.stdout:3/177: unlink db/d14/c35 0 2024-03-23T08:23:44.452 INFO:tasks.workunit.client.0.smithi070.stdout:8/106: sync 2024-03-23T08:23:44.459 INFO:tasks.workunit.client.0.smithi070.stdout:3/178: symlink db/df/d15/l46 0 2024-03-23T08:23:44.462 INFO:tasks.workunit.client.0.smithi070.stdout:8/107: mkdir d5/d8/d16/d28 0 2024-03-23T08:23:44.467 INFO:tasks.workunit.client.0.smithi070.stdout:3/179: dwrite db/df/d1a/f37 [4194304,4194304] 0 2024-03-23T08:23:44.468 INFO:tasks.workunit.client.0.smithi070.stdout:8/108: write d5/d8/f17 [389429,99240] 0 2024-03-23T08:23:44.469 INFO:tasks.workunit.client.0.smithi070.stdout:3/180: dread - db/df/d15/d22/f32 zero size 2024-03-23T08:23:44.476 INFO:tasks.workunit.client.0.smithi070.stdout:8/109: getdents d5 0 2024-03-23T08:23:44.477 INFO:tasks.workunit.client.0.smithi070.stdout:3/181: dwrite db/d20/f45 [0,4194304] 0 2024-03-23T08:23:44.478 INFO:tasks.workunit.client.0.smithi070.stdout:3/182: stat db/df/l1d 0 2024-03-23T08:23:44.481 INFO:tasks.workunit.client.0.smithi070.stdout:8/110: symlink d5/d8/d26/l29 0 2024-03-23T08:23:44.483 INFO:tasks.workunit.client.0.smithi070.stdout:3/183: mknod db/df/d15/d3d/c47 0 2024-03-23T08:23:44.484 INFO:tasks.workunit.client.0.smithi070.stdout:3/184: write db/f36 [983613,110470] 0 2024-03-23T08:23:44.486 INFO:tasks.workunit.client.0.smithi070.stdout:8/111: chown d5/d8/d16/l20 124 1 2024-03-23T08:23:44.487 INFO:tasks.workunit.client.0.smithi070.stdout:3/185: creat db/d20/f48 x:0 0 0 2024-03-23T08:23:44.490 INFO:tasks.workunit.client.0.smithi070.stdout:3/186: mkdir db/d20/d49 0 2024-03-23T08:23:44.495 INFO:tasks.workunit.client.0.smithi070.stdout:3/187: fsync db/d14/f1b 0 2024-03-23T08:23:44.500 INFO:tasks.workunit.client.0.smithi070.stdout:8/112: rmdir d5/d8/d16/d28 0 2024-03-23T08:23:44.501 INFO:tasks.workunit.client.0.smithi070.stdout:9/135: write db/f1b [993123,76120] 0 2024-03-23T08:23:44.501 INFO:tasks.workunit.client.0.smithi070.stdout:0/281: dwrite f3 [0,4194304] 0 2024-03-23T08:23:44.504 INFO:tasks.workunit.client.0.smithi070.stdout:0/282: truncate d16/d22/f2f 5038350 0 2024-03-23T08:23:44.512 INFO:tasks.workunit.client.0.smithi070.stdout:8/113: creat d5/f2a x:0 0 0 2024-03-23T08:23:44.515 INFO:tasks.workunit.client.0.smithi070.stdout:0/283: creat d16/d22/d43/f62 x:0 0 0 2024-03-23T08:23:44.515 INFO:tasks.workunit.client.0.smithi070.stdout:3/188: creat db/d14/d43/f4a x:0 0 0 2024-03-23T08:23:44.518 INFO:tasks.workunit.client.0.smithi070.stdout:9/136: rmdir db/d12/d22 39 2024-03-23T08:23:44.519 INFO:tasks.workunit.client.0.smithi070.stdout:8/114: creat d5/d8/d16/f2b x:0 0 0 2024-03-23T08:23:44.525 INFO:tasks.workunit.client.0.smithi070.stdout:3/189: creat db/d14/f4b x:0 0 0 2024-03-23T08:23:44.525 INFO:tasks.workunit.client.0.smithi070.stdout:8/115: creat d5/d8/f2c x:0 0 0 2024-03-23T08:23:44.527 INFO:tasks.workunit.client.0.smithi070.stdout:9/137: write db/d12/d22/f27 [731056,44291] 0 2024-03-23T08:23:44.528 INFO:tasks.workunit.client.0.smithi070.stdout:0/284: dread d16/d22/d43/f5a [0,4194304] 0 2024-03-23T08:23:44.528 INFO:tasks.workunit.client.0.smithi070.stdout:8/116: fdatasync d5/d8/f25 0 2024-03-23T08:23:44.529 INFO:tasks.workunit.client.0.smithi070.stdout:3/190: dread - db/d14/f4b zero size 2024-03-23T08:23:44.532 INFO:tasks.workunit.client.0.smithi070.stdout:9/138: chown db/d12/d22/d25/l1c 14 1 2024-03-23T08:23:44.533 INFO:tasks.workunit.client.0.smithi070.stdout:8/117: mknod d5/d8/d26/c2d 0 2024-03-23T08:23:44.534 INFO:tasks.workunit.client.0.smithi070.stdout:3/191: write db/f23 [588692,90166] 0 2024-03-23T08:23:44.535 INFO:tasks.workunit.client.0.smithi070.stdout:8/118: stat d5/d8/d16/f22 0 2024-03-23T08:23:44.538 INFO:tasks.workunit.client.0.smithi070.stdout:0/285: mknod d16/d26/c63 0 2024-03-23T08:23:44.539 INFO:tasks.workunit.client.0.smithi070.stdout:8/119: truncate d5/d8/d16/f24 30282 0 2024-03-23T08:23:44.543 INFO:tasks.workunit.client.0.smithi070.stdout:8/120: truncate d5/fe 777068 0 2024-03-23T08:23:44.545 INFO:tasks.workunit.client.0.smithi070.stdout:9/139: creat db/d12/f2b x:0 0 0 2024-03-23T08:23:44.548 INFO:tasks.workunit.client.0.smithi070.stdout:3/192: dwrite db/df/d15/d22/f2c [0,4194304] 0 2024-03-23T08:23:44.549 INFO:tasks.workunit.client.0.smithi070.stdout:0/286: dread d16/d26/d27/f45 [0,4194304] 0 2024-03-23T08:23:44.550 INFO:tasks.workunit.client.0.smithi070.stdout:0/287: fsync d16/f34 0 2024-03-23T08:23:44.556 INFO:tasks.workunit.client.0.smithi070.stdout:8/121: dwrite d5/d8/fb [0,4194304] 0 2024-03-23T08:23:44.556 INFO:tasks.workunit.client.0.smithi070.stdout:9/140: creat db/d12/d22/d25/d26/f2c x:0 0 0 2024-03-23T08:23:44.559 INFO:tasks.workunit.client.0.smithi070.stdout:3/193: dwrite db/df/d15/d22/f2c [0,4194304] 0 2024-03-23T08:23:44.560 INFO:tasks.workunit.client.0.smithi070.stdout:0/288: rename d16/d26/d27/d3d/f49 to d16/d26/d27/d3d/f64 0 2024-03-23T08:23:44.561 INFO:tasks.workunit.client.0.smithi070.stdout:3/194: write db/df/d15/f3e [153085,43290] 0 2024-03-23T08:23:44.563 INFO:tasks.workunit.client.0.smithi070.stdout:8/122: mknod d5/c2e 0 2024-03-23T08:23:44.564 INFO:tasks.workunit.client.0.smithi070.stdout:8/123: stat d5/d8/d16/f22 0 2024-03-23T08:23:44.565 INFO:tasks.workunit.client.0.smithi070.stdout:8/124: dread - d5/d8/f2c zero size 2024-03-23T08:23:44.565 INFO:tasks.workunit.client.0.smithi070.stdout:8/125: read d5/d8/f13 [3808691,57366] 0 2024-03-23T08:23:44.566 INFO:tasks.workunit.client.0.smithi070.stdout:9/141: write db/d12/d22/d25/f24 [4165196,13083] 0 2024-03-23T08:23:44.567 INFO:tasks.workunit.client.0.smithi070.stdout:9/142: fsync db/f21 0 2024-03-23T08:23:44.569 INFO:tasks.workunit.client.0.smithi070.stdout:3/195: unlink db/f2a 0 2024-03-23T08:23:44.570 INFO:tasks.workunit.client.0.smithi070.stdout:3/196: fsync db/f17 0 2024-03-23T08:23:44.573 INFO:tasks.workunit.client.0.smithi070.stdout:8/126: creat d5/d8/d26/f2f x:0 0 0 2024-03-23T08:23:44.578 INFO:tasks.workunit.client.0.smithi070.stdout:9/143: rmdir db/d12/d22/d25/d26 39 2024-03-23T08:23:44.585 INFO:tasks.workunit.client.0.smithi070.stdout:3/197: symlink db/l4c 0 2024-03-23T08:23:44.586 INFO:tasks.workunit.client.0.smithi070.stdout:3/198: write db/d20/f48 [498417,102306] 0 2024-03-23T08:23:44.593 INFO:tasks.workunit.client.0.smithi070.stdout:8/127: rename d5/d8/d26 to d5/d8/d16/d30 0 2024-03-23T08:23:44.595 INFO:tasks.workunit.client.0.smithi070.stdout:3/199: mkdir db/df/d4d 0 2024-03-23T08:23:44.596 INFO:tasks.workunit.client.0.smithi070.stdout:8/128: creat d5/f31 x:0 0 0 2024-03-23T08:23:44.597 INFO:tasks.workunit.client.0.smithi070.stdout:0/289: truncate d16/d22/d43/f5a 4189270 0 2024-03-23T08:23:44.598 INFO:tasks.workunit.client.0.smithi070.stdout:0/290: truncate d16/f30 683549 0 2024-03-23T08:23:44.599 INFO:tasks.workunit.client.0.smithi070.stdout:8/129: mkdir d5/d8/d16/d32 0 2024-03-23T08:23:44.601 INFO:tasks.workunit.client.0.smithi070.stdout:3/200: mkdir db/d4e 0 2024-03-23T08:23:44.601 INFO:tasks.workunit.client.0.smithi070.stdout:3/201: chown db/d14 1 1 2024-03-23T08:23:44.601 INFO:tasks.workunit.client.0.smithi070.stdout:9/144: truncate db/d12/d22/d25/f24 977179 0 2024-03-23T08:23:44.602 INFO:tasks.workunit.client.0.smithi070.stdout:3/202: truncate db/d14/d43/f4a 744338 0 2024-03-23T08:23:44.603 INFO:tasks.workunit.client.0.smithi070.stdout:0/291: readlink d16/d26/d27/d4c/d60/l61 0 2024-03-23T08:23:44.605 INFO:tasks.workunit.client.0.smithi070.stdout:8/130: rename d5/f31 to d5/d8/f33 0 2024-03-23T08:23:44.608 INFO:tasks.workunit.client.0.smithi070.stdout:9/145: mknod db/d12/d14/c2d 0 2024-03-23T08:23:44.612 INFO:tasks.workunit.client.0.smithi070.stdout:8/131: mknod d5/d8/c34 0 2024-03-23T08:23:44.613 INFO:tasks.workunit.client.0.smithi070.stdout:8/132: chown d5/d8 10900 1 2024-03-23T08:23:44.614 INFO:tasks.workunit.client.0.smithi070.stdout:9/146: unlink db/d12/d22/d25/f19 0 2024-03-23T08:23:44.627 INFO:tasks.workunit.client.0.smithi070.stdout:3/203: link l8 db/df/d4d/l4f 0 2024-03-23T08:23:44.629 INFO:tasks.workunit.client.0.smithi070.stdout:9/147: rmdir db/d12/d29 0 2024-03-23T08:23:44.632 INFO:tasks.workunit.client.0.smithi070.stdout:3/204: creat db/df/d1a/f50 x:0 0 0 2024-03-23T08:23:44.634 INFO:tasks.workunit.client.0.smithi070.stdout:9/148: mknod db/d12/d22/d25/d26/c2e 0 2024-03-23T08:23:44.636 INFO:tasks.workunit.client.0.smithi070.stdout:3/205: mkdir db/df/d4d/d51 0 2024-03-23T08:23:44.637 INFO:tasks.workunit.client.0.smithi070.stdout:3/206: readlink db/df/l2f 0 2024-03-23T08:23:44.640 INFO:tasks.workunit.client.0.smithi070.stdout:9/149: dwrite db/f1f [0,4194304] 0 2024-03-23T08:23:44.641 INFO:tasks.workunit.client.0.smithi070.stdout:3/207: symlink db/d4e/l52 0 2024-03-23T08:23:44.642 INFO:tasks.workunit.client.0.smithi070.stdout:9/150: dread - db/d12/d22/d25/d26/f2c zero size 2024-03-23T08:23:44.645 INFO:tasks.workunit.client.0.smithi070.stdout:3/208: creat db/df/d15/d3d/f53 x:0 0 0 2024-03-23T08:23:44.646 INFO:tasks.workunit.client.0.smithi070.stdout:9/151: chown db/d12/d22/d25/f24 30213 1 2024-03-23T08:23:44.649 INFO:tasks.workunit.client.0.smithi070.stdout:3/209: unlink db/df/d15/l46 0 2024-03-23T08:23:44.652 INFO:tasks.workunit.client.0.smithi070.stdout:3/210: rename f6 to db/d14/d43/f54 0 2024-03-23T08:23:44.655 INFO:tasks.workunit.client.0.smithi070.stdout:9/152: creat db/d12/d22/f2f x:0 0 0 2024-03-23T08:23:44.656 INFO:tasks.workunit.client.0.smithi070.stdout:9/153: readlink db/lf 0 2024-03-23T08:23:44.658 INFO:tasks.workunit.client.0.smithi070.stdout:3/211: unlink db/df/d15/d22/f42 0 2024-03-23T08:23:44.658 INFO:tasks.workunit.client.0.smithi070.stdout:3/212: chown db/df/d15/d22/l3b 13 1 2024-03-23T08:23:44.659 INFO:tasks.workunit.client.0.smithi070.stdout:3/213: read db/df/d15/d22/f2c [539457,46022] 0 2024-03-23T08:23:44.660 INFO:tasks.workunit.client.0.smithi070.stdout:9/154: rmdir db/d12/d22/d25/d26 39 2024-03-23T08:23:44.666 INFO:tasks.workunit.client.0.smithi070.stdout:9/155: creat db/d12/d22/d25/d26/f30 x:0 0 0 2024-03-23T08:23:44.673 INFO:tasks.workunit.client.0.smithi070.stdout:9/156: dwrite db/d12/d14/f23 [0,4194304] 0 2024-03-23T08:23:44.675 INFO:tasks.workunit.client.0.smithi070.stdout:9/157: write db/d12/d22/f2f [238722,14249] 0 2024-03-23T08:23:44.682 INFO:tasks.workunit.client.0.smithi070.stdout:3/214: dwrite db/df/d15/f41 [0,4194304] 0 2024-03-23T08:23:44.683 INFO:tasks.workunit.client.0.smithi070.stdout:3/215: chown db/df/d1a/l30 456902209 1 2024-03-23T08:23:44.687 INFO:tasks.workunit.client.0.smithi070.stdout:3/216: creat db/df/f55 x:0 0 0 2024-03-23T08:23:44.696 INFO:tasks.workunit.client.0.smithi070.stdout:3/217: dwrite db/df/d15/f3e [0,4194304] 0 2024-03-23T08:23:44.700 INFO:tasks.workunit.client.0.smithi070.stdout:3/218: creat db/df/d15/d3d/f56 x:0 0 0 2024-03-23T08:23:44.701 INFO:tasks.workunit.client.0.smithi070.stdout:3/219: write db/f36 [1143180,68532] 0 2024-03-23T08:23:44.704 INFO:tasks.workunit.client.0.smithi070.stdout:3/220: rmdir db/df/d15 39 2024-03-23T08:23:44.706 INFO:tasks.workunit.client.0.smithi070.stdout:3/221: stat db/df/d15/d3d/f56 0 2024-03-23T08:23:44.711 INFO:tasks.workunit.client.0.smithi070.stdout:3/222: dwrite db/d20/f33 [0,4194304] 0 2024-03-23T08:23:44.721 INFO:tasks.workunit.client.0.smithi070.stdout:3/223: truncate db/df/d15/f41 4142684 0 2024-03-23T08:23:44.724 INFO:tasks.workunit.client.0.smithi070.stdout:3/224: symlink db/l57 0 2024-03-23T08:23:44.727 INFO:tasks.workunit.client.0.smithi070.stdout:3/225: link db/d20/f48 db/df/d1a/f58 0 2024-03-23T08:23:44.729 INFO:tasks.workunit.client.0.smithi070.stdout:3/226: symlink db/d14/d43/l59 0 2024-03-23T08:23:44.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:44 smithi070 ceph-mon[39699]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:44.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:44 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/4286382734' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:23:44.750 INFO:tasks.workunit.client.0.smithi070.stdout:6/285: sync 2024-03-23T08:23:44.750 INFO:tasks.workunit.client.0.smithi070.stdout:1/219: sync 2024-03-23T08:23:44.750 INFO:tasks.workunit.client.0.smithi070.stdout:3/227: dwrite db/df/d1a/f50 [0,4194304] 0 2024-03-23T08:23:44.751 INFO:tasks.workunit.client.0.smithi070.stdout:6/286: dread - d0/d23/d4d/f54 zero size 2024-03-23T08:23:44.753 INFO:tasks.workunit.client.0.smithi070.stdout:1/220: creat d8/d11/f4d x:0 0 0 2024-03-23T08:23:44.753 INFO:tasks.workunit.client.0.smithi070.stdout:3/228: fsync db/df/d1a/f58 0 2024-03-23T08:23:44.754 INFO:tasks.workunit.client.0.smithi070.stdout:1/221: stat d8/db/d40/d43/d44 0 2024-03-23T08:23:44.756 INFO:tasks.workunit.client.0.smithi070.stdout:6/287: creat d0/d23/d28/d35/d3a/d3c/f60 x:0 0 0 2024-03-23T08:23:44.759 INFO:tasks.workunit.client.0.smithi070.stdout:3/229: creat db/d14/d43/f5a x:0 0 0 2024-03-23T08:23:44.760 INFO:tasks.workunit.client.0.smithi070.stdout:1/222: mkdir d8/d42/d4e 0 2024-03-23T08:23:44.765 INFO:tasks.workunit.client.0.smithi070.stdout:1/223: creat d8/db/d40/f4f x:0 0 0 2024-03-23T08:23:44.767 INFO:tasks.workunit.client.0.smithi070.stdout:6/288: dread d0/d23/d1c/f5d [0,4194304] 0 2024-03-23T08:23:44.768 INFO:tasks.workunit.client.0.smithi070.stdout:3/230: dwrite db/df/d1a/f37 [8388608,4194304] 0 2024-03-23T08:23:44.770 INFO:tasks.workunit.client.0.smithi070.stdout:1/224: unlink d8/db/d40/f4f 0 2024-03-23T08:23:44.774 INFO:tasks.workunit.client.0.smithi070.stdout:6/289: mknod d0/d23/d4d/c61 0 2024-03-23T08:23:44.774 INFO:tasks.workunit.client.0.smithi070.stdout:3/231: read db/df/d15/d22/f2c [4082954,10709] 0 2024-03-23T08:23:44.776 INFO:tasks.workunit.client.0.smithi070.stdout:6/290: write d0/d23/d28/d35/d3a/d3c/f60 [409051,42403] 0 2024-03-23T08:23:44.776 INFO:tasks.workunit.client.0.smithi070.stdout:6/291: readlink d0/d3/d2c/l42 0 2024-03-23T08:23:44.777 INFO:tasks.workunit.client.0.smithi070.stdout:3/232: symlink db/l5b 0 2024-03-23T08:23:44.781 INFO:tasks.workunit.client.0.smithi070.stdout:3/233: mknod db/c5c 0 2024-03-23T08:23:44.782 INFO:tasks.workunit.client.0.smithi070.stdout:3/234: dread - db/d14/f4b zero size 2024-03-23T08:23:44.785 INFO:tasks.workunit.client.0.smithi070.stdout:6/292: dwrite d0/d23/d28/d35/d3a/d3c/f51 [0,4194304] 0 2024-03-23T08:23:44.785 INFO:tasks.workunit.client.0.smithi070.stdout:3/235: write db/df/d15/f41 [602612,1192] 0 2024-03-23T08:23:44.789 INFO:tasks.workunit.client.0.smithi070.stdout:3/236: mkdir db/df/d5d 0 2024-03-23T08:23:44.789 INFO:tasks.workunit.client.0.smithi070.stdout:3/237: stat db/d20/d49 0 2024-03-23T08:23:44.794 INFO:tasks.workunit.client.0.smithi070.stdout:3/238: dread db/d20/f45 [0,4194304] 0 2024-03-23T08:23:44.796 INFO:tasks.workunit.client.0.smithi070.stdout:3/239: rmdir db/df/d1a 39 2024-03-23T08:23:44.800 INFO:tasks.workunit.client.0.smithi070.stdout:3/240: rename db/df/d1a/c38 to db/d14/c5e 0 2024-03-23T08:23:44.801 INFO:tasks.workunit.client.0.smithi070.stdout:3/241: dread - db/d14/f3f zero size 2024-03-23T08:23:44.805 INFO:tasks.workunit.client.0.smithi070.stdout:3/242: mknod db/df/d5d/c5f 0 2024-03-23T08:23:44.811 INFO:tasks.workunit.client.0.smithi070.stdout:3/243: creat db/df/f60 x:0 0 0 2024-03-23T08:23:44.812 INFO:tasks.workunit.client.0.smithi070.stdout:3/244: truncate db/d14/f3f 636408 0 2024-03-23T08:23:44.813 INFO:tasks.workunit.client.0.smithi070.stdout:3/245: write db/d14/f3f [343716,109810] 0 2024-03-23T08:23:44.820 INFO:tasks.workunit.client.0.smithi070.stdout:3/246: dwrite db/d20/f45 [4194304,4194304] 0 2024-03-23T08:23:44.824 INFO:tasks.workunit.client.0.smithi070.stdout:3/247: rename db/df/l29 to db/df/d4d/l61 0 2024-03-23T08:23:44.824 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:44 smithi193 ceph-mon[48370]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:44.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:44 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/4286382734' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:23:44.825 INFO:tasks.workunit.client.0.smithi070.stdout:3/248: chown db/df/d15/c2d 454431 1 2024-03-23T08:23:44.826 INFO:tasks.workunit.client.0.smithi070.stdout:3/249: stat db/df/d15/d3d 0 2024-03-23T08:23:44.827 INFO:tasks.workunit.client.0.smithi070.stdout:3/250: write db/d14/f3f [1098653,22713] 0 2024-03-23T08:23:44.840 INFO:tasks.workunit.client.0.smithi070.stdout:3/251: getdents db/df/d1a 0 2024-03-23T08:23:44.842 INFO:tasks.workunit.client.0.smithi070.stdout:3/252: mkdir db/d14/d43/d62 0 2024-03-23T08:23:44.846 INFO:tasks.workunit.client.0.smithi070.stdout:3/253: write db/d14/f1b [1601172,10679] 0 2024-03-23T08:23:44.848 INFO:tasks.workunit.client.0.smithi070.stdout:3/254: mkdir db/d20/d63 0 2024-03-23T08:23:44.849 INFO:tasks.workunit.client.0.smithi070.stdout:3/255: write db/df/d15/d22/f32 [368998,33127] 0 2024-03-23T08:23:44.852 INFO:tasks.workunit.client.0.smithi070.stdout:3/256: rename db/c34 to db/df/d4d/c64 0 2024-03-23T08:23:44.855 INFO:tasks.workunit.client.0.smithi070.stdout:3/257: dread db/df/d1a/f37 [8388608,4194304] 0 2024-03-23T08:23:44.856 INFO:tasks.workunit.client.0.smithi070.stdout:3/258: fdatasync db/d14/f3f 0 2024-03-23T08:23:44.860 INFO:tasks.workunit.client.0.smithi070.stdout:3/259: mkdir db/d14/d65 0 2024-03-23T08:23:44.872 INFO:tasks.workunit.client.0.smithi070.stdout:3/260: dwrite db/df/d1a/f58 [0,4194304] 0 2024-03-23T08:23:44.882 INFO:tasks.workunit.client.0.smithi070.stdout:3/261: dwrite db/df/d1a/f58 [0,4194304] 0 2024-03-23T08:23:44.882 INFO:tasks.workunit.client.0.smithi070.stdout:3/262: fdatasync db/d14/f3f 0 2024-03-23T08:23:44.883 INFO:tasks.workunit.client.0.smithi070.stdout:3/263: chown db/df/d1a 8 1 2024-03-23T08:23:44.886 INFO:tasks.workunit.client.0.smithi070.stdout:3/264: dread db/df/d15/f41 [0,4194304] 0 2024-03-23T08:23:44.890 INFO:tasks.workunit.client.0.smithi070.stdout:3/265: rename db/df/d4d/c64 to db/df/d15/c66 0 2024-03-23T08:23:44.896 INFO:tasks.workunit.client.0.smithi070.stdout:3/266: dwrite db/d14/f4b [0,4194304] 0 2024-03-23T08:23:44.897 INFO:tasks.workunit.client.0.smithi070.stdout:3/267: readlink db/df/d1a/l30 0 2024-03-23T08:23:44.898 INFO:tasks.workunit.client.0.smithi070.stdout:3/268: write db/d20/f48 [4034556,130179] 0 2024-03-23T08:23:44.899 INFO:tasks.workunit.client.0.smithi070.stdout:3/269: unlink db/l10 0 2024-03-23T08:23:44.904 INFO:tasks.workunit.client.0.smithi070.stdout:3/270: dwrite db/df/d15/f3e [0,4194304] 0 2024-03-23T08:23:44.906 INFO:tasks.workunit.client.0.smithi070.stdout:3/271: mkdir db/d14/d67 0 2024-03-23T08:23:44.908 INFO:tasks.workunit.client.0.smithi070.stdout:3/272: write db/d20/f33 [2032913,28313] 0 2024-03-23T08:23:44.910 INFO:tasks.workunit.client.0.smithi070.stdout:3/273: write db/df/d1a/f37 [12933378,76711] 0 2024-03-23T08:23:44.911 INFO:tasks.workunit.client.0.smithi070.stdout:3/274: stat db/df/d15/d3d/f53 0 2024-03-23T08:23:44.913 INFO:tasks.workunit.client.0.smithi070.stdout:3/275: symlink db/d14/d67/l68 0 2024-03-23T08:23:44.915 INFO:tasks.workunit.client.0.smithi070.stdout:3/276: creat db/d14/d43/d62/f69 x:0 0 0 2024-03-23T08:23:44.917 INFO:tasks.workunit.client.0.smithi070.stdout:3/277: mkdir db/df/d6a 0 2024-03-23T08:23:44.919 INFO:tasks.workunit.client.0.smithi070.stdout:3/278: rename db/df/c28 to db/d14/d67/c6b 0 2024-03-23T08:23:44.920 INFO:tasks.workunit.client.0.smithi070.stdout:3/279: mknod db/df/d6a/c6c 0 2024-03-23T08:23:44.927 INFO:tasks.workunit.client.0.smithi070.stdout:3/280: dwrite db/df/d15/d3d/f56 [0,4194304] 0 2024-03-23T08:23:44.928 INFO:tasks.workunit.client.0.smithi070.stdout:3/281: chown db/df/d1a/f37 1 1 2024-03-23T08:23:44.929 INFO:tasks.workunit.client.0.smithi070.stdout:3/282: write db/d20/f45 [7728774,60623] 0 2024-03-23T08:23:44.930 INFO:tasks.workunit.client.0.smithi070.stdout:3/283: write db/d14/f4b [1349466,86012] 0 2024-03-23T08:23:44.933 INFO:tasks.workunit.client.0.smithi070.stdout:7/74: sync 2024-03-23T08:23:44.934 INFO:tasks.workunit.client.0.smithi070.stdout:7/75: stat l6 0 2024-03-23T08:23:44.946 INFO:tasks.workunit.client.0.smithi070.stdout:3/284: dwrite db/d14/d43/f5a [0,4194304] 0 2024-03-23T08:23:44.949 INFO:tasks.workunit.client.0.smithi070.stdout:3/285: truncate db/df/d15/f41 4445875 0 2024-03-23T08:23:44.956 INFO:tasks.workunit.client.0.smithi070.stdout:3/286: dread db/df/d1a/f37 [0,4194304] 0 2024-03-23T08:23:44.958 INFO:tasks.workunit.client.0.smithi070.stdout:3/287: write db/df/d1a/f50 [1394987,17340] 0 2024-03-23T08:23:44.962 INFO:tasks.workunit.client.0.smithi070.stdout:3/288: dread db/df/d15/d22/f32 [0,4194304] 0 2024-03-23T08:23:44.963 INFO:tasks.workunit.client.0.smithi070.stdout:3/289: dread - db/df/d1a/f44 zero size 2024-03-23T08:23:44.966 INFO:tasks.workunit.client.0.smithi070.stdout:3/290: rmdir db/df 39 2024-03-23T08:23:44.972 INFO:tasks.workunit.client.0.smithi070.stdout:3/291: dwrite db/d20/f48 [0,4194304] 0 2024-03-23T08:23:44.980 INFO:tasks.workunit.client.0.smithi070.stdout:3/292: dwrite db/d14/d43/d62/f69 [0,4194304] 0 2024-03-23T08:23:45.163 INFO:tasks.workunit.client.0.smithi070.stdout:5/75: sync 2024-03-23T08:23:45.163 INFO:tasks.workunit.client.0.smithi070.stdout:2/67: sync 2024-03-23T08:23:45.163 INFO:tasks.workunit.client.0.smithi070.stdout:4/67: sync 2024-03-23T08:23:45.172 INFO:tasks.workunit.client.0.smithi070.stdout:4/68: dwrite d3/fc [0,4194304] 0 2024-03-23T08:23:45.173 INFO:tasks.workunit.client.0.smithi070.stdout:5/76: dwrite dc/f16 [0,4194304] 0 2024-03-23T08:23:45.176 INFO:tasks.workunit.client.0.smithi070.stdout:4/69: fdatasync f1 0 2024-03-23T08:23:45.179 INFO:tasks.workunit.client.0.smithi070.stdout:5/77: write dc/f16 [2841570,55148] 0 2024-03-23T08:23:45.181 INFO:tasks.workunit.client.0.smithi070.stdout:5/78: write f1 [4369268,74544] 0 2024-03-23T08:23:45.191 INFO:tasks.workunit.client.0.smithi070.stdout:2/68: dwrite d1/fd [4194304,4194304] 0 2024-03-23T08:23:45.198 INFO:tasks.workunit.client.0.smithi070.stdout:2/69: creat d1/f16 x:0 0 0 2024-03-23T08:23:45.205 INFO:tasks.workunit.client.0.smithi070.stdout:2/70: dwrite d1/fe [0,4194304] 0 2024-03-23T08:23:45.206 INFO:tasks.workunit.client.0.smithi070.stdout:2/71: truncate d1/d7/f8 2336384 0 2024-03-23T08:23:45.209 INFO:tasks.workunit.client.0.smithi070.stdout:2/72: creat d1/d7/f17 x:0 0 0 2024-03-23T08:23:45.218 INFO:tasks.workunit.client.0.smithi070.stdout:2/73: truncate d1/fc 3981702 0 2024-03-23T08:23:45.235 INFO:tasks.workunit.client.0.smithi070.stdout:2/74: dread d1/fd [0,4194304] 0 2024-03-23T08:23:45.247 INFO:tasks.workunit.client.0.smithi070.stdout:2/75: dread d1/fc [0,4194304] 0 2024-03-23T08:23:45.251 INFO:tasks.workunit.client.0.smithi070.stdout:2/76: rmdir d1/d6/d10 0 2024-03-23T08:23:45.262 INFO:tasks.workunit.client.0.smithi070.stdout:8/133: sync 2024-03-23T08:23:45.263 INFO:tasks.workunit.client.0.smithi070.stdout:0/292: sync 2024-03-23T08:23:45.263 INFO:tasks.workunit.client.0.smithi070.stdout:9/158: sync 2024-03-23T08:23:45.264 INFO:tasks.workunit.client.0.smithi070.stdout:2/77: dwrite d1/d7/f8 [0,4194304] 0 2024-03-23T08:23:45.264 INFO:tasks.workunit.client.0.smithi070.stdout:0/293: fsync d16/d26/d27/f35 0 2024-03-23T08:23:45.265 INFO:tasks.workunit.client.0.smithi070.stdout:9/159: chown db/d12/d22/d25 1715 1 2024-03-23T08:23:45.265 INFO:tasks.workunit.client.0.smithi070.stdout:8/134: write d5/f2a [246316,107629] 0 2024-03-23T08:23:45.266 INFO:tasks.workunit.client.0.smithi070.stdout:9/160: write f6 [7633741,30213] 0 2024-03-23T08:23:45.270 INFO:tasks.workunit.client.0.smithi070.stdout:8/135: unlink d5/d8/f10 0 2024-03-23T08:23:45.272 INFO:tasks.workunit.client.0.smithi070.stdout:0/294: symlink d16/d22/d43/d47/l65 0 2024-03-23T08:23:45.275 INFO:tasks.workunit.client.0.smithi070.stdout:9/161: dwrite db/d12/d22/f2f [0,4194304] 0 2024-03-23T08:23:45.276 INFO:tasks.workunit.client.0.smithi070.stdout:2/78: mkdir d1/d18 0 2024-03-23T08:23:45.278 INFO:tasks.workunit.client.0.smithi070.stdout:2/79: dread - d1/f16 zero size 2024-03-23T08:23:45.279 INFO:tasks.workunit.client.0.smithi070.stdout:2/80: readlink d1/la 0 2024-03-23T08:23:45.280 INFO:tasks.workunit.client.0.smithi070.stdout:8/136: mknod d5/c35 0 2024-03-23T08:23:45.280 INFO:tasks.workunit.client.0.smithi070.stdout:0/295: dwrite f11 [0,4194304] 0 2024-03-23T08:23:45.282 INFO:tasks.workunit.client.0.smithi070.stdout:2/81: read d1/fd [975507,255] 0 2024-03-23T08:23:45.283 INFO:tasks.workunit.client.0.smithi070.stdout:8/137: write d5/d8/f17 [2082610,107579] 0 2024-03-23T08:23:45.284 INFO:tasks.workunit.client.0.smithi070.stdout:8/138: stat d5/f2a 0 2024-03-23T08:23:45.286 INFO:tasks.workunit.client.0.smithi070.stdout:9/162: rmdir db/d12 39 2024-03-23T08:23:45.286 INFO:tasks.workunit.client.0.smithi070.stdout:8/139: truncate d5/d8/f1e 1796196 0 2024-03-23T08:23:45.286 INFO:tasks.workunit.client.0.smithi070.stdout:0/296: chown d16/d22/f2c 490956 1 2024-03-23T08:23:45.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/82: symlink d1/d7/l19 0 2024-03-23T08:23:45.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/83: chown d1/d7/d11 477 1 2024-03-23T08:23:45.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/84: stat d1/fe 0 2024-03-23T08:23:45.289 INFO:tasks.workunit.client.0.smithi070.stdout:2/85: fsync d1/fe 0 2024-03-23T08:23:45.291 INFO:tasks.workunit.client.0.smithi070.stdout:8/140: mknod d5/d8/d16/d32/c36 0 2024-03-23T08:23:45.294 INFO:tasks.workunit.client.0.smithi070.stdout:0/297: creat d16/d26/d27/d3d/d4d/f66 x:0 0 0 2024-03-23T08:23:45.294 INFO:tasks.workunit.client.0.smithi070.stdout:0/298: stat l10 0 2024-03-23T08:23:45.297 INFO:tasks.workunit.client.0.smithi070.stdout:0/299: mkdir d16/d26/d4a/d67 0 2024-03-23T08:23:45.300 INFO:tasks.workunit.client.0.smithi070.stdout:9/163: rename db/d12/d22/f2f to db/f31 0 2024-03-23T08:23:45.303 INFO:tasks.workunit.client.0.smithi070.stdout:0/300: creat d16/f68 x:0 0 0 2024-03-23T08:23:45.303 INFO:tasks.workunit.client.0.smithi070.stdout:2/86: link d1/d6/l15 d1/d7/l1a 0 2024-03-23T08:23:45.307 INFO:tasks.workunit.client.0.smithi070.stdout:9/164: rmdir db/d12 39 2024-03-23T08:23:45.307 INFO:tasks.workunit.client.0.smithi070.stdout:9/165: chown db/f1b 26044624 1 2024-03-23T08:23:45.309 INFO:tasks.workunit.client.0.smithi070.stdout:0/301: creat d16/d26/d27/d4c/d60/f69 x:0 0 0 2024-03-23T08:23:45.309 INFO:tasks.workunit.client.0.smithi070.stdout:0/302: fdatasync d16/d22/d43/f62 0 2024-03-23T08:23:45.310 INFO:tasks.workunit.client.0.smithi070.stdout:0/303: read d16/f18 [972021,35130] 0 2024-03-23T08:23:45.311 INFO:tasks.workunit.client.0.smithi070.stdout:2/87: unlink d1/d7/f8 0 2024-03-23T08:23:45.316 INFO:tasks.workunit.client.0.smithi070.stdout:9/166: chown db/d12/d22/d25/f24 16155 1 2024-03-23T08:23:45.317 INFO:tasks.workunit.client.0.smithi070.stdout:0/304: dwrite d16/d22/d43/f56 [0,4194304] 0 2024-03-23T08:23:45.321 INFO:tasks.workunit.client.0.smithi070.stdout:8/141: truncate d5/d8/f19 78302 0 2024-03-23T08:23:45.322 INFO:tasks.workunit.client.0.smithi070.stdout:9/167: dread f8 [0,4194304] 0 2024-03-23T08:23:45.324 INFO:tasks.workunit.client.0.smithi070.stdout:2/88: fdatasync f0 0 2024-03-23T08:23:45.325 INFO:tasks.workunit.client.0.smithi070.stdout:1/225: sync 2024-03-23T08:23:45.326 INFO:tasks.workunit.client.0.smithi070.stdout:6/293: sync 2024-03-23T08:23:45.326 INFO:tasks.workunit.client.0.smithi070.stdout:4/70: sync 2024-03-23T08:23:45.326 INFO:tasks.workunit.client.0.smithi070.stdout:3/293: sync 2024-03-23T08:23:45.326 INFO:tasks.workunit.client.0.smithi070.stdout:7/76: sync 2024-03-23T08:23:45.329 INFO:tasks.workunit.client.0.smithi070.stdout:1/226: write d8/db/d2c/f3c [430299,73005] 0 2024-03-23T08:23:45.331 INFO:tasks.workunit.client.0.smithi070.stdout:0/305: symlink d16/d22/d43/l6a 0 2024-03-23T08:23:45.333 INFO:tasks.workunit.client.0.smithi070.stdout:0/306: read - d16/f68 zero size 2024-03-23T08:23:45.337 INFO:tasks.workunit.client.0.smithi070.stdout:2/89: dwrite d1/fe [0,4194304] 0 2024-03-23T08:23:45.338 INFO:tasks.workunit.client.0.smithi070.stdout:7/77: dwrite f12 [0,4194304] 0 2024-03-23T08:23:45.338 INFO:tasks.workunit.client.0.smithi070.stdout:3/294: dread db/d14/d43/f54 [0,4194304] 0 2024-03-23T08:23:45.339 INFO:tasks.workunit.client.0.smithi070.stdout:9/168: mkdir db/d12/d22/d25/d26/d32 0 2024-03-23T08:23:45.340 INFO:tasks.workunit.client.0.smithi070.stdout:9/169: dread - db/d12/d22/f2a zero size 2024-03-23T08:23:45.341 INFO:tasks.workunit.client.0.smithi070.stdout:9/170: readlink l2 0 2024-03-23T08:23:45.343 INFO:tasks.workunit.client.0.smithi070.stdout:6/294: mknod d0/d23/d4d/c62 0 2024-03-23T08:23:45.346 INFO:tasks.workunit.client.0.smithi070.stdout:2/90: dwrite d1/f16 [0,4194304] 0 2024-03-23T08:23:45.351 INFO:tasks.workunit.client.0.smithi070.stdout:0/307: creat d16/d26/f6b x:0 0 0 2024-03-23T08:23:45.353 INFO:tasks.workunit.client.0.smithi070.stdout:8/142: link d5/f23 d5/d8/d16/d32/f37 0 2024-03-23T08:23:45.354 INFO:tasks.workunit.client.0.smithi070.stdout:8/143: dread - d5/d8/f2c zero size 2024-03-23T08:23:45.356 INFO:tasks.workunit.client.0.smithi070.stdout:7/78: symlink l1b 0 2024-03-23T08:23:45.358 INFO:tasks.workunit.client.0.smithi070.stdout:7/79: read f8 [758159,73] 0 2024-03-23T08:23:45.358 INFO:tasks.workunit.client.0.smithi070.stdout:7/80: rmdir - no directory 2024-03-23T08:23:45.365 INFO:tasks.workunit.client.0.smithi070.stdout:6/295: rmdir d0/d23/d28/d35/d3a/d3c/d5b 39 2024-03-23T08:23:45.368 INFO:tasks.workunit.client.0.smithi070.stdout:6/296: write d0/d3/d2c/f58 [901597,55942] 0 2024-03-23T08:23:45.373 INFO:tasks.workunit.client.0.smithi070.stdout:0/308: rename d16/d22/d43/d47/l5c to d16/d5f/l6c 0 2024-03-23T08:23:45.374 INFO:tasks.workunit.client.0.smithi070.stdout:3/295: mknod db/c6d 0 2024-03-23T08:23:45.378 INFO:tasks.workunit.client.0.smithi070.stdout:8/144: mkdir d5/d8/d16/d30/d38 0 2024-03-23T08:23:45.378 INFO:tasks.workunit.client.0.smithi070.stdout:6/297: dwrite d0/d23/d28/f4a [0,4194304] 0 2024-03-23T08:23:45.379 INFO:tasks.workunit.client.0.smithi070.stdout:6/298: chown d0/d23/d28/d35/d3a/d3c/l41 2 1 2024-03-23T08:23:45.379 INFO:tasks.workunit.client.0.smithi070.stdout:8/145: chown d5/d8/d16/d32/f37 2617 1 2024-03-23T08:23:45.382 INFO:tasks.workunit.client.0.smithi070.stdout:1/227: truncate d8/d1b/f22 1151431 0 2024-03-23T08:23:45.383 INFO:tasks.workunit.client.0.smithi070.stdout:2/91: creat d1/d18/f1b x:0 0 0 2024-03-23T08:23:45.383 INFO:tasks.workunit.client.0.smithi070.stdout:3/296: dwrite db/d20/f45 [4194304,4194304] 0 2024-03-23T08:23:45.385 INFO:tasks.workunit.client.0.smithi070.stdout:2/92: write d1/fe [2815339,104767] 0 2024-03-23T08:23:45.390 INFO:tasks.workunit.client.0.smithi070.stdout:7/81: rename l7 to l1c 0 2024-03-23T08:23:45.391 INFO:tasks.workunit.client.0.smithi070.stdout:6/299: creat d0/d23/d4d/f63 x:0 0 0 2024-03-23T08:23:45.392 INFO:tasks.workunit.client.0.smithi070.stdout:7/82: write f16 [592750,8459] 0 2024-03-23T08:23:45.393 INFO:tasks.workunit.client.0.smithi070.stdout:1/228: symlink d8/db/d2c/d3f/l50 0 2024-03-23T08:23:45.401 INFO:tasks.workunit.client.0.smithi070.stdout:3/297: truncate db/df/f55 5869 0 2024-03-23T08:23:45.402 INFO:tasks.workunit.client.0.smithi070.stdout:3/298: chown db/d20/l3a 18 1 2024-03-23T08:23:45.407 INFO:tasks.workunit.client.0.smithi070.stdout:0/309: truncate d16/d22/f24 110777 0 2024-03-23T08:23:45.407 INFO:tasks.workunit.client.0.smithi070.stdout:0/310: readlink d16/l31 0 2024-03-23T08:23:45.408 INFO:tasks.workunit.client.0.smithi070.stdout:9/171: link db/c15 db/d12/d22/d25/c33 0 2024-03-23T08:23:45.408 INFO:tasks.workunit.client.0.smithi070.stdout:9/172: chown db/c15 300760 1 2024-03-23T08:23:45.411 INFO:tasks.workunit.client.0.smithi070.stdout:7/83: creat f1d x:0 0 0 2024-03-23T08:23:45.412 INFO:tasks.workunit.client.0.smithi070.stdout:9/173: dread db/f1f [0,4194304] 0 2024-03-23T08:23:45.412 INFO:tasks.workunit.client.0.smithi070.stdout:9/174: fdatasync db/f1f 0 2024-03-23T08:23:45.412 INFO:tasks.workunit.client.0.smithi070.stdout:1/229: fsync d8/f2f 0 2024-03-23T08:23:45.414 INFO:tasks.workunit.client.0.smithi070.stdout:9/175: chown db/d12/f1d 305763675 1 2024-03-23T08:23:45.420 INFO:tasks.workunit.client.0.smithi070.stdout:2/93: dwrite f0 [0,4194304] 0 2024-03-23T08:23:45.426 INFO:tasks.workunit.client.0.smithi070.stdout:6/300: mknod d0/c64 0 2024-03-23T08:23:45.428 INFO:tasks.workunit.client.0.smithi070.stdout:6/301: fsync d0/d23/f17 0 2024-03-23T08:23:45.428 INFO:tasks.workunit.client.0.smithi070.stdout:7/84: symlink l1e 0 2024-03-23T08:23:45.430 INFO:tasks.workunit.client.0.smithi070.stdout:2/94: dwrite d1/fe [0,4194304] 0 2024-03-23T08:23:45.431 INFO:tasks.workunit.client.0.smithi070.stdout:2/95: chown d1/d7/df/l13 1543833540 1 2024-03-23T08:23:45.431 INFO:tasks.workunit.client.0.smithi070.stdout:1/230: mkdir d8/d42/d51 0 2024-03-23T08:23:45.434 INFO:tasks.workunit.client.0.smithi070.stdout:9/176: fsync db/f31 0 2024-03-23T08:23:45.436 INFO:tasks.workunit.client.0.smithi070.stdout:2/96: dread d1/f16 [0,4194304] 0 2024-03-23T08:23:45.440 INFO:tasks.workunit.client.0.smithi070.stdout:8/146: getdents d5 0 2024-03-23T08:23:45.445 INFO:tasks.workunit.client.0.smithi070.stdout:0/311: dwrite d16/d22/f29 [0,4194304] 0 2024-03-23T08:23:45.446 INFO:tasks.workunit.client.0.smithi070.stdout:7/85: read fd [2323635,79899] 0 2024-03-23T08:23:45.453 INFO:tasks.workunit.client.0.smithi070.stdout:5/79: sync 2024-03-23T08:23:45.456 INFO:tasks.workunit.client.0.smithi070.stdout:2/97: mknod d1/d18/c1c 0 2024-03-23T08:23:45.458 INFO:tasks.workunit.client.0.smithi070.stdout:7/86: dwrite f8 [4194304,4194304] 0 2024-03-23T08:23:45.458 INFO:tasks.workunit.client.0.smithi070.stdout:8/147: creat d5/d8/d16/d30/f39 x:0 0 0 2024-03-23T08:23:45.458 INFO:tasks.workunit.client.0.smithi070.stdout:3/299: getdents db/df/d6a 0 2024-03-23T08:23:45.461 INFO:tasks.workunit.client.0.smithi070.stdout:1/231: dwrite d8/d1a/f28 [4194304,4194304] 0 2024-03-23T08:23:45.463 INFO:tasks.workunit.client.0.smithi070.stdout:6/302: mknod d0/d23/d28/d35/d3a/d3c/c65 0 2024-03-23T08:23:45.463 INFO:tasks.workunit.client.0.smithi070.stdout:1/232: chown d8/d1b/l3a 110631542 1 2024-03-23T08:23:45.464 INFO:tasks.workunit.client.0.smithi070.stdout:0/312: write d16/f3b [731193,46415] 0 2024-03-23T08:23:45.468 INFO:tasks.workunit.client.0.smithi070.stdout:5/80: creat dc/f17 x:0 0 0 2024-03-23T08:23:45.471 INFO:tasks.workunit.client.0.smithi070.stdout:9/177: mkdir db/d12/d34 0 2024-03-23T08:23:45.476 INFO:tasks.workunit.client.0.smithi070.stdout:9/178: chown db/d12/d22/d25/f24 380 1 2024-03-23T08:23:45.478 INFO:tasks.workunit.client.0.smithi070.stdout:7/87: symlink l1f 0 2024-03-23T08:23:45.478 INFO:tasks.workunit.client.0.smithi070.stdout:3/300: rmdir db/df/d15/d3d 39 2024-03-23T08:23:45.482 INFO:tasks.workunit.client.0.smithi070.stdout:5/81: dwrite f1 [0,4194304] 0 2024-03-23T08:23:45.482 INFO:tasks.workunit.client.0.smithi070.stdout:3/301: write db/d20/f48 [2682083,107058] 0 2024-03-23T08:23:45.482 INFO:tasks.workunit.client.0.smithi070.stdout:7/88: chown f12 214 1 2024-03-23T08:23:45.482 INFO:tasks.workunit.client.0.smithi070.stdout:6/303: symlink d0/d23/d28/l66 0 2024-03-23T08:23:45.483 INFO:tasks.workunit.client.0.smithi070.stdout:8/148: creat d5/d8/f3a x:0 0 0 2024-03-23T08:23:45.484 INFO:tasks.workunit.client.0.smithi070.stdout:7/89: chown l1e 17 1 2024-03-23T08:23:45.488 INFO:tasks.workunit.client.0.smithi070.stdout:1/233: unlink d8/db/l18 0 2024-03-23T08:23:45.497 INFO:tasks.workunit.client.0.smithi070.stdout:6/304: mknod d0/d3/c67 0 2024-03-23T08:23:45.500 INFO:tasks.workunit.client.0.smithi070.stdout:3/302: dwrite db/d20/f33 [0,4194304] 0 2024-03-23T08:23:45.501 INFO:tasks.workunit.client.0.smithi070.stdout:8/149: mkdir d5/d8/d16/d3b 0 2024-03-23T08:23:45.502 INFO:tasks.workunit.client.0.smithi070.stdout:5/82: rename dc/f17 to dc/f18 0 2024-03-23T08:23:45.504 INFO:tasks.workunit.client.0.smithi070.stdout:7/90: dwrite f8 [8388608,4194304] 0 2024-03-23T08:23:45.504 INFO:tasks.workunit.client.0.smithi070.stdout:7/91: fdatasync f1d 0 2024-03-23T08:23:45.508 INFO:tasks.workunit.client.0.smithi070.stdout:6/305: mknod d0/d23/d4d/c68 0 2024-03-23T08:23:45.509 INFO:tasks.workunit.client.0.smithi070.stdout:3/303: chown l8 1922 1 2024-03-23T08:23:45.512 INFO:tasks.workunit.client.0.smithi070.stdout:1/234: symlink d8/db/d2c/d3f/d48/l52 0 2024-03-23T08:23:45.514 INFO:tasks.workunit.client.0.smithi070.stdout:5/83: mkdir dc/d19 0 2024-03-23T08:23:45.516 INFO:tasks.workunit.client.0.smithi070.stdout:5/84: write dc/f16 [5201677,107755] 0 2024-03-23T08:23:45.517 INFO:tasks.workunit.client.0.smithi070.stdout:2/98: getdents d1/d7/d11 0 2024-03-23T08:23:45.521 INFO:tasks.workunit.client.0.smithi070.stdout:6/306: rmdir d0/d23/d28 39 2024-03-23T08:23:45.522 INFO:tasks.workunit.client.0.smithi070.stdout:1/235: readlink d8/l36 0 2024-03-23T08:23:45.524 INFO:tasks.workunit.client.0.smithi070.stdout:1/236: chown d8/db/d2c/f3c 23 1 2024-03-23T08:23:45.525 INFO:tasks.workunit.client.0.smithi070.stdout:5/85: symlink dc/l1a 0 2024-03-23T08:23:45.526 INFO:tasks.workunit.client.0.smithi070.stdout:7/92: dread f16 [0,4194304] 0 2024-03-23T08:23:45.527 INFO:tasks.workunit.client.0.smithi070.stdout:3/304: mknod db/df/d15/c6e 0 2024-03-23T08:23:45.528 INFO:tasks.workunit.client.0.smithi070.stdout:6/307: truncate d0/d23/d1c/d2f/f4c 556689 0 2024-03-23T08:23:45.530 INFO:tasks.workunit.client.0.smithi070.stdout:5/86: mkdir dc/d1b 0 2024-03-23T08:23:45.531 INFO:tasks.workunit.client.0.smithi070.stdout:5/87: truncate f8 838038 0 2024-03-23T08:23:45.533 INFO:tasks.workunit.client.0.smithi070.stdout:7/93: creat f20 x:0 0 0 2024-03-23T08:23:45.535 INFO:tasks.workunit.client.0.smithi070.stdout:9/179: dread f4 [0,4194304] 0 2024-03-23T08:23:45.536 INFO:tasks.workunit.client.0.smithi070.stdout:6/308: chown d0/d23/d28/d35/d3a/d3c/f51 106 1 2024-03-23T08:23:45.538 INFO:tasks.workunit.client.0.smithi070.stdout:3/305: mkdir db/d14/d6f 0 2024-03-23T08:23:45.538 INFO:tasks.workunit.client.0.smithi070.stdout:0/313: write d16/d22/f2c [88423,80845] 0 2024-03-23T08:23:45.539 INFO:tasks.workunit.client.0.smithi070.stdout:2/99: creat d1/f1d x:0 0 0 2024-03-23T08:23:45.542 INFO:tasks.workunit.client.0.smithi070.stdout:1/237: symlink d8/d42/d4e/l53 0 2024-03-23T08:23:45.545 INFO:tasks.workunit.client.0.smithi070.stdout:7/94: mknod c21 0 2024-03-23T08:23:45.546 INFO:tasks.workunit.client.0.smithi070.stdout:3/306: dwrite db/d14/d43/f4a [0,4194304] 0 2024-03-23T08:23:45.547 INFO:tasks.workunit.client.0.smithi070.stdout:9/180: creat db/d12/d14/f35 x:0 0 0 2024-03-23T08:23:45.549 INFO:tasks.workunit.client.0.smithi070.stdout:2/100: creat d1/d7/f1e x:0 0 0 2024-03-23T08:23:45.549 INFO:tasks.workunit.client.0.smithi070.stdout:8/150: write d5/fe [768253,65692] 0 2024-03-23T08:23:45.550 INFO:tasks.workunit.client.0.smithi070.stdout:5/88: mknod dc/d1b/c1c 0 2024-03-23T08:23:45.550 INFO:tasks.workunit.client.0.smithi070.stdout:5/89: chown dc/d1b 173 1 2024-03-23T08:23:45.551 INFO:tasks.workunit.client.0.smithi070.stdout:2/101: write d1/d7/f1e [186612,110468] 0 2024-03-23T08:23:45.553 INFO:tasks.workunit.client.0.smithi070.stdout:1/238: creat d8/db/d40/d43/f54 x:0 0 0 2024-03-23T08:23:45.554 INFO:tasks.workunit.client.0.smithi070.stdout:7/95: creat f22 x:0 0 0 2024-03-23T08:23:45.555 INFO:tasks.workunit.client.0.smithi070.stdout:3/307: symlink db/d14/d43/d62/l70 0 2024-03-23T08:23:45.555 INFO:tasks.workunit.client.0.smithi070.stdout:7/96: dread - f1d zero size 2024-03-23T08:23:45.555 INFO:tasks.workunit.client.0.smithi070.stdout:5/90: truncate dc/f18 94000 0 2024-03-23T08:23:45.556 INFO:tasks.workunit.client.0.smithi070.stdout:7/97: fsync f1d 0 2024-03-23T08:23:45.558 INFO:tasks.workunit.client.0.smithi070.stdout:8/151: creat d5/d8/d16/d32/f3c x:0 0 0 2024-03-23T08:23:45.558 INFO:tasks.workunit.client.0.smithi070.stdout:3/308: truncate db/df/f55 347186 0 2024-03-23T08:23:45.560 INFO:tasks.workunit.client.0.smithi070.stdout:2/102: creat d1/d6/f1f x:0 0 0 2024-03-23T08:23:45.561 INFO:tasks.workunit.client.0.smithi070.stdout:2/103: stat d1/f1d 0 2024-03-23T08:23:45.564 INFO:tasks.workunit.client.0.smithi070.stdout:7/98: rename f22 to f23 0 2024-03-23T08:23:45.565 INFO:tasks.workunit.client.0.smithi070.stdout:3/309: dwrite db/d14/d43/f5a [0,4194304] 0 2024-03-23T08:23:45.566 INFO:tasks.workunit.client.0.smithi070.stdout:8/152: symlink d5/l3d 0 2024-03-23T08:23:45.569 INFO:tasks.workunit.client.0.smithi070.stdout:1/239: symlink d8/d26/d49/l55 0 2024-03-23T08:23:45.573 INFO:tasks.workunit.client.0.smithi070.stdout:1/240: dread - d8/d1a/f47 zero size 2024-03-23T08:23:45.574 INFO:tasks.workunit.client.0.smithi070.stdout:1/241: stat d8/db/f32 0 2024-03-23T08:23:45.575 INFO:tasks.workunit.client.0.smithi070.stdout:2/104: rename d1/d6/f1f to d1/d7/f20 0 2024-03-23T08:23:45.576 INFO:tasks.workunit.client.0.smithi070.stdout:8/153: dwrite d5/d8/d16/d30/f2f [0,4194304] 0 2024-03-23T08:23:45.578 INFO:tasks.workunit.client.0.smithi070.stdout:7/99: creat f24 x:0 0 0 2024-03-23T08:23:45.578 INFO:tasks.workunit.client.0.smithi070.stdout:7/100: stat f16 0 2024-03-23T08:23:45.578 INFO:tasks.workunit.client.0.smithi070.stdout:7/101: fsync f8 0 2024-03-23T08:23:45.579 INFO:tasks.workunit.client.0.smithi070.stdout:8/154: truncate d5/d8/d16/d32/f37 586487 0 2024-03-23T08:23:45.583 INFO:tasks.workunit.client.0.smithi070.stdout:9/181: link db/d12/d22/d25/d26/c2e db/d12/c36 0 2024-03-23T08:23:45.588 INFO:tasks.workunit.client.0.smithi070.stdout:2/105: stat d1/d6/l15 0 2024-03-23T08:23:45.591 INFO:tasks.workunit.client.0.smithi070.stdout:1/242: creat d8/db/d2c/d3f/f56 x:0 0 0 2024-03-23T08:23:45.595 INFO:tasks.workunit.client.0.smithi070.stdout:7/102: mkdir d25 0 2024-03-23T08:23:45.595 INFO:tasks.workunit.client.0.smithi070.stdout:7/103: chown lc 119817483 1 2024-03-23T08:23:45.598 INFO:tasks.workunit.client.0.smithi070.stdout:3/310: symlink db/df/d15/l71 0 2024-03-23T08:23:45.600 INFO:tasks.workunit.client.0.smithi070.stdout:5/91: truncate dc/f16 3128379 0 2024-03-23T08:23:45.601 INFO:tasks.workunit.client.0.smithi070.stdout:3/311: write db/df/d1a/f58 [3521350,32883] 0 2024-03-23T08:23:45.604 INFO:tasks.workunit.client.0.smithi070.stdout:8/155: dwrite d5/d8/f13 [4194304,4194304] 0 2024-03-23T08:23:45.604 INFO:tasks.workunit.client.0.smithi070.stdout:1/243: rename d8/d26/d49/l55 to d8/d1a/l57 0 2024-03-23T08:23:45.605 INFO:tasks.workunit.client.0.smithi070.stdout:1/244: fdatasync d8/db/f21 0 2024-03-23T08:23:45.611 INFO:tasks.workunit.client.0.smithi070.stdout:3/312: mknod db/df/d6a/c72 0 2024-03-23T08:23:45.612 INFO:tasks.workunit.client.0.smithi070.stdout:7/104: symlink d25/l26 0 2024-03-23T08:23:45.614 INFO:tasks.workunit.client.0.smithi070.stdout:8/156: unlink d5/c6 0 2024-03-23T08:23:45.620 INFO:tasks.workunit.client.0.smithi070.stdout:1/245: rename d8/f2f to d8/db/d2c/d3f/d48/f58 0 2024-03-23T08:23:45.626 INFO:tasks.workunit.client.0.smithi070.stdout:7/105: fdatasync fd 0 2024-03-23T08:23:45.628 INFO:tasks.workunit.client.0.smithi070.stdout:3/313: dread db/df/d15/d3d/f56 [0,4194304] 0 2024-03-23T08:23:45.630 INFO:tasks.workunit.client.0.smithi070.stdout:3/314: read db/d14/d43/f54 [2307255,20802] 0 2024-03-23T08:23:45.631 INFO:tasks.workunit.client.0.smithi070.stdout:9/182: getdents db 0 2024-03-23T08:23:45.631 INFO:tasks.workunit.client.0.smithi070.stdout:9/183: readlink db/lc 0 2024-03-23T08:23:45.633 INFO:tasks.workunit.client.0.smithi070.stdout:1/246: creat d8/d42/d4e/f59 x:0 0 0 2024-03-23T08:23:45.634 INFO:tasks.workunit.client.0.smithi070.stdout:2/106: getdents d1/d7/df 0 2024-03-23T08:23:45.637 INFO:tasks.workunit.client.0.smithi070.stdout:7/106: mkdir d25/d27 0 2024-03-23T08:23:45.637 INFO:tasks.workunit.client.0.smithi070.stdout:7/107: stat c5 0 2024-03-23T08:23:45.641 INFO:tasks.workunit.client.0.smithi070.stdout:8/157: fsync d5/d8/d16/d32/f37 0 2024-03-23T08:23:45.641 INFO:tasks.workunit.client.0.smithi070.stdout:3/315: creat db/d4e/f73 x:0 0 0 2024-03-23T08:23:45.643 INFO:tasks.workunit.client.0.smithi070.stdout:8/158: dread d5/f2a [0,4194304] 0 2024-03-23T08:23:45.644 INFO:tasks.workunit.client.0.smithi070.stdout:1/247: symlink d8/d1a/l5a 0 2024-03-23T08:23:45.646 INFO:tasks.workunit.client.0.smithi070.stdout:5/92: dread dc/f16 [0,4194304] 0 2024-03-23T08:23:45.646 INFO:tasks.workunit.client.0.smithi070.stdout:7/108: mknod d25/c28 0 2024-03-23T08:23:45.648 INFO:tasks.workunit.client.0.smithi070.stdout:9/184: mknod db/d12/d34/c37 0 2024-03-23T08:23:45.650 INFO:tasks.workunit.client.0.smithi070.stdout:8/159: creat d5/d8/d16/d30/f3e x:0 0 0 2024-03-23T08:23:45.651 INFO:tasks.workunit.client.0.smithi070.stdout:8/160: chown d5/d8/d16/d32/f37 23606557 1 2024-03-23T08:23:45.651 INFO:tasks.workunit.client.0.smithi070.stdout:1/248: mknod d8/d1a/c5b 0 2024-03-23T08:23:45.653 INFO:tasks.workunit.client.0.smithi070.stdout:8/161: truncate d5/d8/f21 792177 0 2024-03-23T08:23:45.653 INFO:tasks.workunit.client.0.smithi070.stdout:9/185: dread db/f31 [0,4194304] 0 2024-03-23T08:23:45.653 INFO:tasks.workunit.client.0.smithi070.stdout:2/107: mkdir d1/d21 0 2024-03-23T08:23:45.654 INFO:tasks.workunit.client.0.smithi070.stdout:9/186: chown db/d12/f2b 79845 1 2024-03-23T08:23:45.655 INFO:tasks.workunit.client.0.smithi070.stdout:2/108: read d1/d7/f1e [54065,504] 0 2024-03-23T08:23:45.655 INFO:tasks.workunit.client.0.smithi070.stdout:9/187: chown db/d12/d22/d25/c33 129318 1 2024-03-23T08:23:45.656 INFO:tasks.workunit.client.0.smithi070.stdout:7/109: rename c21 to d25/d27/c29 0 2024-03-23T08:23:45.657 INFO:tasks.workunit.client.0.smithi070.stdout:8/162: mknod d5/d8/c3f 0 2024-03-23T08:23:45.657 INFO:tasks.workunit.client.0.smithi070.stdout:5/93: getdents dc/d19 0 2024-03-23T08:23:45.660 INFO:tasks.workunit.client.0.smithi070.stdout:2/109: unlink d1/f1d 0 2024-03-23T08:23:45.661 INFO:tasks.workunit.client.0.smithi070.stdout:9/188: rmdir db 39 2024-03-23T08:23:45.661 INFO:tasks.workunit.client.0.smithi070.stdout:2/110: stat d1/d7/d11/f12 0 2024-03-23T08:23:45.661 INFO:tasks.workunit.client.0.smithi070.stdout:2/111: readlink d1/d7/l19 0 2024-03-23T08:23:45.662 INFO:tasks.workunit.client.0.smithi070.stdout:2/112: rename d1/d21 to d1/d21/d22 22 2024-03-23T08:23:45.663 INFO:tasks.workunit.client.0.smithi070.stdout:5/94: mkdir dc/d1b/d1d 0 2024-03-23T08:23:45.665 INFO:tasks.workunit.client.0.smithi070.stdout:8/163: mknod d5/d8/d16/d30/c40 0 2024-03-23T08:23:45.668 INFO:tasks.workunit.client.0.smithi070.stdout:3/316: getdents db/df/d15 0 2024-03-23T08:23:45.668 INFO:tasks.workunit.client.0.smithi070.stdout:2/113: mknod d1/d7/c23 0 2024-03-23T08:23:45.669 INFO:tasks.workunit.client.0.smithi070.stdout:3/317: write db/df/d1a/f50 [1718663,19258] 0 2024-03-23T08:23:45.670 INFO:tasks.workunit.client.0.smithi070.stdout:8/164: symlink d5/d8/d16/d30/l41 0 2024-03-23T08:23:45.674 INFO:tasks.workunit.client.0.smithi070.stdout:5/95: creat dc/d1b/d1d/f1e x:0 0 0 2024-03-23T08:23:45.674 INFO:tasks.workunit.client.0.smithi070.stdout:5/96: stat f1 0 2024-03-23T08:23:45.677 INFO:tasks.workunit.client.0.smithi070.stdout:2/114: dwrite f0 [0,4194304] 0 2024-03-23T08:23:45.677 INFO:tasks.workunit.client.0.smithi070.stdout:3/318: dread db/d20/f45 [4194304,4194304] 0 2024-03-23T08:23:45.679 INFO:tasks.workunit.client.0.smithi070.stdout:2/115: write d1/fe [443396,113996] 0 2024-03-23T08:23:45.680 INFO:tasks.workunit.client.0.smithi070.stdout:2/116: chown d1/d7/f17 710722119 1 2024-03-23T08:23:45.683 INFO:tasks.workunit.client.0.smithi070.stdout:5/97: rename dc/d1b/d1d/f1e to dc/f1f 0 2024-03-23T08:23:45.683 INFO:tasks.workunit.client.0.smithi070.stdout:8/165: symlink d5/d8/d16/d30/d38/l42 0 2024-03-23T08:23:45.686 INFO:tasks.workunit.client.0.smithi070.stdout:3/319: dwrite db/df/f60 [0,4194304] 0 2024-03-23T08:23:45.687 INFO:tasks.workunit.client.0.smithi070.stdout:2/117: mknod d1/d7/c24 0 2024-03-23T08:23:45.690 INFO:tasks.workunit.client.0.smithi070.stdout:5/98: symlink dc/d1b/d1d/l20 0 2024-03-23T08:23:45.692 INFO:tasks.workunit.client.0.smithi070.stdout:8/166: rename d5/d8/d16/d32/f37 to d5/f43 0 2024-03-23T08:23:45.694 INFO:tasks.workunit.client.0.smithi070.stdout:3/320: creat db/d14/d67/f74 x:0 0 0 2024-03-23T08:23:45.695 INFO:tasks.workunit.client.0.smithi070.stdout:3/321: chown db/df/d1a/f37 30178346 1 2024-03-23T08:23:45.696 INFO:tasks.workunit.client.0.smithi070.stdout:2/118: creat d1/d7/d11/f25 x:0 0 0 2024-03-23T08:23:45.698 INFO:tasks.workunit.client.0.smithi070.stdout:9/189: truncate f6 5934469 0 2024-03-23T08:23:45.700 INFO:tasks.workunit.client.0.smithi070.stdout:8/167: rename d5/d8/d16/d30/f27 to d5/d8/d16/f44 0 2024-03-23T08:23:45.703 INFO:tasks.workunit.client.0.smithi070.stdout:5/99: mkdir dc/d19/d21 0 2024-03-23T08:23:45.708 INFO:tasks.workunit.client.0.smithi070.stdout:2/119: rename d1/d18/f1b to d1/d7/f26 0 2024-03-23T08:23:45.709 INFO:tasks.workunit.client.0.smithi070.stdout:8/168: mknod d5/d8/d16/d30/c45 0 2024-03-23T08:23:45.712 INFO:tasks.workunit.client.0.smithi070.stdout:5/100: rename dc/fd to dc/d1b/f22 0 2024-03-23T08:23:45.715 INFO:tasks.workunit.client.0.smithi070.stdout:2/120: creat d1/d7/df/f27 x:0 0 0 2024-03-23T08:23:45.719 INFO:tasks.workunit.client.0.smithi070.stdout:8/169: creat d5/d8/d16/d32/f46 x:0 0 0 2024-03-23T08:23:45.721 INFO:tasks.workunit.client.0.smithi070.stdout:8/170: dread - d5/d8/d16/d30/f39 zero size 2024-03-23T08:23:45.721 INFO:tasks.workunit.client.0.smithi070.stdout:5/101: dwrite f1 [4194304,4194304] 0 2024-03-23T08:23:45.725 INFO:tasks.workunit.client.0.smithi070.stdout:3/322: getdents db/d4e 0 2024-03-23T08:23:45.727 INFO:tasks.workunit.client.0.smithi070.stdout:2/121: write d1/fd [2482233,91566] 0 2024-03-23T08:23:45.729 INFO:tasks.workunit.client.0.smithi070.stdout:2/122: read - d1/d7/f20 zero size 2024-03-23T08:23:45.730 INFO:tasks.workunit.client.0.smithi070.stdout:2/123: readlink d1/la 0 2024-03-23T08:23:45.731 INFO:tasks.workunit.client.0.smithi070.stdout:9/190: dwrite db/d12/d22/d25/f24 [0,4194304] 0 2024-03-23T08:23:45.731 INFO:tasks.workunit.client.0.smithi070.stdout:2/124: readlink d1/d6/l15 0 2024-03-23T08:23:45.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:45 smithi070 ceph-mon[39699]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:45.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:45 smithi070 ceph-mon[39699]: pgmap v32: 65 pgs: 65 active+clean; 1.3 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 32 MiB/s rd, 76 MiB/s wr, 286 op/s 2024-03-23T08:23:45.739 INFO:tasks.workunit.client.0.smithi070.stdout:9/191: dwrite db/d12/d14/f23 [0,4194304] 0 2024-03-23T08:23:45.757 INFO:tasks.workunit.client.0.smithi070.stdout:5/102: rmdir dc/d1b/d1d 39 2024-03-23T08:23:45.758 INFO:tasks.workunit.client.0.smithi070.stdout:5/103: stat dc/cf 0 2024-03-23T08:23:45.759 INFO:tasks.workunit.client.0.smithi070.stdout:8/171: unlink d5/d8/d16/d30/c2d 0 2024-03-23T08:23:45.762 INFO:tasks.workunit.client.0.smithi070.stdout:3/323: rename db/d20/f33 to db/d20/d63/f75 0 2024-03-23T08:23:45.763 INFO:tasks.workunit.client.0.smithi070.stdout:3/324: write db/df/d1a/f3c [4139394,21698] 0 2024-03-23T08:23:45.764 INFO:tasks.workunit.client.0.smithi070.stdout:3/325: dread - db/d4e/f73 zero size 2024-03-23T08:23:45.764 INFO:tasks.workunit.client.0.smithi070.stdout:3/326: chown db/d14/d6f 929 1 2024-03-23T08:23:45.771 INFO:tasks.workunit.client.0.smithi070.stdout:2/125: dwrite d1/d7/f20 [0,4194304] 0 2024-03-23T08:23:45.775 INFO:tasks.workunit.client.0.smithi070.stdout:9/192: mkdir db/d12/d22/d25/d38 0 2024-03-23T08:23:45.782 INFO:tasks.workunit.client.0.smithi070.stdout:5/104: rename dc/d1b/c1c to dc/d1b/c23 0 2024-03-23T08:23:45.783 INFO:tasks.workunit.client.0.smithi070.stdout:2/126: link d1/d7/d11/f12 d1/d7/d11/f28 0 2024-03-23T08:23:45.784 INFO:tasks.workunit.client.0.smithi070.stdout:2/127: stat d1/d7/d11/f28 0 2024-03-23T08:23:45.784 INFO:tasks.workunit.client.0.smithi070.stdout:3/327: link l8 db/df/d4d/d51/l76 0 2024-03-23T08:23:45.786 INFO:tasks.workunit.client.0.smithi070.stdout:3/328: write db/df/d1a/f44 [742994,70935] 0 2024-03-23T08:23:45.786 INFO:tasks.workunit.client.0.smithi070.stdout:9/193: rename db/d12/f1d to db/d12/f39 0 2024-03-23T08:23:45.791 INFO:tasks.workunit.client.0.smithi070.stdout:3/329: fdatasync db/d14/f4b 0 2024-03-23T08:23:45.791 INFO:tasks.workunit.client.0.smithi070.stdout:2/128: mkdir d1/d29 0 2024-03-23T08:23:45.791 INFO:tasks.workunit.client.0.smithi070.stdout:9/194: write db/d12/d22/f2a [203220,51282] 0 2024-03-23T08:23:45.791 INFO:tasks.workunit.client.0.smithi070.stdout:9/195: fdatasync db/f31 0 2024-03-23T08:23:45.791 INFO:tasks.workunit.client.0.smithi070.stdout:2/129: dread - d1/d7/df/f27 zero size 2024-03-23T08:23:45.791 INFO:tasks.workunit.client.0.smithi070.stdout:3/330: creat db/d14/d65/f77 x:0 0 0 2024-03-23T08:23:45.791 INFO:tasks.workunit.client.0.smithi070.stdout:3/331: chown db/d20 56818370 1 2024-03-23T08:23:45.793 INFO:tasks.workunit.client.0.smithi070.stdout:2/130: unlink d1/d7/c23 0 2024-03-23T08:23:45.794 INFO:tasks.workunit.client.0.smithi070.stdout:9/196: dread db/d12/d14/f23 [0,4194304] 0 2024-03-23T08:23:45.794 INFO:tasks.workunit.client.0.smithi070.stdout:2/131: chown d1/d6 215940639 1 2024-03-23T08:23:45.794 INFO:tasks.workunit.client.0.smithi070.stdout:3/332: fdatasync db/df/d15/d3d/f56 0 2024-03-23T08:23:45.795 INFO:tasks.workunit.client.0.smithi070.stdout:2/132: chown d1/d7/df/l13 377117 1 2024-03-23T08:23:45.800 INFO:tasks.workunit.client.0.smithi070.stdout:2/133: dread d1/d7/d11/f28 [0,4194304] 0 2024-03-23T08:23:45.801 INFO:tasks.workunit.client.0.smithi070.stdout:3/333: getdents db/df/d15 0 2024-03-23T08:23:45.802 INFO:tasks.workunit.client.0.smithi070.stdout:3/334: chown db/c6d 5602 1 2024-03-23T08:23:45.802 INFO:tasks.workunit.client.0.smithi070.stdout:9/197: dwrite db/d12/f17 [0,4194304] 0 2024-03-23T08:23:45.803 INFO:tasks.workunit.client.0.smithi070.stdout:9/198: stat db/d12/f17 0 2024-03-23T08:23:45.805 INFO:tasks.workunit.client.0.smithi070.stdout:9/199: write db/d12/f1a [1771830,2592] 0 2024-03-23T08:23:45.807 INFO:tasks.workunit.client.0.smithi070.stdout:3/335: rename db/df/d15/d22/f32 to db/d14/d65/f78 0 2024-03-23T08:23:45.814 INFO:tasks.workunit.client.0.smithi070.stdout:9/200: creat db/d12/f3a x:0 0 0 2024-03-23T08:23:45.815 INFO:tasks.workunit.client.0.smithi070.stdout:9/201: readlink db/d12/l28 0 2024-03-23T08:23:45.816 INFO:tasks.workunit.client.0.smithi070.stdout:3/336: getdents db/df/d5d 0 2024-03-23T08:23:45.819 INFO:tasks.workunit.client.0.smithi070.stdout:3/337: rename db/df/d15/c2d to db/d14/d6f/c79 0 2024-03-23T08:23:45.821 INFO:tasks.workunit.client.0.smithi070.stdout:3/338: write db/d20/f48 [3581743,28930] 0 2024-03-23T08:23:45.824 INFO:tasks.workunit.client.0.smithi070.stdout:9/202: dwrite db/d12/f2b [0,4194304] 0 2024-03-23T08:23:45.824 INFO:tasks.workunit.client.0.smithi070.stdout:3/339: mkdir db/d20/d7a 0 2024-03-23T08:23:45.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:45 smithi193 ceph-mon[48370]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:23:45.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:45 smithi193 ceph-mon[48370]: pgmap v32: 65 pgs: 65 active+clean; 1.3 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 32 MiB/s rd, 76 MiB/s wr, 286 op/s 2024-03-23T08:23:45.829 INFO:tasks.workunit.client.0.smithi070.stdout:3/340: unlink db/df/f60 0 2024-03-23T08:23:45.833 INFO:tasks.workunit.client.0.smithi070.stdout:9/203: mkdir db/d12/d22/d25/d26/d3b 0 2024-03-23T08:23:45.836 INFO:tasks.workunit.client.0.smithi070.stdout:3/341: mknod db/d20/d49/c7b 0 2024-03-23T08:23:45.837 INFO:tasks.workunit.client.0.smithi070.stdout:3/342: write db/d14/d43/f5a [153754,117739] 0 2024-03-23T08:23:45.839 INFO:tasks.workunit.client.0.smithi070.stdout:3/343: write db/df/d15/d3d/f53 [828790,26219] 0 2024-03-23T08:23:45.840 INFO:tasks.workunit.client.0.smithi070.stdout:9/204: creat db/d12/d22/f3c x:0 0 0 2024-03-23T08:23:45.842 INFO:tasks.workunit.client.0.smithi070.stdout:3/344: symlink db/df/d1a/l7c 0 2024-03-23T08:23:45.844 INFO:tasks.workunit.client.0.smithi070.stdout:3/345: mkdir db/d14/d43/d7d 0 2024-03-23T08:23:45.845 INFO:tasks.workunit.client.0.smithi070.stdout:9/205: creat db/d12/f3d x:0 0 0 2024-03-23T08:23:45.847 INFO:tasks.workunit.client.0.smithi070.stdout:9/206: symlink db/l3e 0 2024-03-23T08:23:45.851 INFO:tasks.workunit.client.0.smithi070.stdout:3/346: dwrite db/d14/f1b [0,4194304] 0 2024-03-23T08:23:45.856 INFO:tasks.workunit.client.0.smithi070.stdout:9/207: write db/f31 [3741810,16884] 0 2024-03-23T08:23:45.858 INFO:tasks.workunit.client.0.smithi070.stdout:9/208: chown db/c15 126307 1 2024-03-23T08:23:45.860 INFO:tasks.workunit.client.0.smithi070.stdout:9/209: symlink db/d12/d22/d25/d26/l3f 0 2024-03-23T08:23:45.868 INFO:tasks.workunit.client.0.smithi070.stdout:9/210: write db/f1f [5210299,74748] 0 2024-03-23T08:23:45.876 INFO:tasks.workunit.client.0.smithi070.stdout:9/211: dwrite db/d12/d22/d25/d26/f30 [0,4194304] 0 2024-03-23T08:23:45.883 INFO:tasks.workunit.client.0.smithi070.stdout:9/212: rename db/d12/c36 to db/d12/d34/c40 0 2024-03-23T08:23:45.884 INFO:tasks.workunit.client.0.smithi070.stdout:9/213: write db/f1f [4155231,64430] 0 2024-03-23T08:23:45.885 INFO:tasks.workunit.client.0.smithi070.stdout:9/214: chown db/d12/f1a 7473 1 2024-03-23T08:23:45.886 INFO:tasks.workunit.client.0.smithi070.stdout:3/347: dread db/f17 [4194304,4194304] 0 2024-03-23T08:23:45.889 INFO:tasks.workunit.client.0.smithi070.stdout:3/348: write db/d14/d43/f54 [4570287,124020] 0 2024-03-23T08:23:45.891 INFO:tasks.workunit.client.0.smithi070.stdout:9/215: dread db/f1b [0,4194304] 0 2024-03-23T08:23:45.891 INFO:tasks.workunit.client.0.smithi070.stdout:3/349: mknod db/d20/d63/c7e 0 2024-03-23T08:23:45.900 INFO:tasks.workunit.client.0.smithi070.stdout:3/350: dwrite db/d14/d65/f77 [0,4194304] 0 2024-03-23T08:23:45.900 INFO:tasks.workunit.client.0.smithi070.stdout:9/216: dwrite db/d12/d22/f27 [0,4194304] 0 2024-03-23T08:23:45.908 INFO:tasks.workunit.client.0.smithi070.stdout:3/351: mkdir db/d7f 0 2024-03-23T08:23:45.910 INFO:tasks.workunit.client.0.smithi070.stdout:9/217: dread db/d12/d22/d25/f24 [0,4194304] 0 2024-03-23T08:23:45.913 INFO:tasks.workunit.client.0.smithi070.stdout:3/352: mkdir db/d20/d7a/d80 0 2024-03-23T08:23:45.914 INFO:tasks.workunit.client.0.smithi070.stdout:9/218: unlink db/d12/f3d 0 2024-03-23T08:23:45.915 INFO:tasks.workunit.client.0.smithi070.stdout:3/353: mknod db/df/d15/c81 0 2024-03-23T08:23:45.917 INFO:tasks.workunit.client.0.smithi070.stdout:9/219: dread - db/f1e zero size 2024-03-23T08:23:45.919 INFO:tasks.workunit.client.0.smithi070.stdout:9/220: fdatasync db/f1b 0 2024-03-23T08:23:45.921 INFO:tasks.workunit.client.0.smithi070.stdout:3/354: getdents db/d20/d49 0 2024-03-23T08:23:45.923 INFO:tasks.workunit.client.0.smithi070.stdout:3/355: chown db/d20/d63/c7e 15416963 1 2024-03-23T08:23:45.925 INFO:tasks.workunit.client.0.smithi070.stdout:3/356: chown db/df/f55 128068109 1 2024-03-23T08:23:45.928 INFO:tasks.workunit.client.0.smithi070.stdout:9/221: dwrite db/d12/d22/d25/d26/f30 [0,4194304] 0 2024-03-23T08:23:45.933 INFO:tasks.workunit.client.0.smithi070.stdout:3/357: dwrite db/df/d1a/f58 [0,4194304] 0 2024-03-23T08:23:45.938 INFO:tasks.workunit.client.0.smithi070.stdout:9/222: rename db/d12/d22/d25/d38 to db/d12/d22/d41 0 2024-03-23T08:23:45.942 INFO:tasks.workunit.client.0.smithi070.stdout:3/358: dwrite db/df/d1a/f50 [4194304,4194304] 0 2024-03-23T08:23:45.948 INFO:tasks.workunit.client.0.smithi070.stdout:9/223: dwrite db/d12/d22/f2a [0,4194304] 0 2024-03-23T08:23:45.951 INFO:tasks.workunit.client.0.smithi070.stdout:3/359: dread db/d20/f45 [4194304,4194304] 0 2024-03-23T08:23:45.952 INFO:tasks.workunit.client.0.smithi070.stdout:9/224: mkdir db/d12/d22/d25/d26/d3b/d42 0 2024-03-23T08:23:45.952 INFO:tasks.workunit.client.0.smithi070.stdout:3/360: mknod db/d7f/c82 0 2024-03-23T08:23:45.954 INFO:tasks.workunit.client.0.smithi070.stdout:9/225: fsync db/d12/d22/d25/d26/f30 0 2024-03-23T08:23:45.955 INFO:tasks.workunit.client.0.smithi070.stdout:3/361: getdents db/d20/d7a/d80 0 2024-03-23T08:23:45.955 INFO:tasks.workunit.client.0.smithi070.stdout:9/226: fsync db/d12/f3a 0 2024-03-23T08:23:45.996 INFO:tasks.workunit.client.0.smithi070.stdout:4/71: sync 2024-03-23T08:23:45.996 INFO:tasks.workunit.client.0.smithi070.stdout:3/362: rmdir db/d14/d6f 39 2024-03-23T08:23:45.998 INFO:tasks.workunit.client.0.smithi070.stdout:9/227: write db/d12/f39 [243320,23079] 0 2024-03-23T08:23:46.001 INFO:tasks.workunit.client.0.smithi070.stdout:3/363: mkdir db/d20/d63/d83 0 2024-03-23T08:23:46.003 INFO:tasks.workunit.client.0.smithi070.stdout:9/228: fdatasync f8 0 2024-03-23T08:23:46.004 INFO:tasks.workunit.client.0.smithi070.stdout:4/72: dwrite d3/d6/f8 [0,4194304] 0 2024-03-23T08:23:46.008 INFO:tasks.workunit.client.0.smithi070.stdout:3/364: dread db/d20/f48 [0,4194304] 0 2024-03-23T08:23:46.009 INFO:tasks.workunit.client.0.smithi070.stdout:4/73: dread d3/fc [0,4194304] 0 2024-03-23T08:23:46.010 INFO:tasks.workunit.client.0.smithi070.stdout:4/74: truncate d3/d6/f8 4614853 0 2024-03-23T08:23:46.011 INFO:tasks.workunit.client.0.smithi070.stdout:4/75: chown d3/d6 532031 1 2024-03-23T08:23:46.013 INFO:tasks.workunit.client.0.smithi070.stdout:3/365: mknod db/df/d4d/c84 0 2024-03-23T08:23:46.017 INFO:tasks.workunit.client.0.smithi070.stdout:9/229: creat db/d12/d22/d25/d26/d32/f43 x:0 0 0 2024-03-23T08:23:46.020 INFO:tasks.workunit.client.0.smithi070.stdout:3/366: dwrite db/d14/d43/f5a [0,4194304] 0 2024-03-23T08:23:46.021 INFO:tasks.workunit.client.0.smithi070.stdout:3/367: stat db/df/d6a/c72 0 2024-03-23T08:23:46.025 INFO:tasks.workunit.client.0.smithi070.stdout:9/230: rename db/d12/d22/f3c to db/d12/d22/d25/f44 0 2024-03-23T08:23:46.026 INFO:tasks.workunit.client.0.smithi070.stdout:3/368: fsync db/d14/d65/f78 0 2024-03-23T08:23:46.030 INFO:tasks.workunit.client.0.smithi070.stdout:9/231: rmdir db/d12/d14 39 2024-03-23T08:23:46.032 INFO:tasks.workunit.client.0.smithi070.stdout:3/369: rename db/d20/f48 to db/d14/d65/f85 0 2024-03-23T08:23:46.032 INFO:tasks.workunit.client.0.smithi070.stdout:3/370: stat db/d14/d67 0 2024-03-23T08:23:46.033 INFO:tasks.workunit.client.0.smithi070.stdout:3/371: readlink db/d14/d43/d62/l70 0 2024-03-23T08:23:46.034 INFO:tasks.workunit.client.0.smithi070.stdout:9/232: fsync db/d12/d22/d25/d26/d32/f43 0 2024-03-23T08:23:46.034 INFO:tasks.workunit.client.0.smithi070.stdout:4/76: write d3/fd [224632,87732] 0 2024-03-23T08:23:46.036 INFO:tasks.workunit.client.0.smithi070.stdout:4/77: readlink d3/d6/d9/l15 0 2024-03-23T08:23:46.036 INFO:tasks.workunit.client.0.smithi070.stdout:4/78: chown f2 7090792 1 2024-03-23T08:23:46.038 INFO:tasks.workunit.client.0.smithi070.stdout:9/233: unlink db/f1e 0 2024-03-23T08:23:46.041 INFO:tasks.workunit.client.0.smithi070.stdout:3/372: mknod db/d14/d43/d7d/c86 0 2024-03-23T08:23:46.043 INFO:tasks.workunit.client.0.smithi070.stdout:4/79: rename d3/d5/ce to d3/d5/c16 0 2024-03-23T08:23:46.045 INFO:tasks.workunit.client.0.smithi070.stdout:3/373: mknod db/d7f/c87 0 2024-03-23T08:23:46.046 INFO:tasks.workunit.client.0.smithi070.stdout:4/80: creat d3/f17 x:0 0 0 2024-03-23T08:23:46.048 INFO:tasks.workunit.client.0.smithi070.stdout:3/374: symlink db/d20/l88 0 2024-03-23T08:23:46.050 INFO:tasks.workunit.client.0.smithi070.stdout:4/81: creat d3/d6/d11/f18 x:0 0 0 2024-03-23T08:23:46.050 INFO:tasks.workunit.client.0.smithi070.stdout:4/82: stat d3/d6/d11/f13 0 2024-03-23T08:23:46.051 INFO:tasks.workunit.client.0.smithi070.stdout:3/375: dread db/df/f55 [0,4194304] 0 2024-03-23T08:23:46.052 INFO:tasks.workunit.client.0.smithi070.stdout:3/376: chown db/d20/d7a/d80 171 1 2024-03-23T08:23:46.054 INFO:tasks.workunit.client.0.smithi070.stdout:9/234: rename db/f21 to db/d12/d22/d25/f45 0 2024-03-23T08:23:46.058 INFO:tasks.workunit.client.0.smithi070.stdout:4/83: rename d3/d5 to d3/d19 0 2024-03-23T08:23:46.059 INFO:tasks.workunit.client.0.smithi070.stdout:3/377: dwrite db/d14/d43/d62/f69 [4194304,4194304] 0 2024-03-23T08:23:46.062 INFO:tasks.workunit.client.0.smithi070.stdout:9/235: chown db/f1b 2573613 1 2024-03-23T08:23:46.063 INFO:tasks.workunit.client.0.smithi070.stdout:4/84: mkdir d3/d6/d11/d1a 0 2024-03-23T08:23:46.064 INFO:tasks.workunit.client.0.smithi070.stdout:3/378: mknod db/d20/c89 0 2024-03-23T08:23:46.066 INFO:tasks.workunit.client.0.smithi070.stdout:9/236: unlink l1 0 2024-03-23T08:23:46.068 INFO:tasks.workunit.client.0.smithi070.stdout:9/237: dread f8 [0,4194304] 0 2024-03-23T08:23:46.071 INFO:tasks.workunit.client.0.smithi070.stdout:3/379: symlink db/df/d4d/l8a 0 2024-03-23T08:23:46.076 INFO:tasks.workunit.client.0.smithi070.stdout:3/380: chown db/d14/c5e 3 1 2024-03-23T08:23:46.078 INFO:tasks.workunit.client.0.smithi070.stdout:9/238: creat db/d12/d22/d25/d26/d3b/d42/f46 x:0 0 0 2024-03-23T08:23:46.080 INFO:tasks.workunit.client.0.smithi070.stdout:3/381: symlink db/d20/d7a/l8b 0 2024-03-23T08:23:46.085 INFO:tasks.workunit.client.0.smithi070.stdout:4/85: dwrite f1 [0,4194304] 0 2024-03-23T08:23:46.087 INFO:tasks.workunit.client.0.smithi070.stdout:9/239: dread db/d12/d22/f27 [0,4194304] 0 2024-03-23T08:23:46.089 INFO:tasks.workunit.client.0.smithi070.stdout:4/86: write d3/d6/f7 [3046280,106290] 0 2024-03-23T08:23:46.091 INFO:tasks.workunit.client.0.smithi070.stdout:3/382: dwrite db/d14/f1b [0,4194304] 0 2024-03-23T08:23:46.091 INFO:tasks.workunit.client.0.smithi070.stdout:9/240: mkdir db/d12/d34/d47 0 2024-03-23T08:23:46.092 INFO:tasks.workunit.client.0.smithi070.stdout:4/87: write d3/d6/d11/f18 [57655,7987] 0 2024-03-23T08:23:46.101 INFO:tasks.workunit.client.0.smithi070.stdout:4/88: write d3/fc [5063460,74473] 0 2024-03-23T08:23:46.105 INFO:tasks.workunit.client.0.smithi070.stdout:9/241: truncate db/d12/d14/f23 1876029 0 2024-03-23T08:23:46.106 INFO:tasks.workunit.client.0.smithi070.stdout:4/89: mkdir d3/d6/d11/d1a/d1b 0 2024-03-23T08:23:46.106 INFO:tasks.workunit.client.0.smithi070.stdout:4/90: chown f1 2305 1 2024-03-23T08:23:46.106 INFO:tasks.workunit.client.0.smithi070.stdout:9/242: fsync db/d12/d22/d25/f44 0 2024-03-23T08:23:46.109 INFO:tasks.workunit.client.0.smithi070.stdout:9/243: symlink db/d12/d14/l48 0 2024-03-23T08:23:46.110 INFO:tasks.workunit.client.0.smithi070.stdout:4/91: link f1 d3/d6/d11/d1a/d1b/f1c 0 2024-03-23T08:23:46.111 INFO:tasks.workunit.client.0.smithi070.stdout:9/244: creat db/d12/d14/f49 x:0 0 0 2024-03-23T08:23:46.112 INFO:tasks.workunit.client.0.smithi070.stdout:4/92: rename d3/l12 to d3/d6/d11/d1a/d1b/l1d 0 2024-03-23T08:23:46.113 INFO:tasks.workunit.client.0.smithi070.stdout:4/93: chown d3/d6 4827 1 2024-03-23T08:23:46.113 INFO:tasks.workunit.client.0.smithi070.stdout:4/94: chown d3/d19 7397 1 2024-03-23T08:23:46.115 INFO:tasks.workunit.client.0.smithi070.stdout:9/245: rename db/d12/l28 to db/d12/d22/d25/d26/d3b/l4a 0 2024-03-23T08:23:46.205 INFO:tasks.workunit.client.0.smithi070.stdout:9/246: symlink db/d12/d22/d25/d26/d32/l4b 0 2024-03-23T08:23:46.206 INFO:tasks.workunit.client.0.smithi070.stdout:9/247: stat db/d12/d22/f27 0 2024-03-23T08:23:46.207 INFO:tasks.workunit.client.0.smithi070.stdout:0/314: sync 2024-03-23T08:23:46.207 INFO:tasks.workunit.client.0.smithi070.stdout:6/309: sync 2024-03-23T08:23:46.208 INFO:tasks.workunit.client.0.smithi070.stdout:4/95: link d3/d19/c16 d3/d6/d11/c1e 0 2024-03-23T08:23:46.208 INFO:tasks.workunit.client.0.smithi070.stdout:4/96: chown f2 384031 1 2024-03-23T08:23:46.209 INFO:tasks.workunit.client.0.smithi070.stdout:9/248: rmdir db/d12/d22/d25 39 2024-03-23T08:23:46.209 INFO:tasks.workunit.client.0.smithi070.stdout:4/97: readlink d3/d6/d9/l15 0 2024-03-23T08:23:46.210 INFO:tasks.workunit.client.0.smithi070.stdout:4/98: write d3/d6/f7 [2248425,105951] 0 2024-03-23T08:23:46.212 INFO:tasks.workunit.client.0.smithi070.stdout:0/315: mkdir d16/d26/d27/d3d/d6d 0 2024-03-23T08:23:46.212 INFO:tasks.workunit.client.0.smithi070.stdout:0/316: chown l0 73 1 2024-03-23T08:23:46.214 INFO:tasks.workunit.client.0.smithi070.stdout:0/317: read d16/d26/d27/d3d/f64 [1128960,54370] 0 2024-03-23T08:23:46.218 INFO:tasks.workunit.client.0.smithi070.stdout:4/99: symlink d3/d6/d11/d1a/d1b/l1f 0 2024-03-23T08:23:46.219 INFO:tasks.workunit.client.0.smithi070.stdout:4/100: dread - d3/d6/d11/f13 zero size 2024-03-23T08:23:46.221 INFO:tasks.workunit.client.0.smithi070.stdout:0/318: dwrite d16/d22/d43/f56 [0,4194304] 0 2024-03-23T08:23:46.222 INFO:tasks.workunit.client.0.smithi070.stdout:0/319: chown d16/f68 35388057 1 2024-03-23T08:23:46.223 INFO:tasks.workunit.client.0.smithi070.stdout:4/101: rmdir d3/d6/d9 39 2024-03-23T08:23:46.224 INFO:tasks.workunit.client.0.smithi070.stdout:0/320: mknod d16/d26/d27/c6e 0 2024-03-23T08:23:46.230 INFO:tasks.workunit.client.0.smithi070.stdout:0/321: rename d16/d22/d43/f52 to d16/f6f 0 2024-03-23T08:23:46.231 INFO:tasks.workunit.client.0.smithi070.stdout:6/310: truncate d0/d23/d28/d35/d3a/d3c/f51 962359 0 2024-03-23T08:23:46.233 INFO:tasks.workunit.client.0.smithi070.stdout:4/102: write f2 [52467,85217] 0 2024-03-23T08:23:46.237 INFO:tasks.workunit.client.0.smithi070.stdout:0/322: link d16/d26/d27/f35 d16/d22/d43/d47/f70 0 2024-03-23T08:23:46.239 INFO:tasks.workunit.client.0.smithi070.stdout:4/103: unlink d3/d19/c16 0 2024-03-23T08:23:46.240 INFO:tasks.workunit.client.0.smithi070.stdout:4/104: dread - d3/d6/d11/f13 zero size 2024-03-23T08:23:46.246 INFO:tasks.workunit.client.0.smithi070.stdout:6/311: dwrite d0/d23/d1c/d2f/f39 [0,4194304] 0 2024-03-23T08:23:46.248 INFO:tasks.workunit.client.0.smithi070.stdout:4/105: getdents d3 0 2024-03-23T08:23:46.252 INFO:tasks.workunit.client.0.smithi070.stdout:4/106: creat d3/f20 x:0 0 0 2024-03-23T08:23:46.252 INFO:tasks.workunit.client.0.smithi070.stdout:0/323: truncate f13 3609411 0 2024-03-23T08:23:46.254 INFO:tasks.workunit.client.0.smithi070.stdout:6/312: truncate d0/d23/d1c/d2f/f4c 324995 0 2024-03-23T08:23:46.270 INFO:tasks.workunit.client.0.smithi070.stdout:0/324: write d16/d22/f23 [1403507,23643] 0 2024-03-23T08:23:46.279 INFO:tasks.workunit.client.0.smithi070.stdout:0/325: dwrite d16/d26/d27/d3d/d4d/f66 [0,4194304] 0 2024-03-23T08:23:46.281 INFO:tasks.workunit.client.0.smithi070.stdout:6/313: dread d0/d3/fa [0,4194304] 0 2024-03-23T08:23:46.284 INFO:tasks.workunit.client.0.smithi070.stdout:0/326: write d16/f1a [4215541,5495] 0 2024-03-23T08:23:46.286 INFO:tasks.workunit.client.0.smithi070.stdout:4/107: link d3/d6/d11/c1e d3/d6/d9/c21 0 2024-03-23T08:23:46.289 INFO:tasks.workunit.client.0.smithi070.stdout:4/108: creat d3/d6/d9/f22 x:0 0 0 2024-03-23T08:23:46.291 INFO:tasks.workunit.client.0.smithi070.stdout:6/314: link d0/d3/l22 d0/d23/d28/d35/d3a/d3c/l69 0 2024-03-23T08:23:46.292 INFO:tasks.workunit.client.0.smithi070.stdout:6/315: chown d0/d23/d28/d35/c4f 5697 1 2024-03-23T08:23:46.293 INFO:tasks.workunit.client.0.smithi070.stdout:4/109: creat d3/f23 x:0 0 0 2024-03-23T08:23:46.293 INFO:tasks.workunit.client.0.smithi070.stdout:0/327: link d16/l2d d16/d26/d27/d3d/d6d/l71 0 2024-03-23T08:23:46.296 INFO:tasks.workunit.client.0.smithi070.stdout:4/110: symlink d3/l24 0 2024-03-23T08:23:46.297 INFO:tasks.workunit.client.0.smithi070.stdout:0/328: creat d16/d26/d27/d3d/f72 x:0 0 0 2024-03-23T08:23:46.298 INFO:tasks.workunit.client.0.smithi070.stdout:0/329: write d16/d22/d43/f62 [62326,115533] 0 2024-03-23T08:23:46.301 INFO:tasks.workunit.client.0.smithi070.stdout:4/111: creat d3/f25 x:0 0 0 2024-03-23T08:23:46.303 INFO:tasks.workunit.client.0.smithi070.stdout:0/330: fsync d16/f1d 0 2024-03-23T08:23:46.305 INFO:tasks.workunit.client.0.smithi070.stdout:0/331: mknod d16/d26/d27/d3d/d4d/c73 0 2024-03-23T08:23:46.311 INFO:tasks.workunit.client.0.smithi070.stdout:6/316: dwrite d0/d23/d1c/f2b [0,4194304] 0 2024-03-23T08:23:46.312 INFO:tasks.workunit.client.0.smithi070.stdout:0/332: link d16/c1b d16/d26/d4a/c74 0 2024-03-23T08:23:46.314 INFO:tasks.workunit.client.0.smithi070.stdout:6/317: mkdir d0/d23/d28/d35/d6a 0 2024-03-23T08:23:46.318 INFO:tasks.workunit.client.0.smithi070.stdout:0/333: mkdir d16/d26/d27/d4c/d60/d75 0 2024-03-23T08:23:46.322 INFO:tasks.workunit.client.0.smithi070.stdout:6/318: creat d0/d23/d1c/d38/d52/f6b x:0 0 0 2024-03-23T08:23:46.323 INFO:tasks.workunit.client.0.smithi070.stdout:0/334: creat d16/d26/d27/d4c/d60/d75/f76 x:0 0 0 2024-03-23T08:23:46.324 INFO:tasks.workunit.client.0.smithi070.stdout:6/319: write d0/d23/d1c/d2f/f39 [1664509,88834] 0 2024-03-23T08:23:46.325 INFO:tasks.workunit.client.0.smithi070.stdout:6/320: write d0/d23/d4d/f63 [314523,7490] 0 2024-03-23T08:23:46.326 INFO:tasks.workunit.client.0.smithi070.stdout:0/335: symlink d16/d26/d27/l77 0 2024-03-23T08:23:46.329 INFO:tasks.workunit.client.0.smithi070.stdout:0/336: creat d16/d22/d43/f78 x:0 0 0 2024-03-23T08:23:46.334 INFO:tasks.workunit.client.0.smithi070.stdout:0/337: unlink fb 0 2024-03-23T08:23:46.336 INFO:tasks.workunit.client.0.smithi070.stdout:0/338: creat d16/d26/d27/d4c/d60/f79 x:0 0 0 2024-03-23T08:23:46.340 INFO:tasks.workunit.client.0.smithi070.stdout:6/321: write d0/d3/f45 [396599,80562] 0 2024-03-23T08:23:46.345 INFO:tasks.workunit.client.0.smithi070.stdout:6/322: dread d0/d23/d1c/d2f/f30 [0,4194304] 0 2024-03-23T08:23:46.349 INFO:tasks.workunit.client.0.smithi070.stdout:6/323: dread d0/d23/d28/f37 [0,4194304] 0 2024-03-23T08:23:46.350 INFO:tasks.workunit.client.0.smithi070.stdout:6/324: dread - d0/d23/f55 zero size 2024-03-23T08:23:46.350 INFO:tasks.workunit.client.0.smithi070.stdout:0/339: dread d16/d22/f2f [0,4194304] 0 2024-03-23T08:23:46.411 INFO:tasks.workunit.client.0.smithi070.stdout:1/249: sync 2024-03-23T08:23:46.411 INFO:tasks.workunit.client.0.smithi070.stdout:7/110: sync 2024-03-23T08:23:46.453 INFO:tasks.workunit.client.0.smithi070.stdout:0/340: symlink d16/d22/l7a 0 2024-03-23T08:23:46.455 INFO:tasks.workunit.client.0.smithi070.stdout:6/325: write d0/d23/d28/d35/d3a/d3c/d5b/f5e [914952,36612] 0 2024-03-23T08:23:46.455 INFO:tasks.workunit.client.0.smithi070.stdout:7/111: symlink d25/l2a 0 2024-03-23T08:23:46.456 INFO:tasks.workunit.client.0.smithi070.stdout:7/112: truncate f1d 224596 0 2024-03-23T08:23:46.457 INFO:tasks.workunit.client.0.smithi070.stdout:0/341: readlink lc 0 2024-03-23T08:23:46.459 INFO:tasks.workunit.client.0.smithi070.stdout:1/250: truncate d8/db/f1f 958100 0 2024-03-23T08:23:46.461 INFO:tasks.workunit.client.0.smithi070.stdout:0/342: rename l10 to d16/d5f/l7b 0 2024-03-23T08:23:46.462 INFO:tasks.workunit.client.0.smithi070.stdout:1/251: creat d8/d11/f5c x:0 0 0 2024-03-23T08:23:46.462 INFO:tasks.workunit.client.0.smithi070.stdout:0/343: write d16/f5b [288967,35863] 0 2024-03-23T08:23:46.463 INFO:tasks.workunit.client.0.smithi070.stdout:6/326: dwrite d0/d23/d1c/d2f/f39 [0,4194304] 0 2024-03-23T08:23:46.464 INFO:tasks.workunit.client.0.smithi070.stdout:0/344: fdatasync d16/d22/f39 0 2024-03-23T08:23:46.471 INFO:tasks.workunit.client.0.smithi070.stdout:0/345: mkdir d16/d26/d27/d4c/d60/d7c 0 2024-03-23T08:23:46.472 INFO:tasks.workunit.client.0.smithi070.stdout:0/346: fsync d16/d22/f39 0 2024-03-23T08:23:46.474 INFO:tasks.workunit.client.0.smithi070.stdout:1/252: mkdir d8/db/d40/d43/d44/d5d 0 2024-03-23T08:23:46.477 INFO:tasks.workunit.client.0.smithi070.stdout:0/347: mkdir d16/d26/d4a/d67/d7d 0 2024-03-23T08:23:46.478 INFO:tasks.workunit.client.0.smithi070.stdout:1/253: creat d8/d11/f5e x:0 0 0 2024-03-23T08:23:46.481 INFO:tasks.workunit.client.0.smithi070.stdout:1/254: creat d8/db/d2c/d3f/f5f x:0 0 0 2024-03-23T08:23:46.483 INFO:tasks.workunit.client.0.smithi070.stdout:0/348: link d16/d22/f2f d16/d26/d27/d4c/d60/f7e 0 2024-03-23T08:23:46.485 INFO:tasks.workunit.client.0.smithi070.stdout:1/255: symlink d8/d42/l60 0 2024-03-23T08:23:46.489 INFO:tasks.workunit.client.0.smithi070.stdout:1/256: mkdir d8/d11/d61 0 2024-03-23T08:23:46.491 INFO:tasks.workunit.client.0.smithi070.stdout:0/349: rename d16/c1b to d16/d26/d27/d4c/d60/c7f 0 2024-03-23T08:23:46.492 INFO:tasks.workunit.client.0.smithi070.stdout:0/350: write d16/d26/d27/d4c/d60/f79 [279587,9549] 0 2024-03-23T08:23:46.494 INFO:tasks.workunit.client.0.smithi070.stdout:0/351: mknod d16/d26/d27/d4c/d60/d75/c80 0 2024-03-23T08:23:46.498 INFO:tasks.workunit.client.0.smithi070.stdout:0/352: dread d16/d22/f2c [0,4194304] 0 2024-03-23T08:23:46.500 INFO:tasks.workunit.client.0.smithi070.stdout:0/353: creat d16/d22/d43/d47/f81 x:0 0 0 2024-03-23T08:23:46.506 INFO:tasks.workunit.client.0.smithi070.stdout:0/354: fsync f13 0 2024-03-23T08:23:46.508 INFO:tasks.workunit.client.0.smithi070.stdout:6/327: write d0/d3/fa [2583071,49284] 0 2024-03-23T08:23:46.512 INFO:tasks.workunit.client.0.smithi070.stdout:0/355: rename d16/l28 to d16/d22/d43/d47/l82 0 2024-03-23T08:23:46.513 INFO:tasks.workunit.client.0.smithi070.stdout:6/328: symlink d0/d23/l6c 0 2024-03-23T08:23:46.516 INFO:tasks.workunit.client.0.smithi070.stdout:6/329: dread d0/d3/f45 [0,4194304] 0 2024-03-23T08:23:46.517 INFO:tasks.workunit.client.0.smithi070.stdout:0/356: creat d16/d26/d27/d4c/d60/d7c/f83 x:0 0 0 2024-03-23T08:23:46.519 INFO:tasks.workunit.client.0.smithi070.stdout:0/357: fdatasync d16/d22/f2c 0 2024-03-23T08:23:46.521 INFO:tasks.workunit.client.0.smithi070.stdout:6/330: symlink d0/d23/d28/d35/d6a/l6d 0 2024-03-23T08:23:46.522 INFO:tasks.workunit.client.0.smithi070.stdout:0/358: readlink d16/d5f/l6c 0 2024-03-23T08:23:46.529 INFO:tasks.workunit.client.0.smithi070.stdout:0/359: dwrite d16/f46 [0,4194304] 0 2024-03-23T08:23:46.529 INFO:tasks.workunit.client.0.smithi070.stdout:6/331: truncate d0/d23/d1c/f5d 39367 0 2024-03-23T08:23:46.530 INFO:tasks.workunit.client.0.smithi070.stdout:0/360: readlink d16/d22/d43/d47/l65 0 2024-03-23T08:23:46.531 INFO:tasks.workunit.client.0.smithi070.stdout:0/361: fsync d16/d22/f23 0 2024-03-23T08:23:46.536 INFO:tasks.workunit.client.0.smithi070.stdout:6/332: creat d0/d23/d4d/f6e x:0 0 0 2024-03-23T08:23:46.543 INFO:tasks.workunit.client.0.smithi070.stdout:0/362: unlink d16/l2e 0 2024-03-23T08:23:46.546 INFO:tasks.workunit.client.0.smithi070.stdout:6/333: chown d0/d23/d28/d35/d3a/d3c/l69 437841732 1 2024-03-23T08:23:46.546 INFO:tasks.workunit.client.0.smithi070.stdout:0/363: write d16/d26/d27/d4c/d60/d7c/f83 [745564,40329] 0 2024-03-23T08:23:46.554 INFO:tasks.workunit.client.0.smithi070.stdout:0/364: symlink d16/d22/d43/l84 0 2024-03-23T08:23:46.555 INFO:tasks.workunit.client.0.smithi070.stdout:0/365: chown f11 14646 1 2024-03-23T08:23:46.557 INFO:tasks.workunit.client.0.smithi070.stdout:0/366: fsync d16/f5b 0 2024-03-23T08:23:46.564 INFO:tasks.workunit.client.0.smithi070.stdout:6/334: dwrite d0/d23/d28/d35/f46 [0,4194304] 0 2024-03-23T08:23:46.570 INFO:tasks.workunit.client.0.smithi070.stdout:0/367: mknod d16/d26/d27/d4c/d60/d75/c85 0 2024-03-23T08:23:46.577 INFO:tasks.workunit.client.0.smithi070.stdout:6/335: link d0/c2 d0/d23/d4d/c6f 0 2024-03-23T08:23:46.578 INFO:tasks.workunit.client.0.smithi070.stdout:0/368: dread d16/d22/f24 [0,4194304] 0 2024-03-23T08:23:46.578 INFO:tasks.workunit.client.0.smithi070.stdout:6/336: readlink d0/d23/d28/l29 0 2024-03-23T08:23:46.579 INFO:tasks.workunit.client.0.smithi070.stdout:0/369: readlink d16/d5f/l6c 0 2024-03-23T08:23:46.581 INFO:tasks.workunit.client.0.smithi070.stdout:6/337: dread d0/d3/f45 [0,4194304] 0 2024-03-23T08:23:46.598 INFO:tasks.workunit.client.0.smithi070.stdout:0/370: dwrite d16/d26/d27/d4c/d60/f79 [0,4194304] 0 2024-03-23T08:23:46.603 INFO:tasks.workunit.client.0.smithi070.stdout:0/371: rename d16/d26/d4a to d16/d26/d27/d86 0 2024-03-23T08:23:46.605 INFO:tasks.workunit.client.0.smithi070.stdout:6/338: fsync d0/d23/d1c/d2f/f4c 0 2024-03-23T08:23:46.613 INFO:tasks.workunit.client.0.smithi070.stdout:6/339: rename d0/d23/d4d/c6f to d0/d23/d1c/c70 0 2024-03-23T08:23:46.619 INFO:tasks.workunit.client.0.smithi070.stdout:0/372: dwrite d16/f1d [0,4194304] 0 2024-03-23T08:23:46.623 INFO:tasks.workunit.client.0.smithi070.stdout:0/373: mkdir d16/d22/d43/d47/d87 0 2024-03-23T08:23:46.624 INFO:tasks.workunit.client.0.smithi070.stdout:6/340: dread d0/d3/f4 [0,4194304] 0 2024-03-23T08:23:46.627 INFO:tasks.workunit.client.0.smithi070.stdout:6/341: mknod d0/d3/c71 0 2024-03-23T08:23:46.628 INFO:tasks.workunit.client.0.smithi070.stdout:0/374: write d16/d26/d27/f35 [2380253,60355] 0 2024-03-23T08:23:46.633 INFO:tasks.workunit.client.0.smithi070.stdout:0/375: dread d16/d26/d27/d4c/d60/f7e [0,4194304] 0 2024-03-23T08:23:46.634 INFO:tasks.workunit.client.0.smithi070.stdout:0/376: write d16/f5b [63872,106485] 0 2024-03-23T08:23:46.635 INFO:tasks.workunit.client.0.smithi070.stdout:0/377: write d16/d26/d27/d4c/d60/f79 [1438979,109082] 0 2024-03-23T08:23:46.824 INFO:tasks.workunit.client.0.smithi070.stdout:5/105: sync 2024-03-23T08:23:46.824 INFO:tasks.workunit.client.0.smithi070.stdout:2/134: sync 2024-03-23T08:23:46.824 INFO:tasks.workunit.client.0.smithi070.stdout:8/172: sync 2024-03-23T08:23:46.824 INFO:tasks.workunit.client.0.smithi070.stdout:3/383: sync 2024-03-23T08:23:46.826 INFO:tasks.workunit.client.0.smithi070.stdout:8/173: truncate d5/d8/d16/f24 782985 0 2024-03-23T08:23:46.826 INFO:tasks.workunit.client.0.smithi070.stdout:2/135: dread d1/d7/d11/f28 [0,4194304] 0 2024-03-23T08:23:46.827 INFO:tasks.workunit.client.0.smithi070.stdout:5/106: symlink dc/l24 0 2024-03-23T08:23:46.828 INFO:tasks.workunit.client.0.smithi070.stdout:2/136: fsync d1/fe 0 2024-03-23T08:23:46.829 INFO:tasks.workunit.client.0.smithi070.stdout:3/384: dread db/d14/d43/f5a [0,4194304] 0 2024-03-23T08:23:46.831 INFO:tasks.workunit.client.0.smithi070.stdout:5/107: unlink l3 0 2024-03-23T08:23:46.834 INFO:tasks.workunit.client.0.smithi070.stdout:8/174: fdatasync d5/d8/f19 0 2024-03-23T08:23:46.834 INFO:tasks.workunit.client.0.smithi070.stdout:5/108: truncate dc/f18 816492 0 2024-03-23T08:23:46.834 INFO:tasks.workunit.client.0.smithi070.stdout:3/385: symlink db/d20/d63/l8c 0 2024-03-23T08:23:46.836 INFO:tasks.workunit.client.0.smithi070.stdout:5/109: chown dc/f1f 0 1 2024-03-23T08:23:46.836 INFO:tasks.workunit.client.0.smithi070.stdout:2/137: dwrite d1/d7/d11/f25 [0,4194304] 0 2024-03-23T08:23:46.847 INFO:tasks.workunit.client.0.smithi070.stdout:2/138: mknod d1/d7/df/c2a 0 2024-03-23T08:23:46.848 INFO:tasks.workunit.client.0.smithi070.stdout:2/139: stat d1/d7/d11 0 2024-03-23T08:23:46.851 INFO:tasks.workunit.client.0.smithi070.stdout:5/110: mkdir dc/d1b/d1d/d25 0 2024-03-23T08:23:46.856 INFO:tasks.workunit.client.0.smithi070.stdout:3/386: getdents db/d20 0 2024-03-23T08:23:46.860 INFO:tasks.workunit.client.0.smithi070.stdout:5/111: dwrite dc/d1b/f22 [0,4194304] 0 2024-03-23T08:23:46.862 INFO:tasks.workunit.client.0.smithi070.stdout:8/175: truncate d5/f43 124374 0 2024-03-23T08:23:46.862 INFO:tasks.workunit.client.0.smithi070.stdout:5/112: stat dc/c13 0 2024-03-23T08:23:46.865 INFO:tasks.workunit.client.0.smithi070.stdout:2/140: write d1/f16 [4276502,52871] 0 2024-03-23T08:23:46.866 INFO:tasks.workunit.client.0.smithi070.stdout:8/176: mknod d5/d8/d16/d32/c47 0 2024-03-23T08:23:46.867 INFO:tasks.workunit.client.0.smithi070.stdout:5/113: rename dc/l11 to dc/l26 0 2024-03-23T08:23:46.870 INFO:tasks.workunit.client.0.smithi070.stdout:3/387: getdents db/d20/d7a 0 2024-03-23T08:23:46.873 INFO:tasks.workunit.client.0.smithi070.stdout:8/177: creat d5/d8/d16/d3b/f48 x:0 0 0 2024-03-23T08:23:46.875 INFO:tasks.workunit.client.0.smithi070.stdout:5/114: creat dc/d19/d21/f27 x:0 0 0 2024-03-23T08:23:46.879 INFO:tasks.workunit.client.0.smithi070.stdout:2/141: dwrite d1/d7/f26 [0,4194304] 0 2024-03-23T08:23:46.880 INFO:tasks.workunit.client.0.smithi070.stdout:3/388: dread db/df/d1a/f58 [0,4194304] 0 2024-03-23T08:23:46.880 INFO:tasks.workunit.client.0.smithi070.stdout:8/178: mknod d5/c49 0 2024-03-23T08:23:46.882 INFO:tasks.workunit.client.0.smithi070.stdout:2/142: fsync d1/d7/df/f27 0 2024-03-23T08:23:46.883 INFO:tasks.workunit.client.0.smithi070.stdout:3/389: chown db/d14/d43/f4a 26 1 2024-03-23T08:23:46.884 INFO:tasks.workunit.client.0.smithi070.stdout:8/179: dread - d5/d8/d16/d32/f3c zero size 2024-03-23T08:23:46.884 INFO:tasks.workunit.client.0.smithi070.stdout:2/143: dread - d1/d7/df/f27 zero size 2024-03-23T08:23:46.887 INFO:tasks.workunit.client.0.smithi070.stdout:5/115: dwrite dc/d19/d21/f27 [0,4194304] 0 2024-03-23T08:23:46.888 INFO:tasks.workunit.client.0.smithi070.stdout:5/116: readlink l2 0 2024-03-23T08:23:46.892 INFO:tasks.workunit.client.0.smithi070.stdout:8/180: creat d5/d8/f4a x:0 0 0 2024-03-23T08:23:46.893 INFO:tasks.workunit.client.0.smithi070.stdout:2/144: creat d1/d7/d11/f2b x:0 0 0 2024-03-23T08:23:46.894 INFO:tasks.workunit.client.0.smithi070.stdout:2/145: read - d1/d7/df/f27 zero size 2024-03-23T08:23:46.899 INFO:tasks.workunit.client.0.smithi070.stdout:8/181: symlink d5/d8/d16/l4b 0 2024-03-23T08:23:46.901 INFO:tasks.workunit.client.0.smithi070.stdout:2/146: mknod d1/d7/df/c2c 0 2024-03-23T08:23:46.903 INFO:tasks.workunit.client.0.smithi070.stdout:5/117: link dc/c13 dc/d1b/c28 0 2024-03-23T08:23:46.904 INFO:tasks.workunit.client.0.smithi070.stdout:2/147: dread d1/d7/d11/f28 [0,4194304] 0 2024-03-23T08:23:46.906 INFO:tasks.workunit.client.0.smithi070.stdout:3/390: dread db/df/d15/f3e [0,4194304] 0 2024-03-23T08:23:46.907 INFO:tasks.workunit.client.0.smithi070.stdout:5/118: rmdir dc/d1b/d1d 39 2024-03-23T08:23:46.907 INFO:tasks.workunit.client.0.smithi070.stdout:5/119: chown f8 434 1 2024-03-23T08:23:46.909 INFO:tasks.workunit.client.0.smithi070.stdout:3/391: rmdir db/df/d4d 39 2024-03-23T08:23:46.910 INFO:tasks.workunit.client.0.smithi070.stdout:3/392: chown db/d20/l3a 522877714 1 2024-03-23T08:23:46.910 INFO:tasks.workunit.client.0.smithi070.stdout:2/148: creat d1/d29/f2d x:0 0 0 2024-03-23T08:23:46.912 INFO:tasks.workunit.client.0.smithi070.stdout:8/182: truncate d5/d8/f17 1652482 0 2024-03-23T08:23:46.918 INFO:tasks.workunit.client.0.smithi070.stdout:8/183: mknod d5/d8/d16/d32/c4c 0 2024-03-23T08:23:46.918 INFO:tasks.workunit.client.0.smithi070.stdout:3/393: write db/df/f2e [4168595,71377] 0 2024-03-23T08:23:46.920 INFO:tasks.workunit.client.0.smithi070.stdout:5/120: rename dc/d1b/c23 to dc/d19/d21/c29 0 2024-03-23T08:23:46.922 INFO:tasks.workunit.client.0.smithi070.stdout:2/149: write d1/fc [1797129,93653] 0 2024-03-23T08:23:46.924 INFO:tasks.workunit.client.0.smithi070.stdout:8/184: mkdir d5/d8/d16/d3b/d4d 0 2024-03-23T08:23:46.926 INFO:tasks.workunit.client.0.smithi070.stdout:3/394: dwrite db/d14/d65/f77 [0,4194304] 0 2024-03-23T08:23:46.927 INFO:tasks.workunit.client.0.smithi070.stdout:5/121: rename dc/d1b/f22 to dc/d19/d21/f2a 0 2024-03-23T08:23:46.928 INFO:tasks.workunit.client.0.smithi070.stdout:5/122: stat dc/l1a 0 2024-03-23T08:23:46.929 INFO:tasks.workunit.client.0.smithi070.stdout:8/185: dread d5/d8/d16/f24 [0,4194304] 0 2024-03-23T08:23:46.929 INFO:tasks.workunit.client.0.smithi070.stdout:3/395: write db/d14/d65/f77 [304420,18665] 0 2024-03-23T08:23:46.935 INFO:tasks.workunit.client.0.smithi070.stdout:5/123: write dc/f16 [2793026,119645] 0 2024-03-23T08:23:46.937 INFO:tasks.workunit.client.0.smithi070.stdout:5/124: write f8 [1169205,102364] 0 2024-03-23T08:23:46.937 INFO:tasks.workunit.client.0.smithi070.stdout:5/125: read f1 [2044652,14808] 0 2024-03-23T08:23:46.939 INFO:tasks.workunit.client.0.smithi070.stdout:8/186: creat d5/d8/f4e x:0 0 0 2024-03-23T08:23:46.940 INFO:tasks.workunit.client.0.smithi070.stdout:8/187: readlink d5/l14 0 2024-03-23T08:23:46.942 INFO:tasks.workunit.client.0.smithi070.stdout:5/126: dread dc/f16 [0,4194304] 0 2024-03-23T08:23:46.942 INFO:tasks.workunit.client.0.smithi070.stdout:5/127: chown dc/f14 5796561 1 2024-03-23T08:23:46.943 INFO:tasks.workunit.client.0.smithi070.stdout:2/150: truncate d1/d7/f26 2673208 0 2024-03-23T08:23:46.948 INFO:tasks.workunit.client.0.smithi070.stdout:8/188: mknod d5/d8/d16/d32/c4f 0 2024-03-23T08:23:46.952 INFO:tasks.workunit.client.0.smithi070.stdout:5/128: creat dc/f2b x:0 0 0 2024-03-23T08:23:46.953 INFO:tasks.workunit.client.0.smithi070.stdout:5/129: chown dc/f2b 11080 1 2024-03-23T08:23:46.956 INFO:tasks.workunit.client.0.smithi070.stdout:3/396: creat db/df/f8d x:0 0 0 2024-03-23T08:23:46.961 INFO:tasks.workunit.client.0.smithi070.stdout:8/189: creat d5/f50 x:0 0 0 2024-03-23T08:23:46.965 INFO:tasks.workunit.client.0.smithi070.stdout:5/130: creat dc/d19/f2c x:0 0 0 2024-03-23T08:23:46.966 INFO:tasks.workunit.client.0.smithi070.stdout:3/397: creat db/df/d5d/f8e x:0 0 0 2024-03-23T08:23:46.967 INFO:tasks.workunit.client.0.smithi070.stdout:8/190: creat d5/d8/d16/f51 x:0 0 0 2024-03-23T08:23:46.998 INFO:tasks.workunit.client.0.smithi070.stdout:2/151: dread d1/d7/f26 [0,4194304] 0 2024-03-23T08:23:46.998 INFO:tasks.workunit.client.0.smithi070.stdout:2/152: stat d1/d7/d11/f25 0 2024-03-23T08:23:47.013 INFO:tasks.workunit.client.0.smithi070.stdout:9/249: sync 2024-03-23T08:23:47.014 INFO:tasks.workunit.client.0.smithi070.stdout:3/398: rename db/df/d4d/d51/l76 to db/d20/d49/l8f 0 2024-03-23T08:23:47.016 INFO:tasks.workunit.client.0.smithi070.stdout:2/153: symlink d1/d6/l2e 0 2024-03-23T08:23:47.016 INFO:tasks.workunit.client.0.smithi070.stdout:5/131: link l2 dc/d1b/d1d/d25/l2d 0 2024-03-23T08:23:47.017 INFO:tasks.workunit.client.0.smithi070.stdout:2/154: chown d1/d7/d11/f12 0 1 2024-03-23T08:23:47.017 INFO:tasks.workunit.client.0.smithi070.stdout:9/250: dread db/d12/f17 [0,4194304] 0 2024-03-23T08:23:47.021 INFO:tasks.workunit.client.0.smithi070.stdout:5/132: write dc/f18 [1170406,16881] 0 2024-03-23T08:23:47.028 INFO:tasks.workunit.client.0.smithi070.stdout:5/133: creat dc/d19/d21/f2e x:0 0 0 2024-03-23T08:23:47.034 INFO:tasks.workunit.client.0.smithi070.stdout:3/399: link db/d14/d43/f4a db/df/f90 0 2024-03-23T08:23:47.037 INFO:tasks.workunit.client.0.smithi070.stdout:9/251: dwrite db/d12/d22/d25/f45 [0,4194304] 0 2024-03-23T08:23:47.037 INFO:tasks.workunit.client.0.smithi070.stdout:3/400: write db/df/d1a/f44 [1752038,76823] 0 2024-03-23T08:23:47.039 INFO:tasks.workunit.client.0.smithi070.stdout:2/155: creat d1/f2f x:0 0 0 2024-03-23T08:23:47.041 INFO:tasks.workunit.client.0.smithi070.stdout:9/252: write db/d12/d22/d25/d26/d32/f43 [545009,92197] 0 2024-03-23T08:23:47.044 INFO:tasks.workunit.client.0.smithi070.stdout:3/401: rename db/d4e/l52 to db/df/d15/l91 0 2024-03-23T08:23:47.045 INFO:tasks.workunit.client.0.smithi070.stdout:2/156: read d1/d7/f1e [73667,10985] 0 2024-03-23T08:23:47.047 INFO:tasks.workunit.client.0.smithi070.stdout:5/134: creat dc/d1b/f2f x:0 0 0 2024-03-23T08:23:47.048 INFO:tasks.workunit.client.0.smithi070.stdout:3/402: write db/df/d15/f3e [1257788,72525] 0 2024-03-23T08:23:47.049 INFO:tasks.workunit.client.0.smithi070.stdout:2/157: mkdir d1/d6/d30 0 2024-03-23T08:23:47.050 INFO:tasks.workunit.client.0.smithi070.stdout:3/403: write db/d14/f3f [2085229,45478] 0 2024-03-23T08:23:47.052 INFO:tasks.workunit.client.0.smithi070.stdout:2/158: dread d1/d7/d11/f28 [0,4194304] 0 2024-03-23T08:23:47.055 INFO:tasks.workunit.client.0.smithi070.stdout:9/253: getdents db/d12/d22 0 2024-03-23T08:23:47.056 INFO:tasks.workunit.client.0.smithi070.stdout:3/404: dread db/df/f90 [0,4194304] 0 2024-03-23T08:23:47.057 INFO:tasks.workunit.client.0.smithi070.stdout:5/135: rename c9 to dc/d1b/d1d/c30 0 2024-03-23T08:23:47.064 INFO:tasks.workunit.client.0.smithi070.stdout:3/405: rename db/d14/d67 to db/d4e/d92 0 2024-03-23T08:23:47.064 INFO:tasks.workunit.client.0.smithi070.stdout:3/406: stat db/df/d5d/c5f 0 2024-03-23T08:23:47.066 INFO:tasks.workunit.client.0.smithi070.stdout:5/136: symlink dc/d19/d21/l31 0 2024-03-23T08:23:47.068 INFO:tasks.workunit.client.0.smithi070.stdout:9/254: mknod db/d12/d34/d47/c4c 0 2024-03-23T08:23:47.069 INFO:tasks.workunit.client.0.smithi070.stdout:3/407: rename db/d20/c89 to db/d14/d43/d62/c93 0 2024-03-23T08:23:47.071 INFO:tasks.workunit.client.0.smithi070.stdout:5/137: write dc/f1f [14916,72740] 0 2024-03-23T08:23:47.072 INFO:tasks.workunit.client.0.smithi070.stdout:2/159: truncate d1/fc 1774176 0 2024-03-23T08:23:47.074 INFO:tasks.workunit.client.0.smithi070.stdout:3/408: mknod db/d20/c94 0 2024-03-23T08:23:47.075 INFO:tasks.workunit.client.0.smithi070.stdout:9/255: dread db/d12/d14/f23 [0,4194304] 0 2024-03-23T08:23:47.077 INFO:tasks.workunit.client.0.smithi070.stdout:3/409: mkdir db/d14/d43/d62/d95 0 2024-03-23T08:23:47.077 INFO:tasks.workunit.client.0.smithi070.stdout:9/256: write db/d12/d14/f35 [529897,3394] 0 2024-03-23T08:23:47.078 INFO:tasks.workunit.client.0.smithi070.stdout:2/160: symlink d1/d21/l31 0 2024-03-23T08:23:47.084 INFO:tasks.workunit.client.0.smithi070.stdout:5/138: dwrite f1 [0,4194304] 0 2024-03-23T08:23:47.089 INFO:tasks.workunit.client.0.smithi070.stdout:3/410: dwrite db/d4e/f73 [0,4194304] 0 2024-03-23T08:23:47.092 INFO:tasks.workunit.client.0.smithi070.stdout:9/257: rmdir db/d12/d22/d41 0 2024-03-23T08:23:47.098 INFO:tasks.workunit.client.0.smithi070.stdout:5/139: mkdir dc/d19/d21/d32 0 2024-03-23T08:23:47.101 INFO:tasks.workunit.client.0.smithi070.stdout:9/258: fsync db/f31 0 2024-03-23T08:23:47.101 INFO:tasks.workunit.client.0.smithi070.stdout:2/161: dread d1/fd [4194304,4194304] 0 2024-03-23T08:23:47.103 INFO:tasks.workunit.client.0.smithi070.stdout:9/259: dread f8 [0,4194304] 0 2024-03-23T08:23:47.104 INFO:tasks.workunit.client.0.smithi070.stdout:5/140: rename dc/f16 to dc/d1b/d1d/d25/f33 0 2024-03-23T08:23:47.106 INFO:tasks.workunit.client.0.smithi070.stdout:3/411: dwrite db/d20/d63/f75 [0,4194304] 0 2024-03-23T08:23:47.107 INFO:tasks.workunit.client.0.smithi070.stdout:2/162: mknod d1/d7/d11/c32 0 2024-03-23T08:23:47.108 INFO:tasks.workunit.client.0.smithi070.stdout:5/141: symlink dc/d19/d21/l34 0 2024-03-23T08:23:47.111 INFO:tasks.workunit.client.0.smithi070.stdout:4/112: sync 2024-03-23T08:23:47.111 INFO:tasks.workunit.client.0.smithi070.stdout:9/260: dwrite db/d12/f3a [0,4194304] 0 2024-03-23T08:23:47.113 INFO:tasks.workunit.client.0.smithi070.stdout:2/163: mknod d1/d7/c33 0 2024-03-23T08:23:47.115 INFO:tasks.workunit.client.0.smithi070.stdout:3/412: symlink db/d14/d43/d62/d95/l96 0 2024-03-23T08:23:47.116 INFO:tasks.workunit.client.0.smithi070.stdout:5/142: symlink dc/d1b/d1d/d25/l35 0 2024-03-23T08:23:47.123 INFO:tasks.workunit.client.0.smithi070.stdout:3/413: write db/d20/f45 [802061,13526] 0 2024-03-23T08:23:47.125 INFO:tasks.workunit.client.0.smithi070.stdout:5/143: unlink dc/c15 0 2024-03-23T08:23:47.126 INFO:tasks.workunit.client.0.smithi070.stdout:4/113: dwrite d3/d6/d9/f22 [0,4194304] 0 2024-03-23T08:23:47.126 INFO:tasks.workunit.client.0.smithi070.stdout:2/164: dwrite d1/f16 [0,4194304] 0 2024-03-23T08:23:47.129 INFO:tasks.workunit.client.0.smithi070.stdout:3/414: write db/d14/d65/f78 [729621,109558] 0 2024-03-23T08:23:47.131 INFO:tasks.workunit.client.0.smithi070.stdout:1/257: sync 2024-03-23T08:23:47.131 INFO:tasks.workunit.client.0.smithi070.stdout:0/378: sync 2024-03-23T08:23:47.131 INFO:tasks.workunit.client.0.smithi070.stdout:7/113: sync 2024-03-23T08:23:47.131 INFO:tasks.workunit.client.0.smithi070.stdout:8/191: sync 2024-03-23T08:23:47.131 INFO:tasks.workunit.client.0.smithi070.stdout:6/342: sync 2024-03-23T08:23:47.136 INFO:tasks.workunit.client.0.smithi070.stdout:6/343: dread d0/d23/d28/f37 [0,4194304] 0 2024-03-23T08:23:47.139 INFO:tasks.workunit.client.0.smithi070.stdout:2/165: mkdir d1/d6/d34 0 2024-03-23T08:23:47.142 INFO:tasks.workunit.client.0.smithi070.stdout:5/144: rename dc/d19/d21 to dc/d1b/d1d/d25/d36 0 2024-03-23T08:23:47.143 INFO:tasks.workunit.client.0.smithi070.stdout:7/114: mknod d25/d27/c2b 0 2024-03-23T08:23:47.145 INFO:tasks.workunit.client.0.smithi070.stdout:0/379: mknod d16/d26/d27/d3d/d4d/c88 0 2024-03-23T08:23:47.145 INFO:tasks.workunit.client.0.smithi070.stdout:5/145: write f1 [7380586,41379] 0 2024-03-23T08:23:47.148 INFO:tasks.workunit.client.0.smithi070.stdout:0/380: chown d16/d22/f23 7047481 1 2024-03-23T08:23:47.148 INFO:tasks.workunit.client.0.smithi070.stdout:1/258: dwrite d8/db/d2c/d3f/f56 [0,4194304] 0 2024-03-23T08:23:47.149 INFO:tasks.workunit.client.0.smithi070.stdout:6/344: mknod d0/d23/d28/d35/d3a/d3c/d5b/c72 0 2024-03-23T08:23:47.151 INFO:tasks.workunit.client.0.smithi070.stdout:1/259: read - d8/f1c zero size 2024-03-23T08:23:47.153 INFO:tasks.workunit.client.0.smithi070.stdout:3/415: unlink l8 0 2024-03-23T08:23:47.154 INFO:tasks.workunit.client.0.smithi070.stdout:6/345: fsync d0/d23/d1c/d38/d52/f6b 0 2024-03-23T08:23:47.155 INFO:tasks.workunit.client.0.smithi070.stdout:5/146: dread dc/d1b/d1d/d25/d36/f2a [0,4194304] 0 2024-03-23T08:23:47.157 INFO:tasks.workunit.client.0.smithi070.stdout:8/192: rename d5/c49 to d5/d8/d16/d30/d38/c52 0 2024-03-23T08:23:47.157 INFO:tasks.workunit.client.0.smithi070.stdout:8/193: fsync d5/d8/d16/d3b/f48 0 2024-03-23T08:23:47.159 INFO:tasks.workunit.client.0.smithi070.stdout:8/194: stat d5/d8/f4e 0 2024-03-23T08:23:47.160 INFO:tasks.workunit.client.0.smithi070.stdout:7/115: creat d25/f2c x:0 0 0 2024-03-23T08:23:47.160 INFO:tasks.workunit.client.0.smithi070.stdout:8/195: readlink d5/l3d 0 2024-03-23T08:23:47.160 INFO:tasks.workunit.client.0.smithi070.stdout:9/261: link db/d12/d22/d25/c33 db/d12/d22/d25/d26/d32/c4d 0 2024-03-23T08:23:47.165 INFO:tasks.workunit.client.0.smithi070.stdout:1/260: creat d8/db/f62 x:0 0 0 2024-03-23T08:23:47.165 INFO:tasks.workunit.client.0.smithi070.stdout:2/166: dwrite d1/fd [0,4194304] 0 2024-03-23T08:23:47.166 INFO:tasks.workunit.client.0.smithi070.stdout:3/416: mknod db/d4e/d92/c97 0 2024-03-23T08:23:47.166 INFO:tasks.workunit.client.0.smithi070.stdout:1/261: chown d8/db/d2c/d3f/d48 96 1 2024-03-23T08:23:47.173 INFO:tasks.workunit.client.0.smithi070.stdout:4/114: rename d3/f20 to d3/d6/d11/d1a/d1b/f26 0 2024-03-23T08:23:47.175 INFO:tasks.workunit.client.0.smithi070.stdout:4/115: dread - d3/d6/d11/d1a/d1b/f26 zero size 2024-03-23T08:23:47.175 INFO:tasks.workunit.client.0.smithi070.stdout:7/116: fdatasync fd 0 2024-03-23T08:23:47.177 INFO:tasks.workunit.client.0.smithi070.stdout:9/262: readlink db/d12/d22/d25/d26/d3b/l4a 0 2024-03-23T08:23:47.187 INFO:tasks.workunit.client.0.smithi070.stdout:8/196: rename d5/d8/d16/d32/c47 to d5/d8/d16/d3b/d4d/c53 0 2024-03-23T08:23:47.188 INFO:tasks.workunit.client.0.smithi070.stdout:8/197: write d5/fe [12601,7492] 0 2024-03-23T08:23:47.189 INFO:tasks.workunit.client.0.smithi070.stdout:4/116: mkdir d3/d6/d11/d1a/d27 0 2024-03-23T08:23:47.194 INFO:tasks.workunit.client.0.smithi070.stdout:9/263: write db/d12/d22/d25/f24 [2879779,80993] 0 2024-03-23T08:23:47.195 INFO:tasks.workunit.client.0.smithi070.stdout:9/264: chown db/d12/d14/l48 200633353 1 2024-03-23T08:23:47.197 INFO:tasks.workunit.client.0.smithi070.stdout:0/381: link d16/l19 d16/d22/d43/d47/d87/l89 0 2024-03-23T08:23:47.197 INFO:tasks.workunit.client.0.smithi070.stdout:2/167: getdents d1/d6/d34 0 2024-03-23T08:23:47.202 INFO:tasks.workunit.client.0.smithi070.stdout:1/262: fdatasync d8/d1b/f4b 0 2024-03-23T08:23:47.202 INFO:tasks.workunit.client.0.smithi070.stdout:6/346: link d0/d23/d28/l66 d0/d3/l73 0 2024-03-23T08:23:47.202 INFO:tasks.workunit.client.0.smithi070.stdout:6/347: fdatasync d0/d23/d1c/d38/d52/f6b 0 2024-03-23T08:23:47.203 INFO:tasks.workunit.client.0.smithi070.stdout:2/168: dread d1/d7/f26 [0,4194304] 0 2024-03-23T08:23:47.204 INFO:tasks.workunit.client.0.smithi070.stdout:7/117: rename l6 to d25/l2d 0 2024-03-23T08:23:47.209 INFO:tasks.workunit.client.0.smithi070.stdout:0/382: creat d16/d5f/f8a x:0 0 0 2024-03-23T08:23:47.211 INFO:tasks.workunit.client.0.smithi070.stdout:2/169: dwrite d1/d29/f2d [0,4194304] 0 2024-03-23T08:23:47.212 INFO:tasks.workunit.client.0.smithi070.stdout:5/147: link dc/d1b/d1d/d25/f33 dc/d1b/d1d/f37 0 2024-03-23T08:23:47.212 INFO:tasks.workunit.client.0.smithi070.stdout:0/383: chown d16/d5f/l6c 268018 1 2024-03-23T08:23:47.214 INFO:tasks.workunit.client.0.smithi070.stdout:9/265: rename db/f31 to db/d12/d14/f4e 0 2024-03-23T08:23:47.215 INFO:tasks.workunit.client.0.smithi070.stdout:7/118: symlink d25/l2e 0 2024-03-23T08:23:47.217 INFO:tasks.workunit.client.0.smithi070.stdout:7/119: write f20 [1028246,8773] 0 2024-03-23T08:23:47.219 INFO:tasks.workunit.client.0.smithi070.stdout:7/120: fdatasync f12 0 2024-03-23T08:23:47.220 INFO:tasks.workunit.client.0.smithi070.stdout:7/121: chown f24 842 1 2024-03-23T08:23:47.222 INFO:tasks.workunit.client.0.smithi070.stdout:4/117: dwrite d3/d6/d11/f18 [0,4194304] 0 2024-03-23T08:23:47.223 INFO:tasks.workunit.client.0.smithi070.stdout:7/122: write f1d [61949,112309] 0 2024-03-23T08:23:47.226 INFO:tasks.workunit.client.0.smithi070.stdout:3/417: getdents db/d20 0 2024-03-23T08:23:47.231 INFO:tasks.workunit.client.0.smithi070.stdout:7/123: fsync f24 0 2024-03-23T08:23:47.231 INFO:tasks.workunit.client.0.smithi070.stdout:2/170: rename d1/f16 to d1/d29/f35 0 2024-03-23T08:23:47.233 INFO:tasks.workunit.client.0.smithi070.stdout:9/266: dwrite db/d12/d14/f49 [0,4194304] 0 2024-03-23T08:23:47.234 INFO:tasks.workunit.client.0.smithi070.stdout:5/148: mkdir dc/d1b/d1d/d25/d38 0 2024-03-23T08:23:47.239 INFO:tasks.workunit.client.0.smithi070.stdout:0/384: symlink d16/d26/d27/d4c/l8b 0 2024-03-23T08:23:47.239 INFO:tasks.workunit.client.0.smithi070.stdout:5/149: fdatasync dc/f1f 0 2024-03-23T08:23:47.242 INFO:tasks.workunit.client.0.smithi070.stdout:8/198: getdents d5/d8/d16/d3b/d4d 0 2024-03-23T08:23:47.242 INFO:tasks.workunit.client.0.smithi070.stdout:2/171: write d1/d7/f17 [33666,63605] 0 2024-03-23T08:23:47.244 INFO:tasks.workunit.client.0.smithi070.stdout:7/124: mkdir d25/d27/d2f 0 2024-03-23T08:23:47.244 INFO:tasks.workunit.client.0.smithi070.stdout:5/150: write f8 [2024493,11541] 0 2024-03-23T08:23:47.245 INFO:tasks.workunit.client.0.smithi070.stdout:4/118: dwrite d3/fd [0,4194304] 0 2024-03-23T08:23:47.246 INFO:tasks.workunit.client.0.smithi070.stdout:3/418: link db/df/f8d db/df/d15/f98 0 2024-03-23T08:23:47.246 INFO:tasks.workunit.client.0.smithi070.stdout:1/263: dread d8/db/f45 [0,4194304] 0 2024-03-23T08:23:47.247 INFO:tasks.workunit.client.0.smithi070.stdout:0/385: write d16/d26/d27/d4c/d60/f69 [214411,3562] 0 2024-03-23T08:23:47.249 INFO:tasks.workunit.client.0.smithi070.stdout:8/199: write d5/d8/f18 [36920,67663] 0 2024-03-23T08:23:47.252 INFO:tasks.workunit.client.0.smithi070.stdout:1/264: readlink d8/ld 0 2024-03-23T08:23:47.254 INFO:tasks.workunit.client.0.smithi070.stdout:8/200: write d5/d8/f4a [131791,86951] 0 2024-03-23T08:23:47.254 INFO:tasks.workunit.client.0.smithi070.stdout:9/267: mkdir db/d12/d22/d25/d4f 0 2024-03-23T08:23:47.255 INFO:tasks.workunit.client.0.smithi070.stdout:2/172: dread d1/d7/f17 [0,4194304] 0 2024-03-23T08:23:47.257 INFO:tasks.workunit.client.0.smithi070.stdout:0/386: creat d16/d26/d27/d3d/d6d/f8c x:0 0 0 2024-03-23T08:23:47.257 INFO:tasks.workunit.client.0.smithi070.stdout:9/268: write db/f1b [2024191,48660] 0 2024-03-23T08:23:47.260 INFO:tasks.workunit.client.0.smithi070.stdout:1/265: rmdir d8/d42/d4e 39 2024-03-23T08:23:47.263 INFO:tasks.workunit.client.0.smithi070.stdout:4/119: rename d3/d6/d9/ff to d3/d6/d11/d1a/d1b/f28 0 2024-03-23T08:23:47.268 INFO:tasks.workunit.client.0.smithi070.stdout:5/151: dwrite dc/f18 [0,4194304] 0 2024-03-23T08:23:47.268 INFO:tasks.workunit.client.0.smithi070.stdout:7/125: dwrite f8 [0,4194304] 0 2024-03-23T08:23:47.270 INFO:tasks.workunit.client.0.smithi070.stdout:3/419: dwrite db/df/d1a/f3c [0,4194304] 0 2024-03-23T08:23:47.272 INFO:tasks.workunit.client.0.smithi070.stdout:2/173: dread d1/d29/f35 [0,4194304] 0 2024-03-23T08:23:47.274 INFO:tasks.workunit.client.0.smithi070.stdout:2/174: stat d1/d7/c24 0 2024-03-23T08:23:47.275 INFO:tasks.workunit.client.0.smithi070.stdout:9/269: dwrite db/d12/d22/d25/d26/d32/f43 [0,4194304] 0 2024-03-23T08:23:47.284 INFO:tasks.workunit.client.0.smithi070.stdout:9/270: dread db/d12/d22/d25/d26/d32/f43 [0,4194304] 0 2024-03-23T08:23:47.284 INFO:tasks.workunit.client.0.smithi070.stdout:9/271: readlink l5 0 2024-03-23T08:23:47.354 INFO:tasks.workunit.client.0.smithi070.stdout:1/266: truncate d8/d42/d4e/f59 797615 0 2024-03-23T08:23:47.358 INFO:tasks.workunit.client.0.smithi070.stdout:7/126: chown d25/d27/c29 218466055 1 2024-03-23T08:23:47.360 INFO:tasks.workunit.client.0.smithi070.stdout:3/420: symlink db/d14/d65/l99 0 2024-03-23T08:23:47.361 INFO:tasks.workunit.client.0.smithi070.stdout:2/175: creat d1/d7/f36 x:0 0 0 2024-03-23T08:23:47.367 INFO:tasks.workunit.client.0.smithi070.stdout:9/272: write db/d12/d22/f2a [2729938,76176] 0 2024-03-23T08:23:47.368 INFO:tasks.workunit.client.0.smithi070.stdout:1/267: unlink d8/d11/c14 0 2024-03-23T08:23:47.369 INFO:tasks.workunit.client.0.smithi070.stdout:1/268: readlink d8/db/d2c/d3f/d48/l52 0 2024-03-23T08:23:47.370 INFO:tasks.workunit.client.0.smithi070.stdout:1/269: write d8/d11/f3b [474190,88715] 0 2024-03-23T08:23:47.371 INFO:tasks.workunit.client.0.smithi070.stdout:8/201: link d5/d8/d16/f2b d5/d8/d16/d32/f54 0 2024-03-23T08:23:47.376 INFO:tasks.workunit.client.0.smithi070.stdout:7/127: creat d25/d27/f30 x:0 0 0 2024-03-23T08:23:47.379 INFO:tasks.workunit.client.0.smithi070.stdout:3/421: unlink db/d20/l3a 0 2024-03-23T08:23:47.380 INFO:tasks.workunit.client.0.smithi070.stdout:2/176: unlink d1/d7/f26 0 2024-03-23T08:23:47.384 INFO:tasks.workunit.client.0.smithi070.stdout:2/177: dread d1/d29/f35 [0,4194304] 0 2024-03-23T08:23:47.385 INFO:tasks.workunit.client.0.smithi070.stdout:2/178: chown d1/d7/d11/f28 164 1 2024-03-23T08:23:47.389 INFO:tasks.workunit.client.0.smithi070.stdout:1/270: symlink d8/d26/d49/l63 0 2024-03-23T08:23:47.398 INFO:tasks.workunit.client.0.smithi070.stdout:8/202: creat d5/d8/d16/d32/f55 x:0 0 0 2024-03-23T08:23:47.398 INFO:tasks.workunit.client.0.smithi070.stdout:5/152: truncate dc/d1b/d1d/d25/d36/f27 663062 0 2024-03-23T08:23:47.399 INFO:tasks.workunit.client.0.smithi070.stdout:4/120: link d3/d6/d9/c14 d3/d6/d11/d1a/c29 0 2024-03-23T08:23:47.401 INFO:tasks.workunit.client.0.smithi070.stdout:5/153: dread dc/f1f [0,4194304] 0 2024-03-23T08:23:47.402 INFO:tasks.workunit.client.0.smithi070.stdout:0/387: getdents d16/d26/d27/d4c/d60 0 2024-03-23T08:23:47.403 INFO:tasks.workunit.client.0.smithi070.stdout:5/154: chown dc/f1f 2845631 1 2024-03-23T08:23:47.405 INFO:tasks.workunit.client.0.smithi070.stdout:1/271: mkdir d8/db/d40/d43/d64 0 2024-03-23T08:23:47.406 INFO:tasks.workunit.client.0.smithi070.stdout:7/128: symlink d25/d27/d2f/l31 0 2024-03-23T08:23:47.407 INFO:tasks.workunit.client.0.smithi070.stdout:7/129: chown f12 20837 1 2024-03-23T08:23:47.407 INFO:tasks.workunit.client.0.smithi070.stdout:7/130: stat l9 0 2024-03-23T08:23:47.416 INFO:tasks.workunit.client.0.smithi070.stdout:9/273: dwrite f6 [4194304,4194304] 0 2024-03-23T08:23:47.421 INFO:tasks.workunit.client.0.smithi070.stdout:8/203: creat d5/d8/d16/d32/f56 x:0 0 0 2024-03-23T08:23:47.428 INFO:tasks.workunit.client.0.smithi070.stdout:9/274: dwrite db/d12/d22/d25/f24 [0,4194304] 0 2024-03-23T08:23:47.435 INFO:tasks.workunit.client.0.smithi070.stdout:4/121: mkdir d3/d19/d2a 0 2024-03-23T08:23:47.444 INFO:tasks.workunit.client.0.smithi070.stdout:0/388: rename d16/d26/d27/d3d/d4d/f66 to d16/d22/f8d 0 2024-03-23T08:23:47.448 INFO:tasks.workunit.client.0.smithi070.stdout:8/204: dread d5/d8/d16/f22 [0,4194304] 0 2024-03-23T08:23:47.448 INFO:tasks.workunit.client.0.smithi070.stdout:1/272: mkdir d8/d26/d49/d65 0 2024-03-23T08:23:47.451 INFO:tasks.workunit.client.0.smithi070.stdout:8/205: chown d5/d8/d16/d30/l29 33035 1 2024-03-23T08:23:47.453 INFO:tasks.workunit.client.0.smithi070.stdout:0/389: dwrite d16/f30 [0,4194304] 0 2024-03-23T08:23:47.461 INFO:tasks.workunit.client.0.smithi070.stdout:2/179: link d1/d7/d11/f12 d1/d18/f37 0 2024-03-23T08:23:47.463 INFO:tasks.workunit.client.0.smithi070.stdout:3/422: symlink db/df/d4d/l9a 0 2024-03-23T08:23:47.469 INFO:tasks.workunit.client.0.smithi070.stdout:5/155: unlink dc/d1b/d1d/d25/l2d 0 2024-03-23T08:23:47.470 INFO:tasks.workunit.client.0.smithi070.stdout:5/156: chown dc/d1b/f2f 226032 1 2024-03-23T08:23:47.473 INFO:tasks.workunit.client.0.smithi070.stdout:1/273: creat d8/d42/d4e/f66 x:0 0 0 2024-03-23T08:23:47.478 INFO:tasks.workunit.client.0.smithi070.stdout:0/390: symlink d16/d26/d27/d4c/d60/l8e 0 2024-03-23T08:23:47.478 INFO:tasks.workunit.client.0.smithi070.stdout:0/391: chown d16/f34 644 1 2024-03-23T08:23:47.480 INFO:tasks.workunit.client.0.smithi070.stdout:2/180: mkdir d1/d7/d11/d38 0 2024-03-23T08:23:47.480 INFO:tasks.workunit.client.0.smithi070.stdout:2/181: readlink d1/la 0 2024-03-23T08:23:47.483 INFO:tasks.workunit.client.0.smithi070.stdout:3/423: rename db/d14/f4b to db/df/d15/d3d/f9b 0 2024-03-23T08:23:47.484 INFO:tasks.workunit.client.0.smithi070.stdout:3/424: fdatasync db/d14/d65/f78 0 2024-03-23T08:23:47.491 INFO:tasks.workunit.client.0.smithi070.stdout:0/392: symlink d16/d22/d43/l8f 0 2024-03-23T08:23:47.492 INFO:tasks.workunit.client.0.smithi070.stdout:2/182: mkdir d1/d7/d39 0 2024-03-23T08:23:47.493 INFO:tasks.workunit.client.0.smithi070.stdout:4/122: link f1 d3/d19/d2a/f2b 0 2024-03-23T08:23:47.496 INFO:tasks.workunit.client.0.smithi070.stdout:3/425: write db/df/f90 [4217370,120861] 0 2024-03-23T08:23:47.497 INFO:tasks.workunit.client.0.smithi070.stdout:1/274: creat d8/d42/d51/f67 x:0 0 0 2024-03-23T08:23:47.498 INFO:tasks.workunit.client.0.smithi070.stdout:1/275: stat d8/db/d2c/d3f/l50 0 2024-03-23T08:23:47.499 INFO:tasks.workunit.client.0.smithi070.stdout:1/276: chown d8/d1b 0 1 2024-03-23T08:23:47.500 INFO:tasks.workunit.client.0.smithi070.stdout:1/277: chown d8/d1b 213999168 1 2024-03-23T08:23:47.500 INFO:tasks.workunit.client.0.smithi070.stdout:9/275: link db/d12/d22/d25/c33 db/d12/d22/d25/d26/d3b/c50 0 2024-03-23T08:23:47.500 INFO:tasks.workunit.client.0.smithi070.stdout:1/278: chown d8/d42/d4e/f59 15639201 1 2024-03-23T08:23:47.501 INFO:tasks.workunit.client.0.smithi070.stdout:9/276: read db/f1b [1875952,57845] 0 2024-03-23T08:23:47.501 INFO:tasks.workunit.client.0.smithi070.stdout:0/393: rename d16/d26/d27/d3d/f64 to d16/d26/d27/d4c/d60/d7c/f90 0 2024-03-23T08:23:47.502 INFO:tasks.workunit.client.0.smithi070.stdout:0/394: chown d16/d5f 2 1 2024-03-23T08:23:47.506 INFO:tasks.workunit.client.0.smithi070.stdout:8/206: getdents d5/d8 0 2024-03-23T08:23:47.507 INFO:tasks.workunit.client.0.smithi070.stdout:1/279: symlink d8/d26/d49/l68 0 2024-03-23T08:23:47.508 INFO:tasks.workunit.client.0.smithi070.stdout:0/395: dread d16/d26/d27/d4c/d60/f7e [0,4194304] 0 2024-03-23T08:23:47.508 INFO:tasks.workunit.client.0.smithi070.stdout:1/280: truncate d8/d26/f38 307286 0 2024-03-23T08:23:47.508 INFO:tasks.workunit.client.0.smithi070.stdout:0/396: chown f3 354475 1 2024-03-23T08:23:47.509 INFO:tasks.workunit.client.0.smithi070.stdout:2/183: link d1/d7/d11/f25 d1/d6/d30/f3a 0 2024-03-23T08:23:47.510 INFO:tasks.workunit.client.0.smithi070.stdout:0/397: read d16/d22/f8d [337064,53738] 0 2024-03-23T08:23:47.512 INFO:tasks.workunit.client.0.smithi070.stdout:4/123: symlink d3/d6/d11/l2c 0 2024-03-23T08:23:47.512 INFO:tasks.workunit.client.0.smithi070.stdout:2/184: read d1/d7/d11/f25 [79579,2548] 0 2024-03-23T08:23:47.513 INFO:tasks.workunit.client.0.smithi070.stdout:5/157: getdents dc/d1b/d1d/d25/d36 0 2024-03-23T08:23:47.514 INFO:tasks.workunit.client.0.smithi070.stdout:3/426: mkdir db/d14/d9c 0 2024-03-23T08:23:47.518 INFO:tasks.workunit.client.0.smithi070.stdout:5/158: write dc/f2b [65894,42943] 0 2024-03-23T08:23:47.519 INFO:tasks.workunit.client.0.smithi070.stdout:0/398: creat d16/d26/d27/f91 x:0 0 0 2024-03-23T08:23:47.519 INFO:tasks.workunit.client.0.smithi070.stdout:7/131: dwrite f1 [0,4194304] 0 2024-03-23T08:23:47.528 INFO:tasks.workunit.client.0.smithi070.stdout:1/281: dwrite d8/db/d2c/d3f/f56 [0,4194304] 0 2024-03-23T08:23:47.529 INFO:tasks.workunit.client.0.smithi070.stdout:2/185: link d1/d7/d11/f25 d1/d6/f3b 0 2024-03-23T08:23:47.531 INFO:tasks.workunit.client.0.smithi070.stdout:9/277: rename db/d12/d22/d25/d26/d32/c4d to db/d12/c51 0 2024-03-23T08:23:47.531 INFO:tasks.workunit.client.0.smithi070.stdout:3/427: dread db/d14/d43/f5a [0,4194304] 0 2024-03-23T08:23:47.535 INFO:tasks.workunit.client.0.smithi070.stdout:4/124: creat d3/d6/d11/f2d x:0 0 0 2024-03-23T08:23:47.535 INFO:tasks.workunit.client.0.smithi070.stdout:2/186: chown d1/d7/d11/c32 8092 1 2024-03-23T08:23:47.539 INFO:tasks.workunit.client.0.smithi070.stdout:8/207: getdents d5/d8/d16/d3b 0 2024-03-23T08:23:47.541 INFO:tasks.workunit.client.0.smithi070.stdout:7/132: dwrite d25/f2c [0,4194304] 0 2024-03-23T08:23:47.541 INFO:tasks.workunit.client.0.smithi070.stdout:5/159: dread dc/d1b/d1d/d25/d36/f2a [0,4194304] 0 2024-03-23T08:23:47.543 INFO:tasks.workunit.client.0.smithi070.stdout:1/282: rename d8/d26/d41/l4a to d8/d26/d49/l69 0 2024-03-23T08:23:47.545 INFO:tasks.workunit.client.0.smithi070.stdout:9/278: dwrite db/d12/f2b [0,4194304] 0 2024-03-23T08:23:47.551 INFO:tasks.workunit.client.0.smithi070.stdout:6/348: sync 2024-03-23T08:23:47.554 INFO:tasks.workunit.client.0.smithi070.stdout:0/399: creat d16/d22/d43/d47/d87/f92 x:0 0 0 2024-03-23T08:23:47.554 INFO:tasks.workunit.client.0.smithi070.stdout:0/400: chown d16/c1e 56368376 1 2024-03-23T08:23:47.556 INFO:tasks.workunit.client.0.smithi070.stdout:7/133: write f23 [855781,36555] 0 2024-03-23T08:23:47.559 INFO:tasks.workunit.client.0.smithi070.stdout:3/428: rename db/l57 to db/d14/d9c/l9d 0 2024-03-23T08:23:47.563 INFO:tasks.workunit.client.0.smithi070.stdout:9/279: unlink db/d12/d22/f2a 0 2024-03-23T08:23:47.573 INFO:tasks.workunit.client.0.smithi070.stdout:4/125: rename f2 to d3/d6/d9/f2e 0 2024-03-23T08:23:47.573 INFO:tasks.workunit.client.0.smithi070.stdout:3/429: chown db/d20/d49/l8f 754181636 1 2024-03-23T08:23:47.584 INFO:tasks.workunit.client.0.smithi070.stdout:6/349: mkdir d0/d23/d74 0 2024-03-23T08:23:47.593 INFO:tasks.workunit.client.0.smithi070.stdout:9/280: symlink db/d12/d22/l52 0 2024-03-23T08:23:47.599 INFO:tasks.workunit.client.0.smithi070.stdout:7/134: rename l1c to d25/d27/d2f/l32 0 2024-03-23T08:23:47.602 INFO:tasks.workunit.client.0.smithi070.stdout:1/283: link d8/db/f1f d8/db/f6a 0 2024-03-23T08:23:47.608 INFO:tasks.workunit.client.0.smithi070.stdout:4/126: creat d3/d6/f2f x:0 0 0 2024-03-23T08:23:47.610 INFO:tasks.workunit.client.0.smithi070.stdout:3/430: link db/d7f/c82 db/df/d15/d22/c9e 0 2024-03-23T08:23:47.611 INFO:tasks.workunit.client.0.smithi070.stdout:7/135: symlink d25/d27/l33 0 2024-03-23T08:23:47.613 INFO:tasks.workunit.client.0.smithi070.stdout:6/350: creat d0/d23/d28/d35/d3a/d3c/f75 x:0 0 0 2024-03-23T08:23:47.613 INFO:tasks.workunit.client.0.smithi070.stdout:6/351: fdatasync d0/f50 0 2024-03-23T08:23:47.614 INFO:tasks.workunit.client.0.smithi070.stdout:1/284: symlink d8/d42/d51/l6b 0 2024-03-23T08:23:47.614 INFO:tasks.workunit.client.0.smithi070.stdout:1/285: fsync d8/d1b/f33 0 2024-03-23T08:23:47.616 INFO:tasks.workunit.client.0.smithi070.stdout:9/281: symlink db/d12/d22/d25/d26/d3b/d42/l53 0 2024-03-23T08:23:47.620 INFO:tasks.workunit.client.0.smithi070.stdout:7/136: creat d25/f34 x:0 0 0 2024-03-23T08:23:47.623 INFO:tasks.workunit.client.0.smithi070.stdout:6/352: creat d0/d3/d2c/f76 x:0 0 0 2024-03-23T08:23:47.625 INFO:tasks.workunit.client.0.smithi070.stdout:3/431: dwrite db/df/d1a/f58 [0,4194304] 0 2024-03-23T08:23:47.628 INFO:tasks.workunit.client.0.smithi070.stdout:2/187: write d1/d7/f1e [220747,29848] 0 2024-03-23T08:23:47.631 INFO:tasks.workunit.client.0.smithi070.stdout:6/353: creat d0/d23/d28/d35/f77 x:0 0 0 2024-03-23T08:23:47.632 INFO:tasks.workunit.client.0.smithi070.stdout:7/137: dwrite d25/f34 [0,4194304] 0 2024-03-23T08:23:47.637 INFO:tasks.workunit.client.0.smithi070.stdout:9/282: unlink db/d12/d22/f27 0 2024-03-23T08:23:47.637 INFO:tasks.workunit.client.0.smithi070.stdout:6/354: write d0/d23/d28/d35/d3a/d3c/d5b/f5e [59724,46215] 0 2024-03-23T08:23:47.639 INFO:tasks.workunit.client.0.smithi070.stdout:0/401: rmdir d16/d22/d43 39 2024-03-23T08:23:47.642 INFO:tasks.workunit.client.0.smithi070.stdout:3/432: unlink db/df/d5d/f8e 0 2024-03-23T08:23:47.643 INFO:tasks.workunit.client.0.smithi070.stdout:3/433: write db/d14/d43/f4a [1910949,81781] 0 2024-03-23T08:23:47.643 INFO:tasks.workunit.client.0.smithi070.stdout:2/188: creat d1/d18/f3c x:0 0 0 2024-03-23T08:23:47.646 INFO:tasks.workunit.client.0.smithi070.stdout:5/160: write dc/d1b/d1d/d25/d36/f27 [125633,111795] 0 2024-03-23T08:23:47.649 INFO:tasks.workunit.client.0.smithi070.stdout:6/355: dread d0/d23/d28/d35/d3a/d3c/f53 [0,4194304] 0 2024-03-23T08:23:47.654 INFO:tasks.workunit.client.0.smithi070.stdout:1/286: dread d8/d1b/f22 [0,4194304] 0 2024-03-23T08:23:47.658 INFO:tasks.workunit.client.0.smithi070.stdout:3/434: unlink db/df/d15/f98 0 2024-03-23T08:23:47.659 INFO:tasks.workunit.client.0.smithi070.stdout:5/161: dread f1 [4194304,4194304] 0 2024-03-23T08:23:47.660 INFO:tasks.workunit.client.0.smithi070.stdout:3/435: write db/d20/f45 [623946,62836] 0 2024-03-23T08:23:47.661 INFO:tasks.workunit.client.0.smithi070.stdout:5/162: dread - dc/f14 zero size 2024-03-23T08:23:47.663 INFO:tasks.workunit.client.0.smithi070.stdout:5/163: read dc/f18 [570183,61297] 0 2024-03-23T08:23:47.664 INFO:tasks.workunit.client.0.smithi070.stdout:5/164: readlink dc/l1a 0 2024-03-23T08:23:47.665 INFO:tasks.workunit.client.0.smithi070.stdout:2/189: dread d1/fe [0,4194304] 0 2024-03-23T08:23:47.665 INFO:tasks.workunit.client.0.smithi070.stdout:8/208: dwrite d5/d8/fb [0,4194304] 0 2024-03-23T08:23:47.666 INFO:tasks.workunit.client.0.smithi070.stdout:5/165: chown dc/f18 12290023 1 2024-03-23T08:23:47.667 INFO:tasks.workunit.client.0.smithi070.stdout:8/209: truncate d5/d8/f4e 1027874 0 2024-03-23T08:23:47.668 INFO:tasks.workunit.client.0.smithi070.stdout:8/210: write d5/d8/f4a [275083,112048] 0 2024-03-23T08:23:47.670 INFO:tasks.workunit.client.0.smithi070.stdout:7/138: link l1b d25/d27/d2f/l35 0 2024-03-23T08:23:47.676 INFO:tasks.workunit.client.0.smithi070.stdout:3/436: creat db/d14/d9c/f9f x:0 0 0 2024-03-23T08:23:47.679 INFO:tasks.workunit.client.0.smithi070.stdout:9/283: link db/f1b db/d12/d22/d25/d26/d3b/d42/f54 0 2024-03-23T08:23:47.679 INFO:tasks.workunit.client.0.smithi070.stdout:6/356: dread d0/d23/d28/d35/d3a/d3c/f51 [0,4194304] 0 2024-03-23T08:23:47.684 INFO:tasks.workunit.client.0.smithi070.stdout:2/190: dread d1/d7/d11/f28 [0,4194304] 0 2024-03-23T08:23:47.685 INFO:tasks.workunit.client.0.smithi070.stdout:6/357: dwrite d0/f50 [4194304,4194304] 0 2024-03-23T08:23:47.687 INFO:tasks.workunit.client.0.smithi070.stdout:6/358: stat d0/d23/d1c 0 2024-03-23T08:23:47.688 INFO:tasks.workunit.client.0.smithi070.stdout:8/211: creat d5/d8/f57 x:0 0 0 2024-03-23T08:23:47.689 INFO:tasks.workunit.client.0.smithi070.stdout:8/212: dread - d5/d8/f3a zero size 2024-03-23T08:23:47.690 INFO:tasks.workunit.client.0.smithi070.stdout:1/287: link d8/db/f62 d8/db/d40/d43/f6c 0 2024-03-23T08:23:47.691 INFO:tasks.workunit.client.0.smithi070.stdout:4/127: truncate d3/d6/f8 2884832 0 2024-03-23T08:23:47.691 INFO:tasks.workunit.client.0.smithi070.stdout:1/288: dread - d8/d11/f5c zero size 2024-03-23T08:23:47.694 INFO:tasks.workunit.client.0.smithi070.stdout:7/139: mknod d25/d27/c36 0 2024-03-23T08:23:47.698 INFO:tasks.workunit.client.0.smithi070.stdout:8/213: dwrite d5/d8/f1e [0,4194304] 0 2024-03-23T08:23:47.700 INFO:tasks.workunit.client.0.smithi070.stdout:8/214: write d5/d8/d16/d32/f46 [570751,62622] 0 2024-03-23T08:23:47.701 INFO:tasks.workunit.client.0.smithi070.stdout:8/215: truncate d5/d8/f2c 517589 0 2024-03-23T08:23:47.704 INFO:tasks.workunit.client.0.smithi070.stdout:9/284: symlink db/d12/d34/d47/l55 0 2024-03-23T08:23:47.704 INFO:tasks.workunit.client.0.smithi070.stdout:6/359: dread d0/d3/fa [0,4194304] 0 2024-03-23T08:23:47.711 INFO:tasks.workunit.client.0.smithi070.stdout:0/402: getdents d16/d5f 0 2024-03-23T08:23:47.716 INFO:tasks.workunit.client.0.smithi070.stdout:4/128: creat d3/d6/d11/d1a/f30 x:0 0 0 2024-03-23T08:23:47.717 INFO:tasks.workunit.client.0.smithi070.stdout:3/437: dread db/f21 [0,4194304] 0 2024-03-23T08:23:47.718 INFO:tasks.workunit.client.0.smithi070.stdout:1/289: symlink d8/d42/d4e/l6d 0 2024-03-23T08:23:47.720 INFO:tasks.workunit.client.0.smithi070.stdout:7/140: rename d25/d27/f30 to d25/d27/f37 0 2024-03-23T08:23:47.722 INFO:tasks.workunit.client.0.smithi070.stdout:7/141: write f20 [595705,33530] 0 2024-03-23T08:23:47.724 INFO:tasks.workunit.client.0.smithi070.stdout:5/166: truncate f1 3629892 0 2024-03-23T08:23:47.724 INFO:tasks.workunit.client.0.smithi070.stdout:9/285: creat db/d12/d22/d25/d26/d32/f56 x:0 0 0 2024-03-23T08:23:47.725 INFO:tasks.workunit.client.0.smithi070.stdout:2/191: creat d1/d6/d34/f3d x:0 0 0 2024-03-23T08:23:47.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:47 smithi070 ceph-mon[39699]: pgmap v33: 65 pgs: 65 active+clean; 1.3 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 145 op/s 2024-03-23T08:23:47.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:47 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:23:47.735 INFO:tasks.workunit.client.0.smithi070.stdout:7/142: creat d25/f38 x:0 0 0 2024-03-23T08:23:47.735 INFO:tasks.workunit.client.0.smithi070.stdout:1/290: dwrite d8/f9 [4194304,4194304] 0 2024-03-23T08:23:47.739 INFO:tasks.workunit.client.0.smithi070.stdout:9/286: mknod db/d12/d22/d25/d26/c57 0 2024-03-23T08:23:47.741 INFO:tasks.workunit.client.0.smithi070.stdout:9/287: read db/d12/d14/f4e [776033,100030] 0 2024-03-23T08:23:47.745 INFO:tasks.workunit.client.0.smithi070.stdout:4/129: symlink d3/d6/l31 0 2024-03-23T08:23:47.746 INFO:tasks.workunit.client.0.smithi070.stdout:3/438: getdents db/d20/d7a/d80 0 2024-03-23T08:23:47.751 INFO:tasks.workunit.client.0.smithi070.stdout:5/167: mkdir dc/d1b/d1d/d25/d38/d39 0 2024-03-23T08:23:47.752 INFO:tasks.workunit.client.0.smithi070.stdout:0/403: write d16/d22/f24 [277116,39881] 0 2024-03-23T08:23:47.754 INFO:tasks.workunit.client.0.smithi070.stdout:6/360: rename d0/d23/d1c/l5f to d0/d23/l78 0 2024-03-23T08:23:47.756 INFO:tasks.workunit.client.0.smithi070.stdout:4/130: symlink d3/d6/d9/l32 0 2024-03-23T08:23:47.757 INFO:tasks.workunit.client.0.smithi070.stdout:4/131: truncate d3/f17 556880 0 2024-03-23T08:23:47.759 INFO:tasks.workunit.client.0.smithi070.stdout:4/132: write d3/f23 [587609,71789] 0 2024-03-23T08:23:47.759 INFO:tasks.workunit.client.0.smithi070.stdout:5/168: creat dc/d1b/d1d/f3a x:0 0 0 2024-03-23T08:23:47.761 INFO:tasks.workunit.client.0.smithi070.stdout:9/288: mkdir db/d12/d22/d25/d58 0 2024-03-23T08:23:47.764 INFO:tasks.workunit.client.0.smithi070.stdout:9/289: write db/d12/f3a [4789357,64253] 0 2024-03-23T08:23:47.766 INFO:tasks.workunit.client.0.smithi070.stdout:9/290: write db/d12/d22/d25/f24 [4371182,110446] 0 2024-03-23T08:23:47.766 INFO:tasks.workunit.client.0.smithi070.stdout:1/291: rmdir d8/d11/d46 0 2024-03-23T08:23:47.768 INFO:tasks.workunit.client.0.smithi070.stdout:0/404: dread d16/d22/f39 [0,4194304] 0 2024-03-23T08:23:47.769 INFO:tasks.workunit.client.0.smithi070.stdout:7/143: rename lc to d25/d27/l39 0 2024-03-23T08:23:47.769 INFO:tasks.workunit.client.0.smithi070.stdout:2/192: write d1/d6/d30/f3a [1899318,65457] 0 2024-03-23T08:23:47.770 INFO:tasks.workunit.client.0.smithi070.stdout:5/169: mknod dc/d1b/d1d/d25/d38/c3b 0 2024-03-23T08:23:47.771 INFO:tasks.workunit.client.0.smithi070.stdout:7/144: chown d25/d27/l33 263959 1 2024-03-23T08:23:47.771 INFO:tasks.workunit.client.0.smithi070.stdout:5/170: chown dc/d1b/d1d/d25/d38 37 1 2024-03-23T08:23:47.773 INFO:tasks.workunit.client.0.smithi070.stdout:5/171: truncate dc/f1f 416866 0 2024-03-23T08:23:47.775 INFO:tasks.workunit.client.0.smithi070.stdout:1/292: dread - d8/d1b/f4b zero size 2024-03-23T08:23:47.776 INFO:tasks.workunit.client.0.smithi070.stdout:1/293: write d8/db/d40/d43/f54 [771532,5641] 0 2024-03-23T08:23:47.778 INFO:tasks.workunit.client.0.smithi070.stdout:1/294: readlink d8/d26/l2d 0 2024-03-23T08:23:47.778 INFO:tasks.workunit.client.0.smithi070.stdout:9/291: chown db/d12/d22/d25/d26/d3b/d42/f54 348 1 2024-03-23T08:23:47.780 INFO:tasks.workunit.client.0.smithi070.stdout:5/172: dread f8 [0,4194304] 0 2024-03-23T08:23:47.780 INFO:tasks.workunit.client.0.smithi070.stdout:4/133: rename d3/d6/d11/d1a/d1b/l1f to d3/d6/d9/l33 0 2024-03-23T08:23:47.781 INFO:tasks.workunit.client.0.smithi070.stdout:0/405: mknod d16/d26/d27/d3d/c93 0 2024-03-23T08:23:47.781 INFO:tasks.workunit.client.0.smithi070.stdout:5/173: read dc/f1f [89313,90488] 0 2024-03-23T08:23:47.789 INFO:tasks.workunit.client.0.smithi070.stdout:6/361: dwrite d0/d23/d4d/f54 [0,4194304] 0 2024-03-23T08:23:47.789 INFO:tasks.workunit.client.0.smithi070.stdout:1/295: readlink d8/d1a/l57 0 2024-03-23T08:23:47.790 INFO:tasks.workunit.client.0.smithi070.stdout:1/296: stat d8/d11/f5e 0 2024-03-23T08:23:47.790 INFO:tasks.workunit.client.0.smithi070.stdout:6/362: chown d0/d23/f55 1967 1 2024-03-23T08:23:47.790 INFO:tasks.workunit.client.0.smithi070.stdout:6/363: fsync d0/d23/f55 0 2024-03-23T08:23:47.796 INFO:tasks.workunit.client.0.smithi070.stdout:2/193: rename f0 to d1/d18/f3e 0 2024-03-23T08:23:47.797 INFO:tasks.workunit.client.0.smithi070.stdout:2/194: stat d1/d7/df 0 2024-03-23T08:23:47.797 INFO:tasks.workunit.client.0.smithi070.stdout:0/406: write d16/d22/f8d [1457383,32728] 0 2024-03-23T08:23:47.798 INFO:tasks.workunit.client.0.smithi070.stdout:0/407: write d16/f51 [527594,13558] 0 2024-03-23T08:23:47.799 INFO:tasks.workunit.client.0.smithi070.stdout:0/408: fdatasync d16/d26/f6b 0 2024-03-23T08:23:47.801 INFO:tasks.workunit.client.0.smithi070.stdout:3/439: getdents db/df/d4d 0 2024-03-23T08:23:47.802 INFO:tasks.workunit.client.0.smithi070.stdout:0/409: stat d16/d26/d27/d86 0 2024-03-23T08:23:47.804 INFO:tasks.workunit.client.0.smithi070.stdout:3/440: write db/df/d15/d3d/f53 [1238797,1122] 0 2024-03-23T08:23:47.806 INFO:tasks.workunit.client.0.smithi070.stdout:2/195: dwrite d1/d18/f3c [0,4194304] 0 2024-03-23T08:23:47.808 INFO:tasks.workunit.client.0.smithi070.stdout:2/196: write d1/d7/d11/f2b [961422,39796] 0 2024-03-23T08:23:47.808 INFO:tasks.workunit.client.0.smithi070.stdout:9/292: link db/d12/d14/c2d db/d12/d22/d25/d26/d3b/c59 0 2024-03-23T08:23:47.810 INFO:tasks.workunit.client.0.smithi070.stdout:7/145: rename c18 to d25/d27/c3a 0 2024-03-23T08:23:47.811 INFO:tasks.workunit.client.0.smithi070.stdout:4/134: link d3/d6/d11/d1a/c29 d3/d19/c34 0 2024-03-23T08:23:47.811 INFO:tasks.workunit.client.0.smithi070.stdout:9/293: write db/d12/f1a [1162284,10138] 0 2024-03-23T08:23:47.812 INFO:tasks.workunit.client.0.smithi070.stdout:7/146: chown d25/d27/c29 21897 1 2024-03-23T08:23:47.813 INFO:tasks.workunit.client.0.smithi070.stdout:3/441: rmdir db/df/d6a 39 2024-03-23T08:23:47.820 INFO:tasks.workunit.client.0.smithi070.stdout:5/174: rename dc/f18 to dc/d1b/d1d/d25/d38/d39/f3c 0 2024-03-23T08:23:47.824 INFO:tasks.workunit.client.0.smithi070.stdout:4/135: mkdir d3/d6/d35 0 2024-03-23T08:23:47.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:47 smithi193 ceph-mon[48370]: pgmap v33: 65 pgs: 65 active+clean; 1.3 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 6.5 MiB/s rd, 19 MiB/s wr, 145 op/s 2024-03-23T08:23:47.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:47 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:23:47.828 INFO:tasks.workunit.client.0.smithi070.stdout:2/197: dwrite d1/d6/d34/f3d [0,4194304] 0 2024-03-23T08:23:47.829 INFO:tasks.workunit.client.0.smithi070.stdout:7/147: dwrite f1 [0,4194304] 0 2024-03-23T08:23:47.829 INFO:tasks.workunit.client.0.smithi070.stdout:6/364: rename d0/d23/f2a to d0/d23/d28/d35/d3a/d3c/d5b/f79 0 2024-03-23T08:23:47.833 INFO:tasks.workunit.client.0.smithi070.stdout:6/365: chown d0/d3/f45 4403 1 2024-03-23T08:23:47.833 INFO:tasks.workunit.client.0.smithi070.stdout:7/148: write f1d [1261029,40138] 0 2024-03-23T08:23:47.835 INFO:tasks.workunit.client.0.smithi070.stdout:3/442: dwrite db/df/d15/d22/f2c [0,4194304] 0 2024-03-23T08:23:47.840 INFO:tasks.workunit.client.0.smithi070.stdout:1/297: truncate f3 1552774 0 2024-03-23T08:23:47.845 INFO:tasks.workunit.client.0.smithi070.stdout:6/366: creat d0/d3/f7a x:0 0 0 2024-03-23T08:23:47.845 INFO:tasks.workunit.client.0.smithi070.stdout:3/443: symlink db/d20/d7a/la0 0 2024-03-23T08:23:47.845 INFO:tasks.workunit.client.0.smithi070.stdout:7/149: mknod d25/d27/c3b 0 2024-03-23T08:23:47.848 INFO:tasks.workunit.client.0.smithi070.stdout:2/198: dwrite d1/fe [0,4194304] 0 2024-03-23T08:23:47.850 INFO:tasks.workunit.client.0.smithi070.stdout:9/294: getdents db/d12/d22/d25/d26 0 2024-03-23T08:23:47.851 INFO:tasks.workunit.client.0.smithi070.stdout:1/298: creat d8/db/d2c/d3f/f6e x:0 0 0 2024-03-23T08:23:47.852 INFO:tasks.workunit.client.0.smithi070.stdout:2/199: write d1/d7/d11/f2b [578200,114276] 0 2024-03-23T08:23:47.853 INFO:tasks.workunit.client.0.smithi070.stdout:7/150: creat d25/d27/f3c x:0 0 0 2024-03-23T08:23:47.853 INFO:tasks.workunit.client.0.smithi070.stdout:1/299: dread - d8/f37 zero size 2024-03-23T08:23:47.854 INFO:tasks.workunit.client.0.smithi070.stdout:1/300: dread - d8/d1a/f39 zero size 2024-03-23T08:23:47.859 INFO:tasks.workunit.client.0.smithi070.stdout:0/410: rename d16/f3b to d16/d22/f94 0 2024-03-23T08:23:47.860 INFO:tasks.workunit.client.0.smithi070.stdout:5/175: dread dc/d1b/d1d/d25/d38/d39/f3c [0,4194304] 0 2024-03-23T08:23:47.861 INFO:tasks.workunit.client.0.smithi070.stdout:3/444: rmdir db/d20/d49 39 2024-03-23T08:23:47.866 INFO:tasks.workunit.client.0.smithi070.stdout:2/200: mknod d1/d7/df/c3f 0 2024-03-23T08:23:47.867 INFO:tasks.workunit.client.0.smithi070.stdout:2/201: write d1/d6/d34/f3d [1037044,52124] 0 2024-03-23T08:23:47.867 INFO:tasks.workunit.client.0.smithi070.stdout:7/151: symlink d25/d27/d2f/l3d 0 2024-03-23T08:23:47.868 INFO:tasks.workunit.client.0.smithi070.stdout:7/152: fsync f8 0 2024-03-23T08:23:47.869 INFO:tasks.workunit.client.0.smithi070.stdout:1/301: mkdir d8/d42/d51/d6f 0 2024-03-23T08:23:47.873 INFO:tasks.workunit.client.0.smithi070.stdout:9/295: mkdir db/d12/d22/d25/d4f/d5a 0 2024-03-23T08:23:47.874 INFO:tasks.workunit.client.0.smithi070.stdout:9/296: write db/d12/f2b [1875485,10093] 0 2024-03-23T08:23:47.928 INFO:tasks.workunit.client.0.smithi070.stdout:2/202: creat d1/d7/f40 x:0 0 0 2024-03-23T08:23:47.929 INFO:tasks.workunit.client.0.smithi070.stdout:2/203: write d1/d7/df/f27 [473783,36716] 0 2024-03-23T08:23:47.934 INFO:tasks.workunit.client.0.smithi070.stdout:6/367: truncate d0/d23/d28/d35/d3a/d3c/d5b/f5e 510072 0 2024-03-23T08:23:47.936 INFO:tasks.workunit.client.0.smithi070.stdout:3/445: dwrite db/df/f55 [0,4194304] 0 2024-03-23T08:23:47.938 INFO:tasks.workunit.client.0.smithi070.stdout:9/297: mknod db/d12/d22/d25/d26/d3b/d42/c5b 0 2024-03-23T08:23:47.939 INFO:tasks.workunit.client.0.smithi070.stdout:9/298: write db/d12/d14/f49 [1181811,37524] 0 2024-03-23T08:23:47.939 INFO:tasks.workunit.client.0.smithi070.stdout:2/204: creat d1/d7/f41 x:0 0 0 2024-03-23T08:23:47.942 INFO:tasks.workunit.client.0.smithi070.stdout:7/153: rename l1b to d25/d27/d2f/l3e 0 2024-03-23T08:23:47.944 INFO:tasks.workunit.client.0.smithi070.stdout:0/411: truncate d16/d22/f2f 3969631 0 2024-03-23T08:23:47.948 INFO:tasks.workunit.client.0.smithi070.stdout:2/205: dwrite d1/d18/f3c [0,4194304] 0 2024-03-23T08:23:47.948 INFO:tasks.workunit.client.0.smithi070.stdout:9/299: mkdir db/d12/d22/d25/d26/d5c 0 2024-03-23T08:23:47.950 INFO:tasks.workunit.client.0.smithi070.stdout:7/154: write f16 [937214,96633] 0 2024-03-23T08:23:47.950 INFO:tasks.workunit.client.0.smithi070.stdout:2/206: fdatasync d1/d6/d34/f3d 0 2024-03-23T08:23:47.961 INFO:tasks.workunit.client.0.smithi070.stdout:2/207: rename d1/d7/df/c3f to d1/d7/df/c42 0 2024-03-23T08:23:47.965 INFO:tasks.workunit.client.0.smithi070.stdout:3/446: getdents db/d20/d63 0 2024-03-23T08:23:47.966 INFO:tasks.workunit.client.0.smithi070.stdout:0/412: link d16/f3e d16/d22/d43/f95 0 2024-03-23T08:23:47.967 INFO:tasks.workunit.client.0.smithi070.stdout:7/155: dwrite d25/d27/f37 [0,4194304] 0 2024-03-23T08:23:47.975 INFO:tasks.workunit.client.0.smithi070.stdout:9/300: getdents db/d12/d34 0 2024-03-23T08:23:47.976 INFO:tasks.workunit.client.0.smithi070.stdout:0/413: dwrite d16/f46 [0,4194304] 0 2024-03-23T08:23:47.977 INFO:tasks.workunit.client.0.smithi070.stdout:0/414: chown f3 6333 1 2024-03-23T08:23:47.977 INFO:tasks.workunit.client.0.smithi070.stdout:9/301: write db/d12/d22/d25/d26/d32/f56 [80808,122976] 0 2024-03-23T08:23:47.980 INFO:tasks.workunit.client.0.smithi070.stdout:7/156: mknod d25/d27/c3f 0 2024-03-23T08:23:47.982 INFO:tasks.workunit.client.0.smithi070.stdout:2/208: mknod d1/c43 0 2024-03-23T08:23:47.987 INFO:tasks.workunit.client.0.smithi070.stdout:0/415: dwrite d16/d26/d27/d3d/d6d/f8c [0,4194304] 0 2024-03-23T08:23:47.996 INFO:tasks.workunit.client.0.smithi070.stdout:3/447: rename db/df/d15/f41 to db/df/d6a/fa1 0 2024-03-23T08:23:47.996 INFO:tasks.workunit.client.0.smithi070.stdout:2/209: mknod d1/d29/c44 0 2024-03-23T08:23:47.997 INFO:tasks.workunit.client.0.smithi070.stdout:9/302: dread f6 [0,4194304] 0 2024-03-23T08:23:47.998 INFO:tasks.workunit.client.0.smithi070.stdout:3/448: creat db/df/d6a/fa2 x:0 0 0 2024-03-23T08:23:48.003 INFO:tasks.workunit.client.0.smithi070.stdout:7/157: getdents d25/d27/d2f 0 2024-03-23T08:23:48.003 INFO:tasks.workunit.client.0.smithi070.stdout:0/416: dwrite d16/f46 [0,4194304] 0 2024-03-23T08:23:48.005 INFO:tasks.workunit.client.0.smithi070.stdout:7/158: write d25/f38 [654606,26150] 0 2024-03-23T08:23:48.007 INFO:tasks.workunit.client.0.smithi070.stdout:3/449: dread db/df/f2e [0,4194304] 0 2024-03-23T08:23:48.008 INFO:tasks.workunit.client.0.smithi070.stdout:0/417: creat d16/d22/d43/f96 x:0 0 0 2024-03-23T08:23:48.008 INFO:tasks.workunit.client.0.smithi070.stdout:7/159: write d25/f34 [3159301,73841] 0 2024-03-23T08:23:48.016 INFO:tasks.workunit.client.0.smithi070.stdout:0/418: mknod d16/c97 0 2024-03-23T08:23:48.017 INFO:tasks.workunit.client.0.smithi070.stdout:9/303: dread db/d12/d14/f4e [0,4194304] 0 2024-03-23T08:23:48.017 INFO:tasks.workunit.client.0.smithi070.stdout:7/160: mkdir d25/d40 0 2024-03-23T08:23:48.020 INFO:tasks.workunit.client.0.smithi070.stdout:0/419: symlink d16/d26/d27/l98 0 2024-03-23T08:23:48.025 INFO:tasks.workunit.client.0.smithi070.stdout:0/420: write d16/d22/f24 [731103,8016] 0 2024-03-23T08:23:48.028 INFO:tasks.workunit.client.0.smithi070.stdout:0/421: chown d16/l19 1425 1 2024-03-23T08:23:48.028 INFO:tasks.workunit.client.0.smithi070.stdout:7/161: dwrite f24 [0,4194304] 0 2024-03-23T08:23:48.031 INFO:tasks.workunit.client.0.smithi070.stdout:3/450: dread db/df/d15/f3e [0,4194304] 0 2024-03-23T08:23:48.032 INFO:tasks.workunit.client.0.smithi070.stdout:7/162: dread - d25/d27/f3c zero size 2024-03-23T08:23:48.038 INFO:tasks.workunit.client.0.smithi070.stdout:2/210: dwrite d1/d18/f37 [0,4194304] 0 2024-03-23T08:23:48.038 INFO:tasks.workunit.client.0.smithi070.stdout:2/211: chown d1/d29 0 1 2024-03-23T08:23:48.040 INFO:tasks.workunit.client.0.smithi070.stdout:3/451: unlink db/df/d15/d3d/f9b 0 2024-03-23T08:23:48.042 INFO:tasks.workunit.client.0.smithi070.stdout:9/304: dwrite f8 [0,4194304] 0 2024-03-23T08:23:48.044 INFO:tasks.workunit.client.0.smithi070.stdout:9/305: chown db/d12/d22/d25/d26/f2c 0 1 2024-03-23T08:23:48.046 INFO:tasks.workunit.client.0.smithi070.stdout:0/422: creat d16/d26/d27/d86/d67/d7d/f99 x:0 0 0 2024-03-23T08:23:48.047 INFO:tasks.workunit.client.0.smithi070.stdout:0/423: write d16/d22/d43/f96 [1023285,100292] 0 2024-03-23T08:23:48.048 INFO:tasks.workunit.client.0.smithi070.stdout:2/212: mknod d1/d6/c45 0 2024-03-23T08:23:48.050 INFO:tasks.workunit.client.0.smithi070.stdout:0/424: read - d16/d22/d43/d47/d87/f92 zero size 2024-03-23T08:23:48.052 INFO:tasks.workunit.client.0.smithi070.stdout:3/452: rename db/df/d4d/l61 to db/d4e/d92/la3 0 2024-03-23T08:23:48.055 INFO:tasks.workunit.client.0.smithi070.stdout:7/163: link f12 d25/d40/f41 0 2024-03-23T08:23:48.056 INFO:tasks.workunit.client.0.smithi070.stdout:7/164: truncate d25/d27/f3c 429843 0 2024-03-23T08:23:48.057 INFO:tasks.workunit.client.0.smithi070.stdout:0/425: mkdir d16/d26/d27/d3d/d4d/d9a 0 2024-03-23T08:23:48.058 INFO:tasks.workunit.client.0.smithi070.stdout:9/306: creat db/d12/d22/d25/d26/d5c/f5d x:0 0 0 2024-03-23T08:23:48.061 INFO:tasks.workunit.client.0.smithi070.stdout:2/213: link d1/d29/f35 d1/d6/d30/f46 0 2024-03-23T08:23:48.063 INFO:tasks.workunit.client.0.smithi070.stdout:0/426: mknod d16/d26/c9b 0 2024-03-23T08:23:48.064 INFO:tasks.workunit.client.0.smithi070.stdout:9/307: mknod db/d12/d22/d25/d26/d5c/c5e 0 2024-03-23T08:23:48.066 INFO:tasks.workunit.client.0.smithi070.stdout:7/165: dwrite f1d [0,4194304] 0 2024-03-23T08:23:48.068 INFO:tasks.workunit.client.0.smithi070.stdout:7/166: write f23 [550006,37202] 0 2024-03-23T08:23:48.068 INFO:tasks.workunit.client.0.smithi070.stdout:9/308: chown l5 16058 1 2024-03-23T08:23:48.068 INFO:tasks.workunit.client.0.smithi070.stdout:2/214: mkdir d1/d7/d39/d47 0 2024-03-23T08:23:48.069 INFO:tasks.workunit.client.0.smithi070.stdout:3/453: dwrite db/df/d1a/f37 [4194304,4194304] 0 2024-03-23T08:23:48.074 INFO:tasks.workunit.client.0.smithi070.stdout:2/215: write d1/d29/f2d [1914790,48929] 0 2024-03-23T08:23:48.077 INFO:tasks.workunit.client.0.smithi070.stdout:3/454: symlink db/d7f/la4 0 2024-03-23T08:23:48.079 INFO:tasks.workunit.client.0.smithi070.stdout:7/167: write f12 [616982,123753] 0 2024-03-23T08:23:48.080 INFO:tasks.workunit.client.0.smithi070.stdout:2/216: write d1/d6/f3b [3546701,69570] 0 2024-03-23T08:23:48.080 INFO:tasks.workunit.client.0.smithi070.stdout:0/427: dwrite d16/d22/d43/f56 [4194304,4194304] 0 2024-03-23T08:23:48.082 INFO:tasks.workunit.client.0.smithi070.stdout:9/309: creat db/d12/d22/d25/d26/d3b/d42/f5f x:0 0 0 2024-03-23T08:23:48.085 INFO:tasks.workunit.client.0.smithi070.stdout:2/217: read d1/d18/f37 [3350844,9577] 0 2024-03-23T08:23:48.087 INFO:tasks.workunit.client.0.smithi070.stdout:7/168: write d25/d27/f37 [688785,34650] 0 2024-03-23T08:23:48.090 INFO:tasks.workunit.client.0.smithi070.stdout:3/455: rename db/df/d5d/c5f to db/d14/d9c/ca5 0 2024-03-23T08:23:48.092 INFO:tasks.workunit.client.0.smithi070.stdout:9/310: dread db/d12/f39 [0,4194304] 0 2024-03-23T08:23:48.094 INFO:tasks.workunit.client.0.smithi070.stdout:2/218: mkdir d1/d7/d48 0 2024-03-23T08:23:48.097 INFO:tasks.workunit.client.0.smithi070.stdout:3/456: chown db/d4e/d92/c6b 810737 1 2024-03-23T08:23:48.097 INFO:tasks.workunit.client.0.smithi070.stdout:0/428: dwrite d16/d5f/f8a [0,4194304] 0 2024-03-23T08:23:48.098 INFO:tasks.workunit.client.0.smithi070.stdout:3/457: stat db/d4e 0 2024-03-23T08:23:48.104 INFO:tasks.workunit.client.0.smithi070.stdout:2/219: write d1/d18/f3e [4458847,20080] 0 2024-03-23T08:23:48.107 INFO:tasks.workunit.client.0.smithi070.stdout:9/311: rmdir db/d12/d22/d25/d26/d3b/d42 39 2024-03-23T08:23:48.110 INFO:tasks.workunit.client.0.smithi070.stdout:7/169: link le d25/l42 0 2024-03-23T08:23:48.111 INFO:tasks.workunit.client.0.smithi070.stdout:7/170: write f1d [2722997,24874] 0 2024-03-23T08:23:48.116 INFO:tasks.workunit.client.0.smithi070.stdout:2/220: write d1/d6/d30/f46 [1101201,65047] 0 2024-03-23T08:23:48.119 INFO:tasks.workunit.client.0.smithi070.stdout:9/312: creat db/f60 x:0 0 0 2024-03-23T08:23:48.119 INFO:tasks.workunit.client.0.smithi070.stdout:7/171: dwrite f16 [0,4194304] 0 2024-03-23T08:23:48.120 INFO:tasks.workunit.client.0.smithi070.stdout:7/172: readlink l1f 0 2024-03-23T08:23:48.121 INFO:tasks.workunit.client.0.smithi070.stdout:0/429: dread d16/d26/d27/d4c/d60/d7c/f90 [0,4194304] 0 2024-03-23T08:23:48.121 INFO:tasks.workunit.client.0.smithi070.stdout:7/173: write f8 [6771177,128733] 0 2024-03-23T08:23:48.122 INFO:tasks.workunit.client.0.smithi070.stdout:7/174: chown d25/d27/l33 248560085 1 2024-03-23T08:23:48.123 INFO:tasks.workunit.client.0.smithi070.stdout:9/313: symlink db/d12/l61 0 2024-03-23T08:23:48.126 INFO:tasks.workunit.client.0.smithi070.stdout:2/221: truncate d1/d6/d34/f3d 355965 0 2024-03-23T08:23:48.127 INFO:tasks.workunit.client.0.smithi070.stdout:0/430: creat d16/d26/d27/d4c/d60/d75/f9c x:0 0 0 2024-03-23T08:23:48.135 INFO:tasks.workunit.client.0.smithi070.stdout:9/314: creat db/d12/f62 x:0 0 0 2024-03-23T08:23:48.137 INFO:tasks.workunit.client.0.smithi070.stdout:9/315: truncate db/d12/d22/d25/d26/d32/f56 385646 0 2024-03-23T08:23:48.139 INFO:tasks.workunit.client.0.smithi070.stdout:0/431: getdents d16/d22/d43/d47/d87 0 2024-03-23T08:23:48.139 INFO:tasks.workunit.client.0.smithi070.stdout:9/316: stat db/d12/d22/d25/f44 0 2024-03-23T08:23:48.142 INFO:tasks.workunit.client.0.smithi070.stdout:9/317: creat db/d12/d14/f63 x:0 0 0 2024-03-23T08:23:48.142 INFO:tasks.workunit.client.0.smithi070.stdout:2/222: write d1/d6/d34/f3d [1213791,78105] 0 2024-03-23T08:23:48.145 INFO:tasks.workunit.client.0.smithi070.stdout:2/223: truncate d1/d18/f3c 4690135 0 2024-03-23T08:23:48.150 INFO:tasks.workunit.client.0.smithi070.stdout:0/432: getdents d16/d26/d27/d3d/d6d 0 2024-03-23T08:23:48.152 INFO:tasks.workunit.client.0.smithi070.stdout:9/318: dwrite db/d12/d14/f4e [0,4194304] 0 2024-03-23T08:23:48.154 INFO:tasks.workunit.client.0.smithi070.stdout:7/175: dread d25/d40/f41 [0,4194304] 0 2024-03-23T08:23:48.158 INFO:tasks.workunit.client.0.smithi070.stdout:0/433: unlink f11 0 2024-03-23T08:23:48.159 INFO:tasks.workunit.client.0.smithi070.stdout:2/224: dwrite d1/d7/f17 [0,4194304] 0 2024-03-23T08:23:48.161 INFO:tasks.workunit.client.0.smithi070.stdout:7/176: rmdir d25 39 2024-03-23T08:23:48.167 INFO:tasks.workunit.client.0.smithi070.stdout:9/319: dwrite db/f1f [0,4194304] 0 2024-03-23T08:23:48.169 INFO:tasks.workunit.client.0.smithi070.stdout:2/225: symlink d1/d6/d30/l49 0 2024-03-23T08:23:48.174 INFO:tasks.workunit.client.0.smithi070.stdout:7/177: dread d25/f34 [0,4194304] 0 2024-03-23T08:23:48.176 INFO:tasks.workunit.client.0.smithi070.stdout:7/178: read f20 [108003,40747] 0 2024-03-23T08:23:48.178 INFO:tasks.workunit.client.0.smithi070.stdout:9/320: creat db/d12/d22/d25/d26/f64 x:0 0 0 2024-03-23T08:23:48.182 INFO:tasks.workunit.client.0.smithi070.stdout:7/179: creat d25/f43 x:0 0 0 2024-03-23T08:23:48.182 INFO:tasks.workunit.client.0.smithi070.stdout:7/180: chown c19 31351 1 2024-03-23T08:23:48.182 INFO:tasks.workunit.client.0.smithi070.stdout:7/181: chown f24 0 1 2024-03-23T08:23:48.183 INFO:tasks.workunit.client.0.smithi070.stdout:0/434: rename c7 to d16/d22/c9d 0 2024-03-23T08:23:48.190 INFO:tasks.workunit.client.0.smithi070.stdout:7/182: mkdir d25/d27/d2f/d44 0 2024-03-23T08:23:48.191 INFO:tasks.workunit.client.0.smithi070.stdout:0/435: chown d16/d22/d43/f5a 393682 1 2024-03-23T08:23:48.194 INFO:tasks.workunit.client.0.smithi070.stdout:2/226: link d1/d7/c24 d1/d29/c4a 0 2024-03-23T08:23:48.194 INFO:tasks.workunit.client.0.smithi070.stdout:7/183: truncate fd 4387030 0 2024-03-23T08:23:48.195 INFO:tasks.workunit.client.0.smithi070.stdout:2/227: write d1/d7/f17 [2877284,79044] 0 2024-03-23T08:23:48.196 INFO:tasks.workunit.client.0.smithi070.stdout:0/436: rename d16/d5f to d16/d26/d27/d86/d9e 0 2024-03-23T08:23:48.198 INFO:tasks.workunit.client.0.smithi070.stdout:9/321: creat db/f65 x:0 0 0 2024-03-23T08:23:48.198 INFO:tasks.workunit.client.0.smithi070.stdout:2/228: dread d1/d7/df/f27 [0,4194304] 0 2024-03-23T08:23:48.200 INFO:tasks.workunit.client.0.smithi070.stdout:7/184: creat d25/d27/f45 x:0 0 0 2024-03-23T08:23:48.201 INFO:tasks.workunit.client.0.smithi070.stdout:7/185: stat f24 0 2024-03-23T08:23:48.203 INFO:tasks.workunit.client.0.smithi070.stdout:0/437: rmdir d16/d26/d27/d86/d9e 39 2024-03-23T08:23:48.205 INFO:tasks.workunit.client.0.smithi070.stdout:9/322: stat db/d12/c51 0 2024-03-23T08:23:48.206 INFO:tasks.workunit.client.0.smithi070.stdout:0/438: fdatasync d16/d26/f36 0 2024-03-23T08:23:48.207 INFO:tasks.workunit.client.0.smithi070.stdout:2/229: dwrite d1/d7/f41 [0,4194304] 0 2024-03-23T08:23:48.214 INFO:tasks.workunit.client.0.smithi070.stdout:0/439: creat d16/d22/d43/d47/f9f x:0 0 0 2024-03-23T08:23:48.214 INFO:tasks.workunit.client.0.smithi070.stdout:7/186: dwrite d25/f2c [0,4194304] 0 2024-03-23T08:23:48.215 INFO:tasks.workunit.client.0.smithi070.stdout:0/440: write d16/d26/f5d [841895,72663] 0 2024-03-23T08:23:48.220 INFO:tasks.workunit.client.0.smithi070.stdout:9/323: write db/d12/d22/d25/d26/d32/f43 [150038,6089] 0 2024-03-23T08:23:48.223 INFO:tasks.workunit.client.0.smithi070.stdout:2/230: write d1/d7/d11/f28 [2786742,117012] 0 2024-03-23T08:23:48.225 INFO:tasks.workunit.client.0.smithi070.stdout:0/441: getdents d16/d26/d27/d4c 0 2024-03-23T08:23:48.227 INFO:tasks.workunit.client.0.smithi070.stdout:0/442: write d16/d22/f23 [2419711,41006] 0 2024-03-23T08:23:48.230 INFO:tasks.workunit.client.0.smithi070.stdout:0/443: chown d16/d26/l54 429 1 2024-03-23T08:23:48.231 INFO:tasks.workunit.client.0.smithi070.stdout:9/324: dwrite db/d12/d22/d25/d26/d32/f43 [0,4194304] 0 2024-03-23T08:23:48.233 INFO:tasks.workunit.client.0.smithi070.stdout:2/231: dwrite d1/d18/f37 [0,4194304] 0 2024-03-23T08:23:48.234 INFO:tasks.workunit.client.0.smithi070.stdout:0/444: write d16/d22/f94 [1624041,34321] 0 2024-03-23T08:23:48.235 INFO:tasks.workunit.client.0.smithi070.stdout:2/232: chown d1/la 75200 1 2024-03-23T08:23:48.235 INFO:tasks.workunit.client.0.smithi070.stdout:0/445: fdatasync d16/f51 0 2024-03-23T08:23:48.235 INFO:tasks.workunit.client.0.smithi070.stdout:2/233: fdatasync d1/d7/f40 0 2024-03-23T08:23:48.239 INFO:tasks.workunit.client.0.smithi070.stdout:2/234: creat d1/d7/df/f4b x:0 0 0 2024-03-23T08:23:48.239 INFO:tasks.workunit.client.0.smithi070.stdout:9/325: getdents db/d12/d34/d47 0 2024-03-23T08:23:48.241 INFO:tasks.workunit.client.0.smithi070.stdout:9/326: write db/d12/d22/d25/d26/f2c [398666,110502] 0 2024-03-23T08:23:48.247 INFO:tasks.workunit.client.0.smithi070.stdout:9/327: creat db/d12/d22/d25/d26/d3b/f66 x:0 0 0 2024-03-23T08:23:48.251 INFO:tasks.workunit.client.0.smithi070.stdout:9/328: write db/d12/f39 [878076,124375] 0 2024-03-23T08:23:48.252 INFO:tasks.workunit.client.0.smithi070.stdout:2/235: dwrite d1/d6/d34/f3d [0,4194304] 0 2024-03-23T08:23:48.263 INFO:tasks.workunit.client.0.smithi070.stdout:9/329: write db/f1b [3032369,75354] 0 2024-03-23T08:23:48.267 INFO:tasks.workunit.client.0.smithi070.stdout:9/330: truncate db/d12/d22/d25/f44 944128 0 2024-03-23T08:23:48.268 INFO:tasks.workunit.client.0.smithi070.stdout:0/446: dread d16/d26/d27/f35 [0,4194304] 0 2024-03-23T08:23:48.270 INFO:tasks.workunit.client.0.smithi070.stdout:2/236: dwrite d1/fc [0,4194304] 0 2024-03-23T08:23:48.273 INFO:tasks.workunit.client.0.smithi070.stdout:0/447: mknod d16/d26/d27/d86/d67/ca0 0 2024-03-23T08:23:48.277 INFO:tasks.workunit.client.0.smithi070.stdout:9/331: dwrite db/d12/f1a [0,4194304] 0 2024-03-23T08:23:48.279 INFO:tasks.workunit.client.0.smithi070.stdout:0/448: creat d16/d22/d43/fa1 x:0 0 0 2024-03-23T08:23:48.280 INFO:tasks.workunit.client.0.smithi070.stdout:2/237: mkdir d1/d7/d11/d38/d4c 0 2024-03-23T08:23:48.285 INFO:tasks.workunit.client.0.smithi070.stdout:0/449: unlink d16/d22/c53 0 2024-03-23T08:23:48.286 INFO:tasks.workunit.client.0.smithi070.stdout:9/332: dwrite db/d12/d14/f35 [0,4194304] 0 2024-03-23T08:23:48.293 INFO:tasks.workunit.client.0.smithi070.stdout:9/333: mkdir db/d12/d67 0 2024-03-23T08:23:48.301 INFO:tasks.workunit.client.0.smithi070.stdout:9/334: dread db/d12/d22/d25/d26/d3b/d42/f54 [0,4194304] 0 2024-03-23T08:23:48.304 INFO:tasks.workunit.client.0.smithi070.stdout:9/335: symlink db/d12/d22/d25/d26/d3b/l68 0 2024-03-23T08:23:48.305 INFO:tasks.workunit.client.0.smithi070.stdout:9/336: write db/d12/d22/d25/d26/d3b/f66 [518987,38400] 0 2024-03-23T08:23:48.309 INFO:tasks.workunit.client.0.smithi070.stdout:9/337: symlink db/d12/d22/d25/l69 0 2024-03-23T08:23:48.315 INFO:tasks.workunit.client.0.smithi070.stdout:9/338: creat db/d12/d22/d25/f6a x:0 0 0 2024-03-23T08:23:48.317 INFO:tasks.workunit.client.0.smithi070.stdout:9/339: fdatasync db/f1b 0 2024-03-23T08:23:48.319 INFO:tasks.workunit.client.0.smithi070.stdout:9/340: fdatasync db/d12/f17 0 2024-03-23T08:23:48.320 INFO:tasks.workunit.client.0.smithi070.stdout:0/450: dwrite d16/d26/d27/d4c/d60/f7e [0,4194304] 0 2024-03-23T08:23:48.321 INFO:tasks.workunit.client.0.smithi070.stdout:0/451: fsync d16/d26/d27/d4c/d60/d75/f76 0 2024-03-23T08:23:48.322 INFO:tasks.workunit.client.0.smithi070.stdout:0/452: truncate d16/f4b 1233962 0 2024-03-23T08:23:48.323 INFO:tasks.workunit.client.0.smithi070.stdout:0/453: stat d16/d26/d27/l77 0 2024-03-23T08:23:48.325 INFO:tasks.workunit.client.0.smithi070.stdout:9/341: symlink db/d12/d22/d25/l6b 0 2024-03-23T08:23:48.329 INFO:tasks.workunit.client.0.smithi070.stdout:9/342: mkdir db/d12/d22/d25/d26/d32/d6c 0 2024-03-23T08:23:48.334 INFO:tasks.workunit.client.0.smithi070.stdout:9/343: mknod db/d12/d22/c6d 0 2024-03-23T08:23:48.336 INFO:tasks.workunit.client.0.smithi070.stdout:9/344: read db/d12/d22/d25/d26/d32/f56 [231962,127441] 0 2024-03-23T08:23:48.338 INFO:tasks.workunit.client.0.smithi070.stdout:9/345: chown db/d12/d22/d25/d26/c2e 1546723693 1 2024-03-23T08:23:48.339 INFO:tasks.workunit.client.0.smithi070.stdout:9/346: fdatasync db/d12/d22/d25/d26/d3b/f66 0 2024-03-23T08:23:48.340 INFO:tasks.workunit.client.0.smithi070.stdout:9/347: read - db/d12/d22/d25/d26/d3b/d42/f46 zero size 2024-03-23T08:23:48.341 INFO:tasks.workunit.client.0.smithi070.stdout:9/348: read - db/d12/d22/d25/d26/d3b/d42/f46 zero size 2024-03-23T08:23:48.342 INFO:tasks.workunit.client.0.smithi070.stdout:9/349: fsync db/d12/d22/d25/d26/d3b/d42/f46 0 2024-03-23T08:23:48.345 INFO:tasks.workunit.client.0.smithi070.stdout:9/350: mknod db/d12/c6e 0 2024-03-23T08:23:48.347 INFO:tasks.workunit.client.0.smithi070.stdout:9/351: truncate f4 4988956 0 2024-03-23T08:23:48.348 INFO:tasks.workunit.client.0.smithi070.stdout:0/454: dread d16/d22/d43/f95 [0,4194304] 0 2024-03-23T08:23:48.349 INFO:tasks.workunit.client.0.smithi070.stdout:9/352: symlink db/d12/d22/d25/d26/d3b/d42/l6f 0 2024-03-23T08:23:48.354 INFO:tasks.workunit.client.0.smithi070.stdout:0/455: dwrite d16/d22/f39 [0,4194304] 0 2024-03-23T08:23:48.354 INFO:tasks.workunit.client.0.smithi070.stdout:9/353: write db/d12/d22/d25/d26/d32/f56 [293341,95392] 0 2024-03-23T08:23:48.361 INFO:tasks.workunit.client.0.smithi070.stdout:0/456: symlink d16/d26/d27/d4c/la2 0 2024-03-23T08:23:48.365 INFO:tasks.workunit.client.0.smithi070.stdout:9/354: link db/d12/d22/l52 db/d12/d22/d25/d26/d3b/d42/l70 0 2024-03-23T08:23:48.367 INFO:tasks.workunit.client.0.smithi070.stdout:9/355: read db/d12/d22/d25/d26/d32/f56 [4845,58852] 0 2024-03-23T08:23:48.368 INFO:tasks.workunit.client.0.smithi070.stdout:0/457: creat d16/d22/fa3 x:0 0 0 2024-03-23T08:23:48.371 INFO:tasks.workunit.client.0.smithi070.stdout:0/458: rmdir d16/d22/d43/d47 39 2024-03-23T08:23:48.374 INFO:tasks.workunit.client.0.smithi070.stdout:9/356: getdents db/d12/d67 0 2024-03-23T08:23:48.374 INFO:tasks.workunit.client.0.smithi070.stdout:9/357: stat db/d12/d34 0 2024-03-23T08:23:48.381 INFO:tasks.workunit.client.0.smithi070.stdout:9/358: creat db/d12/d22/d25/d4f/d5a/f71 x:0 0 0 2024-03-23T08:23:48.382 INFO:tasks.workunit.client.0.smithi070.stdout:0/459: symlink d16/d26/d27/d4c/d60/d7c/la4 0 2024-03-23T08:23:48.387 INFO:tasks.workunit.client.0.smithi070.stdout:0/460: creat d16/d22/d43/d47/fa5 x:0 0 0 2024-03-23T08:23:48.391 INFO:tasks.workunit.client.0.smithi070.stdout:0/461: creat d16/d22/fa6 x:0 0 0 2024-03-23T08:23:48.392 INFO:tasks.workunit.client.0.smithi070.stdout:9/359: dread db/d12/d22/d25/f45 [0,4194304] 0 2024-03-23T08:23:48.394 INFO:tasks.workunit.client.0.smithi070.stdout:9/360: chown db/d12/d22/d25/d26/d3b/d42/c5b 1930984 1 2024-03-23T08:23:48.394 INFO:tasks.workunit.client.0.smithi070.stdout:0/462: creat d16/d26/d27/d4c/d60/d75/fa7 x:0 0 0 2024-03-23T08:23:48.396 INFO:tasks.workunit.client.0.smithi070.stdout:9/361: creat db/d12/d22/d25/d4f/f72 x:0 0 0 2024-03-23T08:23:48.396 INFO:tasks.workunit.client.0.smithi070.stdout:0/463: truncate d16/d22/d43/d47/f9f 548683 0 2024-03-23T08:23:48.399 INFO:tasks.workunit.client.0.smithi070.stdout:9/362: symlink db/d12/d22/d25/d4f/l73 0 2024-03-23T08:23:48.405 INFO:tasks.workunit.client.0.smithi070.stdout:0/464: dwrite d16/d22/d43/d47/d87/f92 [0,4194304] 0 2024-03-23T08:23:48.412 INFO:tasks.workunit.client.0.smithi070.stdout:0/465: dread d16/d26/d27/d4c/d60/f7e [0,4194304] 0 2024-03-23T08:23:48.416 INFO:tasks.workunit.client.0.smithi070.stdout:9/363: dwrite db/d12/d22/d25/d26/d32/f56 [0,4194304] 0 2024-03-23T08:23:48.419 INFO:tasks.workunit.client.0.smithi070.stdout:0/466: dread d16/d26/d27/f35 [0,4194304] 0 2024-03-23T08:23:48.420 INFO:tasks.workunit.client.0.smithi070.stdout:9/364: creat db/d12/d22/d25/d26/d32/f74 x:0 0 0 2024-03-23T08:23:48.420 INFO:tasks.workunit.client.0.smithi070.stdout:9/365: chown db/d12/f2b 107 1 2024-03-23T08:23:48.421 INFO:tasks.workunit.client.0.smithi070.stdout:0/467: truncate d16/d26/f5d 1123213 0 2024-03-23T08:23:48.423 INFO:tasks.workunit.client.0.smithi070.stdout:9/366: creat db/d12/d22/d25/d26/d32/f75 x:0 0 0 2024-03-23T08:23:48.429 INFO:tasks.workunit.client.0.smithi070.stdout:9/367: rename db/d12/d22/d25/d26/d3b/d42/l70 to db/d12/d22/d25/d26/d3b/l76 0 2024-03-23T08:23:48.436 INFO:tasks.workunit.client.0.smithi070.stdout:0/468: dread d16/f5b [0,4194304] 0 2024-03-23T08:23:48.436 INFO:tasks.workunit.client.0.smithi070.stdout:0/469: fsync d16/d26/d27/d4c/d60/d75/f76 0 2024-03-23T08:23:48.437 INFO:tasks.workunit.client.0.smithi070.stdout:0/470: chown d16/d26/f36 1029 1 2024-03-23T08:23:48.439 INFO:tasks.workunit.client.0.smithi070.stdout:0/471: read d16/d26/d27/d4c/d60/f79 [70751,60555] 0 2024-03-23T08:23:48.506 INFO:tasks.workunit.client.0.smithi070.stdout:8/216: sync 2024-03-23T08:23:48.510 INFO:tasks.workunit.client.0.smithi070.stdout:9/368: rmdir db 39 2024-03-23T08:23:48.511 INFO:tasks.workunit.client.0.smithi070.stdout:0/472: chown d16/d26/d27/d4c/d60/c7f 27 1 2024-03-23T08:23:48.516 INFO:tasks.workunit.client.0.smithi070.stdout:0/473: mkdir d16/d26/da8 0 2024-03-23T08:23:48.520 INFO:tasks.workunit.client.0.smithi070.stdout:0/474: write d16/d22/fa3 [775161,67455] 0 2024-03-23T08:23:48.521 INFO:tasks.workunit.client.0.smithi070.stdout:8/217: dwrite d5/d8/d16/d32/f46 [0,4194304] 0 2024-03-23T08:23:48.530 INFO:tasks.workunit.client.0.smithi070.stdout:0/475: rename d16/d26/d27/d3d/d6d to d16/d22/d43/d47/da9 0 2024-03-23T08:23:48.531 INFO:tasks.workunit.client.0.smithi070.stdout:8/218: dwrite d5/d8/f17 [4194304,4194304] 0 2024-03-23T08:23:48.534 INFO:tasks.workunit.client.0.smithi070.stdout:0/476: write d16/d26/d27/d4c/d60/d7c/f90 [538611,86201] 0 2024-03-23T08:23:48.535 INFO:tasks.workunit.client.0.smithi070.stdout:8/219: creat d5/d8/d16/d32/f58 x:0 0 0 2024-03-23T08:23:48.537 INFO:tasks.workunit.client.0.smithi070.stdout:8/220: symlink d5/d8/d16/d30/l59 0 2024-03-23T08:23:48.539 INFO:tasks.workunit.client.0.smithi070.stdout:8/221: symlink d5/d8/d16/l5a 0 2024-03-23T08:23:48.590 INFO:tasks.workunit.client.0.smithi070.stdout:9/369: write db/d12/d14/f23 [1822124,50789] 0 2024-03-23T08:23:48.593 INFO:tasks.workunit.client.0.smithi070.stdout:0/477: write d16/f18 [4385649,83465] 0 2024-03-23T08:23:48.594 INFO:tasks.workunit.client.0.smithi070.stdout:9/370: dread - db/d12/d22/d25/d26/d3b/d42/f46 zero size 2024-03-23T08:23:48.594 INFO:tasks.workunit.client.0.smithi070.stdout:8/222: truncate d5/d8/f2c 269541 0 2024-03-23T08:23:48.595 INFO:tasks.workunit.client.0.smithi070.stdout:8/223: readlink d5/d8/d16/l5a 0 2024-03-23T08:23:48.597 INFO:tasks.workunit.client.0.smithi070.stdout:9/371: write db/d12/d22/d25/d26/d32/f75 [979792,108209] 0 2024-03-23T08:23:48.599 INFO:tasks.workunit.client.0.smithi070.stdout:8/224: rename d5/d8/f1e to d5/d8/d16/f5b 0 2024-03-23T08:23:48.603 INFO:tasks.workunit.client.0.smithi070.stdout:9/372: symlink db/d12/d22/d25/d4f/d5a/l77 0 2024-03-23T08:23:48.605 INFO:tasks.workunit.client.0.smithi070.stdout:8/225: dwrite d5/d8/f1d [0,4194304] 0 2024-03-23T08:23:48.605 INFO:tasks.workunit.client.0.smithi070.stdout:9/373: fsync db/d12/d22/d25/d26/d5c/f5d 0 2024-03-23T08:23:48.613 INFO:tasks.workunit.client.0.smithi070.stdout:0/478: getdents d16/d22/d43 0 2024-03-23T08:23:48.617 INFO:tasks.workunit.client.0.smithi070.stdout:9/374: creat db/d12/d22/d25/f78 x:0 0 0 2024-03-23T08:23:48.618 INFO:tasks.workunit.client.0.smithi070.stdout:0/479: creat d16/d26/d27/d86/d67/d7d/faa x:0 0 0 2024-03-23T08:23:48.619 INFO:tasks.workunit.client.0.smithi070.stdout:8/226: link d5/d8/d16/f22 d5/d8/d16/d32/f5c 0 2024-03-23T08:23:48.621 INFO:tasks.workunit.client.0.smithi070.stdout:9/375: mkdir db/d12/d22/d25/d26/d32/d6c/d79 0 2024-03-23T08:23:48.622 INFO:tasks.workunit.client.0.smithi070.stdout:9/376: dread - db/f60 zero size 2024-03-23T08:23:48.626 INFO:tasks.workunit.client.0.smithi070.stdout:0/480: dwrite d16/d22/fa6 [0,4194304] 0 2024-03-23T08:23:48.627 INFO:tasks.workunit.client.0.smithi070.stdout:8/227: rmdir d5/d8/d16/d32 39 2024-03-23T08:23:48.628 INFO:tasks.workunit.client.0.smithi070.stdout:9/377: creat db/d12/d14/f7a x:0 0 0 2024-03-23T08:23:48.630 INFO:tasks.workunit.client.0.smithi070.stdout:0/481: creat d16/d26/fab x:0 0 0 2024-03-23T08:23:48.632 INFO:tasks.workunit.client.0.smithi070.stdout:9/378: mknod db/d12/d22/d25/d26/d32/c7b 0 2024-03-23T08:23:48.633 INFO:tasks.workunit.client.0.smithi070.stdout:9/379: write db/d12/d22/d25/d26/d32/f56 [3905851,71207] 0 2024-03-23T08:23:48.634 INFO:tasks.workunit.client.0.smithi070.stdout:8/228: getdents d5/d8/d16/d3b 0 2024-03-23T08:23:48.638 INFO:tasks.workunit.client.0.smithi070.stdout:8/229: dread - d5/d8/d16/d32/f56 zero size 2024-03-23T08:23:48.641 INFO:tasks.workunit.client.0.smithi070.stdout:8/230: creat d5/d8/d16/d3b/f5d x:0 0 0 2024-03-23T08:23:48.642 INFO:tasks.workunit.client.0.smithi070.stdout:9/380: creat db/d12/d22/d25/f7c x:0 0 0 2024-03-23T08:23:48.643 INFO:tasks.workunit.client.0.smithi070.stdout:8/231: read d5/d8/f17 [1070659,113163] 0 2024-03-23T08:23:48.646 INFO:tasks.workunit.client.0.smithi070.stdout:9/381: dread db/d12/d22/d25/d26/d3b/f66 [0,4194304] 0 2024-03-23T08:23:48.648 INFO:tasks.workunit.client.0.smithi070.stdout:9/382: fdatasync db/d12/d22/d25/d26/d3b/d42/f46 0 2024-03-23T08:23:48.651 INFO:tasks.workunit.client.0.smithi070.stdout:0/482: truncate d16/f1a 1165484 0 2024-03-23T08:23:48.652 INFO:tasks.workunit.client.0.smithi070.stdout:8/232: dwrite d5/d8/f57 [0,4194304] 0 2024-03-23T08:23:48.653 INFO:tasks.workunit.client.0.smithi070.stdout:8/233: read - d5/d8/f33 zero size 2024-03-23T08:23:48.654 INFO:tasks.workunit.client.0.smithi070.stdout:9/383: unlink db/d12/d22/d25/d26/d32/l4b 0 2024-03-23T08:23:48.659 INFO:tasks.workunit.client.0.smithi070.stdout:0/483: write d16/d22/d43/d47/f70 [2933374,118048] 0 2024-03-23T08:23:48.662 INFO:tasks.workunit.client.0.smithi070.stdout:8/234: symlink d5/d8/d16/d3b/l5e 0 2024-03-23T08:23:48.665 INFO:tasks.workunit.client.0.smithi070.stdout:8/235: creat d5/d8/d16/d3b/f5f x:0 0 0 2024-03-23T08:23:48.666 INFO:tasks.workunit.client.0.smithi070.stdout:9/384: creat db/d12/d22/d25/d26/d3b/f7d x:0 0 0 2024-03-23T08:23:48.667 INFO:tasks.workunit.client.0.smithi070.stdout:8/236: mknod d5/d8/d16/d3b/c60 0 2024-03-23T08:23:48.669 INFO:tasks.workunit.client.0.smithi070.stdout:8/237: rmdir d5/d8/d16/d30/d38 39 2024-03-23T08:23:48.671 INFO:tasks.workunit.client.0.smithi070.stdout:4/136: sync 2024-03-23T08:23:48.677 INFO:tasks.workunit.client.0.smithi070.stdout:4/137: read d3/d6/f8 [2102421,17185] 0 2024-03-23T08:23:48.677 INFO:tasks.workunit.client.0.smithi070.stdout:9/385: rename db/d12/d22/d25/d26/d32 to db/d12/d22/d25/d26/d5c/d7e 0 2024-03-23T08:23:48.678 INFO:tasks.workunit.client.0.smithi070.stdout:9/386: truncate db/f65 698205 0 2024-03-23T08:23:48.678 INFO:tasks.workunit.client.0.smithi070.stdout:4/138: read - d3/d6/d11/d1a/d1b/f26 zero size 2024-03-23T08:23:48.681 INFO:tasks.workunit.client.0.smithi070.stdout:0/484: dread d16/f18 [0,4194304] 0 2024-03-23T08:23:48.684 INFO:tasks.workunit.client.0.smithi070.stdout:9/387: symlink db/d12/d22/d25/d26/d3b/d42/l7f 0 2024-03-23T08:23:48.686 INFO:tasks.workunit.client.0.smithi070.stdout:8/238: write d5/d8/d16/d32/f5c [1582783,99409] 0 2024-03-23T08:23:48.686 INFO:tasks.workunit.client.0.smithi070.stdout:8/239: dread - d5/d8/d16/f44 zero size 2024-03-23T08:23:48.687 INFO:tasks.workunit.client.0.smithi070.stdout:8/240: chown d5/d8/d16/l20 15992 1 2024-03-23T08:23:48.688 INFO:tasks.workunit.client.0.smithi070.stdout:9/388: mkdir db/d12/d22/d25/d26/d80 0 2024-03-23T08:23:48.689 INFO:tasks.workunit.client.0.smithi070.stdout:0/485: mknod d16/d26/d27/d86/d9e/cac 0 2024-03-23T08:23:48.690 INFO:tasks.workunit.client.0.smithi070.stdout:4/139: write d3/d6/f8 [245801,60524] 0 2024-03-23T08:23:48.690 INFO:tasks.workunit.client.0.smithi070.stdout:0/486: chown d16/d22/f2c 431 1 2024-03-23T08:23:48.692 INFO:tasks.workunit.client.0.smithi070.stdout:8/241: dread d5/f23 [0,4194304] 0 2024-03-23T08:23:48.697 INFO:tasks.workunit.client.0.smithi070.stdout:4/140: creat d3/d6/d11/f36 x:0 0 0 2024-03-23T08:23:48.704 INFO:tasks.workunit.client.0.smithi070.stdout:4/141: stat d3/d6/d11/d1a/d1b/f1c 0 2024-03-23T08:23:48.706 INFO:tasks.workunit.client.0.smithi070.stdout:0/487: dwrite d16/d26/d27/d4c/d60/d75/fa7 [0,4194304] 0 2024-03-23T08:23:48.707 INFO:tasks.workunit.client.0.smithi070.stdout:9/389: dwrite db/d12/d22/d25/f45 [4194304,4194304] 0 2024-03-23T08:23:48.707 INFO:tasks.workunit.client.0.smithi070.stdout:8/242: dwrite d5/d8/d16/d30/f39 [0,4194304] 0 2024-03-23T08:23:48.707 INFO:tasks.workunit.client.0.smithi070.stdout:0/488: chown le 0 1 2024-03-23T08:23:48.711 INFO:tasks.workunit.client.0.smithi070.stdout:8/243: read - d5/d8/d16/f51 zero size 2024-03-23T08:23:48.716 INFO:tasks.workunit.client.0.smithi070.stdout:8/244: chown d5/d8/d16/f5b 1320345093 1 2024-03-23T08:23:48.717 INFO:tasks.workunit.client.0.smithi070.stdout:9/390: fdatasync db/d12/d22/d25/d26/d5c/f5d 0 2024-03-23T08:23:48.721 INFO:tasks.workunit.client.0.smithi070.stdout:0/489: dwrite d16/d22/f29 [0,4194304] 0 2024-03-23T08:23:48.723 INFO:tasks.workunit.client.0.smithi070.stdout:9/391: write db/d12/d22/d25/d4f/f72 [707549,98947] 0 2024-03-23T08:23:48.724 INFO:tasks.workunit.client.0.smithi070.stdout:0/490: stat d16/d22/d43/l84 0 2024-03-23T08:23:48.724 INFO:tasks.workunit.client.0.smithi070.stdout:4/142: dwrite d3/d6/d11/f18 [0,4194304] 0 2024-03-23T08:23:48.725 INFO:tasks.workunit.client.0.smithi070.stdout:0/491: chown d16/d26/c63 74166 1 2024-03-23T08:23:48.727 INFO:tasks.workunit.client.0.smithi070.stdout:8/245: creat d5/d8/d16/d30/f61 x:0 0 0 2024-03-23T08:23:48.731 INFO:tasks.workunit.client.0.smithi070.stdout:4/143: creat d3/d19/d2a/f37 x:0 0 0 2024-03-23T08:23:48.733 INFO:tasks.workunit.client.0.smithi070.stdout:0/492: dread d16/d22/d43/d47/f9f [0,4194304] 0 2024-03-23T08:23:48.735 INFO:tasks.workunit.client.0.smithi070.stdout:0/493: dread d16/d22/d43/d47/f9f [0,4194304] 0 2024-03-23T08:23:48.736 INFO:tasks.workunit.client.0.smithi070.stdout:4/144: readlink d3/d6/d11/d1a/d1b/l1d 0 2024-03-23T08:23:48.737 INFO:tasks.workunit.client.0.smithi070.stdout:0/494: write d16/d26/d27/f35 [3699705,126351] 0 2024-03-23T08:23:48.737 INFO:tasks.workunit.client.0.smithi070.stdout:8/246: write d5/f43 [343131,93668] 0 2024-03-23T08:23:48.739 INFO:tasks.workunit.client.0.smithi070.stdout:8/247: chown d5/f7 2991663 1 2024-03-23T08:23:48.742 INFO:tasks.workunit.client.0.smithi070.stdout:4/145: creat d3/d6/f38 x:0 0 0 2024-03-23T08:23:48.743 INFO:tasks.workunit.client.0.smithi070.stdout:0/495: getdents d16/d26/d27/d86 0 2024-03-23T08:23:48.747 INFO:tasks.workunit.client.0.smithi070.stdout:8/248: dwrite d5/d8/f25 [0,4194304] 0 2024-03-23T08:23:48.749 INFO:tasks.workunit.client.0.smithi070.stdout:4/146: symlink d3/d6/d35/l39 0 2024-03-23T08:23:48.751 INFO:tasks.workunit.client.0.smithi070.stdout:0/496: mkdir d16/d26/d27/d3d/d4d/dad 0 2024-03-23T08:23:48.751 INFO:tasks.workunit.client.0.smithi070.stdout:4/147: fdatasync d3/d6/d11/d1a/f30 0 2024-03-23T08:23:48.758 INFO:tasks.workunit.client.0.smithi070.stdout:8/249: mkdir d5/d8/d16/d30/d38/d62 0 2024-03-23T08:23:48.759 INFO:tasks.workunit.client.0.smithi070.stdout:4/148: dwrite d3/fc [0,4194304] 0 2024-03-23T08:23:48.760 INFO:tasks.workunit.client.0.smithi070.stdout:4/149: chown d3/d6/d11/f2d 133999168 1 2024-03-23T08:23:48.760 INFO:tasks.workunit.client.0.smithi070.stdout:8/250: symlink d5/l63 0 2024-03-23T08:23:48.765 INFO:tasks.workunit.client.0.smithi070.stdout:8/251: rename d5/d8/d16/l4b to d5/d8/d16/d3b/d4d/l64 0 2024-03-23T08:23:48.767 INFO:tasks.workunit.client.0.smithi070.stdout:4/150: dwrite d3/d6/f8 [0,4194304] 0 2024-03-23T08:23:48.767 INFO:tasks.workunit.client.0.smithi070.stdout:0/497: write d16/d22/d43/d47/f9f [671361,47019] 0 2024-03-23T08:23:48.768 INFO:tasks.workunit.client.0.smithi070.stdout:8/252: write d5/d8/f13 [4861433,97056] 0 2024-03-23T08:23:48.770 INFO:tasks.workunit.client.0.smithi070.stdout:4/151: write d3/d6/f2f [340609,94951] 0 2024-03-23T08:23:48.773 INFO:tasks.workunit.client.0.smithi070.stdout:8/253: creat d5/d8/d16/d3b/f65 x:0 0 0 2024-03-23T08:23:48.774 INFO:tasks.workunit.client.0.smithi070.stdout:8/254: fdatasync d5/d8/d16/f51 0 2024-03-23T08:23:48.779 INFO:tasks.workunit.client.0.smithi070.stdout:0/498: creat d16/d26/d27/d4c/fae x:0 0 0 2024-03-23T08:23:48.781 INFO:tasks.workunit.client.0.smithi070.stdout:4/152: link d3/d6/d9/l33 d3/l3a 0 2024-03-23T08:23:48.782 INFO:tasks.workunit.client.0.smithi070.stdout:0/499: mkdir d16/d26/d27/d4c/d60/d7c/daf 0 2024-03-23T08:23:48.785 INFO:tasks.workunit.client.0.smithi070.stdout:8/255: rmdir d5/d8/d16/d30/d38/d62 0 2024-03-23T08:23:48.787 INFO:tasks.workunit.client.0.smithi070.stdout:0/500: dread d16/d22/d43/d47/d87/f92 [0,4194304] 0 2024-03-23T08:23:48.788 INFO:tasks.workunit.client.0.smithi070.stdout:8/256: mknod d5/d8/d16/d30/d38/c66 0 2024-03-23T08:23:48.789 INFO:tasks.workunit.client.0.smithi070.stdout:4/153: creat d3/d6/f3b x:0 0 0 2024-03-23T08:23:48.791 INFO:tasks.workunit.client.0.smithi070.stdout:4/154: dread d3/f17 [0,4194304] 0 2024-03-23T08:23:48.792 INFO:tasks.workunit.client.0.smithi070.stdout:8/257: creat d5/d8/d16/d32/f67 x:0 0 0 2024-03-23T08:23:48.796 INFO:tasks.workunit.client.0.smithi070.stdout:0/501: symlink d16/d26/d27/d3d/d4d/d9a/lb0 0 2024-03-23T08:23:48.799 INFO:tasks.workunit.client.0.smithi070.stdout:4/155: rename d3/d6/f8 to d3/d6/d11/d1a/d1b/f3c 0 2024-03-23T08:23:48.806 INFO:tasks.workunit.client.0.smithi070.stdout:4/156: dwrite d3/d6/d11/d1a/f30 [0,4194304] 0 2024-03-23T08:23:48.812 INFO:tasks.workunit.client.0.smithi070.stdout:8/258: rename d5/d8/d16/d32/f55 to d5/d8/d16/d30/f68 0 2024-03-23T08:23:48.814 INFO:tasks.workunit.client.0.smithi070.stdout:4/157: symlink d3/l3d 0 2024-03-23T08:23:48.817 INFO:tasks.workunit.client.0.smithi070.stdout:0/502: rename d16/c17 to d16/d26/d27/d4c/d60/d7c/daf/cb1 0 2024-03-23T08:23:48.822 INFO:tasks.workunit.client.0.smithi070.stdout:8/259: rmdir d5/d8/d16/d3b 39 2024-03-23T08:23:48.822 INFO:tasks.workunit.client.0.smithi070.stdout:4/158: creat d3/d6/d11/d1a/f3e x:0 0 0 2024-03-23T08:23:48.823 INFO:tasks.workunit.client.0.smithi070.stdout:0/503: rmdir d16/d26/d27/d4c/d60/d7c/daf 39 2024-03-23T08:23:48.823 INFO:tasks.workunit.client.0.smithi070.stdout:8/260: fdatasync d5/d8/f1d 0 2024-03-23T08:23:48.824 INFO:tasks.workunit.client.0.smithi070.stdout:0/504: readlink d16/d22/d43/l8f 0 2024-03-23T08:23:48.835 INFO:tasks.workunit.client.0.smithi070.stdout:4/159: write d3/d6/d11/d1a/d1b/f3c [4538876,46432] 0 2024-03-23T08:23:48.838 INFO:tasks.workunit.client.0.smithi070.stdout:8/261: creat d5/d8/d16/d3b/d4d/f69 x:0 0 0 2024-03-23T08:23:48.839 INFO:tasks.workunit.client.0.smithi070.stdout:8/262: write d5/d8/f25 [701523,14804] 0 2024-03-23T08:23:48.840 INFO:tasks.workunit.client.0.smithi070.stdout:4/160: write d3/d6/d11/d1a/d1b/f28 [1753524,96927] 0 2024-03-23T08:23:48.844 INFO:tasks.workunit.client.0.smithi070.stdout:0/505: getdents d16 0 2024-03-23T08:23:48.844 INFO:tasks.workunit.client.0.smithi070.stdout:4/161: mkdir d3/d6/d35/d3f 0 2024-03-23T08:23:48.847 INFO:tasks.workunit.client.0.smithi070.stdout:8/263: symlink d5/d8/d16/d30/d38/l6a 0 2024-03-23T08:23:48.852 INFO:tasks.workunit.client.0.smithi070.stdout:4/162: rename d3/d6/d11/d1a/d1b/f3c to d3/d6/d11/d1a/f40 0 2024-03-23T08:23:48.853 INFO:tasks.workunit.client.0.smithi070.stdout:8/264: dwrite d5/f23 [0,4194304] 0 2024-03-23T08:23:48.854 INFO:tasks.workunit.client.0.smithi070.stdout:4/163: dread - d3/d6/d11/d1a/f3e zero size 2024-03-23T08:23:48.855 INFO:tasks.workunit.client.0.smithi070.stdout:1/302: sync 2024-03-23T08:23:48.856 INFO:tasks.workunit.client.0.smithi070.stdout:5/176: sync 2024-03-23T08:23:48.856 INFO:tasks.workunit.client.0.smithi070.stdout:6/368: sync 2024-03-23T08:23:48.856 INFO:tasks.workunit.client.0.smithi070.stdout:0/506: symlink d16/d26/d27/d4c/lb2 0 2024-03-23T08:23:48.858 INFO:tasks.workunit.client.0.smithi070.stdout:6/369: chown d0/d23/d1c/d38/d52 0 1 2024-03-23T08:23:48.858 INFO:tasks.workunit.client.0.smithi070.stdout:4/164: dread d3/d6/d9/f2e [0,4194304] 0 2024-03-23T08:23:48.861 INFO:tasks.workunit.client.0.smithi070.stdout:6/370: stat d0 0 2024-03-23T08:23:48.863 INFO:tasks.workunit.client.0.smithi070.stdout:0/507: write d16/f18 [3195771,28600] 0 2024-03-23T08:23:48.866 INFO:tasks.workunit.client.0.smithi070.stdout:8/265: dwrite d5/d8/d16/d32/f67 [0,4194304] 0 2024-03-23T08:23:48.867 INFO:tasks.workunit.client.0.smithi070.stdout:8/266: chown d5/c9 4145675 1 2024-03-23T08:23:48.868 INFO:tasks.workunit.client.0.smithi070.stdout:1/303: mkdir d8/db/d40/d43/d44/d5d/d70 0 2024-03-23T08:23:48.873 INFO:tasks.workunit.client.0.smithi070.stdout:8/267: unlink d5/d8/d16/d3b/l5e 0 2024-03-23T08:23:48.876 INFO:tasks.workunit.client.0.smithi070.stdout:7/187: sync 2024-03-23T08:23:48.876 INFO:tasks.workunit.client.0.smithi070.stdout:2/238: sync 2024-03-23T08:23:48.876 INFO:tasks.workunit.client.0.smithi070.stdout:9/392: sync 2024-03-23T08:23:48.876 INFO:tasks.workunit.client.0.smithi070.stdout:6/371: dread d0/d23/d28/d35/d3a/d3c/f53 [0,4194304] 0 2024-03-23T08:23:48.877 INFO:tasks.workunit.client.0.smithi070.stdout:3/458: sync 2024-03-23T08:23:48.877 INFO:tasks.workunit.client.0.smithi070.stdout:6/372: dread - d0/d23/f55 zero size 2024-03-23T08:23:48.878 INFO:tasks.workunit.client.0.smithi070.stdout:1/304: creat d8/db/d40/d43/d44/f71 x:0 0 0 2024-03-23T08:23:48.878 INFO:tasks.workunit.client.0.smithi070.stdout:7/188: chown d25/c28 4457718 1 2024-03-23T08:23:48.879 INFO:tasks.workunit.client.0.smithi070.stdout:8/268: mkdir d5/d8/d16/d30/d38/d6b 0 2024-03-23T08:23:48.879 INFO:tasks.workunit.client.0.smithi070.stdout:1/305: readlink d8/d1b/l3a 0 2024-03-23T08:23:48.881 INFO:tasks.workunit.client.0.smithi070.stdout:4/165: dread d3/d6/d9/f22 [0,4194304] 0 2024-03-23T08:23:48.881 INFO:tasks.workunit.client.0.smithi070.stdout:6/373: dread d0/d23/d1c/f5d [0,4194304] 0 2024-03-23T08:23:48.882 INFO:tasks.workunit.client.0.smithi070.stdout:4/166: read - d3/d6/f38 zero size 2024-03-23T08:23:48.884 INFO:tasks.workunit.client.0.smithi070.stdout:2/239: creat d1/d6/f4d x:0 0 0 2024-03-23T08:23:48.885 INFO:tasks.workunit.client.0.smithi070.stdout:5/177: write dc/d1b/d1d/f37 [2621671,21774] 0 2024-03-23T08:23:48.887 INFO:tasks.workunit.client.0.smithi070.stdout:3/459: stat db/d14/d9c/ca5 0 2024-03-23T08:23:48.888 INFO:tasks.workunit.client.0.smithi070.stdout:7/189: mknod d25/d27/c46 0 2024-03-23T08:23:48.889 INFO:tasks.workunit.client.0.smithi070.stdout:7/190: chown d25/d27/c29 2401 1 2024-03-23T08:23:48.889 INFO:tasks.workunit.client.0.smithi070.stdout:7/191: chown f16 444038 1 2024-03-23T08:23:48.892 INFO:tasks.workunit.client.0.smithi070.stdout:1/306: chown d8/db/f1f 11 1 2024-03-23T08:23:48.892 INFO:tasks.workunit.client.0.smithi070.stdout:0/508: write d16/d22/f39 [243700,118891] 0 2024-03-23T08:23:48.894 INFO:tasks.workunit.client.0.smithi070.stdout:8/269: symlink d5/d8/d16/d32/l6c 0 2024-03-23T08:23:48.896 INFO:tasks.workunit.client.0.smithi070.stdout:6/374: rename d0/d23/d1c/l43 to d0/d23/d4d/l7b 0 2024-03-23T08:23:48.897 INFO:tasks.workunit.client.0.smithi070.stdout:4/167: mkdir d3/d6/d11/d1a/d41 0 2024-03-23T08:23:48.898 INFO:tasks.workunit.client.0.smithi070.stdout:9/393: symlink db/d12/d22/d25/l81 0 2024-03-23T08:23:48.898 INFO:tasks.workunit.client.0.smithi070.stdout:0/509: dread d16/d22/d43/f95 [0,4194304] 0 2024-03-23T08:23:48.900 INFO:tasks.workunit.client.0.smithi070.stdout:9/394: write db/d12/f39 [1512823,16481] 0 2024-03-23T08:23:48.900 INFO:tasks.workunit.client.0.smithi070.stdout:5/178: creat dc/f3d x:0 0 0 2024-03-23T08:23:48.900 INFO:tasks.workunit.client.0.smithi070.stdout:0/510: stat d16/d26/d27/d86 0 2024-03-23T08:23:48.901 INFO:tasks.workunit.client.0.smithi070.stdout:8/270: readlink d5/lf 0 2024-03-23T08:23:48.903 INFO:tasks.workunit.client.0.smithi070.stdout:7/192: rename d25/l26 to d25/d27/d2f/d44/l47 0 2024-03-23T08:23:48.904 INFO:tasks.workunit.client.0.smithi070.stdout:4/168: readlink d3/l3a 0 2024-03-23T08:23:48.904 INFO:tasks.workunit.client.0.smithi070.stdout:7/193: fdatasync f16 0 2024-03-23T08:23:48.905 INFO:tasks.workunit.client.0.smithi070.stdout:6/375: creat d0/d23/d1c/d38/d52/f7c x:0 0 0 2024-03-23T08:23:48.906 INFO:tasks.workunit.client.0.smithi070.stdout:2/240: creat d1/d7/d11/d38/d4c/f4e x:0 0 0 2024-03-23T08:23:48.909 INFO:tasks.workunit.client.0.smithi070.stdout:3/460: truncate db/df/d15/d3d/f56 610504 0 2024-03-23T08:23:48.910 INFO:tasks.workunit.client.0.smithi070.stdout:5/179: readlink dc/l26 0 2024-03-23T08:23:48.910 INFO:tasks.workunit.client.0.smithi070.stdout:2/241: truncate d1/d7/f20 4837120 0 2024-03-23T08:23:48.912 INFO:tasks.workunit.client.0.smithi070.stdout:3/461: fdatasync db/df/f55 0 2024-03-23T08:23:48.912 INFO:tasks.workunit.client.0.smithi070.stdout:9/395: dwrite db/d12/d14/f35 [0,4194304] 0 2024-03-23T08:23:48.913 INFO:tasks.workunit.client.0.smithi070.stdout:1/307: dread d8/d1b/f25 [0,4194304] 0 2024-03-23T08:23:48.914 INFO:tasks.workunit.client.0.smithi070.stdout:2/242: write d1/d29/f2d [2135833,63520] 0 2024-03-23T08:23:48.916 INFO:tasks.workunit.client.0.smithi070.stdout:0/511: mknod d16/d26/d27/cb3 0 2024-03-23T08:23:48.921 INFO:tasks.workunit.client.0.smithi070.stdout:5/180: dwrite dc/f2b [0,4194304] 0 2024-03-23T08:23:48.923 INFO:tasks.workunit.client.0.smithi070.stdout:7/194: creat d25/d27/f48 x:0 0 0 2024-03-23T08:23:48.926 INFO:tasks.workunit.client.0.smithi070.stdout:6/376: creat d0/d23/d28/f7d x:0 0 0 2024-03-23T08:23:48.927 INFO:tasks.workunit.client.0.smithi070.stdout:4/169: mknod d3/d19/d2a/c42 0 2024-03-23T08:23:48.927 INFO:tasks.workunit.client.0.smithi070.stdout:6/377: write d0/d3/d2c/f58 [445759,55258] 0 2024-03-23T08:23:48.934 INFO:tasks.workunit.client.0.smithi070.stdout:4/170: dwrite d3/d6/f2f [0,4194304] 0 2024-03-23T08:23:49.020 INFO:tasks.workunit.client.0.smithi070.stdout:8/271: creat d5/d8/d16/d30/d38/d6b/f6d x:0 0 0 2024-03-23T08:23:49.023 INFO:tasks.workunit.client.0.smithi070.stdout:2/243: rename d1/d18 to d1/d6/d4f 0 2024-03-23T08:23:49.024 INFO:tasks.workunit.client.0.smithi070.stdout:1/308: dwrite d8/db/f45 [0,4194304] 0 2024-03-23T08:23:49.024 INFO:tasks.workunit.client.0.smithi070.stdout:2/244: fdatasync d1/d29/f2d 0 2024-03-23T08:23:49.087 INFO:tasks.workunit.client.0.smithi070.stdout:0/512: write d16/d22/d43/f5a [4797394,126115] 0 2024-03-23T08:23:49.087 INFO:tasks.workunit.client.0.smithi070.stdout:6/378: creat d0/d3/f7e x:0 0 0 2024-03-23T08:23:49.088 INFO:tasks.workunit.client.0.smithi070.stdout:2/245: symlink d1/d6/d4f/l50 0 2024-03-23T08:23:49.088 INFO:tasks.workunit.client.0.smithi070.stdout:1/309: symlink d8/db/d40/d43/d44/d5d/l72 0 2024-03-23T08:23:49.089 INFO:tasks.workunit.client.0.smithi070.stdout:6/379: stat d0/d23/d1c/d2f 0 2024-03-23T08:23:49.089 INFO:tasks.workunit.client.0.smithi070.stdout:0/513: chown d16/d22/d43/fa1 244 1 2024-03-23T08:23:49.090 INFO:tasks.workunit.client.0.smithi070.stdout:3/462: truncate db/df/d1a/f44 996003 0 2024-03-23T08:23:49.091 INFO:tasks.workunit.client.0.smithi070.stdout:7/195: unlink fd 0 2024-03-23T08:23:49.092 INFO:tasks.workunit.client.0.smithi070.stdout:9/396: rename db/d12/f1a to db/d12/f82 0 2024-03-23T08:23:49.093 INFO:tasks.workunit.client.0.smithi070.stdout:6/380: write d0/d23/d28/d35/f46 [3120849,112268] 0 2024-03-23T08:23:49.095 INFO:tasks.workunit.client.0.smithi070.stdout:8/272: symlink d5/d8/d16/l6e 0 2024-03-23T08:23:49.098 INFO:tasks.workunit.client.0.smithi070.stdout:4/171: creat d3/d6/f43 x:0 0 0 2024-03-23T08:23:49.099 INFO:tasks.workunit.client.0.smithi070.stdout:4/172: fdatasync d3/d6/f7 0 2024-03-23T08:23:49.099 INFO:tasks.workunit.client.0.smithi070.stdout:1/310: rmdir d8/db/d40/d43/d44/d5d 39 2024-03-23T08:23:49.100 INFO:tasks.workunit.client.0.smithi070.stdout:1/311: chown d8/d1b 111 1 2024-03-23T08:23:49.101 INFO:tasks.workunit.client.0.smithi070.stdout:3/463: dwrite db/df/d1a/f58 [0,4194304] 0 2024-03-23T08:23:49.102 INFO:tasks.workunit.client.0.smithi070.stdout:1/312: write d8/d1b/f33 [1184187,110261] 0 2024-03-23T08:23:49.103 INFO:tasks.workunit.client.0.smithi070.stdout:7/196: symlink d25/l49 0 2024-03-23T08:23:49.107 INFO:tasks.workunit.client.0.smithi070.stdout:1/313: dread d8/db/f6a [0,4194304] 0 2024-03-23T08:23:49.110 INFO:tasks.workunit.client.0.smithi070.stdout:0/514: mknod d16/d22/d43/d47/da9/cb4 0 2024-03-23T08:23:49.113 INFO:tasks.workunit.client.0.smithi070.stdout:0/515: read f13 [252437,106142] 0 2024-03-23T08:23:49.113 INFO:tasks.workunit.client.0.smithi070.stdout:2/246: mkdir d1/d7/d48/d51 0 2024-03-23T08:23:49.114 INFO:tasks.workunit.client.0.smithi070.stdout:2/247: chown d1/d21/l31 1519585407 1 2024-03-23T08:23:49.116 INFO:tasks.workunit.client.0.smithi070.stdout:8/273: mkdir d5/d8/d16/d3b/d4d/d6f 0 2024-03-23T08:23:49.119 INFO:tasks.workunit.client.0.smithi070.stdout:4/173: symlink d3/d6/d35/l44 0 2024-03-23T08:23:49.119 INFO:tasks.workunit.client.0.smithi070.stdout:3/464: mknod db/d4e/ca6 0 2024-03-23T08:23:49.121 INFO:tasks.workunit.client.0.smithi070.stdout:7/197: creat d25/d27/d2f/f4a x:0 0 0 2024-03-23T08:23:49.123 INFO:tasks.workunit.client.0.smithi070.stdout:1/314: creat d8/d11/f73 x:0 0 0 2024-03-23T08:23:49.123 INFO:tasks.workunit.client.0.smithi070.stdout:9/397: symlink db/d12/d22/d25/d26/d80/l83 0 2024-03-23T08:23:49.124 INFO:tasks.workunit.client.0.smithi070.stdout:4/174: dread d3/d6/d11/d1a/f40 [0,4194304] 0 2024-03-23T08:23:49.124 INFO:tasks.workunit.client.0.smithi070.stdout:9/398: chown db/d12/d22/d25/d4f/d5a 3 1 2024-03-23T08:23:49.125 INFO:tasks.workunit.client.0.smithi070.stdout:4/175: write d3/d6/d11/d1a/d1b/f28 [376869,104175] 0 2024-03-23T08:23:49.126 INFO:tasks.workunit.client.0.smithi070.stdout:9/399: chown db/d12/d22/d25/d26/d80/l83 48829133 1 2024-03-23T08:23:49.127 INFO:tasks.workunit.client.0.smithi070.stdout:6/381: link d0/d23/d1c/f2b d0/d23/d1c/d38/d52/f7f 0 2024-03-23T08:23:49.127 INFO:tasks.workunit.client.0.smithi070.stdout:9/400: fsync db/d12/d22/d25/d26/d5c/f5d 0 2024-03-23T08:23:49.128 INFO:tasks.workunit.client.0.smithi070.stdout:8/274: mknod d5/d8/d16/d32/c70 0 2024-03-23T08:23:49.129 INFO:tasks.workunit.client.0.smithi070.stdout:8/275: chown d5/d8/d16/l5a 290519793 1 2024-03-23T08:23:49.130 INFO:tasks.workunit.client.0.smithi070.stdout:8/276: dread - d5/d8/f33 zero size 2024-03-23T08:23:49.132 INFO:tasks.workunit.client.0.smithi070.stdout:9/401: dread db/f1f [0,4194304] 0 2024-03-23T08:23:49.136 INFO:tasks.workunit.client.0.smithi070.stdout:3/465: dread db/df/d15/d3d/f56 [0,4194304] 0 2024-03-23T08:23:49.137 INFO:tasks.workunit.client.0.smithi070.stdout:7/198: truncate f16 555672 0 2024-03-23T08:23:49.139 INFO:tasks.workunit.client.0.smithi070.stdout:8/277: rename d5/d8/c3f to d5/d8/c71 0 2024-03-23T08:23:49.139 INFO:tasks.workunit.client.0.smithi070.stdout:8/278: stat d5/d8 0 2024-03-23T08:23:49.142 INFO:tasks.workunit.client.0.smithi070.stdout:6/382: getdents d0/d23/d74 0 2024-03-23T08:23:49.149 INFO:tasks.workunit.client.0.smithi070.stdout:8/279: mkdir d5/d8/d16/d30/d38/d6b/d72 0 2024-03-23T08:23:49.151 INFO:tasks.workunit.client.0.smithi070.stdout:3/466: dread db/d14/f1b [0,4194304] 0 2024-03-23T08:23:49.152 INFO:tasks.workunit.client.0.smithi070.stdout:0/516: getdents d16 0 2024-03-23T08:23:49.155 INFO:tasks.workunit.client.0.smithi070.stdout:1/315: getdents d8 0 2024-03-23T08:23:49.156 INFO:tasks.workunit.client.0.smithi070.stdout:6/383: dread d0/d23/d28/d35/d3a/d3c/d5b/f5e [0,4194304] 0 2024-03-23T08:23:49.157 INFO:tasks.workunit.client.0.smithi070.stdout:6/384: chown d0/d3/c7 24 1 2024-03-23T08:23:49.158 INFO:tasks.workunit.client.0.smithi070.stdout:8/280: creat d5/d8/f73 x:0 0 0 2024-03-23T08:23:49.158 INFO:tasks.workunit.client.0.smithi070.stdout:6/385: dread - d0/d23/d28/d35/f77 zero size 2024-03-23T08:23:49.160 INFO:tasks.workunit.client.0.smithi070.stdout:6/386: truncate d0/d23/d1c/f5d 740897 0 2024-03-23T08:23:49.160 INFO:tasks.workunit.client.0.smithi070.stdout:0/517: creat d16/d26/d27/d86/d9e/fb5 x:0 0 0 2024-03-23T08:23:49.163 INFO:tasks.workunit.client.0.smithi070.stdout:0/518: stat d16/d22/f29 0 2024-03-23T08:23:49.166 INFO:tasks.workunit.client.0.smithi070.stdout:1/316: rename d8/db/d40/d43/d44 to d8/d11/d61/d74 0 2024-03-23T08:23:49.167 INFO:tasks.workunit.client.0.smithi070.stdout:8/281: dwrite d5/d8/d16/f22 [4194304,4194304] 0 2024-03-23T08:23:49.171 INFO:tasks.workunit.client.0.smithi070.stdout:9/402: dwrite db/d12/f17 [0,4194304] 0 2024-03-23T08:23:49.171 INFO:tasks.workunit.client.0.smithi070.stdout:8/282: chown d5/d8/d16/d30/f61 57 1 2024-03-23T08:23:49.171 INFO:tasks.workunit.client.0.smithi070.stdout:3/467: mknod db/df/ca7 0 2024-03-23T08:23:49.173 INFO:tasks.workunit.client.0.smithi070.stdout:8/283: stat d5/d8/d16/d32/c70 0 2024-03-23T08:23:49.178 INFO:tasks.workunit.client.0.smithi070.stdout:7/199: dwrite f16 [0,4194304] 0 2024-03-23T08:23:49.178 INFO:tasks.workunit.client.0.smithi070.stdout:0/519: mkdir d16/d26/d27/d86/db6 0 2024-03-23T08:23:49.182 INFO:tasks.workunit.client.0.smithi070.stdout:1/317: rename d8/d11/f5e to d8/d11/d61/f75 0 2024-03-23T08:23:49.183 INFO:tasks.workunit.client.0.smithi070.stdout:3/468: mkdir db/d14/d65/da8 0 2024-03-23T08:23:49.185 INFO:tasks.workunit.client.0.smithi070.stdout:9/403: chown db/d12/d22/d25/l1c 1490062 1 2024-03-23T08:23:49.187 INFO:tasks.workunit.client.0.smithi070.stdout:9/404: dread db/d12/d22/d25/d26/d3b/f66 [0,4194304] 0 2024-03-23T08:23:49.188 INFO:tasks.workunit.client.0.smithi070.stdout:8/284: unlink d5/d8/f13 0 2024-03-23T08:23:49.189 INFO:tasks.workunit.client.0.smithi070.stdout:6/387: write d0/d23/d28/d35/d3a/d3c/d5b/f5e [1383438,58983] 0 2024-03-23T08:23:49.192 INFO:tasks.workunit.client.0.smithi070.stdout:8/285: dread d5/d8/f4e [0,4194304] 0 2024-03-23T08:23:49.192 INFO:tasks.workunit.client.0.smithi070.stdout:7/200: readlink d25/d27/d2f/l3e 0 2024-03-23T08:23:49.195 INFO:tasks.workunit.client.0.smithi070.stdout:0/520: rename d16/d26/d27/l98 to d16/d26/da8/lb7 0 2024-03-23T08:23:49.196 INFO:tasks.workunit.client.0.smithi070.stdout:7/201: truncate d25/d27/d2f/f4a 49937 0 2024-03-23T08:23:49.197 INFO:tasks.workunit.client.0.smithi070.stdout:9/405: dwrite db/d12/d22/d25/d26/f64 [0,4194304] 0 2024-03-23T08:23:49.198 INFO:tasks.workunit.client.0.smithi070.stdout:1/318: rmdir d8/db 39 2024-03-23T08:23:49.201 INFO:tasks.workunit.client.0.smithi070.stdout:6/388: creat d0/d3/f80 x:0 0 0 2024-03-23T08:23:49.203 INFO:tasks.workunit.client.0.smithi070.stdout:8/286: read d5/d8/f2c [97452,98747] 0 2024-03-23T08:23:49.210 INFO:tasks.workunit.client.0.smithi070.stdout:9/406: rmdir db/d12 39 2024-03-23T08:23:49.213 INFO:tasks.workunit.client.0.smithi070.stdout:6/389: symlink d0/d23/d28/d35/d3a/d3c/d5b/l81 0 2024-03-23T08:23:49.216 INFO:tasks.workunit.client.0.smithi070.stdout:8/287: creat d5/d8/d16/d32/f74 x:0 0 0 2024-03-23T08:23:49.216 INFO:tasks.workunit.client.0.smithi070.stdout:8/288: chown d5/d8/d16/d3b/f48 20 1 2024-03-23T08:23:49.217 INFO:tasks.workunit.client.0.smithi070.stdout:7/202: creat d25/d40/f4b x:0 0 0 2024-03-23T08:23:49.218 INFO:tasks.workunit.client.0.smithi070.stdout:7/203: fdatasync d25/d27/f3c 0 2024-03-23T08:23:49.218 INFO:tasks.workunit.client.0.smithi070.stdout:7/204: stat f1d 0 2024-03-23T08:23:49.221 INFO:tasks.workunit.client.0.smithi070.stdout:3/469: truncate db/f36 272752 0 2024-03-23T08:23:49.223 INFO:tasks.workunit.client.0.smithi070.stdout:6/390: fdatasync d0/d23/d28/f37 0 2024-03-23T08:23:49.225 INFO:tasks.workunit.client.0.smithi070.stdout:1/319: dread d8/d1b/f33 [0,4194304] 0 2024-03-23T08:23:49.228 INFO:tasks.workunit.client.0.smithi070.stdout:0/521: creat d16/d26/d27/d4c/d60/d7c/fb8 x:0 0 0 2024-03-23T08:23:49.232 INFO:tasks.workunit.client.0.smithi070.stdout:3/470: creat db/df/d5d/fa9 x:0 0 0 2024-03-23T08:23:49.233 INFO:tasks.workunit.client.0.smithi070.stdout:7/205: dread f8 [0,4194304] 0 2024-03-23T08:23:49.234 INFO:tasks.workunit.client.0.smithi070.stdout:9/407: rmdir db/d12/d22/d25/d26/d5c/d7e 39 2024-03-23T08:23:49.238 INFO:tasks.workunit.client.0.smithi070.stdout:7/206: dread d25/f34 [0,4194304] 0 2024-03-23T08:23:49.240 INFO:tasks.workunit.client.0.smithi070.stdout:0/522: creat d16/d26/d27/d3d/d4d/d9a/fb9 x:0 0 0 2024-03-23T08:23:49.242 INFO:tasks.workunit.client.0.smithi070.stdout:3/471: mkdir db/d14/d65/daa 0 2024-03-23T08:23:49.246 INFO:tasks.workunit.client.0.smithi070.stdout:9/408: creat db/d12/d34/f84 x:0 0 0 2024-03-23T08:23:49.248 INFO:tasks.workunit.client.0.smithi070.stdout:9/409: truncate db/d12/d22/d25/d26/d5c/f5d 773338 0 2024-03-23T08:23:49.249 INFO:tasks.workunit.client.0.smithi070.stdout:6/391: symlink d0/d23/d28/d35/d3a/d3c/l82 0 2024-03-23T08:23:49.250 INFO:tasks.workunit.client.0.smithi070.stdout:1/320: dread - d8/db/f32 zero size 2024-03-23T08:23:49.251 INFO:tasks.workunit.client.0.smithi070.stdout:6/392: write d0/d23/d1c/d38/d52/f7c [312485,61174] 0 2024-03-23T08:23:49.254 INFO:tasks.workunit.client.0.smithi070.stdout:6/393: write d0/d23/f55 [979755,97630] 0 2024-03-23T08:23:49.255 INFO:tasks.workunit.client.0.smithi070.stdout:0/523: mknod d16/cba 0 2024-03-23T08:23:49.257 INFO:tasks.workunit.client.0.smithi070.stdout:9/410: dwrite db/d12/d14/f63 [0,4194304] 0 2024-03-23T08:23:49.257 INFO:tasks.workunit.client.0.smithi070.stdout:0/524: dread - d16/d26/f6b zero size 2024-03-23T08:23:49.258 INFO:tasks.workunit.client.0.smithi070.stdout:3/472: dread db/df/f90 [0,4194304] 0 2024-03-23T08:23:49.260 INFO:tasks.workunit.client.0.smithi070.stdout:8/289: getdents d5/d8/d16 0 2024-03-23T08:23:49.260 INFO:tasks.workunit.client.0.smithi070.stdout:3/473: fdatasync db/df/d1a/f50 0 2024-03-23T08:23:49.263 INFO:tasks.workunit.client.0.smithi070.stdout:8/290: dread d5/d8/f2c [0,4194304] 0 2024-03-23T08:23:49.273 INFO:tasks.workunit.client.0.smithi070.stdout:8/291: mknod d5/d8/d16/d3b/d4d/c75 0 2024-03-23T08:23:49.276 INFO:tasks.workunit.client.0.smithi070.stdout:0/525: rename d16/d22/d43/d47/l82 to d16/d26/d27/d3d/lbb 0 2024-03-23T08:23:49.277 INFO:tasks.workunit.client.0.smithi070.stdout:7/207: getdents d25/d27/d2f 0 2024-03-23T08:23:49.278 INFO:tasks.workunit.client.0.smithi070.stdout:8/292: unlink d5/lf 0 2024-03-23T08:23:49.279 INFO:tasks.workunit.client.0.smithi070.stdout:8/293: read - d5/d8/d16/f44 zero size 2024-03-23T08:23:49.280 INFO:tasks.workunit.client.0.smithi070.stdout:3/474: rename db/df/l12 to db/d14/d43/d62/lab 0 2024-03-23T08:23:49.282 INFO:tasks.workunit.client.0.smithi070.stdout:7/208: creat d25/d27/f4c x:0 0 0 2024-03-23T08:23:49.284 INFO:tasks.workunit.client.0.smithi070.stdout:8/294: creat d5/d8/d16/d3b/f76 x:0 0 0 2024-03-23T08:23:49.288 INFO:tasks.workunit.client.0.smithi070.stdout:3/475: dwrite db/d14/d65/f85 [0,4194304] 0 2024-03-23T08:23:49.288 INFO:tasks.workunit.client.0.smithi070.stdout:6/394: dread d0/d3/fa [0,4194304] 0 2024-03-23T08:23:49.293 INFO:tasks.workunit.client.0.smithi070.stdout:0/526: link d16/d26/d27/d4c/d60/c7f d16/d26/cbc 0 2024-03-23T08:23:49.294 INFO:tasks.workunit.client.0.smithi070.stdout:8/295: mknod d5/d8/d16/d3b/d4d/d6f/c77 0 2024-03-23T08:23:49.295 INFO:tasks.workunit.client.0.smithi070.stdout:3/476: mkdir db/df/d15/d3d/dac 0 2024-03-23T08:23:49.298 INFO:tasks.workunit.client.0.smithi070.stdout:0/527: creat d16/d26/d27/d3d/d4d/fbd x:0 0 0 2024-03-23T08:23:49.301 INFO:tasks.workunit.client.0.smithi070.stdout:7/209: getdents d25 0 2024-03-23T08:23:49.302 INFO:tasks.workunit.client.0.smithi070.stdout:8/296: rename d5/d8/d16/d30/l41 to d5/d8/d16/d30/d38/d6b/l78 0 2024-03-23T08:23:49.303 INFO:tasks.workunit.client.0.smithi070.stdout:3/477: dwrite db/d14/d65/f85 [0,4194304] 0 2024-03-23T08:23:49.305 INFO:tasks.workunit.client.0.smithi070.stdout:8/297: write d5/d8/d16/d30/f3e [42844,89594] 0 2024-03-23T08:23:49.305 INFO:tasks.workunit.client.0.smithi070.stdout:3/478: chown db/d14/d43/l59 104006 1 2024-03-23T08:23:49.309 INFO:tasks.workunit.client.0.smithi070.stdout:0/528: unlink d16/d26/d27/c6e 0 2024-03-23T08:23:49.313 INFO:tasks.workunit.client.0.smithi070.stdout:7/210: rename f8 to d25/d40/f4d 0 2024-03-23T08:23:49.314 INFO:tasks.workunit.client.0.smithi070.stdout:6/395: getdents d0/d23/d1c/d38 0 2024-03-23T08:23:49.315 INFO:tasks.workunit.client.0.smithi070.stdout:7/211: write d25/d27/f4c [701302,93805] 0 2024-03-23T08:23:49.318 INFO:tasks.workunit.client.0.smithi070.stdout:8/298: creat d5/d8/f79 x:0 0 0 2024-03-23T08:23:49.324 INFO:tasks.workunit.client.0.smithi070.stdout:6/396: mknod d0/d3/d2c/c83 0 2024-03-23T08:23:49.326 INFO:tasks.workunit.client.0.smithi070.stdout:3/479: creat db/d14/d65/daa/fad x:0 0 0 2024-03-23T08:23:49.331 INFO:tasks.workunit.client.0.smithi070.stdout:6/397: rename d0/d23/d28/d35/d3a to d0/d23/d74/d84 0 2024-03-23T08:23:49.334 INFO:tasks.workunit.client.0.smithi070.stdout:3/480: write db/df/d15/f3e [4553969,123143] 0 2024-03-23T08:23:49.335 INFO:tasks.workunit.client.0.smithi070.stdout:3/481: dread - db/d14/d9c/f9f zero size 2024-03-23T08:23:49.338 INFO:tasks.workunit.client.0.smithi070.stdout:6/398: mkdir d0/d23/d28/d35/d85 0 2024-03-23T08:23:49.340 INFO:tasks.workunit.client.0.smithi070.stdout:7/212: truncate d25/f38 561484 0 2024-03-23T08:23:49.342 INFO:tasks.workunit.client.0.smithi070.stdout:0/529: getdents d16/d22/d43/d47 0 2024-03-23T08:23:49.343 INFO:tasks.workunit.client.0.smithi070.stdout:3/482: dwrite db/d14/d65/daa/fad [0,4194304] 0 2024-03-23T08:23:49.345 INFO:tasks.workunit.client.0.smithi070.stdout:7/213: symlink d25/d27/d2f/d44/l4e 0 2024-03-23T08:23:49.350 INFO:tasks.workunit.client.0.smithi070.stdout:6/399: dread d0/f20 [0,4194304] 0 2024-03-23T08:23:49.352 INFO:tasks.workunit.client.0.smithi070.stdout:7/214: rename d25/d27/d2f/f4a to d25/f4f 0 2024-03-23T08:23:49.355 INFO:tasks.workunit.client.0.smithi070.stdout:3/483: chown db/df/d4d/l4f 1025 1 2024-03-23T08:23:49.359 INFO:tasks.workunit.client.0.smithi070.stdout:7/215: symlink d25/l50 0 2024-03-23T08:23:49.360 INFO:tasks.workunit.client.0.smithi070.stdout:0/530: creat d16/d26/d27/d86/d67/fbe x:0 0 0 2024-03-23T08:23:49.361 INFO:tasks.workunit.client.0.smithi070.stdout:0/531: chown d16/d22/c4f 1109 1 2024-03-23T08:23:49.363 INFO:tasks.workunit.client.0.smithi070.stdout:6/400: mkdir d0/d23/d74/d84/d3c/d86 0 2024-03-23T08:23:49.364 INFO:tasks.workunit.client.0.smithi070.stdout:7/216: mknod d25/d27/c51 0 2024-03-23T08:23:49.366 INFO:tasks.workunit.client.0.smithi070.stdout:7/217: dread d25/f4f [0,4194304] 0 2024-03-23T08:23:49.367 INFO:tasks.workunit.client.0.smithi070.stdout:3/484: rename db/df/d4d/d51 to db/d14/d65/dae 0 2024-03-23T08:23:49.368 INFO:tasks.workunit.client.0.smithi070.stdout:0/532: write d16/d22/d43/d47/d87/f92 [1974199,55046] 0 2024-03-23T08:23:49.370 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-23T08:23:49.370 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-23T08:23:49.371 INFO:tasks.workunit.client.0.smithi070.stdout:7/218: symlink d25/d27/d2f/d44/l52 0 2024-03-23T08:23:49.375 INFO:tasks.workunit.client.0.smithi070.stdout:7/219: creat d25/d27/d2f/d44/f53 x:0 0 0 2024-03-23T08:23:49.376 INFO:tasks.workunit.client.0.smithi070.stdout:7/220: truncate d25/d27/f45 231284 0 2024-03-23T08:23:49.378 INFO:tasks.workunit.client.0.smithi070.stdout:6/401: write d0/d23/d1c/d2f/f30 [180298,42249] 0 2024-03-23T08:23:49.379 INFO:tasks.workunit.client.0.smithi070.stdout:7/221: creat d25/d27/d2f/d44/f54 x:0 0 0 2024-03-23T08:23:49.381 INFO:tasks.workunit.client.0.smithi070.stdout:0/533: truncate d16/d22/f94 161377 0 2024-03-23T08:23:49.382 INFO:tasks.workunit.client.0.smithi070.stdout:7/222: truncate d25/d27/f37 4548493 0 2024-03-23T08:23:49.384 INFO:tasks.workunit.client.0.smithi070.stdout:6/402: dread d0/d23/d74/d84/d3c/f60 [0,4194304] 0 2024-03-23T08:23:49.384 INFO:tasks.workunit.client.0.smithi070.stdout:7/223: symlink d25/l55 0 2024-03-23T08:23:49.385 INFO:tasks.workunit.client.0.smithi070.stdout:7/224: fsync f16 0 2024-03-23T08:23:49.386 INFO:tasks.workunit.client.0.smithi070.stdout:0/534: dread - d16/d26/d27/d86/d9e/fb5 zero size 2024-03-23T08:23:49.387 INFO:tasks.workunit.client.0.smithi070.stdout:6/403: mknod d0/d3/c87 0 2024-03-23T08:23:49.388 INFO:tasks.workunit.client.0.smithi070.stdout:3/485: dwrite db/df/f8d [0,4194304] 0 2024-03-23T08:23:49.389 INFO:tasks.workunit.client.0.smithi070.stdout:3/486: readlink db/df/d1a/l24 0 2024-03-23T08:23:49.394 INFO:tasks.workunit.client.0.smithi070.stdout:7/225: creat d25/d27/d2f/d44/f56 x:0 0 0 2024-03-23T08:23:49.394 INFO:tasks.workunit.client.0.smithi070.stdout:0/535: rename d16/d26/d27/d4c/d60/d7c/f90 to d16/d26/da8/fbf 0 2024-03-23T08:23:49.397 INFO:tasks.workunit.client.0.smithi070.stdout:7/226: creat d25/d27/f57 x:0 0 0 2024-03-23T08:23:49.404 INFO:tasks.workunit.client.0.smithi070.stdout:0/536: dwrite d16/d22/d43/f78 [0,4194304] 0 2024-03-23T08:23:49.410 INFO:tasks.workunit.client.0.smithi070.stdout:0/537: creat d16/d26/d27/d4c/d60/d7c/fc0 x:0 0 0 2024-03-23T08:23:49.411 INFO:tasks.workunit.client.0.smithi070.stdout:7/227: dread d25/f2c [0,4194304] 0 2024-03-23T08:23:49.412 INFO:tasks.workunit.client.0.smithi070.stdout:0/538: mknod d16/d26/d27/d3d/d4d/dad/cc1 0 2024-03-23T08:23:49.418 INFO:tasks.workunit.client.0.smithi070.stdout:3/487: dread db/f17 [0,4194304] 0 2024-03-23T08:23:49.418 INFO:tasks.workunit.client.0.smithi070.stdout:7/228: dwrite f16 [0,4194304] 0 2024-03-23T08:23:49.421 INFO:tasks.workunit.client.0.smithi070.stdout:7/229: symlink d25/d40/l58 0 2024-03-23T08:23:49.423 INFO:tasks.workunit.client.0.smithi070.stdout:7/230: mknod d25/d27/c59 0 2024-03-23T08:23:49.424 INFO:tasks.workunit.client.0.smithi070.stdout:7/231: fdatasync f1d 0 2024-03-23T08:23:49.426 INFO:tasks.workunit.client.0.smithi070.stdout:3/488: dwrite db/df/f2e [0,4194304] 0 2024-03-23T08:23:49.431 INFO:tasks.workunit.client.0.smithi070.stdout:3/489: rename db/df/d15/l39 to db/df/d6a/laf 0 2024-03-23T08:23:49.434 INFO:tasks.workunit.client.0.smithi070.stdout:3/490: rmdir db/d20/d49 39 2024-03-23T08:23:49.436 INFO:tasks.workunit.client.0.smithi070.stdout:3/491: mknod db/df/d15/d3d/cb0 0 2024-03-23T08:23:49.439 INFO:tasks.workunit.client.0.smithi070.stdout:3/492: mkdir db/df/d15/d22/db1 0 2024-03-23T08:23:49.440 INFO:tasks.workunit.client.0.smithi070.stdout:3/493: fsync db/d4e/d92/f74 0 2024-03-23T08:23:49.440 INFO:tasks.workunit.client.0.smithi070.stdout:7/232: dread f24 [0,4194304] 0 2024-03-23T08:23:49.442 INFO:tasks.workunit.client.0.smithi070.stdout:3/494: unlink db/d20/f45 0 2024-03-23T08:23:49.443 INFO:tasks.workunit.client.0.smithi070.stdout:3/495: stat db/d14/d65/da8 0 2024-03-23T08:23:49.444 INFO:tasks.workunit.client.0.smithi070.stdout:7/233: unlink d25/d27/f37 0 2024-03-23T08:23:49.445 INFO:tasks.workunit.client.0.smithi070.stdout:7/234: write d25/d40/f4b [700754,105079] 0 2024-03-23T08:23:49.448 INFO:tasks.workunit.client.0.smithi070.stdout:3/496: truncate db/df/d1a/f50 3984631 0 2024-03-23T08:23:49.450 INFO:tasks.workunit.client.0.smithi070.stdout:0/539: dread d16/f30 [0,4194304] 0 2024-03-23T08:23:49.451 INFO:tasks.workunit.client.0.smithi070.stdout:7/235: link d25/d40/f4b d25/d27/d2f/d44/f5a 0 2024-03-23T08:23:49.452 INFO:tasks.workunit.client.0.smithi070.stdout:3/497: mkdir db/df/d15/d22/db2 0 2024-03-23T08:23:49.454 INFO:tasks.workunit.client.0.smithi070.stdout:0/540: fdatasync d16/d26/da8/fbf 0 2024-03-23T08:23:49.457 INFO:tasks.workunit.client.0.smithi070.stdout:7/236: unlink d25/d27/d2f/d44/l47 0 2024-03-23T08:23:49.459 INFO:tasks.workunit.client.0.smithi070.stdout:0/541: unlink d16/d26/d27/d4c/d60/d75/c85 0 2024-03-23T08:23:49.459 INFO:tasks.workunit.client.0.smithi070.stdout:0/542: dread - d16/d26/d27/d3d/f72 zero size 2024-03-23T08:23:49.459 INFO:tasks.workunit.client.0.smithi070.stdout:3/498: mknod db/cb3 0 2024-03-23T08:23:49.461 INFO:tasks.workunit.client.0.smithi070.stdout:7/237: creat d25/d27/d2f/f5b x:0 0 0 2024-03-23T08:23:49.462 INFO:tasks.workunit.client.0.smithi070.stdout:7/238: creat d25/d27/f5c x:0 0 0 2024-03-23T08:23:49.463 INFO:tasks.workunit.client.0.smithi070.stdout:3/499: symlink db/df/d15/d22/db2/lb4 0 2024-03-23T08:23:49.464 INFO:tasks.workunit.client.0.smithi070.stdout:3/500: chown db/cb3 1 1 2024-03-23T08:23:49.467 INFO:tasks.workunit.client.0.smithi070.stdout:3/501: symlink db/df/d4d/lb5 0 2024-03-23T08:23:49.468 INFO:tasks.workunit.client.0.smithi070.stdout:3/502: read db/df/f90 [1734858,37156] 0 2024-03-23T08:23:49.468 INFO:tasks.workunit.client.0.smithi070.stdout:0/543: dwrite d16/d26/d27/d4c/d60/f7e [0,4194304] 0 2024-03-23T08:23:49.472 INFO:tasks.workunit.client.0.smithi070.stdout:7/239: rename d25/d27/c3a to d25/d27/d2f/c5d 0 2024-03-23T08:23:49.473 INFO:tasks.workunit.client.0.smithi070.stdout:3/503: mknod db/d14/d65/daa/cb6 0 2024-03-23T08:23:49.475 INFO:tasks.workunit.client.0.smithi070.stdout:0/544: creat d16/d26/d27/d86/d67/d7d/fc2 x:0 0 0 2024-03-23T08:23:49.476 INFO:tasks.workunit.client.0.smithi070.stdout:7/240: creat d25/d27/d2f/f5e x:0 0 0 2024-03-23T08:23:49.478 INFO:tasks.workunit.client.0.smithi070.stdout:3/504: fdatasync db/d14/f1b 0 2024-03-23T08:23:49.479 INFO:tasks.workunit.client.0.smithi070.stdout:0/545: write d16/f5b [1248135,34042] 0 2024-03-23T08:23:49.479 INFO:tasks.workunit.client.0.smithi070.stdout:7/241: creat d25/d27/d2f/f5f x:0 0 0 2024-03-23T08:23:49.480 INFO:tasks.workunit.client.0.smithi070.stdout:3/505: write db/df/f90 [1586989,106988] 0 2024-03-23T08:23:49.483 INFO:tasks.workunit.client.0.smithi070.stdout:0/546: creat d16/d26/d27/d4c/d60/d75/fc3 x:0 0 0 2024-03-23T08:23:49.483 INFO:tasks.workunit.client.0.smithi070.stdout:7/242: creat d25/d27/f60 x:0 0 0 2024-03-23T08:23:49.487 INFO:tasks.workunit.client.0.smithi070.stdout:0/547: creat d16/d26/d27/d3d/fc4 x:0 0 0 2024-03-23T08:23:49.487 INFO:tasks.workunit.client.0.smithi070.stdout:7/243: creat d25/d40/f61 x:0 0 0 2024-03-23T08:23:49.488 INFO:tasks.workunit.client.0.smithi070.stdout:7/244: stat d25/f34 0 2024-03-23T08:23:49.489 INFO:tasks.workunit.client.0.smithi070.stdout:7/245: chown d25/l55 250 1 2024-03-23T08:23:49.493 INFO:tasks.workunit.client.0.smithi070.stdout:7/246: symlink d25/d40/l62 0 2024-03-23T08:23:49.495 INFO:tasks.workunit.client.0.smithi070.stdout:0/548: dwrite d16/f30 [0,4194304] 0 2024-03-23T08:23:49.496 INFO:tasks.workunit.client.0.smithi070.stdout:7/247: mknod d25/d27/c63 0 2024-03-23T08:23:49.500 INFO:tasks.workunit.client.0.smithi070.stdout:7/248: rename d25/d27/c29 to d25/c64 0 2024-03-23T08:23:49.501 INFO:tasks.workunit.client.0.smithi070.stdout:7/249: dread - d25/f43 zero size 2024-03-23T08:23:49.504 INFO:tasks.workunit.client.0.smithi070.stdout:7/250: mknod d25/d27/d2f/c65 0 2024-03-23T08:23:49.506 INFO:tasks.workunit.client.0.smithi070.stdout:7/251: unlink f1 0 2024-03-23T08:23:49.525 INFO:tasks.workunit.client.0.smithi070.stdout:3/506: truncate db/df/d1a/f50 196847 0 2024-03-23T08:23:49.529 INFO:tasks.workunit.client.0.smithi070.stdout:3/507: creat db/d4e/fb7 x:0 0 0 2024-03-23T08:23:49.531 INFO:tasks.workunit.client.0.smithi070.stdout:7/252: rename d25/d27/d2f to d25/d27/d66 0 2024-03-23T08:23:49.534 INFO:tasks.workunit.client.0.smithi070.stdout:3/508: creat db/d14/d65/dae/fb8 x:0 0 0 2024-03-23T08:23:49.538 INFO:tasks.workunit.client.0.smithi070.stdout:3/509: mknod db/df/d4d/cb9 0 2024-03-23T08:23:49.540 INFO:tasks.workunit.client.0.smithi070.stdout:3/510: write db/d14/d43/d62/f69 [5858721,99993] 0 2024-03-23T08:23:49.545 INFO:tasks.workunit.client.0.smithi070.stdout:3/511: truncate db/d14/d65/f78 316772 0 2024-03-23T08:23:49.574 INFO:tasks.workunit.client.0.smithi070.stdout:3/512: mkdir db/df/d15/d22/db2/dba 0 2024-03-23T08:23:49.575 INFO:tasks.workunit.client.0.smithi070.stdout:3/513: dread - db/df/d6a/fa2 zero size 2024-03-23T08:23:49.577 INFO:tasks.workunit.client.0.smithi070.stdout:3/514: mkdir db/d4e/d92/dbb 0 2024-03-23T08:23:49.577 INFO:tasks.workunit.client.0.smithi070.stdout:3/515: read - db/d14/d9c/f9f zero size 2024-03-23T08:23:49.594 INFO:tasks.workunit.client.0.smithi070.stdout:3/516: dread db/df/d1a/f37 [4194304,4194304] 0 2024-03-23T08:23:49.596 INFO:tasks.workunit.client.0.smithi070.stdout:3/517: unlink db/df/f2e 0 2024-03-23T08:23:49.599 INFO:tasks.workunit.client.0.smithi070.stdout:3/518: creat db/fbc x:0 0 0 2024-03-23T08:23:49.601 INFO:tasks.workunit.client.0.smithi070.stdout:3/519: symlink db/d20/d7a/lbd 0 2024-03-23T08:23:49.603 INFO:tasks.workunit.client.0.smithi070.stdout:3/520: creat db/d7f/fbe x:0 0 0 2024-03-23T08:23:49.604 INFO:tasks.workunit.client.0.smithi070.stdout:3/521: readlink db/d14/d43/d62/lab 0 2024-03-23T08:23:49.607 INFO:tasks.workunit.client.0.smithi070.stdout:3/522: creat db/df/d4d/fbf x:0 0 0 2024-03-23T08:23:49.608 INFO:tasks.workunit.client.0.smithi070.stdout:3/523: truncate db/f21 5124766 0 2024-03-23T08:23:49.609 INFO:tasks.workunit.client.0.smithi070.stdout:3/524: write db/df/d1a/f3c [67352,49046] 0 2024-03-23T08:23:49.619 INFO:tasks.workunit.client.0.smithi070.stdout:3/525: dread db/d14/d43/f5a [0,4194304] 0 2024-03-23T08:23:49.623 INFO:tasks.workunit.client.0.smithi070.stdout:3/526: mkdir db/df/d4d/dc0 0 2024-03-23T08:23:49.625 INFO:tasks.workunit.client.0.smithi070.stdout:3/527: rmdir db/df/d6a 39 2024-03-23T08:23:49.628 INFO:tasks.workunit.client.0.smithi070.stdout:3/528: rename db/df/d15/d3d/f53 to db/d4e/d92/fc1 0 2024-03-23T08:23:49.631 INFO:tasks.workunit.client.0.smithi070.stdout:3/529: rmdir db/d20/d63 39 2024-03-23T08:23:49.635 INFO:tasks.workunit.client.0.smithi070.stdout:3/530: symlink db/lc2 0 2024-03-23T08:23:49.636 INFO:tasks.workunit.client.0.smithi070.stdout:3/531: symlink db/d14/d43/d62/lc3 0 2024-03-23T08:23:49.637 INFO:tasks.workunit.client.0.smithi070.stdout:3/532: truncate db/d4e/fb7 780652 0 2024-03-23T08:23:49.640 INFO:tasks.workunit.client.0.smithi070.stdout:3/533: mkdir db/d4e/dc4 0 2024-03-23T08:23:49.644 INFO:tasks.workunit.client.0.smithi070.stdout:3/534: getdents db/d20/d63/d83 0 2024-03-23T08:23:49.644 INFO:tasks.workunit.client.0.smithi070.stdout:3/535: read db/d14/d43/f5a [3741247,48817] 0 2024-03-23T08:23:49.646 INFO:tasks.workunit.client.0.smithi070.stdout:3/536: chown db/d14/d9c/l9d 1392 1 2024-03-23T08:23:49.648 INFO:tasks.workunit.client.0.smithi070.stdout:3/537: creat db/df/d15/d22/fc5 x:0 0 0 2024-03-23T08:23:49.650 INFO:tasks.workunit.client.0.smithi070.stdout:3/538: mknod db/d20/d7a/cc6 0 2024-03-23T08:23:49.681 INFO:tasks.workunit.client.0.smithi070.stdout:5/181: sync 2024-03-23T08:23:49.682 INFO:tasks.workunit.client.0.smithi070.stdout:5/182: fsync dc/d1b/d1d/d25/f33 0 2024-03-23T08:23:49.689 INFO:tasks.workunit.client.0.smithi070.stdout:5/183: link l2 dc/d1b/d1d/d25/d36/d32/l3e 0 2024-03-23T08:23:49.691 INFO:tasks.workunit.client.0.smithi070.stdout:5/184: truncate dc/d1b/d1d/d25/d36/f27 1526325 0 2024-03-23T08:23:49.693 INFO:tasks.workunit.client.0.smithi070.stdout:5/185: read dc/f1f [16194,2840] 0 2024-03-23T08:23:49.699 INFO:tasks.workunit.client.0.smithi070.stdout:5/186: dwrite dc/d1b/f2f [0,4194304] 0 2024-03-23T08:23:49.699 INFO:tasks.workunit.client.0.smithi070.stdout:5/187: stat dc/f2b 0 2024-03-23T08:23:49.701 INFO:tasks.workunit.client.0.smithi070.stdout:5/188: write dc/d19/f2c [540199,100411] 0 2024-03-23T08:23:49.708 INFO:tasks.workunit.client.0.smithi070.stdout:5/189: dwrite dc/d1b/d1d/d25/d36/f2a [0,4194304] 0 2024-03-23T08:23:49.709 INFO:tasks.workunit.client.0.smithi070.stdout:5/190: chown dc/cf 1697506 1 2024-03-23T08:23:49.709 INFO:tasks.workunit.client.0.smithi070.stdout:5/191: fdatasync dc/f14 0 2024-03-23T08:23:49.713 INFO:tasks.workunit.client.0.smithi070.stdout:5/192: getdents dc/d1b/d1d/d25/d38 0 2024-03-23T08:23:49.714 INFO:tasks.workunit.client.0.smithi070.stdout:5/193: chown dc/l1a 1746935 1 2024-03-23T08:23:49.718 INFO:tasks.workunit.client.0.smithi070.stdout:5/194: dread dc/d1b/f2f [0,4194304] 0 2024-03-23T08:23:49.724 INFO:tasks.workunit.client.0.smithi070.stdout:5/195: write f1 [2205331,105133] 0 2024-03-23T08:23:49.726 INFO:tasks.workunit.client.0.smithi070.stdout:5/196: mknod dc/c3f 0 2024-03-23T08:23:49.728 INFO:tasks.workunit.client.0.smithi070.stdout:5/197: creat dc/d1b/d1d/d25/d38/f40 x:0 0 0 2024-03-23T08:23:49.731 INFO:tasks.workunit.client.0.smithi070.stdout:5/198: mkdir dc/d1b/d1d/d41 0 2024-03-23T08:23:49.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:49 smithi070 ceph-mon[39699]: pgmap v34: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 11 MiB/s rd, 75 MiB/s wr, 254 op/s 2024-03-23T08:23:49.733 INFO:tasks.workunit.client.0.smithi070.stdout:5/199: mknod dc/d1b/d1d/d25/c42 0 2024-03-23T08:23:49.735 INFO:tasks.workunit.client.0.smithi070.stdout:5/200: creat dc/d1b/d1d/d25/f43 x:0 0 0 2024-03-23T08:23:49.741 INFO:tasks.workunit.client.0.smithi070.stdout:5/201: dwrite dc/d1b/d1d/d25/f33 [0,4194304] 0 2024-03-23T08:23:49.745 INFO:tasks.workunit.client.0.smithi070.stdout:5/202: unlink dc/l26 0 2024-03-23T08:23:49.747 INFO:tasks.workunit.client.0.smithi070.stdout:5/203: write dc/d1b/d1d/f3a [654979,85476] 0 2024-03-23T08:23:49.748 INFO:tasks.workunit.client.0.smithi070.stdout:5/204: creat dc/d1b/d1d/f44 x:0 0 0 2024-03-23T08:23:49.751 INFO:tasks.workunit.client.0.smithi070.stdout:5/205: mknod dc/d1b/c45 0 2024-03-23T08:23:49.752 INFO:tasks.workunit.client.0.smithi070.stdout:5/206: write dc/d1b/d1d/d25/d36/f2a [2178904,17000] 0 2024-03-23T08:23:49.753 INFO:tasks.workunit.client.0.smithi070.stdout:5/207: write dc/d1b/d1d/d25/f43 [980722,14853] 0 2024-03-23T08:23:49.755 INFO:tasks.workunit.client.0.smithi070.stdout:5/208: write dc/d1b/d1d/d25/f43 [1218448,123101] 0 2024-03-23T08:23:49.758 INFO:tasks.workunit.client.0.smithi070.stdout:5/209: mknod dc/d1b/d1d/c46 0 2024-03-23T08:23:49.762 INFO:tasks.workunit.client.0.smithi070.stdout:5/210: dread dc/d1b/d1d/f37 [0,4194304] 0 2024-03-23T08:23:49.767 INFO:tasks.workunit.client.0.smithi070.stdout:5/211: unlink l2 0 2024-03-23T08:23:49.768 INFO:tasks.workunit.client.0.smithi070.stdout:5/212: chown dc/d1b/d1d/d25/d36/f2a 172396 1 2024-03-23T08:23:49.769 INFO:tasks.workunit.client.0.smithi070.stdout:5/213: symlink dc/d1b/d1d/d25/l47 0 2024-03-23T08:23:49.770 INFO:tasks.workunit.client.0.smithi070.stdout:5/214: fsync dc/d1b/d1d/d25/d36/f2a 0 2024-03-23T08:23:49.776 INFO:tasks.workunit.client.0.smithi070.stdout:5/215: dwrite dc/f3d [0,4194304] 0 2024-03-23T08:23:49.777 INFO:tasks.workunit.client.0.smithi070.stdout:5/216: fsync dc/d1b/d1d/f44 0 2024-03-23T08:23:49.781 INFO:tasks.workunit.client.0.smithi070.stdout:5/217: mkdir dc/d1b/d1d/d25/d38/d48 0 2024-03-23T08:23:49.785 INFO:tasks.workunit.client.0.smithi070.stdout:5/218: symlink dc/d1b/d1d/l49 0 2024-03-23T08:23:49.787 INFO:tasks.workunit.client.0.smithi070.stdout:5/219: mknod dc/d1b/d1d/d25/c4a 0 2024-03-23T08:23:49.793 INFO:tasks.workunit.client.0.smithi070.stdout:5/220: rmdir dc/d1b/d1d 39 2024-03-23T08:23:49.796 INFO:tasks.workunit.client.0.smithi070.stdout:5/221: creat dc/d1b/d1d/d25/f4b x:0 0 0 2024-03-23T08:23:49.799 INFO:tasks.workunit.client.0.smithi070.stdout:5/222: rename dc/d1b/d1d/d25/c42 to dc/d1b/d1d/d25/d38/d48/c4c 0 2024-03-23T08:23:49.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/223: truncate dc/d1b/d1d/d25/f33 946026 0 2024-03-23T08:23:49.810 INFO:tasks.workunit.client.0.smithi070.stdout:5/224: rename dc/d1b/c28 to dc/d1b/d1d/d25/d38/d39/c4d 0 2024-03-23T08:23:49.813 INFO:tasks.workunit.client.0.smithi070.stdout:5/225: rename dc/d1b/d1d/l20 to dc/d1b/d1d/d25/d36/l4e 0 2024-03-23T08:23:49.814 INFO:tasks.workunit.client.0.smithi070.stdout:5/226: rmdir dc/d1b/d1d/d25/d38 39 2024-03-23T08:23:49.815 INFO:tasks.workunit.client.0.smithi070.stdout:5/227: write dc/f3d [1218244,28198] 0 2024-03-23T08:23:49.821 INFO:tasks.workunit.client.0.smithi070.stdout:5/228: creat dc/d1b/d1d/d25/d38/d39/f4f x:0 0 0 2024-03-23T08:23:49.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:49 smithi193 ceph-mon[48370]: pgmap v34: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 11 MiB/s rd, 75 MiB/s wr, 254 op/s 2024-03-23T08:23:49.832 INFO:tasks.workunit.client.0.smithi070.stdout:5/229: dwrite dc/d1b/d1d/d25/d38/f40 [0,4194304] 0 2024-03-23T08:23:49.838 INFO:tasks.workunit.client.0.smithi070.stdout:5/230: dread dc/f3d [0,4194304] 0 2024-03-23T08:23:49.848 INFO:tasks.workunit.client.0.smithi070.stdout:5/231: fdatasync f8 0 2024-03-23T08:23:49.850 INFO:tasks.workunit.client.0.smithi070.stdout:5/232: read dc/d1b/d1d/f3a [79045,116810] 0 2024-03-23T08:23:49.852 INFO:tasks.workunit.client.0.smithi070.stdout:5/233: symlink dc/d1b/d1d/d25/d38/l50 0 2024-03-23T08:23:49.852 INFO:tasks.workunit.client.0.smithi070.stdout:5/234: fsync dc/f14 0 2024-03-23T08:23:49.856 INFO:tasks.workunit.client.0.smithi070.stdout:5/235: rmdir dc/d1b/d1d/d41 0 2024-03-23T08:23:49.857 INFO:tasks.workunit.client.0.smithi070.stdout:5/236: truncate dc/d1b/d1d/d25/f4b 531658 0 2024-03-23T08:23:49.858 INFO:tasks.workunit.client.0.smithi070.stdout:5/237: readlink dc/d1b/d1d/d25/l47 0 2024-03-23T08:23:49.860 INFO:tasks.workunit.client.0.smithi070.stdout:5/238: rmdir dc/d1b/d1d/d25/d38/d48 39 2024-03-23T08:23:49.863 INFO:tasks.workunit.client.0.smithi070.stdout:5/239: mkdir dc/d1b/d1d/d25/d51 0 2024-03-23T08:23:49.865 INFO:tasks.workunit.client.0.smithi070.stdout:5/240: mknod dc/c52 0 2024-03-23T08:23:49.870 INFO:tasks.workunit.client.0.smithi070.stdout:5/241: mkdir dc/d1b/d1d/d25/d38/d53 0 2024-03-23T08:23:49.875 INFO:tasks.workunit.client.0.smithi070.stdout:5/242: creat dc/d1b/f54 x:0 0 0 2024-03-23T08:23:49.875 INFO:tasks.workunit.client.0.smithi070.stdout:5/243: stat dc/d1b/d1d/d25/d38/d53 0 2024-03-23T08:23:49.879 INFO:tasks.workunit.client.0.smithi070.stdout:5/244: mknod dc/d1b/c55 0 2024-03-23T08:23:49.882 INFO:tasks.workunit.client.0.smithi070.stdout:5/245: creat dc/d1b/d1d/d25/d38/d48/f56 x:0 0 0 2024-03-23T08:23:49.887 INFO:tasks.workunit.client.0.smithi070.stdout:5/246: rename dc/d1b/d1d/d25/f4b to dc/d1b/d1d/d25/d38/d48/f57 0 2024-03-23T08:23:49.909 INFO:tasks.workunit.client.0.smithi070.stdout:4/176: sync 2024-03-23T08:23:49.909 INFO:tasks.workunit.client.0.smithi070.stdout:2/248: sync 2024-03-23T08:23:49.911 INFO:tasks.workunit.client.0.smithi070.stdout:4/177: write d3/d6/f3b [377006,38034] 0 2024-03-23T08:23:49.915 INFO:tasks.workunit.client.0.smithi070.stdout:4/178: mkdir d3/d6/d45 0 2024-03-23T08:23:49.916 INFO:tasks.workunit.client.0.smithi070.stdout:2/249: dwrite d1/d7/f17 [0,4194304] 0 2024-03-23T08:23:49.917 INFO:tasks.workunit.client.0.smithi070.stdout:4/179: dread - d3/d6/d11/f2d zero size 2024-03-23T08:23:49.920 INFO:tasks.workunit.client.0.smithi070.stdout:2/250: rename d1/d7/d48/d51 to d1/d29/d52 0 2024-03-23T08:23:49.920 INFO:tasks.workunit.client.0.smithi070.stdout:2/251: chown d1/d29/d52 4318112 1 2024-03-23T08:23:49.924 INFO:tasks.workunit.client.0.smithi070.stdout:4/180: rename f1 to d3/d6/d11/d1a/d27/f46 0 2024-03-23T08:23:49.927 INFO:tasks.workunit.client.0.smithi070.stdout:2/252: dwrite d1/d6/d30/f46 [0,4194304] 0 2024-03-23T08:23:49.928 INFO:tasks.workunit.client.0.smithi070.stdout:2/253: fdatasync d1/d7/d11/f12 0 2024-03-23T08:23:49.935 INFO:tasks.workunit.client.0.smithi070.stdout:4/181: getdents d3/d6/d9 0 2024-03-23T08:23:49.937 INFO:tasks.workunit.client.0.smithi070.stdout:2/254: symlink d1/d7/d39/d47/l53 0 2024-03-23T08:23:49.938 INFO:tasks.workunit.client.0.smithi070.stdout:2/255: write d1/d29/f2d [3732598,24436] 0 2024-03-23T08:23:49.942 INFO:tasks.workunit.client.0.smithi070.stdout:2/256: mknod d1/d21/c54 0 2024-03-23T08:23:49.942 INFO:tasks.workunit.client.0.smithi070.stdout:4/182: dwrite d3/d6/d11/f18 [0,4194304] 0 2024-03-23T08:23:49.944 INFO:tasks.workunit.client.0.smithi070.stdout:2/257: symlink d1/d21/l55 0 2024-03-23T08:23:49.946 INFO:tasks.workunit.client.0.smithi070.stdout:2/258: rmdir d1/d7/df 39 2024-03-23T08:23:49.948 INFO:tasks.workunit.client.0.smithi070.stdout:4/183: truncate d3/d6/d11/d1a/d27/f46 3363645 0 2024-03-23T08:23:49.948 INFO:tasks.workunit.client.0.smithi070.stdout:2/259: creat d1/d7/d11/f56 x:0 0 0 2024-03-23T08:23:49.957 INFO:tasks.workunit.client.0.smithi070.stdout:2/260: dwrite d1/d6/d4f/f3c [4194304,4194304] 0 2024-03-23T08:23:49.972 INFO:tasks.workunit.client.0.smithi070.stdout:4/184: link d3/d6/d11/d1a/d27/f46 d3/d6/d45/f47 0 2024-03-23T08:23:49.972 INFO:tasks.workunit.client.0.smithi070.stdout:4/185: readlink d3/l24 0 2024-03-23T08:23:49.973 INFO:tasks.workunit.client.0.smithi070.stdout:4/186: dread d3/d6/d9/f2e [0,4194304] 0 2024-03-23T08:23:49.975 INFO:tasks.workunit.client.0.smithi070.stdout:4/187: unlink d3/d6/d11/d1a/d1b/l1d 0 2024-03-23T08:23:49.979 INFO:tasks.workunit.client.0.smithi070.stdout:4/188: dread d3/d6/f2f [0,4194304] 0 2024-03-23T08:23:49.986 INFO:tasks.workunit.client.0.smithi070.stdout:4/189: dwrite d3/d6/f2f [0,4194304] 0 2024-03-23T08:23:49.987 INFO:tasks.workunit.client.0.smithi070.stdout:4/190: chown d3/d6/d9/f22 162598 1 2024-03-23T08:23:49.992 INFO:tasks.workunit.client.0.smithi070.stdout:4/191: dread d3/d6/d9/f22 [0,4194304] 0 2024-03-23T08:23:49.994 INFO:tasks.workunit.client.0.smithi070.stdout:4/192: write d3/d6/d9/f22 [1201686,94163] 0 2024-03-23T08:23:49.996 INFO:tasks.workunit.client.0.smithi070.stdout:4/193: mknod d3/d6/c48 0 2024-03-23T08:23:49.997 INFO:tasks.workunit.client.0.smithi070.stdout:4/194: fdatasync d3/fd 0 2024-03-23T08:23:50.008 INFO:tasks.workunit.client.0.smithi070.stdout:4/195: truncate d3/d6/d11/d1a/f40 1066378 0 2024-03-23T08:23:50.010 INFO:tasks.workunit.client.0.smithi070.stdout:4/196: fdatasync d3/d6/d11/f13 0 2024-03-23T08:23:50.021 INFO:tasks.workunit.client.0.smithi070.stdout:4/197: dread d3/d6/d45/f47 [0,4194304] 0 2024-03-23T08:23:50.022 INFO:tasks.workunit.client.0.smithi070.stdout:4/198: dread - d3/f25 zero size 2024-03-23T08:23:50.025 INFO:tasks.workunit.client.0.smithi070.stdout:4/199: mknod d3/d6/d35/c49 0 2024-03-23T08:23:50.025 INFO:tasks.workunit.client.0.smithi070.stdout:4/200: stat d3/l3a 0 2024-03-23T08:23:50.027 INFO:tasks.workunit.client.0.smithi070.stdout:4/201: creat d3/d6/d9/f4a x:0 0 0 2024-03-23T08:23:50.037 INFO:tasks.workunit.client.0.smithi070.stdout:4/202: dread d3/d6/f7 [0,4194304] 0 2024-03-23T08:23:50.038 INFO:tasks.workunit.client.0.smithi070.stdout:4/203: chown d3/l3a 30908385 1 2024-03-23T08:23:50.039 INFO:tasks.workunit.client.0.smithi070.stdout:4/204: mkdir d3/d6/d11/d1a/d27/d4b 0 2024-03-23T08:23:50.196 INFO:tasks.workunit.client.0.smithi070.stdout:1/321: sync 2024-03-23T08:23:50.201 INFO:tasks.workunit.client.0.smithi070.stdout:1/322: symlink d8/d11/d61/d74/d5d/d70/l76 0 2024-03-23T08:23:50.205 INFO:tasks.workunit.client.0.smithi070.stdout:1/323: creat d8/d26/d49/d65/f77 x:0 0 0 2024-03-23T08:23:50.207 INFO:tasks.workunit.client.0.smithi070.stdout:1/324: rmdir d8/d1a 39 2024-03-23T08:23:50.209 INFO:tasks.workunit.client.0.smithi070.stdout:1/325: creat d8/d11/d61/f78 x:0 0 0 2024-03-23T08:23:50.282 INFO:tasks.workunit.client.0.smithi070.stdout:1/326: rmdir d8/d1a 39 2024-03-23T08:23:50.285 INFO:tasks.workunit.client.0.smithi070.stdout:1/327: symlink d8/l79 0 2024-03-23T08:23:50.290 INFO:tasks.workunit.client.0.smithi070.stdout:1/328: creat d8/db/d40/d43/f7a x:0 0 0 2024-03-23T08:23:50.290 INFO:tasks.workunit.client.0.smithi070.stdout:1/329: read - d8/d26/d49/d65/f77 zero size 2024-03-23T08:23:50.297 INFO:tasks.workunit.client.0.smithi070.stdout:1/330: dwrite d8/d1b/f22 [0,4194304] 0 2024-03-23T08:23:50.299 INFO:tasks.workunit.client.0.smithi070.stdout:1/331: mkdir d8/d26/d7b 0 2024-03-23T08:23:50.311 INFO:tasks.workunit.client.0.smithi070.stdout:1/332: dwrite d8/db/f27 [4194304,4194304] 0 2024-03-23T08:23:50.313 INFO:tasks.workunit.client.0.smithi070.stdout:1/333: dread - d8/d26/d49/d65/f77 zero size 2024-03-23T08:23:50.316 INFO:tasks.workunit.client.0.smithi070.stdout:1/334: mkdir d8/d1a/d7c 0 2024-03-23T08:23:50.318 INFO:tasks.workunit.client.0.smithi070.stdout:1/335: symlink d8/d26/d7b/l7d 0 2024-03-23T08:23:50.319 INFO:tasks.workunit.client.0.smithi070.stdout:1/336: chown d8/db/d2c/c30 167775220 1 2024-03-23T08:23:50.320 INFO:tasks.workunit.client.0.smithi070.stdout:9/411: sync 2024-03-23T08:23:50.320 INFO:tasks.workunit.client.0.smithi070.stdout:8/299: sync 2024-03-23T08:23:50.320 INFO:tasks.workunit.client.0.smithi070.stdout:6/404: sync 2024-03-23T08:23:50.320 INFO:tasks.workunit.client.0.smithi070.stdout:0/549: sync 2024-03-23T08:23:50.320 INFO:tasks.workunit.client.0.smithi070.stdout:1/337: rmdir d8/d26/d49 39 2024-03-23T08:23:50.322 INFO:tasks.workunit.client.0.smithi070.stdout:0/550: fdatasync d16/d26/d27/d86/d67/d7d/f99 0 2024-03-23T08:23:50.325 INFO:tasks.workunit.client.0.smithi070.stdout:6/405: mkdir d0/d23/d88 0 2024-03-23T08:23:50.326 INFO:tasks.workunit.client.0.smithi070.stdout:1/338: mknod d8/d11/d61/c7e 0 2024-03-23T08:23:50.328 INFO:tasks.workunit.client.0.smithi070.stdout:0/551: creat d16/d22/d43/d47/da9/fc5 x:0 0 0 2024-03-23T08:23:50.330 INFO:tasks.workunit.client.0.smithi070.stdout:0/552: write d16/f18 [3794418,10134] 0 2024-03-23T08:23:50.332 INFO:tasks.workunit.client.0.smithi070.stdout:8/300: dwrite d5/d8/d16/d32/f67 [0,4194304] 0 2024-03-23T08:23:50.332 INFO:tasks.workunit.client.0.smithi070.stdout:0/553: write d16/d22/d43/f78 [332172,3239] 0 2024-03-23T08:23:50.337 INFO:tasks.workunit.client.0.smithi070.stdout:9/412: rename db/d12/d22/d25/d26/d3b to db/d12/d22/d85 0 2024-03-23T08:23:50.338 INFO:tasks.workunit.client.0.smithi070.stdout:0/554: dread d16/f51 [0,4194304] 0 2024-03-23T08:23:50.345 INFO:tasks.workunit.client.0.smithi070.stdout:6/406: rename d0/d3/f45 to d0/d23/d28/d35/d85/f89 0 2024-03-23T08:23:50.350 INFO:tasks.workunit.client.0.smithi070.stdout:1/339: creat d8/d26/f7f x:0 0 0 2024-03-23T08:23:50.351 INFO:tasks.workunit.client.0.smithi070.stdout:8/301: mkdir d5/d8/d16/d30/d38/d6b/d72/d7a 0 2024-03-23T08:23:50.352 INFO:tasks.workunit.client.0.smithi070.stdout:6/407: chown d0/d23/d1c/c70 290068 1 2024-03-23T08:23:50.354 INFO:tasks.workunit.client.0.smithi070.stdout:9/413: link db/f60 db/d12/d34/d47/f86 0 2024-03-23T08:23:50.356 INFO:tasks.workunit.client.0.smithi070.stdout:9/414: truncate db/d12/d34/d47/f86 47037 0 2024-03-23T08:23:50.357 INFO:tasks.workunit.client.0.smithi070.stdout:8/302: chown d5/d8/d16/d3b/d4d/l64 70965318 1 2024-03-23T08:23:50.359 INFO:tasks.workunit.client.0.smithi070.stdout:8/303: dread - d5/d8/d16/d3b/f5f zero size 2024-03-23T08:23:50.360 INFO:tasks.workunit.client.0.smithi070.stdout:0/555: dread d16/f1c [0,4194304] 0 2024-03-23T08:23:50.360 INFO:tasks.workunit.client.0.smithi070.stdout:9/415: rename db/d12/d22 to db/d12/d22/d87 22 2024-03-23T08:23:50.360 INFO:tasks.workunit.client.0.smithi070.stdout:8/304: dread - d5/d8/d16/d32/f54 zero size 2024-03-23T08:23:50.362 INFO:tasks.workunit.client.0.smithi070.stdout:8/305: dread - d5/d8/d16/d32/f54 zero size 2024-03-23T08:23:50.362 INFO:tasks.workunit.client.0.smithi070.stdout:6/408: mknod d0/d23/d74/d84/d3c/c8a 0 2024-03-23T08:23:50.367 INFO:tasks.workunit.client.0.smithi070.stdout:0/556: symlink d16/d26/d27/d4c/d60/d75/lc6 0 2024-03-23T08:23:50.368 INFO:tasks.workunit.client.0.smithi070.stdout:9/416: dwrite db/d12/f62 [0,4194304] 0 2024-03-23T08:23:50.370 INFO:tasks.workunit.client.0.smithi070.stdout:9/417: fsync db/d12/d22/d85/f7d 0 2024-03-23T08:23:50.370 INFO:tasks.workunit.client.0.smithi070.stdout:6/409: creat d0/d23/d28/d35/d85/f8b x:0 0 0 2024-03-23T08:23:50.381 INFO:tasks.workunit.client.0.smithi070.stdout:8/306: rename d5/d8/d16/d30/d38/d6b/d72/d7a to d5/d8/d16/d30/d7b 0 2024-03-23T08:23:50.385 INFO:tasks.workunit.client.0.smithi070.stdout:9/418: getdents db/d12/d34 0 2024-03-23T08:23:50.386 INFO:tasks.workunit.client.0.smithi070.stdout:0/557: truncate d16/f18 365178 0 2024-03-23T08:23:50.387 INFO:tasks.workunit.client.0.smithi070.stdout:0/558: chown d16/d22/d43/d47/l65 2363351 1 2024-03-23T08:23:50.389 INFO:tasks.workunit.client.0.smithi070.stdout:9/419: fdatasync f6 0 2024-03-23T08:23:50.393 INFO:tasks.workunit.client.0.smithi070.stdout:8/307: write d5/f2a [824805,39130] 0 2024-03-23T08:23:50.398 INFO:tasks.workunit.client.0.smithi070.stdout:6/410: dwrite d0/d23/d74/d84/f3b [0,4194304] 0 2024-03-23T08:23:50.399 INFO:tasks.workunit.client.0.smithi070.stdout:0/559: dwrite d16/d26/d27/d4c/d60/d75/f76 [0,4194304] 0 2024-03-23T08:23:50.401 INFO:tasks.workunit.client.0.smithi070.stdout:9/420: mknod db/d12/d22/d25/d26/d5c/d7e/c88 0 2024-03-23T08:23:50.402 INFO:tasks.workunit.client.0.smithi070.stdout:0/560: readlink d16/d22/l38 0 2024-03-23T08:23:50.402 INFO:tasks.workunit.client.0.smithi070.stdout:9/421: chown db/d12/d22/d25/c33 198191239 1 2024-03-23T08:23:50.409 INFO:tasks.workunit.client.0.smithi070.stdout:9/422: dwrite db/d12/d22/d25/d26/d5c/f5d [0,4194304] 0 2024-03-23T08:23:50.409 INFO:tasks.workunit.client.0.smithi070.stdout:0/561: rmdir d16/d26/d27/d86 39 2024-03-23T08:23:50.418 INFO:tasks.workunit.client.0.smithi070.stdout:6/411: creat d0/f8c x:0 0 0 2024-03-23T08:23:50.420 INFO:tasks.workunit.client.0.smithi070.stdout:9/423: dwrite db/d12/d22/d85/d42/f54 [0,4194304] 0 2024-03-23T08:23:50.428 INFO:tasks.workunit.client.0.smithi070.stdout:9/424: creat db/d12/f89 x:0 0 0 2024-03-23T08:23:50.429 INFO:tasks.workunit.client.0.smithi070.stdout:6/412: link d0/l33 d0/d23/d74/d84/d3c/d86/l8d 0 2024-03-23T08:23:50.432 INFO:tasks.workunit.client.0.smithi070.stdout:6/413: creat d0/d23/d28/f8e x:0 0 0 2024-03-23T08:23:50.432 INFO:tasks.workunit.client.0.smithi070.stdout:0/562: write f14 [1680561,50678] 0 2024-03-23T08:23:50.437 INFO:tasks.workunit.client.0.smithi070.stdout:9/425: dwrite db/d12/d14/f23 [0,4194304] 0 2024-03-23T08:23:50.438 INFO:tasks.workunit.client.0.smithi070.stdout:9/426: fdatasync db/d12/d22/d25/d26/d5c/d7e/f43 0 2024-03-23T08:23:50.439 INFO:tasks.workunit.client.0.smithi070.stdout:5/247: sync 2024-03-23T08:23:50.439 INFO:tasks.workunit.client.0.smithi070.stdout:3/539: sync 2024-03-23T08:23:50.439 INFO:tasks.workunit.client.0.smithi070.stdout:7/253: sync 2024-03-23T08:23:50.440 INFO:tasks.workunit.client.0.smithi070.stdout:7/254: read f16 [364906,12220] 0 2024-03-23T08:23:50.447 INFO:tasks.workunit.client.0.smithi070.stdout:3/540: creat db/df/d15/d22/db2/fc7 x:0 0 0 2024-03-23T08:23:50.448 INFO:tasks.workunit.client.0.smithi070.stdout:7/255: chown l1a 24 1 2024-03-23T08:23:50.450 INFO:tasks.workunit.client.0.smithi070.stdout:6/414: rename d0/d23/d28/f31 to d0/d23/d1c/f8f 0 2024-03-23T08:23:50.455 INFO:tasks.workunit.client.0.smithi070.stdout:3/541: symlink db/d14/d43/lc8 0 2024-03-23T08:23:50.456 INFO:tasks.workunit.client.0.smithi070.stdout:7/256: rename d25/f2c to d25/d27/d66/d44/f67 0 2024-03-23T08:23:50.458 INFO:tasks.workunit.client.0.smithi070.stdout:9/427: creat db/d12/d22/d25/d26/d5c/f8a x:0 0 0 2024-03-23T08:23:50.459 INFO:tasks.workunit.client.0.smithi070.stdout:9/428: write db/d12/f17 [701888,100752] 0 2024-03-23T08:23:50.460 INFO:tasks.workunit.client.0.smithi070.stdout:5/248: rmdir dc/d1b/d1d/d25/d51 0 2024-03-23T08:23:50.460 INFO:tasks.workunit.client.0.smithi070.stdout:9/429: read db/d12/d22/d85/d42/f54 [734463,9932] 0 2024-03-23T08:23:50.461 INFO:tasks.workunit.client.0.smithi070.stdout:5/249: write dc/f14 [827763,116360] 0 2024-03-23T08:23:50.462 INFO:tasks.workunit.client.0.smithi070.stdout:9/430: write db/d12/f17 [4526135,3290] 0 2024-03-23T08:23:50.463 INFO:tasks.workunit.client.0.smithi070.stdout:3/542: mkdir db/d14/d65/daa/dc9 0 2024-03-23T08:23:50.465 INFO:tasks.workunit.client.0.smithi070.stdout:7/257: creat d25/d40/f68 x:0 0 0 2024-03-23T08:23:50.467 INFO:tasks.workunit.client.0.smithi070.stdout:6/415: mknod d0/d23/c90 0 2024-03-23T08:23:50.467 INFO:tasks.workunit.client.0.smithi070.stdout:7/258: truncate d25/d27/f4c 1241305 0 2024-03-23T08:23:50.471 INFO:tasks.workunit.client.0.smithi070.stdout:0/563: dread f14 [4194304,4194304] 0 2024-03-23T08:23:50.477 INFO:tasks.workunit.client.0.smithi070.stdout:5/250: unlink dc/d1b/d1d/d25/d36/f2e 0 2024-03-23T08:23:50.479 INFO:tasks.workunit.client.0.smithi070.stdout:3/543: mkdir db/df/d15/dca 0 2024-03-23T08:23:50.479 INFO:tasks.workunit.client.0.smithi070.stdout:3/544: chown db/d14/d9c/f9f 11 1 2024-03-23T08:23:50.483 INFO:tasks.workunit.client.0.smithi070.stdout:7/259: symlink d25/d27/l69 0 2024-03-23T08:23:50.484 INFO:tasks.workunit.client.0.smithi070.stdout:7/260: truncate d25/d27/f45 817296 0 2024-03-23T08:23:50.484 INFO:tasks.workunit.client.0.smithi070.stdout:7/261: readlink d25/d27/l69 0 2024-03-23T08:23:50.488 INFO:tasks.workunit.client.0.smithi070.stdout:3/545: mkdir db/d14/d65/daa/dcb 0 2024-03-23T08:23:50.489 INFO:tasks.workunit.client.0.smithi070.stdout:6/416: symlink d0/l91 0 2024-03-23T08:23:50.491 INFO:tasks.workunit.client.0.smithi070.stdout:0/564: symlink d16/d26/d27/lc7 0 2024-03-23T08:23:50.493 INFO:tasks.workunit.client.0.smithi070.stdout:0/565: write d16/d26/fab [569837,8025] 0 2024-03-23T08:23:50.494 INFO:tasks.workunit.client.0.smithi070.stdout:7/262: rename d25/d27/d66 to d25/d27/d6a 0 2024-03-23T08:23:50.496 INFO:tasks.workunit.client.0.smithi070.stdout:3/546: dwrite db/df/d15/d22/fc5 [0,4194304] 0 2024-03-23T08:23:50.502 INFO:tasks.workunit.client.0.smithi070.stdout:7/263: mknod d25/d27/d6a/c6b 0 2024-03-23T08:23:50.503 INFO:tasks.workunit.client.0.smithi070.stdout:7/264: dread d25/f4f [0,4194304] 0 2024-03-23T08:23:50.504 INFO:tasks.workunit.client.0.smithi070.stdout:9/431: getdents db/d12/d14 0 2024-03-23T08:23:50.509 INFO:tasks.workunit.client.0.smithi070.stdout:3/547: read db/f21 [1015543,27195] 0 2024-03-23T08:23:50.511 INFO:tasks.workunit.client.0.smithi070.stdout:7/265: symlink d25/d27/d6a/d44/l6c 0 2024-03-23T08:23:50.513 INFO:tasks.workunit.client.0.smithi070.stdout:6/417: link d0/d23/d74/d84/d3c/d5b/l81 d0/d23/d28/l92 0 2024-03-23T08:23:50.515 INFO:tasks.workunit.client.0.smithi070.stdout:0/566: creat d16/d26/d27/fc8 x:0 0 0 2024-03-23T08:23:50.516 INFO:tasks.workunit.client.0.smithi070.stdout:9/432: dwrite db/d12/d22/d25/d26/f2c [0,4194304] 0 2024-03-23T08:23:50.516 INFO:tasks.workunit.client.0.smithi070.stdout:5/251: dwrite dc/d1b/d1d/d25/d38/d39/f3c [0,4194304] 0 2024-03-23T08:23:50.516 INFO:tasks.workunit.client.0.smithi070.stdout:0/567: dread - d16/d26/f36 zero size 2024-03-23T08:23:50.517 INFO:tasks.workunit.client.0.smithi070.stdout:3/548: mkdir db/d20/d7a/dcc 0 2024-03-23T08:23:50.520 INFO:tasks.workunit.client.0.smithi070.stdout:7/266: creat d25/d27/f6d x:0 0 0 2024-03-23T08:23:50.526 INFO:tasks.workunit.client.0.smithi070.stdout:6/418: mkdir d0/d23/d28/d93 0 2024-03-23T08:23:50.526 INFO:tasks.workunit.client.0.smithi070.stdout:7/267: write d25/d27/f3c [1001675,97082] 0 2024-03-23T08:23:50.527 INFO:tasks.workunit.client.0.smithi070.stdout:0/568: mkdir d16/d26/d27/d3d/d4d/dc9 0 2024-03-23T08:23:50.530 INFO:tasks.workunit.client.0.smithi070.stdout:3/549: rename db/df/d15/d22/db1 to db/d7f/dcd 0 2024-03-23T08:23:50.578 INFO:tasks.workunit.client.0.smithi070.stdout:3/550: rename db/d4e/d92/dbb to db/d4e/d92/dbb/dce 22 2024-03-23T08:23:50.579 INFO:tasks.workunit.client.0.smithi070.stdout:9/433: mknod db/d12/d22/d25/d58/c8b 0 2024-03-23T08:23:50.579 INFO:tasks.workunit.client.0.smithi070.stdout:7/268: mknod d25/d27/d6a/c6e 0 2024-03-23T08:23:50.579 INFO:tasks.workunit.client.0.smithi070.stdout:0/569: mkdir d16/d26/d27/d3d/d4d/dca 0 2024-03-23T08:23:50.582 INFO:tasks.workunit.client.0.smithi070.stdout:0/570: chown d16/f1d 971 1 2024-03-23T08:23:50.583 INFO:tasks.workunit.client.0.smithi070.stdout:7/269: mkdir d25/d27/d6a/d44/d6f 0 2024-03-23T08:23:50.585 INFO:tasks.workunit.client.0.smithi070.stdout:9/434: unlink db/d12/d22/d85/d42/f54 0 2024-03-23T08:23:50.588 INFO:tasks.workunit.client.0.smithi070.stdout:7/270: creat d25/d27/f70 x:0 0 0 2024-03-23T08:23:50.590 INFO:tasks.workunit.client.0.smithi070.stdout:3/551: getdents db/d14/d65/daa/dcb 0 2024-03-23T08:23:50.590 INFO:tasks.workunit.client.0.smithi070.stdout:6/419: dread d0/f50 [4194304,4194304] 0 2024-03-23T08:23:50.591 INFO:tasks.workunit.client.0.smithi070.stdout:0/571: dwrite d16/f51 [0,4194304] 0 2024-03-23T08:23:50.594 INFO:tasks.workunit.client.0.smithi070.stdout:3/552: creat db/df/d5d/fcf x:0 0 0 2024-03-23T08:23:50.596 INFO:tasks.workunit.client.0.smithi070.stdout:0/572: creat d16/d22/d43/d47/d87/fcb x:0 0 0 2024-03-23T08:23:50.603 INFO:tasks.workunit.client.0.smithi070.stdout:7/271: dwrite f12 [0,4194304] 0 2024-03-23T08:23:50.607 INFO:tasks.workunit.client.0.smithi070.stdout:6/420: dwrite d0/d23/d74/d84/d3c/d5b/f79 [4194304,4194304] 0 2024-03-23T08:23:50.607 INFO:tasks.workunit.client.0.smithi070.stdout:0/573: rename d16/f1f to d16/d26/d27/d4c/d60/d75/fcc 0 2024-03-23T08:23:50.610 INFO:tasks.workunit.client.0.smithi070.stdout:3/553: dwrite db/d4e/f73 [4194304,4194304] 0 2024-03-23T08:23:50.615 INFO:tasks.workunit.client.0.smithi070.stdout:7/272: creat d25/d27/d6a/d44/d6f/f71 x:0 0 0 2024-03-23T08:23:50.615 INFO:tasks.workunit.client.0.smithi070.stdout:3/554: rmdir db/d20/d63 39 2024-03-23T08:23:50.618 INFO:tasks.workunit.client.0.smithi070.stdout:6/421: creat d0/d23/d88/f94 x:0 0 0 2024-03-23T08:23:50.622 INFO:tasks.workunit.client.0.smithi070.stdout:3/555: dread db/f17 [0,4194304] 0 2024-03-23T08:23:50.623 INFO:tasks.workunit.client.0.smithi070.stdout:7/273: mkdir d25/d72 0 2024-03-23T08:23:50.624 INFO:tasks.workunit.client.0.smithi070.stdout:0/574: rename d16/d26/d27/d86/d67/ca0 to d16/d26/da8/ccd 0 2024-03-23T08:23:50.624 INFO:tasks.workunit.client.0.smithi070.stdout:4/205: sync 2024-03-23T08:23:50.625 INFO:tasks.workunit.client.0.smithi070.stdout:2/261: sync 2024-03-23T08:23:50.626 INFO:tasks.workunit.client.0.smithi070.stdout:4/206: truncate d3/f23 1245075 0 2024-03-23T08:23:50.627 INFO:tasks.workunit.client.0.smithi070.stdout:6/422: mknod d0/d23/d1c/d38/d52/c95 0 2024-03-23T08:23:50.628 INFO:tasks.workunit.client.0.smithi070.stdout:2/262: read d1/d6/d4f/f37 [1701774,86470] 0 2024-03-23T08:23:50.630 INFO:tasks.workunit.client.0.smithi070.stdout:7/274: creat d25/d27/d6a/f73 x:0 0 0 2024-03-23T08:23:50.631 INFO:tasks.workunit.client.0.smithi070.stdout:0/575: fdatasync d16/d26/d27/d4c/d60/f79 0 2024-03-23T08:23:50.632 INFO:tasks.workunit.client.0.smithi070.stdout:0/576: chown d16/d22/d43/d47 31883 1 2024-03-23T08:23:50.632 INFO:tasks.workunit.client.0.smithi070.stdout:0/577: readlink lc 0 2024-03-23T08:23:50.634 INFO:tasks.workunit.client.0.smithi070.stdout:4/207: rename d3/d6/d11/f36 to d3/d19/f4c 0 2024-03-23T08:23:50.635 INFO:tasks.workunit.client.0.smithi070.stdout:6/423: symlink d0/d23/d4d/l96 0 2024-03-23T08:23:50.639 INFO:tasks.workunit.client.0.smithi070.stdout:7/275: write f24 [1187700,57837] 0 2024-03-23T08:23:50.642 INFO:tasks.workunit.client.0.smithi070.stdout:0/578: dwrite d16/d26/d27/d86/d67/d7d/faa [0,4194304] 0 2024-03-23T08:23:50.642 INFO:tasks.workunit.client.0.smithi070.stdout:4/208: dread d3/d6/f2f [0,4194304] 0 2024-03-23T08:23:50.644 INFO:tasks.workunit.client.0.smithi070.stdout:0/579: dread - d16/d26/d27/d3d/f72 zero size 2024-03-23T08:23:50.648 INFO:tasks.workunit.client.0.smithi070.stdout:0/580: truncate d16/f6f 511306 0 2024-03-23T08:23:50.652 INFO:tasks.workunit.client.0.smithi070.stdout:6/424: mkdir d0/d23/d97 0 2024-03-23T08:23:50.652 INFO:tasks.workunit.client.0.smithi070.stdout:3/556: link db/d20/d63/c7e db/d7f/dcd/cd0 0 2024-03-23T08:23:50.654 INFO:tasks.workunit.client.0.smithi070.stdout:6/425: write d0/d23/f17 [4919383,21515] 0 2024-03-23T08:23:50.656 INFO:tasks.workunit.client.0.smithi070.stdout:2/263: getdents d1/d6 0 2024-03-23T08:23:50.656 INFO:tasks.workunit.client.0.smithi070.stdout:6/426: chown d0/d23/d1c/d38 0 1 2024-03-23T08:23:50.658 INFO:tasks.workunit.client.0.smithi070.stdout:7/276: dread f1d [0,4194304] 0 2024-03-23T08:23:50.659 INFO:tasks.workunit.client.0.smithi070.stdout:0/581: creat d16/d26/d27/d86/db6/fce x:0 0 0 2024-03-23T08:23:50.660 INFO:tasks.workunit.client.0.smithi070.stdout:3/557: mknod db/d20/cd1 0 2024-03-23T08:23:50.662 INFO:tasks.workunit.client.0.smithi070.stdout:2/264: symlink d1/d21/l57 0 2024-03-23T08:23:50.662 INFO:tasks.workunit.client.0.smithi070.stdout:2/265: stat d1/d6/d4f/l50 0 2024-03-23T08:23:50.664 INFO:tasks.workunit.client.0.smithi070.stdout:4/209: dread d3/fc [0,4194304] 0 2024-03-23T08:23:50.666 INFO:tasks.workunit.client.0.smithi070.stdout:7/277: mknod d25/c74 0 2024-03-23T08:23:50.667 INFO:tasks.workunit.client.0.smithi070.stdout:7/278: dread - d25/d27/d6a/d44/f53 zero size 2024-03-23T08:23:50.668 INFO:tasks.workunit.client.0.smithi070.stdout:7/279: write f23 [387924,26817] 0 2024-03-23T08:23:50.669 INFO:tasks.workunit.client.0.smithi070.stdout:3/558: symlink db/d20/d7a/ld2 0 2024-03-23T08:23:50.670 INFO:tasks.workunit.client.0.smithi070.stdout:2/266: mkdir d1/d7/d39/d58 0 2024-03-23T08:23:50.671 INFO:tasks.workunit.client.0.smithi070.stdout:2/267: chown d1/d6/d4f/f37 10624417 1 2024-03-23T08:23:50.678 INFO:tasks.workunit.client.0.smithi070.stdout:3/559: dwrite db/d14/d65/daa/fad [4194304,4194304] 0 2024-03-23T08:23:50.682 INFO:tasks.workunit.client.0.smithi070.stdout:2/268: symlink d1/d6/d34/l59 0 2024-03-23T08:23:50.682 INFO:tasks.workunit.client.0.smithi070.stdout:6/427: dwrite d0/d23/d1c/f2e [4194304,4194304] 0 2024-03-23T08:23:50.685 INFO:tasks.workunit.client.0.smithi070.stdout:3/560: chown db/df/d15/d22/l3b 9 1 2024-03-23T08:23:50.685 INFO:tasks.workunit.client.0.smithi070.stdout:4/210: write d3/f17 [384701,63839] 0 2024-03-23T08:23:50.687 INFO:tasks.workunit.client.0.smithi070.stdout:4/211: stat d3/d6/d9/l32 0 2024-03-23T08:23:50.687 INFO:tasks.workunit.client.0.smithi070.stdout:0/582: creat d16/d26/d27/d4c/d60/d7c/fcf x:0 0 0 2024-03-23T08:23:50.690 INFO:tasks.workunit.client.0.smithi070.stdout:2/269: creat d1/d6/d34/f5a x:0 0 0 2024-03-23T08:23:50.691 INFO:tasks.workunit.client.0.smithi070.stdout:2/270: fdatasync d1/d6/d4f/f3e 0 2024-03-23T08:23:50.693 INFO:tasks.workunit.client.0.smithi070.stdout:2/271: write d1/d6/d30/f3a [1411111,60514] 0 2024-03-23T08:23:50.698 INFO:tasks.workunit.client.0.smithi070.stdout:0/583: chown d16/d26/cbc 60 1 2024-03-23T08:23:50.698 INFO:tasks.workunit.client.0.smithi070.stdout:0/584: fdatasync d16/d22/d43/d47/f81 0 2024-03-23T08:23:50.698 INFO:tasks.workunit.client.0.smithi070.stdout:6/428: mkdir d0/d23/d97/d98 0 2024-03-23T08:23:50.699 INFO:tasks.workunit.client.0.smithi070.stdout:0/585: truncate d16/d22/d43/d47/fa5 104621 0 2024-03-23T08:23:50.699 INFO:tasks.workunit.client.0.smithi070.stdout:3/561: symlink db/d14/d65/daa/dcb/ld3 0 2024-03-23T08:23:50.703 INFO:tasks.workunit.client.0.smithi070.stdout:6/429: rename d0/l91 to d0/d23/d1c/d38/d52/l99 0 2024-03-23T08:23:50.707 INFO:tasks.workunit.client.0.smithi070.stdout:6/430: mkdir d0/d23/d1c/d38/d9a 0 2024-03-23T08:23:50.708 INFO:tasks.workunit.client.0.smithi070.stdout:3/562: creat db/d14/fd4 x:0 0 0 2024-03-23T08:23:50.709 INFO:tasks.workunit.client.0.smithi070.stdout:0/586: dwrite d16/d22/d43/f62 [0,4194304] 0 2024-03-23T08:23:50.709 INFO:tasks.workunit.client.0.smithi070.stdout:6/431: write d0/f8c [642326,106484] 0 2024-03-23T08:23:50.713 INFO:tasks.workunit.client.0.smithi070.stdout:6/432: chown d0/d3/c71 90070 1 2024-03-23T08:23:50.713 INFO:tasks.workunit.client.0.smithi070.stdout:3/563: symlink db/df/d15/d3d/ld5 0 2024-03-23T08:23:50.714 INFO:tasks.workunit.client.0.smithi070.stdout:4/212: write d3/d6/d11/d1a/f40 [1257318,43851] 0 2024-03-23T08:23:50.714 INFO:tasks.workunit.client.0.smithi070.stdout:6/433: read - d0/d23/d88/f94 zero size 2024-03-23T08:23:50.717 INFO:tasks.workunit.client.0.smithi070.stdout:4/213: mknod d3/d19/c4d 0 2024-03-23T08:23:50.718 INFO:tasks.workunit.client.0.smithi070.stdout:4/214: chown d3/d6/f43 1089303985 1 2024-03-23T08:23:50.719 INFO:tasks.workunit.client.0.smithi070.stdout:6/434: symlink d0/d23/d88/l9b 0 2024-03-23T08:23:50.719 INFO:tasks.workunit.client.0.smithi070.stdout:6/435: chown d0/f20 43462266 1 2024-03-23T08:23:50.720 INFO:tasks.workunit.client.0.smithi070.stdout:4/215: fdatasync d3/d6/d11/d1a/d1b/f28 0 2024-03-23T08:23:50.720 INFO:tasks.workunit.client.0.smithi070.stdout:0/587: creat d16/d26/d27/d4c/d60/d7c/fd0 x:0 0 0 2024-03-23T08:23:50.722 INFO:tasks.workunit.client.0.smithi070.stdout:0/588: chown d16/f1d 91043466 1 2024-03-23T08:23:50.723 INFO:tasks.workunit.client.0.smithi070.stdout:2/272: dread d1/d7/d11/f28 [0,4194304] 0 2024-03-23T08:23:50.725 INFO:tasks.workunit.client.0.smithi070.stdout:2/273: fsync d1/d6/d30/f3a 0 2024-03-23T08:23:50.730 INFO:tasks.workunit.client.0.smithi070.stdout:4/216: dread d3/d6/d45/f47 [0,4194304] 0 2024-03-23T08:23:50.730 INFO:tasks.workunit.client.0.smithi070.stdout:0/589: dwrite d16/d22/d43/fa1 [0,4194304] 0 2024-03-23T08:23:50.735 INFO:tasks.workunit.client.0.smithi070.stdout:2/274: dwrite d1/d7/d11/f12 [0,4194304] 0 2024-03-23T08:23:50.737 INFO:tasks.workunit.client.0.smithi070.stdout:4/217: creat d3/d6/d11/d1a/d27/f4e x:0 0 0 2024-03-23T08:23:50.738 INFO:tasks.workunit.client.0.smithi070.stdout:6/436: write d0/d3/d2c/f34 [508360,28034] 0 2024-03-23T08:23:50.746 INFO:tasks.workunit.client.0.smithi070.stdout:0/590: link d16/d26/l4e d16/d26/d27/d3d/d4d/d9a/ld1 0 2024-03-23T08:23:50.750 INFO:tasks.workunit.client.0.smithi070.stdout:2/275: truncate d1/d6/d4f/f3c 3044519 0 2024-03-23T08:23:50.752 INFO:tasks.workunit.client.0.smithi070.stdout:2/276: write d1/fe [518103,116825] 0 2024-03-23T08:23:50.754 INFO:tasks.workunit.client.0.smithi070.stdout:0/591: dwrite d16/d22/d43/d47/fa5 [0,4194304] 0 2024-03-23T08:23:50.762 INFO:tasks.workunit.client.0.smithi070.stdout:6/437: truncate d0/d23/d74/d84/d3c/d5b/f79 3852356 0 2024-03-23T08:23:50.763 INFO:tasks.workunit.client.0.smithi070.stdout:6/438: write d0/d23/f55 [190063,73265] 0 2024-03-23T08:23:50.763 INFO:tasks.workunit.client.0.smithi070.stdout:2/277: read d1/d7/df/f27 [293848,94172] 0 2024-03-23T08:23:50.765 INFO:tasks.workunit.client.0.smithi070.stdout:6/439: chown d0/d3/c7 60231 1 2024-03-23T08:23:50.768 INFO:tasks.workunit.client.0.smithi070.stdout:6/440: truncate d0/d23/d28/d35/f77 836718 0 2024-03-23T08:23:50.771 INFO:tasks.workunit.client.0.smithi070.stdout:0/592: getdents d16/d26/d27/d3d/d4d/dca 0 2024-03-23T08:23:50.772 INFO:tasks.workunit.client.0.smithi070.stdout:2/278: dwrite d1/d7/d11/f25 [0,4194304] 0 2024-03-23T08:23:50.772 INFO:tasks.workunit.client.0.smithi070.stdout:0/593: fdatasync d16/f1a 0 2024-03-23T08:23:50.776 INFO:tasks.workunit.client.0.smithi070.stdout:0/594: write d16/d26/d27/d4c/d60/d75/fc3 [312776,120505] 0 2024-03-23T08:23:50.776 INFO:tasks.workunit.client.0.smithi070.stdout:6/441: dread d0/d23/d1c/d38/d52/f7c [0,4194304] 0 2024-03-23T08:23:50.778 INFO:tasks.workunit.client.0.smithi070.stdout:6/442: read d0/d3/d2c/f58 [820167,50725] 0 2024-03-23T08:23:50.779 INFO:tasks.workunit.client.0.smithi070.stdout:6/443: fdatasync d0/d3/f7e 0 2024-03-23T08:23:50.779 INFO:tasks.workunit.client.0.smithi070.stdout:0/595: truncate d16/d26/d27/d4c/d60/d7c/fcf 732949 0 2024-03-23T08:23:50.781 INFO:tasks.workunit.client.0.smithi070.stdout:2/279: mkdir d1/d7/d48/d5b 0 2024-03-23T08:23:50.782 INFO:tasks.workunit.client.0.smithi070.stdout:2/280: write d1/d6/d30/f3a [2515832,112641] 0 2024-03-23T08:23:50.786 INFO:tasks.workunit.client.0.smithi070.stdout:0/596: creat d16/d26/d27/d4c/d60/d75/fd2 x:0 0 0 2024-03-23T08:23:50.794 INFO:tasks.workunit.client.0.smithi070.stdout:0/597: rmdir d16/d26/d27/d3d/d4d 39 2024-03-23T08:23:50.799 INFO:tasks.workunit.client.0.smithi070.stdout:6/444: creat d0/f9c x:0 0 0 2024-03-23T08:23:50.804 INFO:tasks.workunit.client.0.smithi070.stdout:6/445: symlink d0/d23/l9d 0 2024-03-23T08:23:50.805 INFO:tasks.workunit.client.0.smithi070.stdout:6/446: chown d0/d3/f80 72 1 2024-03-23T08:23:50.806 INFO:tasks.workunit.client.0.smithi070.stdout:0/598: creat d16/d22/fd3 x:0 0 0 2024-03-23T08:23:50.806 INFO:tasks.workunit.client.0.smithi070.stdout:6/447: write d0/d3/f80 [689722,97216] 0 2024-03-23T08:23:50.810 INFO:tasks.workunit.client.0.smithi070.stdout:2/281: truncate d1/d6/f3b 418446 0 2024-03-23T08:23:50.811 INFO:tasks.workunit.client.0.smithi070.stdout:2/282: truncate d1/d7/d11/f2b 1586461 0 2024-03-23T08:23:50.812 INFO:tasks.workunit.client.0.smithi070.stdout:6/448: truncate d0/d23/d1c/f2b 1044480 0 2024-03-23T08:23:50.813 INFO:tasks.workunit.client.0.smithi070.stdout:6/449: chown d0/d23/d1c/d38/l4e 5109081 1 2024-03-23T08:23:50.813 INFO:tasks.workunit.client.0.smithi070.stdout:0/599: creat d16/d26/d27/d4c/d60/d7c/fd4 x:0 0 0 2024-03-23T08:23:50.818 INFO:tasks.workunit.client.0.smithi070.stdout:2/283: dwrite d1/d29/f2d [0,4194304] 0 2024-03-23T08:23:50.819 INFO:tasks.workunit.client.0.smithi070.stdout:2/284: fsync d1/d7/d11/f56 0 2024-03-23T08:23:50.826 INFO:tasks.workunit.client.0.smithi070.stdout:0/600: link d16/d22/f24 d16/d26/d27/d4c/d60/d7c/fd5 0 2024-03-23T08:23:50.826 INFO:tasks.workunit.client.0.smithi070.stdout:6/450: write d0/d3/d2c/f49 [1665446,69017] 0 2024-03-23T08:23:50.833 INFO:tasks.workunit.client.0.smithi070.stdout:6/451: fdatasync d0/d23/d1c/f5d 0 2024-03-23T08:23:50.835 INFO:tasks.workunit.client.0.smithi070.stdout:0/601: stat d16/d26/d27/d4c/d60/d75/f76 0 2024-03-23T08:23:50.837 INFO:tasks.workunit.client.0.smithi070.stdout:2/285: dwrite d1/d6/f4d [0,4194304] 0 2024-03-23T08:23:50.838 INFO:tasks.workunit.client.0.smithi070.stdout:2/286: chown d1/d7/d11/d38/d4c/f4e 2302010 1 2024-03-23T08:23:50.839 INFO:tasks.workunit.client.0.smithi070.stdout:2/287: fdatasync d1/d7/d11/f12 0 2024-03-23T08:23:50.843 INFO:tasks.workunit.client.0.smithi070.stdout:0/602: write d16/f3e [4622881,63120] 0 2024-03-23T08:23:50.846 INFO:tasks.workunit.client.0.smithi070.stdout:2/288: creat d1/d7/d11/d38/d4c/f5c x:0 0 0 2024-03-23T08:23:50.847 INFO:tasks.workunit.client.0.smithi070.stdout:2/289: chown d1/d7/df/c2a 7809353 1 2024-03-23T08:23:50.848 INFO:tasks.workunit.client.0.smithi070.stdout:0/603: creat d16/d22/d43/d47/fd6 x:0 0 0 2024-03-23T08:23:50.853 INFO:tasks.workunit.client.0.smithi070.stdout:0/604: symlink d16/d26/d27/d3d/d4d/dc9/ld7 0 2024-03-23T08:23:50.855 INFO:tasks.workunit.client.0.smithi070.stdout:1/340: sync 2024-03-23T08:23:50.855 INFO:tasks.workunit.client.0.smithi070.stdout:8/308: sync 2024-03-23T08:23:50.855 INFO:tasks.workunit.client.0.smithi070.stdout:0/605: mknod d16/d26/d27/d4c/d60/cd8 0 2024-03-23T08:23:50.856 INFO:tasks.workunit.client.0.smithi070.stdout:8/309: stat d5/d8/f2c 0 2024-03-23T08:23:50.857 INFO:tasks.workunit.client.0.smithi070.stdout:8/310: chown d5/d8/fb 3913 1 2024-03-23T08:23:50.857 INFO:tasks.workunit.client.0.smithi070.stdout:1/341: symlink d8/d26/d7b/l80 0 2024-03-23T08:23:50.859 INFO:tasks.workunit.client.0.smithi070.stdout:0/606: creat d16/d26/d27/d3d/d4d/dad/fd9 x:0 0 0 2024-03-23T08:23:50.860 INFO:tasks.workunit.client.0.smithi070.stdout:0/607: chown d16/d26/d27/d4c/d60/d7c/f83 193087 1 2024-03-23T08:23:50.860 INFO:tasks.workunit.client.0.smithi070.stdout:8/311: mkdir d5/d8/d16/d3b/d4d/d6f/d7c 0 2024-03-23T08:23:50.861 INFO:tasks.workunit.client.0.smithi070.stdout:2/290: fsync d1/d7/d11/d38/d4c/f5c 0 2024-03-23T08:23:50.861 INFO:tasks.workunit.client.0.smithi070.stdout:0/608: readlink d16/d26/d27/d4c/la2 0 2024-03-23T08:23:50.861 INFO:tasks.workunit.client.0.smithi070.stdout:1/342: dread d8/d11/f3b [0,4194304] 0 2024-03-23T08:23:50.863 INFO:tasks.workunit.client.0.smithi070.stdout:0/609: fdatasync d16/d22/d43/f78 0 2024-03-23T08:23:50.863 INFO:tasks.workunit.client.0.smithi070.stdout:2/291: symlink d1/d21/l5d 0 2024-03-23T08:23:50.864 INFO:tasks.workunit.client.0.smithi070.stdout:0/610: write d16/f30 [1814291,62057] 0 2024-03-23T08:23:50.867 INFO:tasks.workunit.client.0.smithi070.stdout:1/343: creat d8/db/d2c/d3f/d48/f81 x:0 0 0 2024-03-23T08:23:50.869 INFO:tasks.workunit.client.0.smithi070.stdout:8/312: mkdir d5/d8/d16/d3b/d4d/d6f/d7c/d7d 0 2024-03-23T08:23:50.870 INFO:tasks.workunit.client.0.smithi070.stdout:1/344: dread - d8/f37 zero size 2024-03-23T08:23:50.873 INFO:tasks.workunit.client.0.smithi070.stdout:0/611: dwrite d16/d26/d27/fc8 [0,4194304] 0 2024-03-23T08:23:50.875 INFO:tasks.workunit.client.0.smithi070.stdout:8/313: creat d5/d8/d16/d3b/d4d/d6f/f7e x:0 0 0 2024-03-23T08:23:50.875 INFO:tasks.workunit.client.0.smithi070.stdout:0/612: dread - d16/d26/d27/d4c/d60/d75/f9c zero size 2024-03-23T08:23:50.876 INFO:tasks.workunit.client.0.smithi070.stdout:0/613: chown d16/d26/d27/d4c/d60/d75 528 1 2024-03-23T08:23:50.876 INFO:tasks.workunit.client.0.smithi070.stdout:0/614: fdatasync d16/d26/d27/d3d/d4d/dad/fd9 0 2024-03-23T08:23:50.880 INFO:tasks.workunit.client.0.smithi070.stdout:0/615: creat d16/fda x:0 0 0 2024-03-23T08:23:50.888 INFO:tasks.workunit.client.0.smithi070.stdout:0/616: rename d16/d26/d27/d4c/d60/d7c/fd5 to d16/d26/d27/d4c/d60/d7c/daf/fdb 0 2024-03-23T08:23:50.891 INFO:tasks.workunit.client.0.smithi070.stdout:2/292: truncate d1/d6/f4d 3486918 0 2024-03-23T08:23:50.892 INFO:tasks.workunit.client.0.smithi070.stdout:0/617: rename d16/d26/d27/d4c/d60/d7c/fd4 to d16/d26/d27/d86/db6/fdc 0 2024-03-23T08:23:50.896 INFO:tasks.workunit.client.0.smithi070.stdout:2/293: unlink d1/d7/f41 0 2024-03-23T08:23:50.900 INFO:tasks.workunit.client.0.smithi070.stdout:2/294: mknod d1/d7/d39/c5e 0 2024-03-23T08:23:50.901 INFO:tasks.workunit.client.0.smithi070.stdout:0/618: dwrite d16/d26/f36 [0,4194304] 0 2024-03-23T08:23:50.902 INFO:tasks.workunit.client.0.smithi070.stdout:8/314: dwrite d5/d8/fb [4194304,4194304] 0 2024-03-23T08:23:50.906 INFO:tasks.workunit.client.0.smithi070.stdout:2/295: mkdir d1/d21/d5f 0 2024-03-23T08:23:50.907 INFO:tasks.workunit.client.0.smithi070.stdout:0/619: dread - d16/d22/d43/d47/fd6 zero size 2024-03-23T08:23:50.907 INFO:tasks.workunit.client.0.smithi070.stdout:8/315: chown d5/d8/d16/d30/f2f 296484392 1 2024-03-23T08:23:50.907 INFO:tasks.workunit.client.0.smithi070.stdout:2/296: chown d1/d7/l1a 23693 1 2024-03-23T08:23:50.910 INFO:tasks.workunit.client.0.smithi070.stdout:2/297: write d1/d7/f17 [4786585,16576] 0 2024-03-23T08:23:50.915 INFO:tasks.workunit.client.0.smithi070.stdout:0/620: dwrite d16/d26/d27/f91 [0,4194304] 0 2024-03-23T08:23:51.021 INFO:tasks.workunit.client.0.smithi070.stdout:8/316: creat d5/d8/d16/d3b/d4d/d6f/d7c/d7d/f7f x:0 0 0 2024-03-23T08:23:51.023 INFO:tasks.workunit.client.0.smithi070.stdout:8/317: write d5/d8/f25 [1661178,110663] 0 2024-03-23T08:23:51.027 INFO:tasks.workunit.client.0.smithi070.stdout:2/298: mknod d1/d7/d11/c60 0 2024-03-23T08:23:51.028 INFO:tasks.workunit.client.0.smithi070.stdout:0/621: mkdir d16/d26/d27/d86/d9e/ddd 0 2024-03-23T08:23:51.032 INFO:tasks.workunit.client.0.smithi070.stdout:8/318: creat d5/d8/d16/d3b/d4d/d6f/d7c/f80 x:0 0 0 2024-03-23T08:23:51.033 INFO:tasks.workunit.client.0.smithi070.stdout:8/319: chown d5/d8/d16/d32/c4c 43310 1 2024-03-23T08:23:51.035 INFO:tasks.workunit.client.0.smithi070.stdout:2/299: read d1/d6/f3b [245243,130322] 0 2024-03-23T08:23:51.036 INFO:tasks.workunit.client.0.smithi070.stdout:0/622: rename d16/d26/da8 to d16/d26/d27/d86/db6/dde 0 2024-03-23T08:23:51.037 INFO:tasks.workunit.client.0.smithi070.stdout:0/623: fsync d16/d26/d27/d86/db6/dde/fbf 0 2024-03-23T08:23:51.038 INFO:tasks.workunit.client.0.smithi070.stdout:8/320: symlink d5/d8/d16/d3b/d4d/d6f/d7c/l81 0 2024-03-23T08:23:51.042 INFO:tasks.workunit.client.0.smithi070.stdout:8/321: dread d5/d8/fb [4194304,4194304] 0 2024-03-23T08:23:51.045 INFO:tasks.workunit.client.0.smithi070.stdout:2/300: creat d1/d7/f61 x:0 0 0 2024-03-23T08:23:51.047 INFO:tasks.workunit.client.0.smithi070.stdout:0/624: dwrite d16/d26/d27/d86/db6/fdc [0,4194304] 0 2024-03-23T08:23:51.048 INFO:tasks.workunit.client.0.smithi070.stdout:8/322: unlink d5/d8/d16/d3b/d4d/c53 0 2024-03-23T08:23:51.049 INFO:tasks.workunit.client.0.smithi070.stdout:8/323: dread - d5/d8/f3a zero size 2024-03-23T08:23:51.049 INFO:tasks.workunit.client.0.smithi070.stdout:0/625: truncate d16/d26/f6b 991164 0 2024-03-23T08:23:51.053 INFO:tasks.workunit.client.0.smithi070.stdout:2/301: mknod d1/d7/d39/d58/c62 0 2024-03-23T08:23:51.053 INFO:tasks.workunit.client.0.smithi070.stdout:0/626: truncate d16/d22/d43/d47/d87/fcb 465987 0 2024-03-23T08:23:51.056 INFO:tasks.workunit.client.0.smithi070.stdout:2/302: symlink d1/d7/df/l63 0 2024-03-23T08:23:51.058 INFO:tasks.workunit.client.0.smithi070.stdout:0/627: creat d16/d26/d27/d3d/d4d/dc9/fdf x:0 0 0 2024-03-23T08:23:51.061 INFO:tasks.workunit.client.0.smithi070.stdout:8/324: dwrite d5/f23 [0,4194304] 0 2024-03-23T08:23:51.063 INFO:tasks.workunit.client.0.smithi070.stdout:2/303: truncate d1/d6/f3b 121538 0 2024-03-23T08:23:51.070 INFO:tasks.workunit.client.0.smithi070.stdout:8/325: dwrite d5/d8/d16/d32/f5c [0,4194304] 0 2024-03-23T08:23:51.078 INFO:tasks.workunit.client.0.smithi070.stdout:2/304: getdents d1/d7/d11/d38/d4c 0 2024-03-23T08:23:51.078 INFO:tasks.workunit.client.0.smithi070.stdout:8/326: getdents d5/d8/d16/d3b/d4d 0 2024-03-23T08:23:51.079 INFO:tasks.workunit.client.0.smithi070.stdout:2/305: mknod d1/d7/d39/d58/c64 0 2024-03-23T08:23:51.080 INFO:tasks.workunit.client.0.smithi070.stdout:5/252: sync 2024-03-23T08:23:51.080 INFO:tasks.workunit.client.0.smithi070.stdout:2/306: read d1/d29/f35 [2977167,90486] 0 2024-03-23T08:23:51.082 INFO:tasks.workunit.client.0.smithi070.stdout:8/327: symlink d5/d8/d16/d30/d38/d6b/d72/l82 0 2024-03-23T08:23:51.094 INFO:tasks.workunit.client.0.smithi070.stdout:5/253: dwrite dc/d1b/d1d/d25/d36/f27 [0,4194304] 0 2024-03-23T08:23:51.096 INFO:tasks.workunit.client.0.smithi070.stdout:2/307: dwrite d1/d7/d11/d38/d4c/f5c [0,4194304] 0 2024-03-23T08:23:51.106 INFO:tasks.workunit.client.0.smithi070.stdout:5/254: creat dc/f58 x:0 0 0 2024-03-23T08:23:51.106 INFO:tasks.workunit.client.0.smithi070.stdout:0/628: dwrite d16/f34 [0,4194304] 0 2024-03-23T08:23:51.108 INFO:tasks.workunit.client.0.smithi070.stdout:5/255: write dc/d1b/d1d/d25/d38/d48/f56 [676230,9637] 0 2024-03-23T08:23:51.109 INFO:tasks.workunit.client.0.smithi070.stdout:0/629: creat d16/d26/fe0 x:0 0 0 2024-03-23T08:23:51.112 INFO:tasks.workunit.client.0.smithi070.stdout:2/308: truncate d1/d6/d4f/f3e 810201 0 2024-03-23T08:23:51.113 INFO:tasks.workunit.client.0.smithi070.stdout:0/630: creat d16/d22/d43/fe1 x:0 0 0 2024-03-23T08:23:51.117 INFO:tasks.workunit.client.0.smithi070.stdout:5/256: dwrite dc/d1b/d1d/d25/d38/d39/f4f [0,4194304] 0 2024-03-23T08:23:51.123 INFO:tasks.workunit.client.0.smithi070.stdout:5/257: creat dc/f59 x:0 0 0 2024-03-23T08:23:51.127 INFO:tasks.workunit.client.0.smithi070.stdout:0/631: write d16/d26/f37 [1020326,15953] 0 2024-03-23T08:23:51.129 INFO:tasks.workunit.client.0.smithi070.stdout:2/309: truncate d1/fc 3586566 0 2024-03-23T08:23:51.130 INFO:tasks.workunit.client.0.smithi070.stdout:0/632: write d16/d26/d27/d86/db6/dde/fbf [4857952,57366] 0 2024-03-23T08:23:51.132 INFO:tasks.workunit.client.0.smithi070.stdout:5/258: creat dc/d1b/d1d/d25/d36/d32/f5a x:0 0 0 2024-03-23T08:23:51.133 INFO:tasks.workunit.client.0.smithi070.stdout:0/633: mkdir d16/d26/d27/d4c/d60/d75/de2 0 2024-03-23T08:23:51.141 INFO:tasks.workunit.client.0.smithi070.stdout:0/634: dwrite d16/f4b [0,4194304] 0 2024-03-23T08:23:51.141 INFO:tasks.workunit.client.0.smithi070.stdout:2/310: write d1/d6/d4f/f3e [959085,122466] 0 2024-03-23T08:23:51.144 INFO:tasks.workunit.client.0.smithi070.stdout:0/635: read d16/d26/f37 [809558,9618] 0 2024-03-23T08:23:51.149 INFO:tasks.workunit.client.0.smithi070.stdout:0/636: creat d16/d26/d27/d3d/d4d/d9a/fe3 x:0 0 0 2024-03-23T08:23:51.152 INFO:tasks.workunit.client.0.smithi070.stdout:2/311: link d1/d7/df/f27 d1/d6/f65 0 2024-03-23T08:23:51.153 INFO:tasks.workunit.client.0.smithi070.stdout:0/637: creat d16/d26/d27/d3d/d4d/dc9/fe4 x:0 0 0 2024-03-23T08:23:51.159 INFO:tasks.workunit.client.0.smithi070.stdout:2/312: dread d1/d7/df/f27 [0,4194304] 0 2024-03-23T08:23:51.161 INFO:tasks.workunit.client.0.smithi070.stdout:0/638: dwrite d16/d26/f6b [0,4194304] 0 2024-03-23T08:23:51.161 INFO:tasks.workunit.client.0.smithi070.stdout:2/313: write d1/d29/f2d [3482167,77771] 0 2024-03-23T08:23:51.164 INFO:tasks.workunit.client.0.smithi070.stdout:0/639: chown d16/d26/fe0 5757034 1 2024-03-23T08:23:51.167 INFO:tasks.workunit.client.0.smithi070.stdout:2/314: unlink d1/d7/d39/d47/l53 0 2024-03-23T08:23:51.170 INFO:tasks.workunit.client.0.smithi070.stdout:0/640: rename d16/d26/fe0 to d16/d26/d27/d86/d67/fe5 0 2024-03-23T08:23:51.174 INFO:tasks.workunit.client.0.smithi070.stdout:0/641: rmdir d16/d26/d27/d3d/d4d/dad 39 2024-03-23T08:23:51.175 INFO:tasks.workunit.client.0.smithi070.stdout:0/642: fsync d16/d22/f29 0 2024-03-23T08:23:51.176 INFO:tasks.workunit.client.0.smithi070.stdout:0/643: fsync d16/d22/d43/d47/d87/f92 0 2024-03-23T08:23:51.179 INFO:tasks.workunit.client.0.smithi070.stdout:2/315: creat d1/d7/f66 x:0 0 0 2024-03-23T08:23:51.180 INFO:tasks.workunit.client.0.smithi070.stdout:2/316: chown d1/d7/df/c2a 1004161 1 2024-03-23T08:23:51.180 INFO:tasks.workunit.client.0.smithi070.stdout:0/644: readlink d16/d26/l4e 0 2024-03-23T08:23:51.181 INFO:tasks.workunit.client.0.smithi070.stdout:0/645: write d16/d22/d43/fe1 [19767,82592] 0 2024-03-23T08:23:51.184 INFO:tasks.workunit.client.0.smithi070.stdout:0/646: creat d16/d22/d43/d47/da9/fe6 x:0 0 0 2024-03-23T08:23:51.187 INFO:tasks.workunit.client.0.smithi070.stdout:2/317: dwrite d1/d7/f36 [0,4194304] 0 2024-03-23T08:23:51.188 INFO:tasks.workunit.client.0.smithi070.stdout:0/647: truncate d16/d26/d27/d3d/f72 103977 0 2024-03-23T08:23:51.207 INFO:tasks.workunit.client.0.smithi070.stdout:2/318: dread d1/fe [0,4194304] 0 2024-03-23T08:23:51.211 INFO:tasks.workunit.client.0.smithi070.stdout:2/319: creat d1/d21/d5f/f67 x:0 0 0 2024-03-23T08:23:51.214 INFO:tasks.workunit.client.0.smithi070.stdout:2/320: dread d1/d6/d30/f3a [0,4194304] 0 2024-03-23T08:23:51.216 INFO:tasks.workunit.client.0.smithi070.stdout:2/321: symlink d1/d7/d39/l68 0 2024-03-23T08:23:51.222 INFO:tasks.workunit.client.0.smithi070.stdout:2/322: dwrite d1/d7/d11/f56 [0,4194304] 0 2024-03-23T08:23:51.223 INFO:tasks.workunit.client.0.smithi070.stdout:2/323: write d1/d7/f40 [178703,59062] 0 2024-03-23T08:23:51.234 INFO:tasks.workunit.client.0.smithi070.stdout:2/324: dwrite d1/d7/d11/d38/d4c/f4e [0,4194304] 0 2024-03-23T08:23:51.237 INFO:tasks.workunit.client.0.smithi070.stdout:2/325: mkdir d1/d7/d48/d69 0 2024-03-23T08:23:51.242 INFO:tasks.workunit.client.0.smithi070.stdout:2/326: creat d1/d7/d11/d38/d4c/f6a x:0 0 0 2024-03-23T08:23:51.243 INFO:tasks.workunit.client.0.smithi070.stdout:9/435: sync 2024-03-23T08:23:51.246 INFO:tasks.workunit.client.0.smithi070.stdout:9/436: mkdir db/d12/d34/d8c 0 2024-03-23T08:23:51.248 INFO:tasks.workunit.client.0.smithi070.stdout:2/327: link d1/d7/c33 d1/d21/d5f/c6b 0 2024-03-23T08:23:51.248 INFO:tasks.workunit.client.0.smithi070.stdout:9/437: rmdir db/d12/d22/d25/d26/d5c/d7e 39 2024-03-23T08:23:51.251 INFO:tasks.workunit.client.0.smithi070.stdout:9/438: mknod db/d12/d22/d25/d4f/c8d 0 2024-03-23T08:23:51.251 INFO:tasks.workunit.client.0.smithi070.stdout:2/328: mknod d1/d7/d48/d5b/c6c 0 2024-03-23T08:23:51.251 INFO:tasks.workunit.client.0.smithi070.stdout:9/439: readlink db/d12/d22/d85/d42/l7f 0 2024-03-23T08:23:51.259 INFO:tasks.workunit.client.0.smithi070.stdout:9/440: dwrite db/d12/d22/d85/d42/f46 [0,4194304] 0 2024-03-23T08:23:51.273 INFO:tasks.workunit.client.0.smithi070.stdout:9/441: dread db/d12/d22/d25/d26/d5c/d7e/f75 [0,4194304] 0 2024-03-23T08:23:51.274 INFO:tasks.workunit.client.0.smithi070.stdout:2/329: write d1/d6/f65 [869925,88242] 0 2024-03-23T08:23:51.282 INFO:tasks.workunit.client.0.smithi070.stdout:2/330: truncate d1/d7/f36 3804045 0 2024-03-23T08:23:51.285 INFO:tasks.workunit.client.0.smithi070.stdout:9/442: dread db/f1f [0,4194304] 0 2024-03-23T08:23:51.286 INFO:tasks.workunit.client.0.smithi070.stdout:2/331: rename d1/d6/c45 to d1/d7/d11/d38/c6d 0 2024-03-23T08:23:51.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/332: fdatasync d1/d7/d11/f25 0 2024-03-23T08:23:51.292 INFO:tasks.workunit.client.0.smithi070.stdout:9/443: dwrite db/d12/f17 [0,4194304] 0 2024-03-23T08:23:51.299 INFO:tasks.workunit.client.0.smithi070.stdout:9/444: dwrite db/d12/d22/d25/f24 [0,4194304] 0 2024-03-23T08:23:51.333 INFO:tasks.workunit.client.0.smithi070.stdout:9/445: symlink db/d12/d22/d85/d42/l8e 0 2024-03-23T08:23:51.335 INFO:tasks.workunit.client.0.smithi070.stdout:2/333: symlink d1/l6e 0 2024-03-23T08:23:51.337 INFO:tasks.workunit.client.0.smithi070.stdout:9/446: mknod db/d12/d22/c8f 0 2024-03-23T08:23:51.338 INFO:tasks.workunit.client.0.smithi070.stdout:2/334: symlink d1/d7/d11/l6f 0 2024-03-23T08:23:51.340 INFO:tasks.workunit.client.0.smithi070.stdout:2/335: creat d1/d29/f70 x:0 0 0 2024-03-23T08:23:51.342 INFO:tasks.workunit.client.0.smithi070.stdout:2/336: read d1/d6/d30/f46 [147265,16242] 0 2024-03-23T08:23:51.342 INFO:tasks.workunit.client.0.smithi070.stdout:9/447: dwrite f6 [4194304,4194304] 0 2024-03-23T08:23:51.345 INFO:tasks.workunit.client.0.smithi070.stdout:9/448: mkdir db/d12/d34/d47/d90 0 2024-03-23T08:23:51.346 INFO:tasks.workunit.client.0.smithi070.stdout:9/449: read f8 [1005069,4879] 0 2024-03-23T08:23:51.347 INFO:tasks.workunit.client.0.smithi070.stdout:2/337: rename d1/d21/l55 to d1/d7/d39/d47/l71 0 2024-03-23T08:23:51.352 INFO:tasks.workunit.client.0.smithi070.stdout:2/338: dread d1/fe [0,4194304] 0 2024-03-23T08:23:51.360 INFO:tasks.workunit.client.0.smithi070.stdout:9/450: dread f8 [0,4194304] 0 2024-03-23T08:23:51.363 INFO:tasks.workunit.client.0.smithi070.stdout:9/451: mkdir db/d12/d91 0 2024-03-23T08:23:51.364 INFO:tasks.workunit.client.0.smithi070.stdout:9/452: write db/d12/d14/f35 [3540211,99767] 0 2024-03-23T08:23:51.464 INFO:tasks.workunit.client.0.smithi070.stdout:3/564: sync 2024-03-23T08:23:51.465 INFO:tasks.workunit.client.0.smithi070.stdout:4/218: sync 2024-03-23T08:23:51.465 INFO:tasks.workunit.client.0.smithi070.stdout:7/280: sync 2024-03-23T08:23:51.465 INFO:tasks.workunit.client.0.smithi070.stdout:7/281: fdatasync f1d 0 2024-03-23T08:23:51.468 INFO:tasks.workunit.client.0.smithi070.stdout:4/219: rename d3/d6/d35/l39 to d3/d6/d11/d1a/d27/l4f 0 2024-03-23T08:23:51.496 INFO:tasks.workunit.client.0.smithi070.stdout:7/282: getdents d25/d27/d6a 0 2024-03-23T08:23:51.497 INFO:tasks.workunit.client.0.smithi070.stdout:4/220: rename d3/d6/d9/c21 to d3/d6/d9/c50 0 2024-03-23T08:23:51.499 INFO:tasks.workunit.client.0.smithi070.stdout:7/283: creat d25/d27/d6a/f75 x:0 0 0 2024-03-23T08:23:51.504 INFO:tasks.workunit.client.0.smithi070.stdout:7/284: creat d25/d40/f76 x:0 0 0 2024-03-23T08:23:51.504 INFO:tasks.workunit.client.0.smithi070.stdout:4/221: mkdir d3/d6/d35/d3f/d51 0 2024-03-23T08:23:51.505 INFO:tasks.workunit.client.0.smithi070.stdout:4/222: truncate d3/d6/d11/d1a/d1b/f26 189474 0 2024-03-23T08:23:51.506 INFO:tasks.workunit.client.0.smithi070.stdout:7/285: creat d25/d27/d6a/d44/f77 x:0 0 0 2024-03-23T08:23:51.510 INFO:tasks.workunit.client.0.smithi070.stdout:7/286: dread f12 [0,4194304] 0 2024-03-23T08:23:51.514 INFO:tasks.workunit.client.0.smithi070.stdout:7/287: write d25/d27/f6d [699818,106548] 0 2024-03-23T08:23:51.514 INFO:tasks.workunit.client.0.smithi070.stdout:4/223: rmdir d3/d6/d9 39 2024-03-23T08:23:51.515 INFO:tasks.workunit.client.0.smithi070.stdout:4/224: dread - d3/d19/d2a/f37 zero size 2024-03-23T08:23:51.515 INFO:tasks.workunit.client.0.smithi070.stdout:7/288: creat d25/d27/d6a/d44/f78 x:0 0 0 2024-03-23T08:23:51.517 INFO:tasks.workunit.client.0.smithi070.stdout:7/289: unlink d25/d27/c46 0 2024-03-23T08:23:51.518 INFO:tasks.workunit.client.0.smithi070.stdout:7/290: dread - d25/d27/d6a/f5f zero size 2024-03-23T08:23:51.519 INFO:tasks.workunit.client.0.smithi070.stdout:4/225: symlink d3/d6/l52 0 2024-03-23T08:23:51.521 INFO:tasks.workunit.client.0.smithi070.stdout:7/291: unlink c19 0 2024-03-23T08:23:51.523 INFO:tasks.workunit.client.0.smithi070.stdout:4/226: chown d3/d6/d9/f2e 58959247 1 2024-03-23T08:23:51.530 INFO:tasks.workunit.client.0.smithi070.stdout:4/227: mknod d3/d6/d11/d1a/d27/d4b/c53 0 2024-03-23T08:23:51.532 INFO:tasks.workunit.client.0.smithi070.stdout:7/292: dread f24 [0,4194304] 0 2024-03-23T08:23:51.533 INFO:tasks.workunit.client.0.smithi070.stdout:7/293: chown d25/d27/d6a/f5b 12 1 2024-03-23T08:23:51.533 INFO:tasks.workunit.client.0.smithi070.stdout:4/228: mknod d3/d6/d11/d1a/d27/d4b/c54 0 2024-03-23T08:23:51.536 INFO:tasks.workunit.client.0.smithi070.stdout:7/294: creat d25/d27/d6a/d44/f79 x:0 0 0 2024-03-23T08:23:51.539 INFO:tasks.workunit.client.0.smithi070.stdout:7/295: symlink d25/d27/d6a/d44/l7a 0 2024-03-23T08:23:51.539 INFO:tasks.workunit.client.0.smithi070.stdout:7/296: dread - d25/d27/f48 zero size 2024-03-23T08:23:51.541 INFO:tasks.workunit.client.0.smithi070.stdout:4/229: creat d3/d6/d11/f55 x:0 0 0 2024-03-23T08:23:51.545 INFO:tasks.workunit.client.0.smithi070.stdout:4/230: truncate d3/d6/d11/d1a/f3e 956578 0 2024-03-23T08:23:51.545 INFO:tasks.workunit.client.0.smithi070.stdout:7/297: creat d25/d27/d6a/d44/d6f/f7b x:0 0 0 2024-03-23T08:23:51.546 INFO:tasks.workunit.client.0.smithi070.stdout:4/231: mkdir d3/d6/d11/d1a/d41/d56 0 2024-03-23T08:23:51.547 INFO:tasks.workunit.client.0.smithi070.stdout:4/232: chown d3/f25 10 1 2024-03-23T08:23:51.548 INFO:tasks.workunit.client.0.smithi070.stdout:7/298: write d25/d27/d6a/d44/f53 [278041,110744] 0 2024-03-23T08:23:51.555 INFO:tasks.workunit.client.0.smithi070.stdout:6/452: sync 2024-03-23T08:23:51.556 INFO:tasks.workunit.client.0.smithi070.stdout:7/299: mkdir d25/d27/d6a/d44/d7c 0 2024-03-23T08:23:51.556 INFO:tasks.workunit.client.0.smithi070.stdout:4/233: symlink d3/d19/d2a/l57 0 2024-03-23T08:23:51.557 INFO:tasks.workunit.client.0.smithi070.stdout:7/300: read - d25/d40/f76 zero size 2024-03-23T08:23:51.565 INFO:tasks.workunit.client.0.smithi070.stdout:4/234: creat d3/d19/f58 x:0 0 0 2024-03-23T08:23:51.568 INFO:tasks.workunit.client.0.smithi070.stdout:4/235: write d3/d6/d11/d1a/f3e [1623549,3004] 0 2024-03-23T08:23:51.569 INFO:tasks.workunit.client.0.smithi070.stdout:4/236: write d3/d6/d11/f2d [250197,56388] 0 2024-03-23T08:23:51.585 INFO:tasks.workunit.client.0.smithi070.stdout:6/453: dread d0/d23/d4d/f54 [0,4194304] 0 2024-03-23T08:23:51.607 INFO:tasks.workunit.client.0.smithi070.stdout:6/454: creat d0/d23/d28/d35/d85/f9e x:0 0 0 2024-03-23T08:23:51.610 INFO:tasks.workunit.client.0.smithi070.stdout:6/455: chown d0/d23/d28/f4a 2799 1 2024-03-23T08:23:51.611 INFO:tasks.workunit.client.0.smithi070.stdout:4/237: link d3/d6/d11/f18 d3/d6/d11/d1a/d1b/f59 0 2024-03-23T08:23:51.621 INFO:tasks.workunit.client.0.smithi070.stdout:4/238: dread d3/d6/d9/f2e [0,4194304] 0 2024-03-23T08:23:51.636 INFO:tasks.workunit.client.0.smithi070.stdout:6/456: link d0/d23/d74/d84/d3c/d5b/l81 d0/d23/d1c/d38/d52/l9f 0 2024-03-23T08:23:51.644 INFO:tasks.workunit.client.0.smithi070.stdout:6/457: dwrite d0/d23/d1c/d38/d52/f6b [0,4194304] 0 2024-03-23T08:23:51.645 INFO:tasks.workunit.client.0.smithi070.stdout:6/458: dread - d0/d23/d4d/f6e zero size 2024-03-23T08:23:51.657 INFO:tasks.workunit.client.0.smithi070.stdout:6/459: mknod d0/d23/d1c/d38/d9a/ca0 0 2024-03-23T08:23:51.658 INFO:tasks.workunit.client.0.smithi070.stdout:4/239: dwrite d3/d6/d11/d1a/d1b/f1c [0,4194304] 0 2024-03-23T08:23:51.659 INFO:tasks.workunit.client.0.smithi070.stdout:4/240: dread - d3/d6/d11/f55 zero size 2024-03-23T08:23:51.660 INFO:tasks.workunit.client.0.smithi070.stdout:4/241: stat d3/d6/l31 0 2024-03-23T08:23:51.664 INFO:tasks.workunit.client.0.smithi070.stdout:6/460: symlink d0/d23/d1c/d38/d9a/la1 0 2024-03-23T08:23:51.669 INFO:tasks.workunit.client.0.smithi070.stdout:4/242: creat d3/d6/d11/d1a/d41/d56/f5a x:0 0 0 2024-03-23T08:23:51.671 INFO:tasks.workunit.client.0.smithi070.stdout:6/461: link d0/d3/f7a d0/d23/d97/fa2 0 2024-03-23T08:23:51.676 INFO:tasks.workunit.client.0.smithi070.stdout:6/462: chown d0/d23/d1c/d38/d52/l99 13580 1 2024-03-23T08:23:51.677 INFO:tasks.workunit.client.0.smithi070.stdout:1/345: sync 2024-03-23T08:23:51.678 INFO:tasks.workunit.client.0.smithi070.stdout:8/328: sync 2024-03-23T08:23:51.678 INFO:tasks.workunit.client.0.smithi070.stdout:5/259: sync 2024-03-23T08:23:51.678 INFO:tasks.workunit.client.0.smithi070.stdout:6/463: rmdir d0/d23/d74/d84 39 2024-03-23T08:23:51.680 INFO:tasks.workunit.client.0.smithi070.stdout:6/464: fsync d0/d23/d28/d35/f46 0 2024-03-23T08:23:51.682 INFO:tasks.workunit.client.0.smithi070.stdout:8/329: dread d5/d8/f4a [0,4194304] 0 2024-03-23T08:23:51.684 INFO:tasks.workunit.client.0.smithi070.stdout:8/330: write d5/d8/d16/d32/f5c [8842446,108477] 0 2024-03-23T08:23:51.685 INFO:tasks.workunit.client.0.smithi070.stdout:8/331: write d5/d8/d16/d3b/d4d/d6f/d7c/f80 [974047,129954] 0 2024-03-23T08:23:51.685 INFO:tasks.workunit.client.0.smithi070.stdout:5/260: fdatasync f8 0 2024-03-23T08:23:51.686 INFO:tasks.workunit.client.0.smithi070.stdout:4/243: truncate d3/d6/d9/f22 3767349 0 2024-03-23T08:23:51.687 INFO:tasks.workunit.client.0.smithi070.stdout:4/244: stat d3/d6/f3b 0 2024-03-23T08:23:51.690 INFO:tasks.workunit.client.0.smithi070.stdout:4/245: write d3/d6/d11/f13 [557451,70013] 0 2024-03-23T08:23:51.693 INFO:tasks.workunit.client.0.smithi070.stdout:6/465: rename d0/d23/d74/d84/d3c/c65 to d0/d23/d28/d35/d6a/ca3 0 2024-03-23T08:23:51.693 INFO:tasks.workunit.client.0.smithi070.stdout:4/246: write d3/d6/d11/f13 [1177475,52407] 0 2024-03-23T08:23:51.694 INFO:tasks.workunit.client.0.smithi070.stdout:4/247: fdatasync d3/d6/f43 0 2024-03-23T08:23:51.697 INFO:tasks.workunit.client.0.smithi070.stdout:5/261: mknod dc/d1b/d1d/d25/d38/d53/c5b 0 2024-03-23T08:23:51.700 INFO:tasks.workunit.client.0.smithi070.stdout:4/248: symlink d3/l5b 0 2024-03-23T08:23:51.704 INFO:tasks.workunit.client.0.smithi070.stdout:6/466: mkdir d0/d3/d2c/da4 0 2024-03-23T08:23:51.718 INFO:tasks.workunit.client.0.smithi070.stdout:8/332: link d5/l3d d5/d8/d16/d30/l83 0 2024-03-23T08:23:51.721 INFO:tasks.workunit.client.0.smithi070.stdout:5/262: mkdir dc/d1b/d1d/d25/d36/d32/d5c 0 2024-03-23T08:23:51.724 INFO:tasks.workunit.client.0.smithi070.stdout:6/467: write d0/d3/f7a [721664,21993] 0 2024-03-23T08:23:51.726 INFO:tasks.workunit.client.0.smithi070.stdout:6/468: readlink d0/d3/l22 0 2024-03-23T08:23:51.726 INFO:tasks.workunit.client.0.smithi070.stdout:5/263: creat dc/d1b/d1d/d25/d36/d32/d5c/f5d x:0 0 0 2024-03-23T08:23:51.727 INFO:tasks.workunit.client.0.smithi070.stdout:5/264: fdatasync dc/d1b/d1d/d25/f43 0 2024-03-23T08:23:51.728 INFO:tasks.workunit.client.0.smithi070.stdout:5/265: chown dc/d1b/d1d/d25/c4a 380 1 2024-03-23T08:23:51.729 INFO:tasks.workunit.client.0.smithi070.stdout:6/469: mkdir d0/d23/d4d/da5 0 2024-03-23T08:23:51.732 INFO:tasks.workunit.client.0.smithi070.stdout:5/266: mkdir dc/d1b/d1d/d25/d36/d32/d5c/d5e 0 2024-03-23T08:23:51.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:51 smithi070 ceph-mon[39699]: pgmap v35: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 4.9 MiB/s rd, 63 MiB/s wr, 183 op/s 2024-03-23T08:23:51.737 INFO:tasks.workunit.client.0.smithi070.stdout:6/470: dwrite d0/d23/d28/d35/d85/f8b [0,4194304] 0 2024-03-23T08:23:51.739 INFO:tasks.workunit.client.0.smithi070.stdout:8/333: dread d5/d8/d16/d32/f5c [4194304,4194304] 0 2024-03-23T08:23:51.739 INFO:tasks.workunit.client.0.smithi070.stdout:5/267: creat dc/d1b/d1d/d25/d38/d53/f5f x:0 0 0 2024-03-23T08:23:51.741 INFO:tasks.workunit.client.0.smithi070.stdout:8/334: dread - d5/d8/d16/d32/f56 zero size 2024-03-23T08:23:51.744 INFO:tasks.workunit.client.0.smithi070.stdout:5/268: creat dc/d1b/d1d/d25/d38/d53/f60 x:0 0 0 2024-03-23T08:23:51.745 INFO:tasks.workunit.client.0.smithi070.stdout:8/335: mkdir d5/d8/d16/d32/d84 0 2024-03-23T08:23:51.746 INFO:tasks.workunit.client.0.smithi070.stdout:6/471: mknod d0/d23/d28/d93/ca6 0 2024-03-23T08:23:51.750 INFO:tasks.workunit.client.0.smithi070.stdout:6/472: dread d0/d23/d1c/d38/d52/f6b [0,4194304] 0 2024-03-23T08:23:51.751 INFO:tasks.workunit.client.0.smithi070.stdout:8/336: unlink d5/d8/d16/d32/f67 0 2024-03-23T08:23:51.758 INFO:tasks.workunit.client.0.smithi070.stdout:8/337: mkdir d5/d8/d16/d3b/d85 0 2024-03-23T08:23:51.759 INFO:tasks.workunit.client.0.smithi070.stdout:8/338: dread - d5/d8/d16/d30/f68 zero size 2024-03-23T08:23:51.761 INFO:tasks.workunit.client.0.smithi070.stdout:5/269: rmdir dc/d1b/d1d/d25/d36/d32/d5c/d5e 0 2024-03-23T08:23:51.761 INFO:tasks.workunit.client.0.smithi070.stdout:5/270: write dc/f58 [631579,14082] 0 2024-03-23T08:23:51.763 INFO:tasks.workunit.client.0.smithi070.stdout:8/339: symlink d5/d8/d16/d32/l86 0 2024-03-23T08:23:51.765 INFO:tasks.workunit.client.0.smithi070.stdout:6/473: creat d0/fa7 x:0 0 0 2024-03-23T08:23:51.768 INFO:tasks.workunit.client.0.smithi070.stdout:5/271: dwrite dc/f2b [0,4194304] 0 2024-03-23T08:23:51.769 INFO:tasks.workunit.client.0.smithi070.stdout:5/272: write dc/f58 [114602,42306] 0 2024-03-23T08:23:51.770 INFO:tasks.workunit.client.0.smithi070.stdout:5/273: dread - dc/d1b/d1d/f44 zero size 2024-03-23T08:23:51.771 INFO:tasks.workunit.client.0.smithi070.stdout:6/474: mknod d0/d23/d28/d35/ca8 0 2024-03-23T08:23:51.779 INFO:tasks.workunit.client.0.smithi070.stdout:8/340: rmdir d5/d8/d16/d3b/d85 0 2024-03-23T08:23:51.782 INFO:tasks.workunit.client.0.smithi070.stdout:6/475: fsync d0/d23/d74/d84/d3c/d5b/f79 0 2024-03-23T08:23:51.785 INFO:tasks.workunit.client.0.smithi070.stdout:6/476: symlink d0/d23/d1c/d38/la9 0 2024-03-23T08:23:51.788 INFO:tasks.workunit.client.0.smithi070.stdout:6/477: creat d0/d23/d74/d84/faa x:0 0 0 2024-03-23T08:23:51.790 INFO:tasks.workunit.client.0.smithi070.stdout:6/478: dread - d0/d23/d4d/f6e zero size 2024-03-23T08:23:51.791 INFO:tasks.workunit.client.0.smithi070.stdout:8/341: dwrite d5/f7 [0,4194304] 0 2024-03-23T08:23:51.793 INFO:tasks.workunit.client.0.smithi070.stdout:5/274: truncate dc/d19/f2c 338371 0 2024-03-23T08:23:51.794 INFO:tasks.workunit.client.0.smithi070.stdout:5/275: fsync dc/d1b/d1d/d25/d38/d39/f3c 0 2024-03-23T08:23:51.794 INFO:tasks.workunit.client.0.smithi070.stdout:6/479: unlink d0/d23/d1c/f2e 0 2024-03-23T08:23:51.796 INFO:tasks.workunit.client.0.smithi070.stdout:6/480: chown d0/d23/d1c/d38/d52/f7c 88216 1 2024-03-23T08:23:51.796 INFO:tasks.workunit.client.0.smithi070.stdout:8/342: mknod d5/d8/c87 0 2024-03-23T08:23:51.796 INFO:tasks.workunit.client.0.smithi070.stdout:6/481: chown d0/d23/d74 449996714 1 2024-03-23T08:23:51.798 INFO:tasks.workunit.client.0.smithi070.stdout:5/276: mkdir dc/d1b/d1d/d61 0 2024-03-23T08:23:51.801 INFO:tasks.workunit.client.0.smithi070.stdout:8/343: mkdir d5/d8/d16/d30/d38/d6b/d72/d88 0 2024-03-23T08:23:51.802 INFO:tasks.workunit.client.0.smithi070.stdout:6/482: dread d0/d23/d1c/d38/d52/f6b [0,4194304] 0 2024-03-23T08:23:51.805 INFO:tasks.workunit.client.0.smithi070.stdout:8/344: read d5/d8/f4e [549253,1369] 0 2024-03-23T08:23:51.810 INFO:tasks.workunit.client.0.smithi070.stdout:6/483: dwrite d0/d3/f80 [0,4194304] 0 2024-03-23T08:23:51.811 INFO:tasks.workunit.client.0.smithi070.stdout:8/345: creat d5/d8/d16/d3b/d4d/d6f/f89 x:0 0 0 2024-03-23T08:23:51.816 INFO:tasks.workunit.client.0.smithi070.stdout:6/484: mkdir d0/d23/d1c/d2f/dab 0 2024-03-23T08:23:51.824 INFO:tasks.workunit.client.0.smithi070.stdout:5/277: dwrite dc/d1b/d1d/f3a [0,4194304] 0 2024-03-23T08:23:51.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:51 smithi193 ceph-mon[48370]: pgmap v35: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 4.9 MiB/s rd, 63 MiB/s wr, 183 op/s 2024-03-23T08:23:51.825 INFO:tasks.workunit.client.0.smithi070.stdout:6/485: dread d0/d3/fa [0,4194304] 0 2024-03-23T08:23:51.827 INFO:tasks.workunit.client.0.smithi070.stdout:5/278: truncate dc/d1b/d1d/d25/d38/d48/f56 748939 0 2024-03-23T08:23:51.828 INFO:tasks.workunit.client.0.smithi070.stdout:5/279: readlink dc/d1b/d1d/d25/d36/l31 0 2024-03-23T08:23:51.829 INFO:tasks.workunit.client.0.smithi070.stdout:5/280: chown dc/d1b/d1d/d25/d38/d53 140293383 1 2024-03-23T08:23:51.829 INFO:tasks.workunit.client.0.smithi070.stdout:5/281: write dc/d1b/f54 [644524,82364] 0 2024-03-23T08:23:51.830 INFO:tasks.workunit.client.0.smithi070.stdout:8/346: write d5/d8/d16/d30/f2f [230611,108932] 0 2024-03-23T08:23:51.831 INFO:tasks.workunit.client.0.smithi070.stdout:5/282: write dc/f59 [128845,120184] 0 2024-03-23T08:23:51.846 INFO:tasks.workunit.client.0.smithi070.stdout:8/347: unlink d5/f43 0 2024-03-23T08:23:51.848 INFO:tasks.workunit.client.0.smithi070.stdout:8/348: truncate d5/d8/d16/d3b/d4d/d6f/f89 932167 0 2024-03-23T08:23:51.849 INFO:tasks.workunit.client.0.smithi070.stdout:8/349: chown d5/d8/d16/f24 0 1 2024-03-23T08:23:51.850 INFO:tasks.workunit.client.0.smithi070.stdout:5/283: fsync dc/d19/f2c 0 2024-03-23T08:23:51.851 INFO:tasks.workunit.client.0.smithi070.stdout:8/350: dread d5/d8/f2c [0,4194304] 0 2024-03-23T08:23:51.855 INFO:tasks.workunit.client.0.smithi070.stdout:5/284: dread dc/d1b/d1d/d25/d38/d48/f56 [0,4194304] 0 2024-03-23T08:23:51.858 INFO:tasks.workunit.client.0.smithi070.stdout:8/351: dwrite d5/d8/d16/d32/f46 [0,4194304] 0 2024-03-23T08:23:51.860 INFO:tasks.workunit.client.0.smithi070.stdout:5/285: creat dc/d1b/d1d/f62 x:0 0 0 2024-03-23T08:23:51.864 INFO:tasks.workunit.client.0.smithi070.stdout:8/352: rmdir d5/d8 39 2024-03-23T08:23:51.874 INFO:tasks.workunit.client.0.smithi070.stdout:5/286: link dc/l1a dc/d1b/d1d/d25/d38/d53/l63 0 2024-03-23T08:23:51.880 INFO:tasks.workunit.client.0.smithi070.stdout:5/287: dwrite dc/f14 [0,4194304] 0 2024-03-23T08:23:51.882 INFO:tasks.workunit.client.0.smithi070.stdout:8/353: dwrite d5/d8/f2c [0,4194304] 0 2024-03-23T08:23:51.887 INFO:tasks.workunit.client.0.smithi070.stdout:5/288: symlink dc/d1b/d1d/d25/d36/d32/d5c/l64 0 2024-03-23T08:23:51.888 INFO:tasks.workunit.client.0.smithi070.stdout:5/289: truncate dc/d1b/d1d/f62 605867 0 2024-03-23T08:23:51.892 INFO:tasks.workunit.client.0.smithi070.stdout:8/354: creat d5/d8/d16/d30/d38/d6b/d72/d88/f8a x:0 0 0 2024-03-23T08:23:51.894 INFO:tasks.workunit.client.0.smithi070.stdout:8/355: rmdir d5 39 2024-03-23T08:23:51.897 INFO:tasks.workunit.client.0.smithi070.stdout:5/290: dwrite dc/f58 [0,4194304] 0 2024-03-23T08:23:51.898 INFO:tasks.workunit.client.0.smithi070.stdout:8/356: read - d5/d8/f33 zero size 2024-03-23T08:23:51.900 INFO:tasks.workunit.client.0.smithi070.stdout:5/291: fdatasync dc/f3d 0 2024-03-23T08:23:51.902 INFO:tasks.workunit.client.0.smithi070.stdout:8/357: rename d5/d8/d16/d3b/d4d/d6f/d7c/l81 to d5/d8/d16/d30/d38/d6b/d72/l8b 0 2024-03-23T08:23:51.937 INFO:tasks.workunit.client.0.smithi070.stdout:3/565: sync 2024-03-23T08:23:51.937 INFO:tasks.workunit.client.0.smithi070.stdout:9/453: sync 2024-03-23T08:23:51.937 INFO:tasks.workunit.client.0.smithi070.stdout:2/339: sync 2024-03-23T08:23:51.937 INFO:tasks.workunit.client.0.smithi070.stdout:0/648: sync 2024-03-23T08:23:51.938 INFO:tasks.workunit.client.0.smithi070.stdout:8/358: write d5/d8/d16/d32/f46 [405030,68618] 0 2024-03-23T08:23:51.941 INFO:tasks.workunit.client.0.smithi070.stdout:3/566: dread - db/df/d5d/fcf zero size 2024-03-23T08:23:51.943 INFO:tasks.workunit.client.0.smithi070.stdout:0/649: fsync d16/d26/d27/d86/db6/fce 0 2024-03-23T08:23:51.943 INFO:tasks.workunit.client.0.smithi070.stdout:9/454: creat db/d12/d34/f92 x:0 0 0 2024-03-23T08:23:51.945 INFO:tasks.workunit.client.0.smithi070.stdout:8/359: mkdir d5/d8/d16/d3b/d8c 0 2024-03-23T08:23:51.945 INFO:tasks.workunit.client.0.smithi070.stdout:5/292: dwrite dc/d1b/d1d/d25/d38/f40 [0,4194304] 0 2024-03-23T08:23:51.948 INFO:tasks.workunit.client.0.smithi070.stdout:9/455: dread - db/d12/d22/d25/f6a zero size 2024-03-23T08:23:51.949 INFO:tasks.workunit.client.0.smithi070.stdout:3/567: rename db/df/d4d/dc0 to db/d4e/dd6 0 2024-03-23T08:23:51.951 INFO:tasks.workunit.client.0.smithi070.stdout:2/340: dwrite d1/d7/f17 [0,4194304] 0 2024-03-23T08:23:51.951 INFO:tasks.workunit.client.0.smithi070.stdout:0/650: mkdir d16/d22/d43/d47/da9/de7 0 2024-03-23T08:23:51.952 INFO:tasks.workunit.client.0.smithi070.stdout:9/456: dread - db/d12/d22/d25/f6a zero size 2024-03-23T08:23:51.952 INFO:tasks.workunit.client.0.smithi070.stdout:5/293: write dc/d1b/d1d/d25/d36/d32/f5a [985549,58404] 0 2024-03-23T08:23:51.954 INFO:tasks.workunit.client.0.smithi070.stdout:0/651: readlink d16/d22/d43/l8f 0 2024-03-23T08:23:51.958 INFO:tasks.workunit.client.0.smithi070.stdout:5/294: fsync dc/f3d 0 2024-03-23T08:23:51.960 INFO:tasks.workunit.client.0.smithi070.stdout:5/295: readlink l5 0 2024-03-23T08:23:51.966 INFO:tasks.workunit.client.0.smithi070.stdout:8/360: dwrite d5/fe [0,4194304] 0 2024-03-23T08:23:51.967 INFO:tasks.workunit.client.0.smithi070.stdout:2/341: dread d1/d29/f35 [0,4194304] 0 2024-03-23T08:23:51.969 INFO:tasks.workunit.client.0.smithi070.stdout:9/457: dwrite db/d12/d14/f7a [0,4194304] 0 2024-03-23T08:23:51.970 INFO:tasks.workunit.client.0.smithi070.stdout:5/296: creat dc/d1b/d1d/d25/d36/d32/d5c/f65 x:0 0 0 2024-03-23T08:23:51.971 INFO:tasks.workunit.client.0.smithi070.stdout:9/458: chown db/d12/d22/d25/f78 69 1 2024-03-23T08:23:51.974 INFO:tasks.workunit.client.0.smithi070.stdout:0/652: rename d16/d26/d27/d86/d9e/fb5 to d16/d26/d27/d3d/d4d/dad/fe8 0 2024-03-23T08:23:51.974 INFO:tasks.workunit.client.0.smithi070.stdout:0/653: fdatasync d16/d22/d43/d47/fd6 0 2024-03-23T08:23:51.975 INFO:tasks.workunit.client.0.smithi070.stdout:8/361: mkdir d5/d8/d16/d8d 0 2024-03-23T08:23:51.979 INFO:tasks.workunit.client.0.smithi070.stdout:3/568: write db/f21 [5367549,15065] 0 2024-03-23T08:23:51.980 INFO:tasks.workunit.client.0.smithi070.stdout:3/569: write db/f17 [6906560,9804] 0 2024-03-23T08:23:51.983 INFO:tasks.workunit.client.0.smithi070.stdout:0/654: truncate f13 1418 0 2024-03-23T08:23:51.985 INFO:tasks.workunit.client.0.smithi070.stdout:0/655: dread - d16/d26/d27/d86/d67/d7d/fc2 zero size 2024-03-23T08:23:51.988 INFO:tasks.workunit.client.0.smithi070.stdout:3/570: mkdir db/d4e/dd6/dd7 0 2024-03-23T08:23:51.991 INFO:tasks.workunit.client.0.smithi070.stdout:3/571: write db/d14/d65/daa/fad [4614755,27465] 0 2024-03-23T08:23:51.998 INFO:tasks.workunit.client.0.smithi070.stdout:9/459: dread db/d12/d14/f35 [0,4194304] 0 2024-03-23T08:23:51.998 INFO:tasks.workunit.client.0.smithi070.stdout:0/656: dwrite d16/d26/d27/d4c/d60/d75/f76 [4194304,4194304] 0 2024-03-23T08:23:52.002 INFO:tasks.workunit.client.0.smithi070.stdout:8/362: chown d5/d8/d16/d32/f56 159473 1 2024-03-23T08:23:52.003 INFO:tasks.workunit.client.0.smithi070.stdout:5/297: dwrite f8 [0,4194304] 0 2024-03-23T08:23:52.007 INFO:tasks.workunit.client.0.smithi070.stdout:2/342: truncate d1/d6/d34/f3d 2849535 0 2024-03-23T08:23:52.008 INFO:tasks.workunit.client.0.smithi070.stdout:3/572: chown db/df/d4d/l8a 2 1 2024-03-23T08:23:52.014 INFO:tasks.workunit.client.0.smithi070.stdout:8/363: mknod d5/d8/d16/d30/d7b/c8e 0 2024-03-23T08:23:52.018 INFO:tasks.workunit.client.0.smithi070.stdout:9/460: creat db/d12/d22/f93 x:0 0 0 2024-03-23T08:23:52.019 INFO:tasks.workunit.client.0.smithi070.stdout:2/343: dwrite d1/d7/f40 [0,4194304] 0 2024-03-23T08:23:52.020 INFO:tasks.workunit.client.0.smithi070.stdout:2/344: chown d1/d7/d39/c5e 492387406 1 2024-03-23T08:23:52.023 INFO:tasks.workunit.client.0.smithi070.stdout:3/573: truncate db/d14/f1b 58357 0 2024-03-23T08:23:52.029 INFO:tasks.workunit.client.0.smithi070.stdout:9/461: rename db/d12/l61 to db/d12/d91/l94 0 2024-03-23T08:23:52.030 INFO:tasks.workunit.client.0.smithi070.stdout:8/364: symlink d5/d8/d16/d3b/d4d/d6f/d7c/l8f 0 2024-03-23T08:23:52.032 INFO:tasks.workunit.client.0.smithi070.stdout:5/298: dread dc/f3d [0,4194304] 0 2024-03-23T08:23:52.036 INFO:tasks.workunit.client.0.smithi070.stdout:9/462: dread db/d12/d22/d85/f66 [0,4194304] 0 2024-03-23T08:23:52.039 INFO:tasks.workunit.client.0.smithi070.stdout:3/574: mkdir db/df/d15/d22/dd8 0 2024-03-23T08:23:52.043 INFO:tasks.workunit.client.0.smithi070.stdout:5/299: creat dc/d1b/d1d/d25/d38/d39/f66 x:0 0 0 2024-03-23T08:23:52.045 INFO:tasks.workunit.client.0.smithi070.stdout:2/345: write d1/d6/d30/f46 [738637,73977] 0 2024-03-23T08:23:52.046 INFO:tasks.workunit.client.0.smithi070.stdout:2/346: read d1/d7/f17 [1742666,88503] 0 2024-03-23T08:23:52.048 INFO:tasks.workunit.client.0.smithi070.stdout:9/463: rename db/d12/d22/d25/d26/d5c/f5d to db/d12/d14/f95 0 2024-03-23T08:23:52.050 INFO:tasks.workunit.client.0.smithi070.stdout:3/575: rmdir db/d14/d65/daa 39 2024-03-23T08:23:52.051 INFO:tasks.workunit.client.0.smithi070.stdout:3/576: chown db/d14/d65/dae/fb8 4975 1 2024-03-23T08:23:52.052 INFO:tasks.workunit.client.0.smithi070.stdout:2/347: dread d1/d7/d11/f56 [0,4194304] 0 2024-03-23T08:23:52.052 INFO:tasks.workunit.client.0.smithi070.stdout:5/300: fdatasync dc/d19/f2c 0 2024-03-23T08:23:52.054 INFO:tasks.workunit.client.0.smithi070.stdout:5/301: chown dc/d1b/d1d/d25/d38/d53 0 1 2024-03-23T08:23:52.055 INFO:tasks.workunit.client.0.smithi070.stdout:5/302: chown dc/d1b/d1d/d25 266133 1 2024-03-23T08:23:52.060 INFO:tasks.workunit.client.0.smithi070.stdout:3/577: dwrite db/f21 [4194304,4194304] 0 2024-03-23T08:23:52.060 INFO:tasks.workunit.client.0.smithi070.stdout:9/464: unlink db/d12/f62 0 2024-03-23T08:23:52.061 INFO:tasks.workunit.client.0.smithi070.stdout:3/578: fsync db/df/d5d/fcf 0 2024-03-23T08:23:52.067 INFO:tasks.workunit.client.0.smithi070.stdout:2/348: creat d1/d6/f72 x:0 0 0 2024-03-23T08:23:52.068 INFO:tasks.workunit.client.0.smithi070.stdout:2/349: readlink d1/d7/l1a 0 2024-03-23T08:23:52.070 INFO:tasks.workunit.client.0.smithi070.stdout:3/579: dwrite db/d14/d9c/f9f [0,4194304] 0 2024-03-23T08:23:52.070 INFO:tasks.workunit.client.0.smithi070.stdout:5/303: dread dc/d1b/d1d/d25/d38/d48/f56 [0,4194304] 0 2024-03-23T08:23:52.074 INFO:tasks.workunit.client.0.smithi070.stdout:3/580: write db/d4e/d92/f74 [852379,125250] 0 2024-03-23T08:23:52.082 INFO:tasks.workunit.client.0.smithi070.stdout:8/365: link d5/l14 d5/d8/d16/d30/l90 0 2024-03-23T08:23:52.083 INFO:tasks.workunit.client.0.smithi070.stdout:5/304: dwrite dc/d1b/d1d/d25/d38/d39/f3c [0,4194304] 0 2024-03-23T08:23:52.087 INFO:tasks.workunit.client.0.smithi070.stdout:3/581: mkdir db/d4e/dd6/dd7/dd9 0 2024-03-23T08:23:52.088 INFO:tasks.workunit.client.0.smithi070.stdout:8/366: creat d5/d8/d16/d3b/d4d/d6f/d7c/f91 x:0 0 0 2024-03-23T08:23:52.088 INFO:tasks.workunit.client.0.smithi070.stdout:5/305: write dc/d1b/d1d/d25/d38/d53/f5f [1021649,69457] 0 2024-03-23T08:23:52.090 INFO:tasks.workunit.client.0.smithi070.stdout:8/367: dread - d5/d8/d16/d3b/f76 zero size 2024-03-23T08:23:52.093 INFO:tasks.workunit.client.0.smithi070.stdout:9/465: dread db/d12/d22/d25/d26/f64 [0,4194304] 0 2024-03-23T08:23:52.093 INFO:tasks.workunit.client.0.smithi070.stdout:5/306: fdatasync dc/d19/f2c 0 2024-03-23T08:23:52.094 INFO:tasks.workunit.client.0.smithi070.stdout:9/466: chown db/d12/d22/d25/l81 25977 1 2024-03-23T08:23:52.099 INFO:tasks.workunit.client.0.smithi070.stdout:9/467: truncate db/d12/d22/d25/d26/f2c 4576381 0 2024-03-23T08:23:52.099 INFO:tasks.workunit.client.0.smithi070.stdout:5/307: fdatasync dc/d1b/d1d/d25/d38/d48/f57 0 2024-03-23T08:23:52.100 INFO:tasks.workunit.client.0.smithi070.stdout:2/350: dread d1/d6/d34/f3d [0,4194304] 0 2024-03-23T08:23:52.100 INFO:tasks.workunit.client.0.smithi070.stdout:8/368: dwrite d5/d8/f57 [0,4194304] 0 2024-03-23T08:23:52.141 INFO:tasks.workunit.client.0.smithi070.stdout:9/468: mknod db/d12/d22/d25/d26/d80/c96 0 2024-03-23T08:23:52.142 INFO:tasks.workunit.client.0.smithi070.stdout:5/308: mknod dc/d1b/d1d/d61/c67 0 2024-03-23T08:23:52.144 INFO:tasks.workunit.client.0.smithi070.stdout:2/351: mknod d1/d7/d48/d69/c73 0 2024-03-23T08:23:52.144 INFO:tasks.workunit.client.0.smithi070.stdout:5/309: write dc/d1b/f54 [685963,8155] 0 2024-03-23T08:23:52.145 INFO:tasks.workunit.client.0.smithi070.stdout:9/469: creat db/d12/d91/f97 x:0 0 0 2024-03-23T08:23:52.150 INFO:tasks.workunit.client.0.smithi070.stdout:8/369: dwrite d5/d8/d16/d3b/f76 [0,4194304] 0 2024-03-23T08:23:52.150 INFO:tasks.workunit.client.0.smithi070.stdout:2/352: creat d1/d6/d4f/f74 x:0 0 0 2024-03-23T08:23:52.151 INFO:tasks.workunit.client.0.smithi070.stdout:8/370: chown d5/d8/d16/d32/f74 7248579 1 2024-03-23T08:23:52.153 INFO:tasks.workunit.client.0.smithi070.stdout:8/371: read d5/d8/d16/d3b/f76 [3902673,3821] 0 2024-03-23T08:23:52.154 INFO:tasks.workunit.client.0.smithi070.stdout:5/310: mkdir dc/d1b/d1d/d25/d68 0 2024-03-23T08:23:52.154 INFO:tasks.workunit.client.0.smithi070.stdout:5/311: chown dc/d1b/d1d/f3a 1678250 1 2024-03-23T08:23:52.155 INFO:tasks.workunit.client.0.smithi070.stdout:5/312: dread - dc/d1b/d1d/f44 zero size 2024-03-23T08:23:52.155 INFO:tasks.workunit.client.0.smithi070.stdout:2/353: unlink d1/la 0 2024-03-23T08:23:52.156 INFO:tasks.workunit.client.0.smithi070.stdout:2/354: readlink d1/d7/df/l63 0 2024-03-23T08:23:52.162 INFO:tasks.workunit.client.0.smithi070.stdout:8/372: write d5/d8/d16/d30/f68 [223094,74096] 0 2024-03-23T08:23:52.166 INFO:tasks.workunit.client.0.smithi070.stdout:2/355: dwrite d1/d7/f20 [0,4194304] 0 2024-03-23T08:23:52.167 INFO:tasks.workunit.client.0.smithi070.stdout:9/470: link db/d12/d22/d85/d42/c5b db/d12/d22/d25/d4f/c98 0 2024-03-23T08:23:52.169 INFO:tasks.workunit.client.0.smithi070.stdout:9/471: truncate db/d12/d22/d25/f24 5232624 0 2024-03-23T08:23:52.173 INFO:tasks.workunit.client.0.smithi070.stdout:5/313: creat dc/d1b/d1d/d25/d38/f69 x:0 0 0 2024-03-23T08:23:52.175 INFO:tasks.workunit.client.0.smithi070.stdout:5/314: stat dc/d1b/d1d/d25/d38/f69 0 2024-03-23T08:23:52.177 INFO:tasks.workunit.client.0.smithi070.stdout:9/472: dwrite db/d12/d22/d85/d42/f46 [0,4194304] 0 2024-03-23T08:23:52.178 INFO:tasks.workunit.client.0.smithi070.stdout:5/315: fdatasync dc/d1b/d1d/d25/d36/d32/d5c/f5d 0 2024-03-23T08:23:52.179 INFO:tasks.workunit.client.0.smithi070.stdout:2/356: link d1/d7/d11/f2b d1/d7/d39/d58/f75 0 2024-03-23T08:23:52.183 INFO:tasks.workunit.client.0.smithi070.stdout:9/473: creat db/d12/d22/d25/d26/d5c/d7e/d6c/f99 x:0 0 0 2024-03-23T08:23:52.186 INFO:tasks.workunit.client.0.smithi070.stdout:8/373: getdents d5/d8 0 2024-03-23T08:23:52.187 INFO:tasks.workunit.client.0.smithi070.stdout:8/374: fdatasync d5/d8/f25 0 2024-03-23T08:23:52.189 INFO:tasks.workunit.client.0.smithi070.stdout:9/474: write db/d12/d22/d85/d42/f5f [331052,41175] 0 2024-03-23T08:23:52.190 INFO:tasks.workunit.client.0.smithi070.stdout:5/316: getdents dc/d1b/d1d/d25/d38/d53 0 2024-03-23T08:23:52.191 INFO:tasks.workunit.client.0.smithi070.stdout:8/375: chown d5/d8/c87 0 1 2024-03-23T08:23:52.192 INFO:tasks.workunit.client.0.smithi070.stdout:2/357: dwrite d1/d21/d5f/f67 [0,4194304] 0 2024-03-23T08:23:52.192 INFO:tasks.workunit.client.0.smithi070.stdout:5/317: chown dc/d1b/f2f 32424 1 2024-03-23T08:23:52.196 INFO:tasks.workunit.client.0.smithi070.stdout:5/318: dread dc/d1b/d1d/f62 [0,4194304] 0 2024-03-23T08:23:52.197 INFO:tasks.workunit.client.0.smithi070.stdout:8/376: unlink d5/d8/d16/d3b/f48 0 2024-03-23T08:23:52.203 INFO:tasks.workunit.client.0.smithi070.stdout:2/358: dwrite d1/d7/df/f27 [0,4194304] 0 2024-03-23T08:23:52.204 INFO:tasks.workunit.client.0.smithi070.stdout:2/359: readlink d1/d6/l15 0 2024-03-23T08:23:52.209 INFO:tasks.workunit.client.0.smithi070.stdout:5/319: dwrite dc/d1b/d1d/d25/d38/d39/f66 [0,4194304] 0 2024-03-23T08:23:52.212 INFO:tasks.workunit.client.0.smithi070.stdout:8/377: symlink d5/d8/d16/d30/d7b/l92 0 2024-03-23T08:23:52.213 INFO:tasks.workunit.client.0.smithi070.stdout:8/378: read d5/d8/d16/d32/f46 [3336529,119777] 0 2024-03-23T08:23:52.214 INFO:tasks.workunit.client.0.smithi070.stdout:5/320: chown dc/d1b/d1d/d25/d38/c3b 7 1 2024-03-23T08:23:52.216 INFO:tasks.workunit.client.0.smithi070.stdout:8/379: dread - d5/d8/d16/d30/d38/d6b/f6d zero size 2024-03-23T08:23:52.219 INFO:tasks.workunit.client.0.smithi070.stdout:5/321: dread dc/f14 [0,4194304] 0 2024-03-23T08:23:52.225 INFO:tasks.workunit.client.0.smithi070.stdout:9/475: truncate db/d12/d14/f63 1987782 0 2024-03-23T08:23:52.226 INFO:tasks.workunit.client.0.smithi070.stdout:2/360: fsync d1/d6/f65 0 2024-03-23T08:23:52.226 INFO:tasks.workunit.client.0.smithi070.stdout:5/322: mknod dc/d1b/d1d/d25/d38/d39/c6a 0 2024-03-23T08:23:52.227 INFO:tasks.workunit.client.0.smithi070.stdout:8/380: dwrite d5/d8/d16/d3b/d4d/d6f/d7c/d7d/f7f [0,4194304] 0 2024-03-23T08:23:52.229 INFO:tasks.workunit.client.0.smithi070.stdout:8/381: dread - d5/d8/d16/d3b/d4d/d6f/f7e zero size 2024-03-23T08:23:52.236 INFO:tasks.workunit.client.0.smithi070.stdout:2/361: dwrite d1/d7/f66 [0,4194304] 0 2024-03-23T08:23:52.237 INFO:tasks.workunit.client.0.smithi070.stdout:8/382: symlink d5/d8/d16/d32/l93 0 2024-03-23T08:23:52.239 INFO:tasks.workunit.client.0.smithi070.stdout:8/383: dread d5/d8/f4a [0,4194304] 0 2024-03-23T08:23:52.244 INFO:tasks.workunit.client.0.smithi070.stdout:2/362: mkdir d1/d7/d11/d38/d4c/d76 0 2024-03-23T08:23:52.245 INFO:tasks.workunit.client.0.smithi070.stdout:2/363: dread - d1/d6/d4f/f74 zero size 2024-03-23T08:23:52.246 INFO:tasks.workunit.client.0.smithi070.stdout:8/384: creat d5/d8/d16/d30/d7b/f94 x:0 0 0 2024-03-23T08:23:52.248 INFO:tasks.workunit.client.0.smithi070.stdout:9/476: creat db/d12/f9a x:0 0 0 2024-03-23T08:23:52.251 INFO:tasks.workunit.client.0.smithi070.stdout:8/385: symlink d5/d8/d16/d30/d38/d6b/d72/l95 0 2024-03-23T08:23:52.251 INFO:tasks.workunit.client.0.smithi070.stdout:2/364: dread d1/d7/df/f27 [0,4194304] 0 2024-03-23T08:23:52.251 INFO:tasks.workunit.client.0.smithi070.stdout:8/386: stat d5/d8/f2c 0 2024-03-23T08:23:52.253 INFO:tasks.workunit.client.0.smithi070.stdout:8/387: stat d5/d8/d16/d3b/d4d/d6f/d7c/l8f 0 2024-03-23T08:23:52.254 INFO:tasks.workunit.client.0.smithi070.stdout:2/365: creat d1/d7/d48/d69/f77 x:0 0 0 2024-03-23T08:23:52.256 INFO:tasks.workunit.client.0.smithi070.stdout:8/388: readlink d5/d8/d16/d30/d38/d6b/d72/l8b 0 2024-03-23T08:23:52.257 INFO:tasks.workunit.client.0.smithi070.stdout:2/366: mknod d1/d29/c78 0 2024-03-23T08:23:52.259 INFO:tasks.workunit.client.0.smithi070.stdout:2/367: symlink d1/d29/d52/l79 0 2024-03-23T08:23:52.260 INFO:tasks.workunit.client.0.smithi070.stdout:8/389: creat d5/d8/d16/d32/d84/f96 x:0 0 0 2024-03-23T08:23:52.262 INFO:tasks.workunit.client.0.smithi070.stdout:2/368: mkdir d1/d7/d48/d69/d7a 0 2024-03-23T08:23:52.264 INFO:tasks.workunit.client.0.smithi070.stdout:2/369: mkdir d1/d7/d39/d47/d7b 0 2024-03-23T08:23:52.270 INFO:tasks.workunit.client.0.smithi070.stdout:8/390: dwrite d5/d8/f33 [0,4194304] 0 2024-03-23T08:23:52.273 INFO:tasks.workunit.client.0.smithi070.stdout:2/370: dwrite d1/d7/df/f4b [0,4194304] 0 2024-03-23T08:23:52.275 INFO:tasks.workunit.client.0.smithi070.stdout:8/391: dread d5/d8/f19 [0,4194304] 0 2024-03-23T08:23:52.277 INFO:tasks.workunit.client.0.smithi070.stdout:8/392: dread - d5/d8/d16/d3b/f5d zero size 2024-03-23T08:23:52.278 INFO:tasks.workunit.client.0.smithi070.stdout:2/371: getdents d1/d7/d39/d58 0 2024-03-23T08:23:52.280 INFO:tasks.workunit.client.0.smithi070.stdout:8/393: mknod d5/d8/d16/d3b/d4d/d6f/c97 0 2024-03-23T08:23:52.281 INFO:tasks.workunit.client.0.smithi070.stdout:8/394: chown d5/d8/f17 4638752 1 2024-03-23T08:23:52.282 INFO:tasks.workunit.client.0.smithi070.stdout:2/372: symlink d1/d7/d48/d5b/l7c 0 2024-03-23T08:23:52.284 INFO:tasks.workunit.client.0.smithi070.stdout:2/373: chown d1/d6/f4d 58 1 2024-03-23T08:23:52.285 INFO:tasks.workunit.client.0.smithi070.stdout:8/395: mkdir d5/d8/d16/d32/d84/d98 0 2024-03-23T08:23:52.290 INFO:tasks.workunit.client.0.smithi070.stdout:8/396: creat d5/d8/d16/d3b/d8c/f99 x:0 0 0 2024-03-23T08:23:52.293 INFO:tasks.workunit.client.0.smithi070.stdout:2/374: getdents d1/d7/d48/d69 0 2024-03-23T08:23:52.295 INFO:tasks.workunit.client.0.smithi070.stdout:2/375: mkdir d1/d7/d39/d47/d7d 0 2024-03-23T08:23:52.297 INFO:tasks.workunit.client.0.smithi070.stdout:2/376: write d1/d7/d11/d38/d4c/f6a [42450,118851] 0 2024-03-23T08:23:52.306 INFO:tasks.workunit.client.0.smithi070.stdout:2/377: write d1/d7/f17 [4826145,63133] 0 2024-03-23T08:23:52.310 INFO:tasks.workunit.client.0.smithi070.stdout:2/378: creat d1/d7/f7e x:0 0 0 2024-03-23T08:23:52.313 INFO:tasks.workunit.client.0.smithi070.stdout:2/379: dread d1/d7/df/f27 [0,4194304] 0 2024-03-23T08:23:52.314 INFO:tasks.workunit.client.0.smithi070.stdout:2/380: fsync d1/f2f 0 2024-03-23T08:23:52.316 INFO:tasks.workunit.client.0.smithi070.stdout:2/381: write d1/d7/d11/f56 [154606,17813] 0 2024-03-23T08:23:52.317 INFO:tasks.workunit.client.0.smithi070.stdout:2/382: write d1/d7/d11/d38/d4c/f6a [845206,10689] 0 2024-03-23T08:23:52.319 INFO:tasks.workunit.client.0.smithi070.stdout:7/301: sync 2024-03-23T08:23:52.319 INFO:tasks.workunit.client.0.smithi070.stdout:1/346: sync 2024-03-23T08:23:52.319 INFO:tasks.workunit.client.0.smithi070.stdout:4/249: sync 2024-03-23T08:23:52.319 INFO:tasks.workunit.client.0.smithi070.stdout:2/383: creat d1/d29/f7f x:0 0 0 2024-03-23T08:23:52.320 INFO:tasks.workunit.client.0.smithi070.stdout:7/302: dread - d25/d40/f68 zero size 2024-03-23T08:23:52.321 INFO:tasks.workunit.client.0.smithi070.stdout:7/303: stat d25/f4f 0 2024-03-23T08:23:52.356 INFO:tasks.workunit.client.0.smithi070.stdout:1/347: read d8/d1b/f22 [3798513,69230] 0 2024-03-23T08:23:52.356 INFO:tasks.workunit.client.0.smithi070.stdout:4/250: mkdir d3/d6/d11/d1a/d27/d4b/d5c 0 2024-03-23T08:23:52.357 INFO:tasks.workunit.client.0.smithi070.stdout:1/348: readlink d8/d1b/l35 0 2024-03-23T08:23:52.357 INFO:tasks.workunit.client.0.smithi070.stdout:2/384: symlink d1/l80 0 2024-03-23T08:23:52.359 INFO:tasks.workunit.client.0.smithi070.stdout:7/304: truncate d25/d27/d6a/d44/d6f/f71 733886 0 2024-03-23T08:23:52.359 INFO:tasks.workunit.client.0.smithi070.stdout:7/305: chown d25/l49 32695426 1 2024-03-23T08:23:52.359 INFO:tasks.workunit.client.0.smithi070.stdout:1/349: rmdir d8/db 39 2024-03-23T08:23:52.362 INFO:tasks.workunit.client.0.smithi070.stdout:7/306: creat d25/d27/d6a/f7d x:0 0 0 2024-03-23T08:23:52.367 INFO:tasks.workunit.client.0.smithi070.stdout:1/350: mknod d8/d26/c82 0 2024-03-23T08:23:52.371 INFO:tasks.workunit.client.0.smithi070.stdout:4/251: getdents d3/d6/d11 0 2024-03-23T08:23:52.371 INFO:tasks.workunit.client.0.smithi070.stdout:4/252: readlink d3/l3d 0 2024-03-23T08:23:52.372 INFO:tasks.workunit.client.0.smithi070.stdout:4/253: chown d3/d6/d11/d1a/d27/d4b/c54 142430485 1 2024-03-23T08:23:52.372 INFO:tasks.workunit.client.0.smithi070.stdout:1/351: mknod d8/d11/d61/d74/d5d/c83 0 2024-03-23T08:23:52.377 INFO:tasks.workunit.client.0.smithi070.stdout:4/254: symlink d3/d6/d11/d1a/d41/d56/l5d 0 2024-03-23T08:23:52.379 INFO:tasks.workunit.client.0.smithi070.stdout:4/255: write d3/d6/d11/d1a/d27/f4e [657107,54984] 0 2024-03-23T08:23:52.379 INFO:tasks.workunit.client.0.smithi070.stdout:2/385: dwrite d1/d6/f4d [0,4194304] 0 2024-03-23T08:23:52.380 INFO:tasks.workunit.client.0.smithi070.stdout:1/352: creat d8/db/d40/f84 x:0 0 0 2024-03-23T08:23:52.380 INFO:tasks.workunit.client.0.smithi070.stdout:4/256: readlink d3/l3a 0 2024-03-23T08:23:52.386 INFO:tasks.workunit.client.0.smithi070.stdout:7/307: dwrite d25/f38 [0,4194304] 0 2024-03-23T08:23:52.387 INFO:tasks.workunit.client.0.smithi070.stdout:7/308: truncate d25/d27/f6d 1072476 0 2024-03-23T08:23:52.393 INFO:tasks.workunit.client.0.smithi070.stdout:7/309: dwrite d25/d27/f3c [0,4194304] 0 2024-03-23T08:23:52.394 INFO:tasks.workunit.client.0.smithi070.stdout:7/310: chown d25/d27/d6a 104695038 1 2024-03-23T08:23:52.400 INFO:tasks.workunit.client.0.smithi070.stdout:7/311: dwrite d25/d27/f3c [0,4194304] 0 2024-03-23T08:23:52.598 INFO:tasks.workunit.client.0.smithi070.stdout:2/386: dwrite d1/d7/f36 [4194304,4194304] 0 2024-03-23T08:23:52.599 INFO:tasks.workunit.client.0.smithi070.stdout:0/657: sync 2024-03-23T08:23:52.599 INFO:tasks.workunit.client.0.smithi070.stdout:6/486: sync 2024-03-23T08:23:52.599 INFO:tasks.workunit.client.0.smithi070.stdout:5/323: sync 2024-03-23T08:23:52.599 INFO:tasks.workunit.client.0.smithi070.stdout:3/582: sync 2024-03-23T08:23:52.601 INFO:tasks.workunit.client.0.smithi070.stdout:5/324: write dc/d1b/d1d/d25/d38/d53/f60 [939862,79118] 0 2024-03-23T08:23:52.608 INFO:tasks.workunit.client.0.smithi070.stdout:7/312: rename d25/d27/d6a/d44/d6f/f7b to d25/d27/d6a/d44/f7e 0 2024-03-23T08:23:52.611 INFO:tasks.workunit.client.0.smithi070.stdout:1/353: mkdir d8/db/d40/d43/d64/d85 0 2024-03-23T08:23:52.617 INFO:tasks.workunit.client.0.smithi070.stdout:2/387: mknod d1/d7/d11/c81 0 2024-03-23T08:23:52.618 INFO:tasks.workunit.client.0.smithi070.stdout:2/388: readlink d1/d7/df/l13 0 2024-03-23T08:23:52.620 INFO:tasks.workunit.client.0.smithi070.stdout:6/487: readlink d0/d23/l78 0 2024-03-23T08:23:52.622 INFO:tasks.workunit.client.0.smithi070.stdout:0/658: creat d16/d26/d27/d4c/d60/d7c/daf/fe9 x:0 0 0 2024-03-23T08:23:52.622 INFO:tasks.workunit.client.0.smithi070.stdout:0/659: dread - d16/d22/fd3 zero size 2024-03-23T08:23:52.623 INFO:tasks.workunit.client.0.smithi070.stdout:3/583: dread db/f23 [0,4194304] 0 2024-03-23T08:23:52.623 INFO:tasks.workunit.client.0.smithi070.stdout:0/660: readlink d16/d22/d43/l8f 0 2024-03-23T08:23:52.626 INFO:tasks.workunit.client.0.smithi070.stdout:3/584: write db/d14/d65/dae/fb8 [182557,17813] 0 2024-03-23T08:23:52.629 INFO:tasks.workunit.client.0.smithi070.stdout:3/585: stat db/df/d15/c66 0 2024-03-23T08:23:52.632 INFO:tasks.workunit.client.0.smithi070.stdout:2/389: creat d1/d7/d39/f82 x:0 0 0 2024-03-23T08:23:52.632 INFO:tasks.workunit.client.0.smithi070.stdout:0/661: dwrite d16/d22/fa6 [0,4194304] 0 2024-03-23T08:23:52.633 INFO:tasks.workunit.client.0.smithi070.stdout:2/390: dread - d1/d29/f70 zero size 2024-03-23T08:23:52.633 INFO:tasks.workunit.client.0.smithi070.stdout:0/662: readlink d16/d26/d27/d4c/lb2 0 2024-03-23T08:23:52.637 INFO:tasks.workunit.client.0.smithi070.stdout:6/488: rename d0/d23/d1c/d38/d52/f7c to d0/d23/d1c/d2f/dab/fac 0 2024-03-23T08:23:52.651 INFO:tasks.workunit.client.0.smithi070.stdout:6/489: dwrite d0/d23/d28/d35/f46 [4194304,4194304] 0 2024-03-23T08:23:52.655 INFO:tasks.workunit.client.0.smithi070.stdout:3/586: fsync db/f36 0 2024-03-23T08:23:52.660 INFO:tasks.workunit.client.0.smithi070.stdout:6/490: write d0/d23/d74/d84/d3c/f60 [338926,73531] 0 2024-03-23T08:23:52.684 INFO:tasks.workunit.client.0.smithi070.stdout:6/491: dread d0/d23/d74/d84/d3c/d5b/f79 [0,4194304] 0 2024-03-23T08:23:52.686 INFO:tasks.workunit.client.0.smithi070.stdout:4/257: truncate d3/d6/d11/d1a/d1b/f1c 1603780 0 2024-03-23T08:23:52.687 INFO:tasks.workunit.client.0.smithi070.stdout:6/492: creat d0/d23/d1c/d38/d9a/fad x:0 0 0 2024-03-23T08:23:52.688 INFO:tasks.workunit.client.0.smithi070.stdout:7/313: rmdir d25/d72 0 2024-03-23T08:23:52.689 INFO:tasks.workunit.client.0.smithi070.stdout:7/314: write d25/f43 [984320,22131] 0 2024-03-23T08:23:52.689 INFO:tasks.workunit.client.0.smithi070.stdout:4/258: mknod d3/d6/d11/d1a/d41/c5e 0 2024-03-23T08:23:52.695 INFO:tasks.workunit.client.0.smithi070.stdout:7/315: write d25/d27/d6a/d44/f78 [826756,99068] 0 2024-03-23T08:23:52.698 INFO:tasks.workunit.client.0.smithi070.stdout:7/316: chown d25/d27/d6a/f5f 72 1 2024-03-23T08:23:52.700 INFO:tasks.workunit.client.0.smithi070.stdout:5/325: write dc/d1b/d1d/d25/d38/d48/f56 [89004,47065] 0 2024-03-23T08:23:52.700 INFO:tasks.workunit.client.0.smithi070.stdout:6/493: dread d0/d23/d4d/f63 [0,4194304] 0 2024-03-23T08:23:52.705 INFO:tasks.workunit.client.0.smithi070.stdout:6/494: write d0/f9c [982543,18332] 0 2024-03-23T08:23:52.710 INFO:tasks.workunit.client.0.smithi070.stdout:7/317: creat d25/d27/d6a/f7f x:0 0 0 2024-03-23T08:23:52.712 INFO:tasks.workunit.client.0.smithi070.stdout:6/495: dread d0/d23/d4d/f63 [0,4194304] 0 2024-03-23T08:23:52.714 INFO:tasks.workunit.client.0.smithi070.stdout:4/259: dwrite d3/d6/d9/f4a [0,4194304] 0 2024-03-23T08:23:52.723 INFO:tasks.workunit.client.0.smithi070.stdout:6/496: truncate d0/d3/d2c/f58 162699 0 2024-03-23T08:23:52.725 INFO:tasks.workunit.client.0.smithi070.stdout:0/663: rename d16/d22/c48 to d16/d26/d27/cea 0 2024-03-23T08:23:52.726 INFO:tasks.workunit.client.0.smithi070.stdout:5/326: dwrite dc/d1b/d1d/d25/d38/d39/f66 [0,4194304] 0 2024-03-23T08:23:52.727 INFO:tasks.workunit.client.0.smithi070.stdout:8/397: sync 2024-03-23T08:23:52.727 INFO:tasks.workunit.client.0.smithi070.stdout:9/477: sync 2024-03-23T08:23:52.732 INFO:tasks.workunit.client.0.smithi070.stdout:7/318: dwrite d25/d27/f48 [0,4194304] 0 2024-03-23T08:23:52.734 INFO:tasks.workunit.client.0.smithi070.stdout:8/398: mkdir d5/d8/d9a 0 2024-03-23T08:23:52.738 INFO:tasks.workunit.client.0.smithi070.stdout:8/399: read - d5/d8/d16/d32/f74 zero size 2024-03-23T08:23:52.738 INFO:tasks.workunit.client.0.smithi070.stdout:0/664: chown d16/d26/d27/d3d/lbb 33492762 1 2024-03-23T08:23:52.741 INFO:tasks.workunit.client.0.smithi070.stdout:2/391: truncate d1/fd 2443551 0 2024-03-23T08:23:52.741 INFO:tasks.workunit.client.0.smithi070.stdout:5/327: creat dc/d1b/d1d/d25/d38/d39/f6b x:0 0 0 2024-03-23T08:23:52.742 INFO:tasks.workunit.client.0.smithi070.stdout:0/665: chown d16/d22/d43/f78 2056616 1 2024-03-23T08:23:52.744 INFO:tasks.workunit.client.0.smithi070.stdout:9/478: creat db/d12/d22/d25/d26/d5c/f9b x:0 0 0 2024-03-23T08:23:52.746 INFO:tasks.workunit.client.0.smithi070.stdout:7/319: write d25/d40/f41 [507825,99904] 0 2024-03-23T08:23:52.749 INFO:tasks.workunit.client.0.smithi070.stdout:7/320: dread - d25/d27/d6a/d44/f54 zero size 2024-03-23T08:23:52.750 INFO:tasks.workunit.client.0.smithi070.stdout:5/328: read dc/d1b/f2f [1814295,80265] 0 2024-03-23T08:23:52.750 INFO:tasks.workunit.client.0.smithi070.stdout:0/666: creat d16/d26/d27/d86/d9e/ddd/feb x:0 0 0 2024-03-23T08:23:52.752 INFO:tasks.workunit.client.0.smithi070.stdout:0/667: stat d16/d22/d43 0 2024-03-23T08:23:52.754 INFO:tasks.workunit.client.0.smithi070.stdout:0/668: write d16/d22/d43/d47/d87/fcb [711110,122378] 0 2024-03-23T08:23:52.757 INFO:tasks.workunit.client.0.smithi070.stdout:2/392: dread d1/d7/f20 [0,4194304] 0 2024-03-23T08:23:52.757 INFO:tasks.workunit.client.0.smithi070.stdout:8/400: link d5/d8/d16/d30/c40 d5/d8/d16/d3b/d4d/c9b 0 2024-03-23T08:23:52.757 INFO:tasks.workunit.client.0.smithi070.stdout:7/321: write d25/d27/f6d [639593,114570] 0 2024-03-23T08:23:52.761 INFO:tasks.workunit.client.0.smithi070.stdout:5/329: dread dc/d1b/d1d/d25/d38/d53/f5f [0,4194304] 0 2024-03-23T08:23:52.761 INFO:tasks.workunit.client.0.smithi070.stdout:0/669: mkdir d16/d26/d27/d86/db6/dec 0 2024-03-23T08:23:52.763 INFO:tasks.workunit.client.0.smithi070.stdout:8/401: symlink d5/d8/d16/d30/d7b/l9c 0 2024-03-23T08:23:52.764 INFO:tasks.workunit.client.0.smithi070.stdout:5/330: chown dc/d1b/d1d/d25/d36/f2a 1 1 2024-03-23T08:23:52.764 INFO:tasks.workunit.client.0.smithi070.stdout:5/331: readlink dc/l12 0 2024-03-23T08:23:52.765 INFO:tasks.workunit.client.0.smithi070.stdout:7/322: creat d25/d27/d6a/f80 x:0 0 0 2024-03-23T08:23:52.766 INFO:tasks.workunit.client.0.smithi070.stdout:5/332: dread dc/d1b/d1d/d25/d38/d48/f56 [0,4194304] 0 2024-03-23T08:23:52.768 INFO:tasks.workunit.client.0.smithi070.stdout:5/333: read dc/d1b/d1d/d25/d38/d39/f3c [3927926,26462] 0 2024-03-23T08:23:52.768 INFO:tasks.workunit.client.0.smithi070.stdout:0/670: rename d16/d22/c40 to d16/ced 0 2024-03-23T08:23:52.770 INFO:tasks.workunit.client.0.smithi070.stdout:8/402: creat d5/d8/f9d x:0 0 0 2024-03-23T08:23:52.772 INFO:tasks.workunit.client.0.smithi070.stdout:8/403: read d5/d8/d16/d32/f5c [8061159,13065] 0 2024-03-23T08:23:52.772 INFO:tasks.workunit.client.0.smithi070.stdout:2/393: mknod d1/d7/d39/d47/d7b/c83 0 2024-03-23T08:23:52.774 INFO:tasks.workunit.client.0.smithi070.stdout:7/323: symlink d25/d40/l81 0 2024-03-23T08:23:52.776 INFO:tasks.workunit.client.0.smithi070.stdout:5/334: dwrite dc/d1b/d1d/d25/d36/f2a [0,4194304] 0 2024-03-23T08:23:52.776 INFO:tasks.workunit.client.0.smithi070.stdout:2/394: write d1/d7/d11/d38/d4c/f4e [4295885,125423] 0 2024-03-23T08:23:52.779 INFO:tasks.workunit.client.0.smithi070.stdout:5/335: chown dc/c52 414549 1 2024-03-23T08:23:52.779 INFO:tasks.workunit.client.0.smithi070.stdout:7/324: chown d25/d27/f48 8705007 1 2024-03-23T08:23:52.786 INFO:tasks.workunit.client.0.smithi070.stdout:8/404: dwrite d5/fe [0,4194304] 0 2024-03-23T08:23:52.788 INFO:tasks.workunit.client.0.smithi070.stdout:2/395: link d1/d7/d48/d69/f77 d1/d7/d11/f84 0 2024-03-23T08:23:52.790 INFO:tasks.workunit.client.0.smithi070.stdout:0/671: getdents d16/d26/d27/d86 0 2024-03-23T08:23:52.792 INFO:tasks.workunit.client.0.smithi070.stdout:5/336: dwrite dc/d1b/d1d/d25/d38/d39/f4f [0,4194304] 0 2024-03-23T08:23:52.798 INFO:tasks.workunit.client.0.smithi070.stdout:2/396: creat d1/d6/f85 x:0 0 0 2024-03-23T08:23:52.802 INFO:tasks.workunit.client.0.smithi070.stdout:7/325: getdents d25 0 2024-03-23T08:23:52.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/337: dwrite dc/d1b/d1d/d25/d38/d39/f66 [0,4194304] 0 2024-03-23T08:23:52.806 INFO:tasks.workunit.client.0.smithi070.stdout:0/672: mkdir d16/d26/d27/dee 0 2024-03-23T08:23:52.808 INFO:tasks.workunit.client.0.smithi070.stdout:2/397: mkdir d1/d7/d39/d47/d7b/d86 0 2024-03-23T08:23:52.812 INFO:tasks.workunit.client.0.smithi070.stdout:8/405: link d5/d8/d16/d30/d38/d6b/l78 d5/d8/d16/d8d/l9e 0 2024-03-23T08:23:52.814 INFO:tasks.workunit.client.0.smithi070.stdout:0/673: mkdir d16/d26/d27/d86/def 0 2024-03-23T08:23:52.814 INFO:tasks.workunit.client.0.smithi070.stdout:7/326: dread d25/d27/d6a/d44/f5a [0,4194304] 0 2024-03-23T08:23:52.817 INFO:tasks.workunit.client.0.smithi070.stdout:7/327: chown d25/d27/d6a/d44/l4e 10726 1 2024-03-23T08:23:52.817 INFO:tasks.workunit.client.0.smithi070.stdout:0/674: chown d16/d22/d43/d47/da9/fe6 15677 1 2024-03-23T08:23:52.817 INFO:tasks.workunit.client.0.smithi070.stdout:8/406: rmdir d5/d8/d16/d30 39 2024-03-23T08:23:52.818 INFO:tasks.workunit.client.0.smithi070.stdout:7/328: write d25/d27/d6a/f75 [80041,72133] 0 2024-03-23T08:23:52.822 INFO:tasks.workunit.client.0.smithi070.stdout:7/329: write d25/d27/d6a/d44/f54 [945007,25153] 0 2024-03-23T08:23:52.826 INFO:tasks.workunit.client.0.smithi070.stdout:0/675: dread d16/d26/d27/d4c/d60/d7c/daf/fdb [0,4194304] 0 2024-03-23T08:23:52.830 INFO:tasks.workunit.client.0.smithi070.stdout:9/479: write db/d12/d14/f23 [4744241,45541] 0 2024-03-23T08:23:52.832 INFO:tasks.workunit.client.0.smithi070.stdout:5/338: link dc/d1b/d1d/d25/d36/d32/l3e dc/l6c 0 2024-03-23T08:23:52.835 INFO:tasks.workunit.client.0.smithi070.stdout:8/407: write d5/d8/d16/d3b/d4d/d6f/f89 [1387093,13156] 0 2024-03-23T08:23:52.835 INFO:tasks.workunit.client.0.smithi070.stdout:9/480: read - db/d12/d22/d25/d26/d5c/d7e/f74 zero size 2024-03-23T08:23:52.841 INFO:tasks.workunit.client.0.smithi070.stdout:7/330: dwrite d25/d27/f60 [0,4194304] 0 2024-03-23T08:23:52.842 INFO:tasks.workunit.client.0.smithi070.stdout:9/481: chown db/d12/d22/d25/d26/d5c/d7e/f56 3101 1 2024-03-23T08:23:52.842 INFO:tasks.workunit.client.0.smithi070.stdout:7/331: stat l1a 0 2024-03-23T08:23:52.843 INFO:tasks.workunit.client.0.smithi070.stdout:7/332: chown d25/d27/f57 1035643656 1 2024-03-23T08:23:52.855 INFO:tasks.workunit.client.0.smithi070.stdout:1/354: sync 2024-03-23T08:23:52.856 INFO:tasks.workunit.client.0.smithi070.stdout:1/355: chown d8/db/d40 110050 1 2024-03-23T08:23:52.856 INFO:tasks.workunit.client.0.smithi070.stdout:4/260: write d3/d19/d2a/f2b [1129780,98456] 0 2024-03-23T08:23:52.864 INFO:tasks.workunit.client.0.smithi070.stdout:5/339: symlink dc/d1b/d1d/d25/d38/d39/l6d 0 2024-03-23T08:23:52.864 INFO:tasks.workunit.client.0.smithi070.stdout:5/340: readlink l5 0 2024-03-23T08:23:52.866 INFO:tasks.workunit.client.0.smithi070.stdout:8/408: truncate d5/d8/d16/d30/d7b/f94 188853 0 2024-03-23T08:23:52.866 INFO:tasks.workunit.client.0.smithi070.stdout:0/676: write d16/d26/d27/d86/db6/fce [817975,49781] 0 2024-03-23T08:23:52.867 INFO:tasks.workunit.client.0.smithi070.stdout:7/333: symlink d25/d27/d6a/d44/l82 0 2024-03-23T08:23:52.868 INFO:tasks.workunit.client.0.smithi070.stdout:7/334: write d25/d40/f41 [1973085,29764] 0 2024-03-23T08:23:52.870 INFO:tasks.workunit.client.0.smithi070.stdout:0/677: dread d16/d26/d27/d4c/d60/d7c/daf/fdb [0,4194304] 0 2024-03-23T08:23:52.872 INFO:tasks.workunit.client.0.smithi070.stdout:0/678: chown d16/d22/d43/f96 4077990 1 2024-03-23T08:23:52.873 INFO:tasks.workunit.client.0.smithi070.stdout:9/482: link db/d12/d22/d25/f78 db/d12/d22/d25/d4f/d5a/f9c 0 2024-03-23T08:23:52.873 INFO:tasks.workunit.client.0.smithi070.stdout:0/679: chown d16/d22/d43/d47 2 1 2024-03-23T08:23:52.876 INFO:tasks.workunit.client.0.smithi070.stdout:2/398: dwrite d1/fc [0,4194304] 0 2024-03-23T08:23:52.876 INFO:tasks.workunit.client.0.smithi070.stdout:0/680: write d16/d26/d27/d4c/d60/d7c/fcf [649246,56334] 0 2024-03-23T08:23:52.877 INFO:tasks.workunit.client.0.smithi070.stdout:2/399: read d1/d29/f35 [1414039,35532] 0 2024-03-23T08:23:52.881 INFO:tasks.workunit.client.0.smithi070.stdout:7/335: rename d25/d40/f41 to d25/f83 0 2024-03-23T08:23:52.881 INFO:tasks.workunit.client.0.smithi070.stdout:3/587: sync 2024-03-23T08:23:52.881 INFO:tasks.workunit.client.0.smithi070.stdout:6/497: sync 2024-03-23T08:23:52.882 INFO:tasks.workunit.client.0.smithi070.stdout:7/336: readlink d25/d27/d6a/d44/l52 0 2024-03-23T08:23:52.882 INFO:tasks.workunit.client.0.smithi070.stdout:5/341: getdents dc/d1b/d1d/d25/d68 0 2024-03-23T08:23:52.883 INFO:tasks.workunit.client.0.smithi070.stdout:6/498: readlink d0/d23/d74/d84/d3c/l69 0 2024-03-23T08:23:52.883 INFO:tasks.workunit.client.0.smithi070.stdout:5/342: stat dc/d1b/f2f 0 2024-03-23T08:23:52.884 INFO:tasks.workunit.client.0.smithi070.stdout:8/409: mkdir d5/d8/d16/d32/d84/d9f 0 2024-03-23T08:23:52.884 INFO:tasks.workunit.client.0.smithi070.stdout:7/337: chown d25/d27/d6a/c6b 222 1 2024-03-23T08:23:52.886 INFO:tasks.workunit.client.0.smithi070.stdout:9/483: readlink db/d12/d91/l94 0 2024-03-23T08:23:52.887 INFO:tasks.workunit.client.0.smithi070.stdout:1/356: rmdir d8/d1a/d7c 0 2024-03-23T08:23:52.889 INFO:tasks.workunit.client.0.smithi070.stdout:1/357: dread - d8/d11/d61/d74/f71 zero size 2024-03-23T08:23:52.893 INFO:tasks.workunit.client.0.smithi070.stdout:0/681: read d16/f18 [316242,19241] 0 2024-03-23T08:23:52.893 INFO:tasks.workunit.client.0.smithi070.stdout:9/484: dwrite db/d12/d22/f93 [0,4194304] 0 2024-03-23T08:23:52.894 INFO:tasks.workunit.client.0.smithi070.stdout:3/588: creat db/df/d5d/fda x:0 0 0 2024-03-23T08:23:52.894 INFO:tasks.workunit.client.0.smithi070.stdout:9/485: fsync db/d12/d22/d85/d42/f46 0 2024-03-23T08:23:52.896 INFO:tasks.workunit.client.0.smithi070.stdout:5/343: symlink dc/d1b/d1d/d25/d38/d53/l6e 0 2024-03-23T08:23:52.898 INFO:tasks.workunit.client.0.smithi070.stdout:7/338: rename d25/d27/d6a/f75 to d25/f84 0 2024-03-23T08:23:52.899 INFO:tasks.workunit.client.0.smithi070.stdout:7/339: chown d25/d27/d6a/c6e 1149719 1 2024-03-23T08:23:52.900 INFO:tasks.workunit.client.0.smithi070.stdout:8/410: creat d5/d8/d16/d3b/d4d/d6f/fa0 x:0 0 0 2024-03-23T08:23:52.902 INFO:tasks.workunit.client.0.smithi070.stdout:4/261: getdents d3/d6/d35/d3f 0 2024-03-23T08:23:52.902 INFO:tasks.workunit.client.0.smithi070.stdout:7/340: dread d25/f84 [0,4194304] 0 2024-03-23T08:23:52.902 INFO:tasks.workunit.client.0.smithi070.stdout:4/262: dread - d3/d6/f38 zero size 2024-03-23T08:23:52.905 INFO:tasks.workunit.client.0.smithi070.stdout:1/358: creat d8/d1b/f86 x:0 0 0 2024-03-23T08:23:52.908 INFO:tasks.workunit.client.0.smithi070.stdout:5/344: dread dc/d1b/d1d/d25/f43 [0,4194304] 0 2024-03-23T08:23:52.910 INFO:tasks.workunit.client.0.smithi070.stdout:3/589: creat db/d7f/fdb x:0 0 0 2024-03-23T08:23:52.915 INFO:tasks.workunit.client.0.smithi070.stdout:8/411: symlink d5/d8/d16/d3b/d4d/d6f/d7c/d7d/la1 0 2024-03-23T08:23:52.916 INFO:tasks.workunit.client.0.smithi070.stdout:8/412: fdatasync d5/d8/d16/d3b/f5f 0 2024-03-23T08:23:52.917 INFO:tasks.workunit.client.0.smithi070.stdout:9/486: dwrite db/d12/d14/f95 [0,4194304] 0 2024-03-23T08:23:52.918 INFO:tasks.workunit.client.0.smithi070.stdout:7/341: unlink d25/d27/d6a/f7d 0 2024-03-23T08:23:52.921 INFO:tasks.workunit.client.0.smithi070.stdout:2/400: link d1/d7/d39/d58/f75 d1/d7/d11/d38/f87 0 2024-03-23T08:23:52.924 INFO:tasks.workunit.client.0.smithi070.stdout:5/345: rmdir dc 39 2024-03-23T08:23:52.925 INFO:tasks.workunit.client.0.smithi070.stdout:0/682: mkdir d16/d22/df0 0 2024-03-23T08:23:52.936 INFO:tasks.workunit.client.0.smithi070.stdout:7/342: mkdir d25/d27/d6a/d44/d85 0 2024-03-23T08:23:52.937 INFO:tasks.workunit.client.0.smithi070.stdout:2/401: rename d1/d21/l57 to d1/d6/d34/l88 0 2024-03-23T08:23:52.942 INFO:tasks.workunit.client.0.smithi070.stdout:6/499: write d0/d3/d2c/f58 [1000038,69902] 0 2024-03-23T08:23:52.946 INFO:tasks.workunit.client.0.smithi070.stdout:8/413: mkdir d5/d8/d9a/da2 0 2024-03-23T08:23:52.946 INFO:tasks.workunit.client.0.smithi070.stdout:1/359: truncate d8/d1b/f25 2953712 0 2024-03-23T08:23:52.947 INFO:tasks.workunit.client.0.smithi070.stdout:3/590: dwrite db/d14/f3f [0,4194304] 0 2024-03-23T08:23:52.949 INFO:tasks.workunit.client.0.smithi070.stdout:7/343: creat d25/d27/d6a/f86 x:0 0 0 2024-03-23T08:23:52.950 INFO:tasks.workunit.client.0.smithi070.stdout:1/360: dread - d8/d1b/f86 zero size 2024-03-23T08:23:52.951 INFO:tasks.workunit.client.0.smithi070.stdout:2/402: rename d1/d6/f85 to d1/d7/d39/d58/f89 0 2024-03-23T08:23:52.956 INFO:tasks.workunit.client.0.smithi070.stdout:5/346: creat dc/d1b/d1d/f6f x:0 0 0 2024-03-23T08:23:52.956 INFO:tasks.workunit.client.0.smithi070.stdout:0/683: symlink d16/d22/d43/d47/da9/de7/lf1 0 2024-03-23T08:23:52.960 INFO:tasks.workunit.client.0.smithi070.stdout:9/487: fsync db/d12/d34/d47/f86 0 2024-03-23T08:23:52.960 INFO:tasks.workunit.client.0.smithi070.stdout:8/414: dwrite d5/f7 [0,4194304] 0 2024-03-23T08:23:52.964 INFO:tasks.workunit.client.0.smithi070.stdout:2/403: dwrite d1/d7/f17 [0,4194304] 0 2024-03-23T08:23:52.964 INFO:tasks.workunit.client.0.smithi070.stdout:8/415: readlink d5/d8/d16/d3b/d4d/d6f/d7c/l8f 0 2024-03-23T08:23:52.966 INFO:tasks.workunit.client.0.smithi070.stdout:7/344: creat d25/f87 x:0 0 0 2024-03-23T08:23:52.969 INFO:tasks.workunit.client.0.smithi070.stdout:1/361: unlink d8/d1a/c5b 0 2024-03-23T08:23:52.974 INFO:tasks.workunit.client.0.smithi070.stdout:8/416: dwrite d5/d8/d16/d30/f61 [0,4194304] 0 2024-03-23T08:23:52.974 INFO:tasks.workunit.client.0.smithi070.stdout:6/500: symlink d0/d23/d4d/da5/lae 0 2024-03-23T08:23:52.979 INFO:tasks.workunit.client.0.smithi070.stdout:3/591: rename db/d7f/c87 to db/d20/d49/cdc 0 2024-03-23T08:23:52.981 INFO:tasks.workunit.client.0.smithi070.stdout:8/417: dread d5/d8/f4e [0,4194304] 0 2024-03-23T08:23:52.983 INFO:tasks.workunit.client.0.smithi070.stdout:9/488: dread db/d12/f39 [0,4194304] 0 2024-03-23T08:23:52.983 INFO:tasks.workunit.client.0.smithi070.stdout:7/345: mknod d25/d27/d6a/d44/d6f/c88 0 2024-03-23T08:23:52.985 INFO:tasks.workunit.client.0.smithi070.stdout:5/347: write dc/f3d [592616,28263] 0 2024-03-23T08:23:52.992 INFO:tasks.workunit.client.0.smithi070.stdout:2/404: rename d1/d6/l2e to d1/d7/d48/d69/d7a/l8a 0 2024-03-23T08:23:52.995 INFO:tasks.workunit.client.0.smithi070.stdout:6/501: write d0/d23/d74/d84/d3c/d5b/f79 [595961,35583] 0 2024-03-23T08:23:53.001 INFO:tasks.workunit.client.0.smithi070.stdout:1/362: link d8/db/f6a d8/d11/d61/d74/d5d/f87 0 2024-03-23T08:23:53.005 INFO:tasks.workunit.client.0.smithi070.stdout:0/684: dread d16/d22/f8d [0,4194304] 0 2024-03-23T08:23:53.006 INFO:tasks.workunit.client.0.smithi070.stdout:3/592: mknod db/d4e/dd6/dd7/dd9/cdd 0 2024-03-23T08:23:53.007 INFO:tasks.workunit.client.0.smithi070.stdout:8/418: mkdir d5/d8/d16/d8d/da3 0 2024-03-23T08:23:53.009 INFO:tasks.workunit.client.0.smithi070.stdout:9/489: creat db/d12/d22/d25/d26/d5c/d7e/d6c/d79/f9d x:0 0 0 2024-03-23T08:23:53.010 INFO:tasks.workunit.client.0.smithi070.stdout:6/502: unlink d0/d3/fa 0 2024-03-23T08:23:53.010 INFO:tasks.workunit.client.0.smithi070.stdout:9/490: read db/d12/d14/f4e [2551818,71718] 0 2024-03-23T08:23:53.011 INFO:tasks.workunit.client.0.smithi070.stdout:6/503: chown d0/d23/d74 13707098 1 2024-03-23T08:23:53.013 INFO:tasks.workunit.client.0.smithi070.stdout:1/363: chown d8/d42/d4e/f59 37 1 2024-03-23T08:23:53.013 INFO:tasks.workunit.client.0.smithi070.stdout:2/405: mkdir d1/d8b 0 2024-03-23T08:23:53.016 INFO:tasks.workunit.client.0.smithi070.stdout:7/346: rename d25/d27/l39 to d25/d27/d6a/d44/l89 0 2024-03-23T08:23:53.020 INFO:tasks.workunit.client.0.smithi070.stdout:1/364: truncate d8/d11/d61/d74/f71 425744 0 2024-03-23T08:23:53.020 INFO:tasks.workunit.client.0.smithi070.stdout:3/593: dread db/df/f8d [0,4194304] 0 2024-03-23T08:23:53.020 INFO:tasks.workunit.client.0.smithi070.stdout:6/504: dwrite d0/f8c [0,4194304] 0 2024-03-23T08:23:53.020 INFO:tasks.workunit.client.0.smithi070.stdout:8/419: unlink d5/d8/fb 0 2024-03-23T08:23:53.022 INFO:tasks.workunit.client.0.smithi070.stdout:3/594: truncate db/d7f/fbe 79552 0 2024-03-23T08:23:53.023 INFO:tasks.workunit.client.0.smithi070.stdout:6/505: truncate d0/d23/d28/f7d 324666 0 2024-03-23T08:23:53.024 INFO:tasks.workunit.client.0.smithi070.stdout:3/595: write db/d7f/fbe [348522,114711] 0 2024-03-23T08:23:53.028 INFO:tasks.workunit.client.0.smithi070.stdout:2/406: symlink d1/d6/d34/l8c 0 2024-03-23T08:23:53.033 INFO:tasks.workunit.client.0.smithi070.stdout:1/365: symlink d8/db/d40/d43/l88 0 2024-03-23T08:23:53.036 INFO:tasks.workunit.client.0.smithi070.stdout:8/420: creat d5/d8/d16/d30/d38/fa4 x:0 0 0 2024-03-23T08:23:53.036 INFO:tasks.workunit.client.0.smithi070.stdout:7/347: dwrite d25/f4f [0,4194304] 0 2024-03-23T08:23:53.045 INFO:tasks.workunit.client.0.smithi070.stdout:6/506: dread d0/d23/d1c/d2f/dab/fac [0,4194304] 0 2024-03-23T08:23:53.045 INFO:tasks.workunit.client.0.smithi070.stdout:7/348: dwrite d25/d27/f60 [0,4194304] 0 2024-03-23T08:23:53.050 INFO:tasks.workunit.client.0.smithi070.stdout:9/491: rename db/d12/d22/d85/d42/l6f to db/d12/d22/d25/l9e 0 2024-03-23T08:23:53.051 INFO:tasks.workunit.client.0.smithi070.stdout:6/507: write d0/f8c [146440,84263] 0 2024-03-23T08:23:53.054 INFO:tasks.workunit.client.0.smithi070.stdout:7/349: symlink d25/d27/d6a/d44/d6f/l8a 0 2024-03-23T08:23:53.055 INFO:tasks.workunit.client.0.smithi070.stdout:9/492: dread - db/d12/d22/d25/d26/d5c/d7e/d6c/f99 zero size 2024-03-23T08:23:53.055 INFO:tasks.workunit.client.0.smithi070.stdout:6/508: write d0/d23/d28/d35/f46 [3853062,114406] 0 2024-03-23T08:23:53.056 INFO:tasks.workunit.client.0.smithi070.stdout:1/366: fdatasync d8/f9 0 2024-03-23T08:23:53.058 INFO:tasks.workunit.client.0.smithi070.stdout:2/407: rmdir d1/d7/d39/d47/d7b/d86 0 2024-03-23T08:23:53.059 INFO:tasks.workunit.client.0.smithi070.stdout:1/367: dread - d8/db/d40/d43/f7a zero size 2024-03-23T08:23:53.063 INFO:tasks.workunit.client.0.smithi070.stdout:0/685: rename d16/d22/d43/d47/l65 to d16/d26/d27/d86/d67/d7d/lf2 0 2024-03-23T08:23:53.069 INFO:tasks.workunit.client.0.smithi070.stdout:6/509: symlink d0/d23/d74/d84/laf 0 2024-03-23T08:23:53.071 INFO:tasks.workunit.client.0.smithi070.stdout:1/368: dread - d8/d11/f5c zero size 2024-03-23T08:23:53.072 INFO:tasks.workunit.client.0.smithi070.stdout:8/421: dread d5/d8/d16/d30/f2f [0,4194304] 0 2024-03-23T08:23:53.075 INFO:tasks.workunit.client.0.smithi070.stdout:7/350: rename d25/d27/d6a/d44/f53 to d25/d27/d6a/d44/d7c/f8b 0 2024-03-23T08:23:53.078 INFO:tasks.workunit.client.0.smithi070.stdout:9/493: dread db/f1f [0,4194304] 0 2024-03-23T08:23:53.078 INFO:tasks.workunit.client.0.smithi070.stdout:7/351: truncate d25/d40/f76 439875 0 2024-03-23T08:23:53.079 INFO:tasks.workunit.client.0.smithi070.stdout:6/510: mknod d0/d23/d74/d84/d3c/d5b/cb0 0 2024-03-23T08:23:53.080 INFO:tasks.workunit.client.0.smithi070.stdout:7/352: chown d25/d27/d6a 13 1 2024-03-23T08:23:53.080 INFO:tasks.workunit.client.0.smithi070.stdout:2/408: getdents d1/d8b 0 2024-03-23T08:23:53.080 INFO:tasks.workunit.client.0.smithi070.stdout:6/511: chown d0/d3/d2c/f34 779771980 1 2024-03-23T08:23:53.081 INFO:tasks.workunit.client.0.smithi070.stdout:8/422: readlink d5/l3d 0 2024-03-23T08:23:53.083 INFO:tasks.workunit.client.0.smithi070.stdout:3/596: truncate db/d20/d63/f75 615848 0 2024-03-23T08:23:53.085 INFO:tasks.workunit.client.0.smithi070.stdout:1/369: creat d8/d1b/f89 x:0 0 0 2024-03-23T08:23:53.089 INFO:tasks.workunit.client.0.smithi070.stdout:0/686: rename d16/d22/d43/fa1 to d16/d26/d27/d86/d9e/ddd/ff3 0 2024-03-23T08:23:53.091 INFO:tasks.workunit.client.0.smithi070.stdout:0/687: chown d16/d26/d27/cb3 1945 1 2024-03-23T08:23:53.091 INFO:tasks.workunit.client.0.smithi070.stdout:2/409: unlink d1/d6/d4f/f37 0 2024-03-23T08:23:53.093 INFO:tasks.workunit.client.0.smithi070.stdout:1/370: dwrite d8/db/d40/f84 [0,4194304] 0 2024-03-23T08:23:53.093 INFO:tasks.workunit.client.0.smithi070.stdout:2/410: read - d1/d7/f61 zero size 2024-03-23T08:23:53.101 INFO:tasks.workunit.client.0.smithi070.stdout:3/597: dread db/d14/d65/f78 [0,4194304] 0 2024-03-23T08:23:53.105 INFO:tasks.workunit.client.0.smithi070.stdout:9/494: rename db/d12/d91/l94 to db/d12/d14/l9f 0 2024-03-23T08:23:53.110 INFO:tasks.workunit.client.0.smithi070.stdout:2/411: truncate d1/d6/d34/f3d 2376598 0 2024-03-23T08:23:53.110 INFO:tasks.workunit.client.0.smithi070.stdout:3/598: dwrite db/d7f/fbe [0,4194304] 0 2024-03-23T08:23:53.123 INFO:tasks.workunit.client.0.smithi070.stdout:1/371: rename d8/db/f62 to d8/d42/d51/d6f/f8a 0 2024-03-23T08:23:53.123 INFO:tasks.workunit.client.0.smithi070.stdout:9/495: symlink db/d12/d22/d25/d58/la0 0 2024-03-23T08:23:53.125 INFO:tasks.workunit.client.0.smithi070.stdout:9/496: fsync db/d12/d14/f95 0 2024-03-23T08:23:53.126 INFO:tasks.workunit.client.0.smithi070.stdout:8/423: truncate d5/d8/d16/d32/f58 786282 0 2024-03-23T08:23:53.126 INFO:tasks.workunit.client.0.smithi070.stdout:0/688: link d16/d22/l7a d16/lf4 0 2024-03-23T08:23:53.127 INFO:tasks.workunit.client.0.smithi070.stdout:0/689: write d16/d22/d43/d47/da9/f8c [654063,94366] 0 2024-03-23T08:23:53.129 INFO:tasks.workunit.client.0.smithi070.stdout:3/599: rename db/lc2 to db/d14/d9c/lde 0 2024-03-23T08:23:53.131 INFO:tasks.workunit.client.0.smithi070.stdout:3/600: truncate db/df/d1a/f3c 4634014 0 2024-03-23T08:23:53.133 INFO:tasks.workunit.client.0.smithi070.stdout:3/601: write db/d14/fd4 [438544,105854] 0 2024-03-23T08:23:53.138 INFO:tasks.workunit.client.0.smithi070.stdout:9/497: mkdir db/d12/da1 0 2024-03-23T08:23:53.140 INFO:tasks.workunit.client.0.smithi070.stdout:8/424: symlink d5/d8/la5 0 2024-03-23T08:23:53.141 INFO:tasks.workunit.client.0.smithi070.stdout:2/412: dread d1/d7/d11/f56 [0,4194304] 0 2024-03-23T08:23:53.146 INFO:tasks.workunit.client.0.smithi070.stdout:0/690: mkdir d16/d22/df0/df5 0 2024-03-23T08:23:53.146 INFO:tasks.workunit.client.0.smithi070.stdout:0/691: write d16/d22/d43/f56 [7324202,90481] 0 2024-03-23T08:23:53.147 INFO:tasks.workunit.client.0.smithi070.stdout:9/498: rename db/d12/d22/d25 to db/d12/d34/d8c/da2 0 2024-03-23T08:23:53.152 INFO:tasks.workunit.client.0.smithi070.stdout:1/372: getdents d8/d26/d7b 0 2024-03-23T08:23:53.157 INFO:tasks.workunit.client.0.smithi070.stdout:3/602: write db/df/d1a/f50 [1162406,20128] 0 2024-03-23T08:23:53.159 INFO:tasks.workunit.client.0.smithi070.stdout:3/603: write db/d14/d65/f85 [844707,17487] 0 2024-03-23T08:23:53.160 INFO:tasks.workunit.client.0.smithi070.stdout:9/499: creat db/d12/d34/d47/fa3 x:0 0 0 2024-03-23T08:23:53.161 INFO:tasks.workunit.client.0.smithi070.stdout:8/425: chown d5/d8/c71 31301 1 2024-03-23T08:23:53.162 INFO:tasks.workunit.client.0.smithi070.stdout:8/426: read - d5/d8/d16/d3b/d8c/f99 zero size 2024-03-23T08:23:53.163 INFO:tasks.workunit.client.0.smithi070.stdout:1/373: fsync d8/f9 0 2024-03-23T08:23:53.165 INFO:tasks.workunit.client.0.smithi070.stdout:8/427: chown d5/d8/d16/d3b/d4d/d6f/f89 1 1 2024-03-23T08:23:53.166 INFO:tasks.workunit.client.0.smithi070.stdout:3/604: mknod db/df/d15/d3d/cdf 0 2024-03-23T08:23:53.166 INFO:tasks.workunit.client.0.smithi070.stdout:8/428: write d5/d8/d16/d3b/d8c/f99 [374520,9574] 0 2024-03-23T08:23:53.172 INFO:tasks.workunit.client.0.smithi070.stdout:0/692: rename d16/d26/f6b to d16/d26/d27/d3d/ff6 0 2024-03-23T08:23:53.175 INFO:tasks.workunit.client.0.smithi070.stdout:1/374: fsync d8/db/d2c/d3f/f56 0 2024-03-23T08:23:53.180 INFO:tasks.workunit.client.0.smithi070.stdout:2/413: getdents d1/d6/d4f 0 2024-03-23T08:23:53.185 INFO:tasks.workunit.client.0.smithi070.stdout:0/693: creat d16/d26/d27/d3d/d4d/dad/ff7 x:0 0 0 2024-03-23T08:23:53.187 INFO:tasks.workunit.client.0.smithi070.stdout:0/694: dread d16/d22/f24 [0,4194304] 0 2024-03-23T08:23:53.192 INFO:tasks.workunit.client.0.smithi070.stdout:1/375: creat d8/db/d40/d43/d64/f8b x:0 0 0 2024-03-23T08:23:53.193 INFO:tasks.workunit.client.0.smithi070.stdout:1/376: readlink d8/d11/d61/d74/d5d/l72 0 2024-03-23T08:23:53.194 INFO:tasks.workunit.client.0.smithi070.stdout:2/414: mkdir d1/d7/d48/d69/d7a/d8d 0 2024-03-23T08:23:53.197 INFO:tasks.workunit.client.0.smithi070.stdout:8/429: write d5/f23 [725075,48667] 0 2024-03-23T08:23:53.198 INFO:tasks.workunit.client.0.smithi070.stdout:9/500: link db/d12/d14/f4e db/d12/d22/d85/d42/fa4 0 2024-03-23T08:23:53.199 INFO:tasks.workunit.client.0.smithi070.stdout:8/430: fsync d5/d8/d16/d30/d38/d6b/d72/d88/f8a 0 2024-03-23T08:23:53.200 INFO:tasks.workunit.client.0.smithi070.stdout:0/695: mkdir d16/d26/d27/d3d/d4d/d9a/df8 0 2024-03-23T08:23:53.202 INFO:tasks.workunit.client.0.smithi070.stdout:0/696: write d16/d26/d27/d86/d67/d7d/fc2 [435828,33075] 0 2024-03-23T08:23:53.205 INFO:tasks.workunit.client.0.smithi070.stdout:3/605: link db/df/d4d/lb5 db/df/d15/d3d/dac/le0 0 2024-03-23T08:23:53.205 INFO:tasks.workunit.client.0.smithi070.stdout:3/606: dread - db/df/d5d/fda zero size 2024-03-23T08:23:53.206 INFO:tasks.workunit.client.0.smithi070.stdout:3/607: dread - db/df/d4d/fbf zero size 2024-03-23T08:23:53.208 INFO:tasks.workunit.client.0.smithi070.stdout:9/501: rename db/f60 to db/d12/d34/d8c/da2/d4f/d5a/fa5 0 2024-03-23T08:23:53.210 INFO:tasks.workunit.client.0.smithi070.stdout:1/377: write d8/db/f6a [1668843,130857] 0 2024-03-23T08:23:53.211 INFO:tasks.workunit.client.0.smithi070.stdout:0/697: creat d16/d22/d43/ff9 x:0 0 0 2024-03-23T08:23:53.213 INFO:tasks.workunit.client.0.smithi070.stdout:0/698: truncate d16/d26/d27/d4c/d60/d75/f9c 808381 0 2024-03-23T08:23:53.214 INFO:tasks.workunit.client.0.smithi070.stdout:0/699: stat d16/d26/d27/d4c/d60/f79 0 2024-03-23T08:23:53.215 INFO:tasks.workunit.client.0.smithi070.stdout:8/431: rename d5/d8/d16/d30/d38/d6b/f6d to d5/d8/d16/d3b/d4d/fa6 0 2024-03-23T08:23:53.216 INFO:tasks.workunit.client.0.smithi070.stdout:3/608: dread db/df/d15/d3d/f56 [0,4194304] 0 2024-03-23T08:23:53.220 INFO:tasks.workunit.client.0.smithi070.stdout:2/415: rmdir d1/d7/d48/d69/d7a/d8d 0 2024-03-23T08:23:53.223 INFO:tasks.workunit.client.0.smithi070.stdout:9/502: dwrite db/d12/f39 [0,4194304] 0 2024-03-23T08:23:53.224 INFO:tasks.workunit.client.0.smithi070.stdout:0/700: mkdir d16/d22/d43/d47/da9/de7/dfa 0 2024-03-23T08:23:53.225 INFO:tasks.workunit.client.0.smithi070.stdout:0/701: chown d16/d22/d43/d47/da9 841859 1 2024-03-23T08:23:53.225 INFO:tasks.workunit.client.0.smithi070.stdout:9/503: fdatasync db/d12/d34/d8c/da2/d26/d5c/d7e/d6c/f99 0 2024-03-23T08:23:53.226 INFO:tasks.workunit.client.0.smithi070.stdout:1/378: write d8/d1b/f4b [223210,28633] 0 2024-03-23T08:23:53.228 INFO:tasks.workunit.client.0.smithi070.stdout:2/416: dread d1/d7/f17 [0,4194304] 0 2024-03-23T08:23:53.231 INFO:tasks.workunit.client.0.smithi070.stdout:1/379: dread - d8/d1b/f86 zero size 2024-03-23T08:23:53.234 INFO:tasks.workunit.client.0.smithi070.stdout:1/380: read - d8/db/d2c/d3f/d48/f81 zero size 2024-03-23T08:23:53.235 INFO:tasks.workunit.client.0.smithi070.stdout:8/432: dwrite d5/d8/f79 [0,4194304] 0 2024-03-23T08:23:53.238 INFO:tasks.workunit.client.0.smithi070.stdout:3/609: rename db/d14/d65/daa/dc9 to db/d20/d49/de1 0 2024-03-23T08:23:53.297 INFO:tasks.workunit.client.0.smithi070.stdout:3/610: chown db/d14/fd4 10208 1 2024-03-23T08:23:53.298 INFO:tasks.workunit.client.0.smithi070.stdout:1/381: write d8/d11/d61/f75 [623106,69712] 0 2024-03-23T08:23:53.298 INFO:tasks.workunit.client.0.smithi070.stdout:2/417: rename d1/f2f to d1/d7/d11/d38/d4c/f8e 0 2024-03-23T08:23:53.300 INFO:tasks.workunit.client.0.smithi070.stdout:9/504: creat db/d12/fa6 x:0 0 0 2024-03-23T08:23:53.301 INFO:tasks.workunit.client.0.smithi070.stdout:8/433: dwrite d5/d8/d16/d32/f58 [0,4194304] 0 2024-03-23T08:23:53.308 INFO:tasks.workunit.client.0.smithi070.stdout:9/505: mkdir db/d12/d34/d8c/da2/d26/da7 0 2024-03-23T08:23:53.310 INFO:tasks.workunit.client.0.smithi070.stdout:2/418: dwrite d1/d7/df/f4b [0,4194304] 0 2024-03-23T08:23:53.315 INFO:tasks.workunit.client.0.smithi070.stdout:8/434: dwrite d5/d8/d16/d3b/d4d/d6f/f89 [0,4194304] 0 2024-03-23T08:23:53.319 INFO:tasks.workunit.client.0.smithi070.stdout:8/435: mkdir d5/d8/d16/d32/d84/da7 0 2024-03-23T08:23:53.321 INFO:tasks.workunit.client.0.smithi070.stdout:8/436: mknod d5/d8/d16/d30/d38/ca8 0 2024-03-23T08:23:53.325 INFO:tasks.workunit.client.0.smithi070.stdout:2/419: dwrite d1/d29/f35 [0,4194304] 0 2024-03-23T08:23:53.329 INFO:tasks.workunit.client.0.smithi070.stdout:2/420: mkdir d1/d7/d39/d58/d8f 0 2024-03-23T08:23:53.331 INFO:tasks.workunit.client.0.smithi070.stdout:8/437: dwrite d5/d8/d16/d30/f61 [0,4194304] 0 2024-03-23T08:23:53.333 INFO:tasks.workunit.client.0.smithi070.stdout:8/438: read d5/d8/d16/d30/f2f [2306225,128195] 0 2024-03-23T08:23:53.337 INFO:tasks.workunit.client.0.smithi070.stdout:2/421: truncate d1/d6/d4f/f3c 1072429 0 2024-03-23T08:23:53.337 INFO:tasks.workunit.client.0.smithi070.stdout:8/439: mkdir d5/d8/d16/d30/d38/d6b/da9 0 2024-03-23T08:23:53.341 INFO:tasks.workunit.client.0.smithi070.stdout:2/422: unlink d1/d6/d34/l59 0 2024-03-23T08:23:53.342 INFO:tasks.workunit.client.0.smithi070.stdout:2/423: chown d1/d7/df/f4b 338814 1 2024-03-23T08:23:53.342 INFO:tasks.workunit.client.0.smithi070.stdout:2/424: stat d1/d21/d5f 0 2024-03-23T08:23:53.345 INFO:tasks.workunit.client.0.smithi070.stdout:2/425: mkdir d1/d7/d39/d47/d90 0 2024-03-23T08:23:53.354 INFO:tasks.workunit.client.0.smithi070.stdout:8/440: write d5/d8/f4e [960445,92388] 0 2024-03-23T08:23:53.356 INFO:tasks.workunit.client.0.smithi070.stdout:8/441: truncate d5/d8/d16/d3b/f5d 335401 0 2024-03-23T08:23:53.356 INFO:tasks.workunit.client.0.smithi070.stdout:8/442: chown c0 366254121 1 2024-03-23T08:23:53.357 INFO:tasks.workunit.client.0.smithi070.stdout:8/443: chown d5/d8/d16/d3b/f5d 115 1 2024-03-23T08:23:53.358 INFO:tasks.workunit.client.0.smithi070.stdout:9/506: dread db/d12/f82 [0,4194304] 0 2024-03-23T08:23:53.368 INFO:tasks.workunit.client.0.smithi070.stdout:9/507: dwrite db/d12/d34/f84 [0,4194304] 0 2024-03-23T08:23:53.368 INFO:tasks.workunit.client.0.smithi070.stdout:2/426: dwrite d1/fd [0,4194304] 0 2024-03-23T08:23:53.370 INFO:tasks.workunit.client.0.smithi070.stdout:2/427: fdatasync d1/d29/f7f 0 2024-03-23T08:23:53.374 INFO:tasks.workunit.client.0.smithi070.stdout:9/508: unlink db/d12/d22/d85/l4a 0 2024-03-23T08:23:53.376 INFO:tasks.workunit.client.0.smithi070.stdout:8/444: dwrite d5/d8/f18 [0,4194304] 0 2024-03-23T08:23:53.377 INFO:tasks.workunit.client.0.smithi070.stdout:9/509: write db/d12/d14/f49 [1451941,17877] 0 2024-03-23T08:23:53.385 INFO:tasks.workunit.client.0.smithi070.stdout:8/445: creat d5/d8/d16/d3b/faa x:0 0 0 2024-03-23T08:23:53.388 INFO:tasks.workunit.client.0.smithi070.stdout:9/510: symlink db/d12/d34/d8c/da2/d26/d5c/la8 0 2024-03-23T08:23:53.391 INFO:tasks.workunit.client.0.smithi070.stdout:8/446: mkdir d5/d8/d16/d8d/da3/dab 0 2024-03-23T08:23:53.391 INFO:tasks.workunit.client.0.smithi070.stdout:9/511: chown db/d12/f9a 69 1 2024-03-23T08:23:53.391 INFO:tasks.workunit.client.0.smithi070.stdout:9/512: stat db/d12/d34/d8c/da2/d26/d80 0 2024-03-23T08:23:53.392 INFO:tasks.workunit.client.0.smithi070.stdout:2/428: truncate d1/d7/d11/d38/d4c/f6a 851851 0 2024-03-23T08:23:53.393 INFO:tasks.workunit.client.0.smithi070.stdout:9/513: creat db/d12/d34/d8c/da2/d4f/fa9 x:0 0 0 2024-03-23T08:23:53.394 INFO:tasks.workunit.client.0.smithi070.stdout:9/514: write db/d12/d34/f92 [628958,48513] 0 2024-03-23T08:23:53.395 INFO:tasks.workunit.client.0.smithi070.stdout:8/447: rename c0 to d5/d8/d16/cac 0 2024-03-23T08:23:53.397 INFO:tasks.workunit.client.0.smithi070.stdout:2/429: fdatasync d1/d7/d11/f2b 0 2024-03-23T08:23:53.399 INFO:tasks.workunit.client.0.smithi070.stdout:2/430: write d1/d7/f7e [1045641,124299] 0 2024-03-23T08:23:53.401 INFO:tasks.workunit.client.0.smithi070.stdout:8/448: creat d5/d8/d16/d8d/da3/dab/fad x:0 0 0 2024-03-23T08:23:53.407 INFO:tasks.workunit.client.0.smithi070.stdout:8/449: chown d5/d8/f25 14086205 1 2024-03-23T08:23:53.408 INFO:tasks.workunit.client.0.smithi070.stdout:2/431: dwrite d1/d6/f4d [0,4194304] 0 2024-03-23T08:23:53.409 INFO:tasks.workunit.client.0.smithi070.stdout:2/432: chown d1/d21/l5d 66 1 2024-03-23T08:23:53.409 INFO:tasks.workunit.client.0.smithi070.stdout:9/515: write db/d12/f3a [3939039,119072] 0 2024-03-23T08:23:53.410 INFO:tasks.workunit.client.0.smithi070.stdout:2/433: stat d1/d6/f72 0 2024-03-23T08:23:53.412 INFO:tasks.workunit.client.0.smithi070.stdout:9/516: stat db/d12/d34/d8c/da2/d26/f30 0 2024-03-23T08:23:53.415 INFO:tasks.workunit.client.0.smithi070.stdout:8/450: rename d5/d8/d16/d30/d7b to d5/d8/d9a/da2/dae 0 2024-03-23T08:23:53.416 INFO:tasks.workunit.client.0.smithi070.stdout:2/434: write d1/d6/f3b [977901,122558] 0 2024-03-23T08:23:53.416 INFO:tasks.workunit.client.0.smithi070.stdout:8/451: stat d5/d8/d9a/da2/dae/c8e 0 2024-03-23T08:23:53.418 INFO:tasks.workunit.client.0.smithi070.stdout:2/435: write d1/d6/d30/f3a [1300043,38303] 0 2024-03-23T08:23:53.419 INFO:tasks.workunit.client.0.smithi070.stdout:8/452: read - d5/d8/f9d zero size 2024-03-23T08:23:53.420 INFO:tasks.workunit.client.0.smithi070.stdout:2/436: stat d1/d29/f7f 0 2024-03-23T08:23:53.420 INFO:tasks.workunit.client.0.smithi070.stdout:2/437: chown d1/d7/d48/d5b 28 1 2024-03-23T08:23:53.422 INFO:tasks.workunit.client.0.smithi070.stdout:8/453: creat d5/d8/faf x:0 0 0 2024-03-23T08:23:53.423 INFO:tasks.workunit.client.0.smithi070.stdout:2/438: creat d1/d7/df/f91 x:0 0 0 2024-03-23T08:23:53.427 INFO:tasks.workunit.client.0.smithi070.stdout:8/454: unlink d5/d8/f33 0 2024-03-23T08:23:53.428 INFO:tasks.workunit.client.0.smithi070.stdout:2/439: unlink d1/d7/d11/d38/d4c/f8e 0 2024-03-23T08:23:53.431 INFO:tasks.workunit.client.0.smithi070.stdout:2/440: creat d1/d29/d52/f92 x:0 0 0 2024-03-23T08:23:53.433 INFO:tasks.workunit.client.0.smithi070.stdout:2/441: stat d1/d7/df/f91 0 2024-03-23T08:23:53.434 INFO:tasks.workunit.client.0.smithi070.stdout:8/455: dwrite d5/d8/d16/d3b/f65 [0,4194304] 0 2024-03-23T08:23:53.438 INFO:tasks.workunit.client.0.smithi070.stdout:2/442: rename d1/d29/d52/l79 to d1/d8b/l93 0 2024-03-23T08:23:53.439 INFO:tasks.workunit.client.0.smithi070.stdout:2/443: chown d1/d7/d39/d47/d90 77232 1 2024-03-23T08:23:53.442 INFO:tasks.workunit.client.0.smithi070.stdout:8/456: read - d5/d8/f3a zero size 2024-03-23T08:23:53.445 INFO:tasks.workunit.client.0.smithi070.stdout:2/444: rename d1/d7/d11 to d1/d7/df/d94 0 2024-03-23T08:23:53.447 INFO:tasks.workunit.client.0.smithi070.stdout:8/457: mkdir d5/d8/d16/d30/d38/d6b/db0 0 2024-03-23T08:23:53.448 INFO:tasks.workunit.client.0.smithi070.stdout:2/445: creat d1/d7/d39/f95 x:0 0 0 2024-03-23T08:23:53.452 INFO:tasks.workunit.client.0.smithi070.stdout:2/446: truncate d1/d7/f1e 281153 0 2024-03-23T08:23:53.456 INFO:tasks.workunit.client.0.smithi070.stdout:2/447: unlink d1/d7/df/d94/d38/d4c/f4e 0 2024-03-23T08:23:53.456 INFO:tasks.workunit.client.0.smithi070.stdout:8/458: dwrite d5/d8/d16/d32/f5c [4194304,4194304] 0 2024-03-23T08:23:53.459 INFO:tasks.workunit.client.0.smithi070.stdout:2/448: creat d1/d7/d48/d5b/f96 x:0 0 0 2024-03-23T08:23:53.465 INFO:tasks.workunit.client.0.smithi070.stdout:8/459: dwrite d5/d8/d16/d32/f74 [0,4194304] 0 2024-03-23T08:23:53.472 INFO:tasks.workunit.client.0.smithi070.stdout:2/449: getdents d1/d29 0 2024-03-23T08:23:53.480 INFO:tasks.workunit.client.0.smithi070.stdout:2/450: dwrite d1/d6/f65 [0,4194304] 0 2024-03-23T08:23:53.481 INFO:tasks.workunit.client.0.smithi070.stdout:8/460: dread d5/d8/d16/d30/f39 [0,4194304] 0 2024-03-23T08:23:53.486 INFO:tasks.workunit.client.0.smithi070.stdout:2/451: mknod d1/c97 0 2024-03-23T08:23:53.487 INFO:tasks.workunit.client.0.smithi070.stdout:8/461: write d5/d8/d16/d30/f61 [1753358,57383] 0 2024-03-23T08:23:53.489 INFO:tasks.workunit.client.0.smithi070.stdout:2/452: mkdir d1/d98 0 2024-03-23T08:23:53.492 INFO:tasks.workunit.client.0.smithi070.stdout:2/453: write d1/d7/df/d94/f56 [4589288,75942] 0 2024-03-23T08:23:53.496 INFO:tasks.workunit.client.0.smithi070.stdout:8/462: dwrite d5/d8/d16/d30/f68 [0,4194304] 0 2024-03-23T08:23:53.499 INFO:tasks.workunit.client.0.smithi070.stdout:8/463: symlink d5/d8/d16/d30/d38/d6b/lb1 0 2024-03-23T08:23:53.500 INFO:tasks.workunit.client.0.smithi070.stdout:2/454: dread d1/d6/d30/f3a [0,4194304] 0 2024-03-23T08:23:53.501 INFO:tasks.workunit.client.0.smithi070.stdout:8/464: readlink d5/d8/d16/d30/d38/l42 0 2024-03-23T08:23:53.505 INFO:tasks.workunit.client.0.smithi070.stdout:2/455: mkdir d1/d7/df/d99 0 2024-03-23T08:23:53.506 INFO:tasks.workunit.client.0.smithi070.stdout:8/465: link d5/fe d5/d8/d16/d8d/da3/fb2 0 2024-03-23T08:23:53.509 INFO:tasks.workunit.client.0.smithi070.stdout:8/466: rmdir d5/d8/d16/d8d/da3/dab 39 2024-03-23T08:23:53.511 INFO:tasks.workunit.client.0.smithi070.stdout:8/467: write d5/d8/d16/d3b/d4d/d6f/f89 [995077,81537] 0 2024-03-23T08:23:53.518 INFO:tasks.workunit.client.0.smithi070.stdout:2/456: dread d1/d7/df/d94/d38/d4c/f5c [0,4194304] 0 2024-03-23T08:23:53.520 INFO:tasks.workunit.client.0.smithi070.stdout:8/468: dwrite d5/d8/d16/d3b/f76 [0,4194304] 0 2024-03-23T08:23:53.525 INFO:tasks.workunit.client.0.smithi070.stdout:8/469: rename d5/d8/d16/d30/d38/l6a to d5/d8/d16/d3b/d8c/lb3 0 2024-03-23T08:23:53.525 INFO:tasks.workunit.client.0.smithi070.stdout:2/457: dwrite d1/d6/d4f/f3e [0,4194304] 0 2024-03-23T08:23:53.527 INFO:tasks.workunit.client.0.smithi070.stdout:8/470: symlink d5/d8/d16/d8d/lb4 0 2024-03-23T08:23:53.541 INFO:tasks.workunit.client.0.smithi070.stdout:2/458: dwrite d1/d7/d39/d58/f75 [0,4194304] 0 2024-03-23T08:23:53.547 INFO:tasks.workunit.client.0.smithi070.stdout:4/263: sync 2024-03-23T08:23:53.547 INFO:tasks.workunit.client.0.smithi070.stdout:5/348: sync 2024-03-23T08:23:53.549 INFO:tasks.workunit.client.0.smithi070.stdout:4/264: write d3/d6/d11/d1a/d41/d56/f5a [123675,48008] 0 2024-03-23T08:23:53.550 INFO:tasks.workunit.client.0.smithi070.stdout:2/459: write d1/d7/d48/d69/f77 [124267,118095] 0 2024-03-23T08:23:53.617 INFO:tasks.workunit.client.0.smithi070.stdout:2/460: link d1/d7/df/d94/f84 d1/d7/d39/d47/d7b/f9a 0 2024-03-23T08:23:53.619 INFO:tasks.workunit.client.0.smithi070.stdout:5/349: rename dc/d1b/d1d/d25/d38/d48/c4c to dc/d1b/d1d/d25/d38/c70 0 2024-03-23T08:23:53.622 INFO:tasks.workunit.client.0.smithi070.stdout:2/461: creat d1/d7/d39/d47/f9b x:0 0 0 2024-03-23T08:23:53.623 INFO:tasks.workunit.client.0.smithi070.stdout:4/265: rename d3/d6/c48 to d3/d6/d35/d3f/c5f 0 2024-03-23T08:23:53.625 INFO:tasks.workunit.client.0.smithi070.stdout:5/350: creat dc/d1b/d1d/d25/d38/d48/f71 x:0 0 0 2024-03-23T08:23:53.626 INFO:tasks.workunit.client.0.smithi070.stdout:4/266: mknod d3/d6/d11/c60 0 2024-03-23T08:23:53.626 INFO:tasks.workunit.client.0.smithi070.stdout:5/351: stat dc/d1b/f54 0 2024-03-23T08:23:53.629 INFO:tasks.workunit.client.0.smithi070.stdout:5/352: creat dc/d1b/d1d/d25/d38/d48/f72 x:0 0 0 2024-03-23T08:23:53.635 INFO:tasks.workunit.client.0.smithi070.stdout:5/353: creat dc/d1b/d1d/d25/d38/f73 x:0 0 0 2024-03-23T08:23:53.640 INFO:tasks.workunit.client.0.smithi070.stdout:5/354: dwrite f1 [0,4194304] 0 2024-03-23T08:23:53.649 INFO:tasks.workunit.client.0.smithi070.stdout:5/355: dwrite dc/d1b/d1d/d25/d36/f27 [4194304,4194304] 0 2024-03-23T08:23:53.653 INFO:tasks.workunit.client.0.smithi070.stdout:5/356: rename dc/f1f to dc/f74 0 2024-03-23T08:23:53.654 INFO:tasks.workunit.client.0.smithi070.stdout:5/357: readlink dc/d1b/d1d/d25/l47 0 2024-03-23T08:23:53.658 INFO:tasks.workunit.client.0.smithi070.stdout:5/358: rename l5 to dc/d1b/d1d/d25/d36/l75 0 2024-03-23T08:23:53.665 INFO:tasks.workunit.client.0.smithi070.stdout:5/359: link dc/d1b/d1d/d25/d38/f40 dc/d1b/d1d/d25/d38/d53/f76 0 2024-03-23T08:23:53.665 INFO:tasks.workunit.client.0.smithi070.stdout:4/267: read d3/d6/d11/d1a/f40 [949979,44620] 0 2024-03-23T08:23:53.666 INFO:tasks.workunit.client.0.smithi070.stdout:5/360: dread - dc/d1b/d1d/d25/d38/f69 zero size 2024-03-23T08:23:53.695 INFO:tasks.workunit.client.0.smithi070.stdout:5/361: dread dc/f74 [0,4194304] 0 2024-03-23T08:23:53.697 INFO:tasks.workunit.client.0.smithi070.stdout:5/362: write dc/d1b/d1d/d25/d38/d39/f66 [2817223,75285] 0 2024-03-23T08:23:53.705 INFO:tasks.workunit.client.0.smithi070.stdout:5/363: dwrite dc/d1b/d1d/d25/d38/d48/f71 [0,4194304] 0 2024-03-23T08:23:53.709 INFO:tasks.workunit.client.0.smithi070.stdout:5/364: mknod dc/d1b/d1d/d25/d36/c77 0 2024-03-23T08:23:53.710 INFO:tasks.workunit.client.0.smithi070.stdout:5/365: dread - dc/d1b/d1d/d25/d36/d32/d5c/f65 zero size 2024-03-23T08:23:53.711 INFO:tasks.workunit.client.0.smithi070.stdout:5/366: dread - dc/d1b/d1d/d25/d36/d32/d5c/f5d zero size 2024-03-23T08:23:53.713 INFO:tasks.workunit.client.0.smithi070.stdout:5/367: read dc/d1b/d1d/f3a [2562791,44681] 0 2024-03-23T08:23:53.718 INFO:tasks.workunit.client.0.smithi070.stdout:5/368: mknod dc/d1b/d1d/d25/d36/c78 0 2024-03-23T08:23:53.725 INFO:tasks.workunit.client.0.smithi070.stdout:4/268: dwrite d3/d6/d9/f22 [0,4194304] 0 2024-03-23T08:23:53.726 INFO:tasks.workunit.client.0.smithi070.stdout:4/269: fdatasync d3/d19/d2a/f2b 0 2024-03-23T08:23:53.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:53 smithi070 ceph-mon[39699]: pgmap v36: 65 pgs: 65 active+clean; 1.1 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 14 MiB/s rd, 99 MiB/s wr, 277 op/s 2024-03-23T08:23:53.733 INFO:tasks.workunit.client.0.smithi070.stdout:4/270: rename d3/d6/d11/d1a/d27/d4b/c53 to d3/d6/d45/c61 0 2024-03-23T08:23:53.737 INFO:tasks.workunit.client.0.smithi070.stdout:4/271: creat d3/f62 x:0 0 0 2024-03-23T08:23:53.740 INFO:tasks.workunit.client.0.smithi070.stdout:4/272: creat d3/d6/d35/d3f/d51/f63 x:0 0 0 2024-03-23T08:23:53.741 INFO:tasks.workunit.client.0.smithi070.stdout:4/273: fsync d3/d6/d11/d1a/f3e 0 2024-03-23T08:23:53.742 INFO:tasks.workunit.client.0.smithi070.stdout:4/274: write d3/d6/d11/d1a/d1b/f1c [681710,11896] 0 2024-03-23T08:23:53.749 INFO:tasks.workunit.client.0.smithi070.stdout:7/353: sync 2024-03-23T08:23:53.750 INFO:tasks.workunit.client.0.smithi070.stdout:4/275: write d3/d6/d11/f2d [733561,80641] 0 2024-03-23T08:23:53.781 INFO:tasks.workunit.client.0.smithi070.stdout:4/276: mknod d3/d6/d11/d1a/d41/d56/c64 0 2024-03-23T08:23:53.785 INFO:tasks.workunit.client.0.smithi070.stdout:7/354: link d25/d27/d6a/d44/f54 d25/d27/d6a/d44/f8c 0 2024-03-23T08:23:53.793 INFO:tasks.workunit.client.0.smithi070.stdout:7/355: unlink f1d 0 2024-03-23T08:23:53.799 INFO:tasks.workunit.client.0.smithi070.stdout:4/277: dwrite d3/d6/d35/d3f/d51/f63 [0,4194304] 0 2024-03-23T08:23:53.813 INFO:tasks.workunit.client.0.smithi070.stdout:7/356: getdents d25/d40 0 2024-03-23T08:23:53.816 INFO:tasks.workunit.client.0.smithi070.stdout:7/357: read f23 [305832,102677] 0 2024-03-23T08:23:53.818 INFO:tasks.workunit.client.0.smithi070.stdout:7/358: truncate d25/d27/d6a/d44/f56 886129 0 2024-03-23T08:23:53.819 INFO:tasks.workunit.client.0.smithi070.stdout:7/359: chown d25/d27/d6a/d44/f8c 21 1 2024-03-23T08:23:53.821 INFO:tasks.workunit.client.0.smithi070.stdout:7/360: fdatasync d25/d27/d6a/d44/f78 0 2024-03-23T08:23:53.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:53 smithi193 ceph-mon[48370]: pgmap v36: 65 pgs: 65 active+clean; 1.1 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 14 MiB/s rd, 99 MiB/s wr, 277 op/s 2024-03-23T08:23:53.827 INFO:tasks.workunit.client.0.smithi070.stdout:7/361: dwrite d25/d27/d6a/f73 [0,4194304] 0 2024-03-23T08:23:53.830 INFO:tasks.workunit.client.0.smithi070.stdout:7/362: creat d25/d27/d6a/d44/d7c/f8d x:0 0 0 2024-03-23T08:23:53.834 INFO:tasks.workunit.client.0.smithi070.stdout:7/363: symlink d25/d27/d6a/d44/d85/l8e 0 2024-03-23T08:23:53.835 INFO:tasks.workunit.client.0.smithi070.stdout:7/364: chown d25/d27 0 1 2024-03-23T08:23:53.837 INFO:tasks.workunit.client.0.smithi070.stdout:7/365: creat d25/d40/f8f x:0 0 0 2024-03-23T08:23:53.839 INFO:tasks.workunit.client.0.smithi070.stdout:7/366: write d25/d27/d6a/d44/f78 [1113301,80943] 0 2024-03-23T08:23:53.845 INFO:tasks.workunit.client.0.smithi070.stdout:5/369: write dc/d1b/d1d/d25/d38/d53/f76 [3239001,108292] 0 2024-03-23T08:23:53.846 INFO:tasks.workunit.client.0.smithi070.stdout:7/367: write d25/d27/f70 [1000002,88967] 0 2024-03-23T08:23:53.847 INFO:tasks.workunit.client.0.smithi070.stdout:7/368: fdatasync d25/d27/d6a/f5f 0 2024-03-23T08:23:53.849 INFO:tasks.workunit.client.0.smithi070.stdout:7/369: dread - d25/d27/d6a/f7f zero size 2024-03-23T08:23:53.849 INFO:tasks.workunit.client.0.smithi070.stdout:7/370: chown f16 6194 1 2024-03-23T08:23:53.869 INFO:tasks.workunit.client.0.smithi070.stdout:5/370: creat dc/d1b/d1d/d25/d38/d39/f79 x:0 0 0 2024-03-23T08:23:53.870 INFO:tasks.workunit.client.0.smithi070.stdout:7/371: creat d25/d27/d6a/d44/f90 x:0 0 0 2024-03-23T08:23:53.872 INFO:tasks.workunit.client.0.smithi070.stdout:5/371: truncate dc/d1b/d1d/d25/d38/d39/f79 851059 0 2024-03-23T08:23:53.877 INFO:tasks.workunit.client.0.smithi070.stdout:7/372: dwrite d25/d27/f70 [0,4194304] 0 2024-03-23T08:23:53.877 INFO:tasks.workunit.client.0.smithi070.stdout:7/373: readlink l1e 0 2024-03-23T08:23:53.878 INFO:tasks.workunit.client.0.smithi070.stdout:7/374: chown d25/d27/d6a/d44/d6f/f71 10294754 1 2024-03-23T08:23:53.881 INFO:tasks.workunit.client.0.smithi070.stdout:4/278: rename d3/d6/d11/d1a to d3/d6/d35/d65 0 2024-03-23T08:23:53.883 INFO:tasks.workunit.client.0.smithi070.stdout:7/375: unlink d25/d27/d6a/d44/d7c/f8d 0 2024-03-23T08:23:53.886 INFO:tasks.workunit.client.0.smithi070.stdout:4/279: creat d3/d6/d35/d65/d1b/f66 x:0 0 0 2024-03-23T08:23:53.888 INFO:tasks.workunit.client.0.smithi070.stdout:4/280: write d3/d6/d35/d65/d27/f4e [156590,8288] 0 2024-03-23T08:23:53.892 INFO:tasks.workunit.client.0.smithi070.stdout:7/376: dwrite f24 [0,4194304] 0 2024-03-23T08:23:53.894 INFO:tasks.workunit.client.0.smithi070.stdout:4/281: truncate d3/d6/d35/d65/f40 722714 0 2024-03-23T08:23:53.900 INFO:tasks.workunit.client.0.smithi070.stdout:7/377: dwrite d25/d27/d6a/f73 [0,4194304] 0 2024-03-23T08:23:53.901 INFO:tasks.workunit.client.0.smithi070.stdout:7/378: dread - d25/d27/d6a/d44/f77 zero size 2024-03-23T08:23:53.902 INFO:tasks.workunit.client.0.smithi070.stdout:7/379: dread - d25/f87 zero size 2024-03-23T08:23:53.904 INFO:tasks.workunit.client.0.smithi070.stdout:4/282: creat d3/d6/d35/f67 x:0 0 0 2024-03-23T08:23:53.904 INFO:tasks.workunit.client.0.smithi070.stdout:7/380: write f24 [3457820,12894] 0 2024-03-23T08:23:53.905 INFO:tasks.workunit.client.0.smithi070.stdout:7/381: fsync d25/d27/f6d 0 2024-03-23T08:23:53.905 INFO:tasks.workunit.client.0.smithi070.stdout:4/283: stat d3/d6/d35/d65/d1b/f28 0 2024-03-23T08:23:53.907 INFO:tasks.workunit.client.0.smithi070.stdout:7/382: write f24 [2432699,42459] 0 2024-03-23T08:23:53.911 INFO:tasks.workunit.client.0.smithi070.stdout:4/284: dread d3/f23 [0,4194304] 0 2024-03-23T08:23:53.913 INFO:tasks.workunit.client.0.smithi070.stdout:4/285: truncate d3/d6/d9/f2e 348595 0 2024-03-23T08:23:53.913 INFO:tasks.workunit.client.0.smithi070.stdout:4/286: chown d3/f25 31246092 1 2024-03-23T08:23:53.917 INFO:tasks.workunit.client.0.smithi070.stdout:4/287: dread d3/d6/d35/d3f/d51/f63 [0,4194304] 0 2024-03-23T08:23:53.920 INFO:tasks.workunit.client.0.smithi070.stdout:7/383: mknod d25/d27/d6a/d44/d7c/c91 0 2024-03-23T08:23:53.921 INFO:tasks.workunit.client.0.smithi070.stdout:7/384: fdatasync d25/d27/d6a/f7f 0 2024-03-23T08:23:53.929 INFO:tasks.workunit.client.0.smithi070.stdout:7/385: rename f16 to d25/d27/f92 0 2024-03-23T08:23:53.931 INFO:tasks.workunit.client.0.smithi070.stdout:7/386: read d25/d27/d6a/d44/f54 [180647,105009] 0 2024-03-23T08:23:53.932 INFO:tasks.workunit.client.0.smithi070.stdout:7/387: dread - d25/d27/d6a/d44/f7e zero size 2024-03-23T08:23:53.933 INFO:tasks.workunit.client.0.smithi070.stdout:7/388: write d25/d40/f61 [438225,39029] 0 2024-03-23T08:23:53.935 INFO:tasks.workunit.client.0.smithi070.stdout:7/389: creat d25/d27/d6a/d44/d85/f93 x:0 0 0 2024-03-23T08:23:53.935 INFO:tasks.workunit.client.0.smithi070.stdout:7/390: dread - d25/d40/f8f zero size 2024-03-23T08:23:53.940 INFO:tasks.workunit.client.0.smithi070.stdout:7/391: rename d25/l50 to d25/d40/l94 0 2024-03-23T08:23:53.940 INFO:tasks.workunit.client.0.smithi070.stdout:7/392: readlink l9 0 2024-03-23T08:23:53.941 INFO:tasks.workunit.client.0.smithi070.stdout:7/393: dread - d25/d27/d6a/d44/f77 zero size 2024-03-23T08:23:53.943 INFO:tasks.workunit.client.0.smithi070.stdout:7/394: stat le 0 2024-03-23T08:23:53.943 INFO:tasks.workunit.client.0.smithi070.stdout:7/395: write d25/f87 [513779,29982] 0 2024-03-23T08:23:53.946 INFO:tasks.workunit.client.0.smithi070.stdout:7/396: fdatasync d25/d27/d6a/d44/d6f/f71 0 2024-03-23T08:23:53.949 INFO:tasks.workunit.client.0.smithi070.stdout:7/397: rmdir d25/d27/d6a/d44/d6f 39 2024-03-23T08:23:54.171 INFO:tasks.workunit.client.0.smithi070.stdout:6/512: sync 2024-03-23T08:23:54.172 INFO:tasks.workunit.client.0.smithi070.stdout:0/702: sync 2024-03-23T08:23:54.172 INFO:tasks.workunit.client.0.smithi070.stdout:0/703: fdatasync d16/d22/d43/d47/d87/fcb 0 2024-03-23T08:23:54.172 INFO:tasks.workunit.client.0.smithi070.stdout:6/513: creat d0/d23/d28/d35/d6a/fb1 x:0 0 0 2024-03-23T08:23:54.264 INFO:tasks.workunit.client.0.smithi070.stdout:6/514: mknod d0/d23/d88/cb2 0 2024-03-23T08:23:54.265 INFO:tasks.workunit.client.0.smithi070.stdout:6/515: chown d0/d23/d28/d35/d85/f89 4840125 1 2024-03-23T08:23:54.267 INFO:tasks.workunit.client.0.smithi070.stdout:0/704: creat d16/d26/d27/d3d/d4d/ffb x:0 0 0 2024-03-23T08:23:54.275 INFO:tasks.workunit.client.0.smithi070.stdout:6/516: rmdir d0/d3/d2c/da4 0 2024-03-23T08:23:54.280 INFO:tasks.workunit.client.0.smithi070.stdout:6/517: rename d0/d23/d1c to d0/d23/d74/d84/db3 0 2024-03-23T08:23:54.283 INFO:tasks.workunit.client.0.smithi070.stdout:6/518: creat d0/d23/d28/fb4 x:0 0 0 2024-03-23T08:23:54.286 INFO:tasks.workunit.client.0.smithi070.stdout:6/519: mknod d0/d23/d74/d84/db3/d38/d9a/cb5 0 2024-03-23T08:23:54.289 INFO:tasks.workunit.client.0.smithi070.stdout:6/520: rmdir d0/d23/d74/d84/db3/d2f 39 2024-03-23T08:23:54.292 INFO:tasks.workunit.client.0.smithi070.stdout:6/521: mkdir d0/d23/d88/db6 0 2024-03-23T08:23:54.296 INFO:tasks.workunit.client.0.smithi070.stdout:6/522: unlink d0/d3/c7 0 2024-03-23T08:23:54.301 INFO:tasks.workunit.client.0.smithi070.stdout:6/523: symlink d0/d23/d97/d98/lb7 0 2024-03-23T08:23:54.308 INFO:tasks.workunit.client.0.smithi070.stdout:6/524: dwrite d0/f50 [0,4194304] 0 2024-03-23T08:23:54.310 INFO:tasks.workunit.client.0.smithi070.stdout:6/525: chown d0/d23/d28/d35/ca8 2532 1 2024-03-23T08:23:54.310 INFO:tasks.workunit.client.0.smithi070.stdout:3/611: sync 2024-03-23T08:23:54.310 INFO:tasks.workunit.client.0.smithi070.stdout:1/382: sync 2024-03-23T08:23:54.310 INFO:tasks.workunit.client.0.smithi070.stdout:9/517: sync 2024-03-23T08:23:54.311 INFO:tasks.workunit.client.0.smithi070.stdout:1/383: read - d8/d11/f5c zero size 2024-03-23T08:23:54.312 INFO:tasks.workunit.client.0.smithi070.stdout:9/518: readlink db/d12/d22/d85/d42/l8e 0 2024-03-23T08:23:54.312 INFO:tasks.workunit.client.0.smithi070.stdout:6/526: write d0/d23/d74/d84/d3c/f60 [1050497,63817] 0 2024-03-23T08:23:54.317 INFO:tasks.workunit.client.0.smithi070.stdout:1/384: symlink d8/d11/d61/l8c 0 2024-03-23T08:23:54.322 INFO:tasks.workunit.client.0.smithi070.stdout:3/612: mkdir db/d4e/d92/dbb/de2 0 2024-03-23T08:23:54.331 INFO:tasks.workunit.client.0.smithi070.stdout:1/385: mknod d8/d26/d7b/c8d 0 2024-03-23T08:23:54.331 INFO:tasks.workunit.client.0.smithi070.stdout:6/527: mkdir d0/d23/d74/d84/db3/d2f/dab/db8 0 2024-03-23T08:23:54.334 INFO:tasks.workunit.client.0.smithi070.stdout:3/613: dwrite db/df/d15/d22/db2/fc7 [0,4194304] 0 2024-03-23T08:23:54.337 INFO:tasks.workunit.client.0.smithi070.stdout:0/705: dwrite f13 [0,4194304] 0 2024-03-23T08:23:54.337 INFO:tasks.workunit.client.0.smithi070.stdout:6/528: symlink d0/d23/d28/d35/lb9 0 2024-03-23T08:23:54.338 INFO:tasks.workunit.client.0.smithi070.stdout:9/519: link db/d12/d22/d85/d42/l53 db/d12/d22/laa 0 2024-03-23T08:23:54.340 INFO:tasks.workunit.client.0.smithi070.stdout:9/520: chown db/d12/d34/d8c/da2/d58/la0 1 1 2024-03-23T08:23:54.343 INFO:tasks.workunit.client.0.smithi070.stdout:0/706: mknod d16/d26/d27/dee/cfc 0 2024-03-23T08:23:54.346 INFO:tasks.workunit.client.0.smithi070.stdout:0/707: read d16/d22/d43/d47/da9/f8c [2164442,118303] 0 2024-03-23T08:23:54.349 INFO:tasks.workunit.client.0.smithi070.stdout:9/521: mkdir db/d12/dab 0 2024-03-23T08:23:54.350 INFO:tasks.workunit.client.0.smithi070.stdout:9/522: readlink db/d12/d14/l48 0 2024-03-23T08:23:54.351 INFO:tasks.workunit.client.0.smithi070.stdout:9/523: chown db/d12/d34/d8c/da2/d26/d5c/c5e 5554642 1 2024-03-23T08:23:54.352 INFO:tasks.workunit.client.0.smithi070.stdout:6/529: link d0/d23/d74/d84/d3c/l41 d0/d23/d74/d84/lba 0 2024-03-23T08:23:54.353 INFO:tasks.workunit.client.0.smithi070.stdout:0/708: fsync d16/d26/d27/d3d/d4d/dad/fe8 0 2024-03-23T08:23:54.356 INFO:tasks.workunit.client.0.smithi070.stdout:6/530: unlink d0/d23/d28/d35/c4f 0 2024-03-23T08:23:54.360 INFO:tasks.workunit.client.0.smithi070.stdout:6/531: fsync d0/d23/d28/d35/d85/f89 0 2024-03-23T08:23:54.362 INFO:tasks.workunit.client.0.smithi070.stdout:9/524: getdents db/d12/d34/d8c/da2/d26/d80 0 2024-03-23T08:23:54.366 INFO:tasks.workunit.client.0.smithi070.stdout:9/525: dread db/d12/f82 [0,4194304] 0 2024-03-23T08:23:54.367 INFO:tasks.workunit.client.0.smithi070.stdout:9/526: stat db/d12/d34/d8c/da2 0 2024-03-23T08:23:54.370 INFO:tasks.workunit.client.0.smithi070.stdout:9/527: creat db/d12/d34/d8c/da2/d26/da7/fac x:0 0 0 2024-03-23T08:23:54.372 INFO:tasks.workunit.client.0.smithi070.stdout:9/528: read db/f1b [1215497,13791] 0 2024-03-23T08:23:54.378 INFO:tasks.workunit.client.0.smithi070.stdout:9/529: write db/d12/d34/d8c/da2/d26/f64 [421209,79547] 0 2024-03-23T08:23:54.380 INFO:tasks.workunit.client.0.smithi070.stdout:9/530: getdents db/d12/d22 0 2024-03-23T08:23:54.381 INFO:tasks.workunit.client.0.smithi070.stdout:6/532: rmdir d0/d23/d28/d35 39 2024-03-23T08:23:54.383 INFO:tasks.workunit.client.0.smithi070.stdout:9/531: unlink db/cd 0 2024-03-23T08:23:54.385 INFO:tasks.workunit.client.0.smithi070.stdout:6/533: mknod d0/d23/d74/d84/db3/d2f/cbb 0 2024-03-23T08:23:54.388 INFO:tasks.workunit.client.0.smithi070.stdout:6/534: chown d0/d23/d28/d35 93564 1 2024-03-23T08:23:54.389 INFO:tasks.workunit.client.0.smithi070.stdout:6/535: fsync d0/d23/d28/d35/d6a/fb1 0 2024-03-23T08:23:54.391 INFO:tasks.workunit.client.0.smithi070.stdout:6/536: mknod d0/d3/d2c/cbc 0 2024-03-23T08:23:54.393 INFO:tasks.workunit.client.0.smithi070.stdout:6/537: mknod d0/d23/d4d/da5/cbd 0 2024-03-23T08:23:54.395 INFO:tasks.workunit.client.0.smithi070.stdout:9/532: dread db/d12/d14/f7a [0,4194304] 0 2024-03-23T08:23:54.404 INFO:tasks.workunit.client.0.smithi070.stdout:9/533: dwrite db/d12/f9a [0,4194304] 0 2024-03-23T08:23:54.408 INFO:tasks.workunit.client.0.smithi070.stdout:6/538: dwrite d0/d23/d74/d84/db3/d38/d52/f7f [0,4194304] 0 2024-03-23T08:23:54.410 INFO:tasks.workunit.client.0.smithi070.stdout:6/539: read - d0/d23/d28/fb4 zero size 2024-03-23T08:23:54.411 INFO:tasks.workunit.client.0.smithi070.stdout:9/534: fdatasync db/d12/d14/f63 0 2024-03-23T08:23:54.412 INFO:tasks.workunit.client.0.smithi070.stdout:6/540: chown d0/d23/d74/d84/d3c/f53 10222417 1 2024-03-23T08:23:54.412 INFO:tasks.workunit.client.0.smithi070.stdout:9/535: readlink db/l3e 0 2024-03-23T08:23:54.414 INFO:tasks.workunit.client.0.smithi070.stdout:6/541: truncate d0/d23/d28/f7d 1085673 0 2024-03-23T08:23:54.424 INFO:tasks.workunit.client.0.smithi070.stdout:9/536: write db/d12/d22/d85/d42/fa4 [4543755,107112] 0 2024-03-23T08:23:54.445 INFO:tasks.workunit.client.0.smithi070.stdout:9/537: dread db/d12/d34/d8c/da2/d26/d5c/d7e/f56 [0,4194304] 0 2024-03-23T08:23:54.453 INFO:tasks.workunit.client.0.smithi070.stdout:9/538: dwrite db/d12/d34/d8c/da2/d26/d5c/d7e/d6c/f99 [0,4194304] 0 2024-03-23T08:23:54.461 INFO:tasks.workunit.client.0.smithi070.stdout:9/539: dwrite db/d12/d14/f95 [0,4194304] 0 2024-03-23T08:23:54.463 INFO:tasks.workunit.client.0.smithi070.stdout:9/540: write db/d12/d14/f23 [2786767,93545] 0 2024-03-23T08:23:54.464 INFO:tasks.workunit.client.0.smithi070.stdout:9/541: dread - db/d12/d34/d8c/da2/d26/d5c/d7e/d6c/d79/f9d zero size 2024-03-23T08:23:54.502 INFO:tasks.workunit.client.0.smithi070.stdout:9/542: creat db/d12/d22/d85/fad x:0 0 0 2024-03-23T08:23:54.503 INFO:tasks.workunit.client.0.smithi070.stdout:9/543: write db/d12/d22/d85/d42/f46 [2490336,63748] 0 2024-03-23T08:23:54.506 INFO:tasks.workunit.client.0.smithi070.stdout:9/544: mknod db/d12/d67/cae 0 2024-03-23T08:23:54.507 INFO:tasks.workunit.client.0.smithi070.stdout:9/545: write db/d12/d14/f23 [619270,50387] 0 2024-03-23T08:23:54.507 INFO:tasks.workunit.client.0.smithi070.stdout:9/546: chown db/d12/d34/d8c 22439147 1 2024-03-23T08:23:54.510 INFO:tasks.workunit.client.0.smithi070.stdout:9/547: rename db/f1f to db/d12/d34/d8c/da2/d26/d5c/d7e/d6c/faf 0 2024-03-23T08:23:54.512 INFO:tasks.workunit.client.0.smithi070.stdout:9/548: write db/d12/d34/d8c/da2/d26/d5c/d7e/d6c/d79/f9d [633136,106110] 0 2024-03-23T08:23:54.514 INFO:tasks.workunit.client.0.smithi070.stdout:9/549: creat db/d12/d22/fb0 x:0 0 0 2024-03-23T08:23:54.516 INFO:tasks.workunit.client.0.smithi070.stdout:9/550: mknod db/d12/d34/d8c/cb1 0 2024-03-23T08:23:54.517 INFO:tasks.workunit.client.0.smithi070.stdout:9/551: chown db/d12/d34/d8c/da2/l1c 603371590 1 2024-03-23T08:23:54.519 INFO:tasks.workunit.client.0.smithi070.stdout:9/552: creat db/d12/d34/d8c/da2/d26/d80/fb2 x:0 0 0 2024-03-23T08:23:54.641 INFO:tasks.workunit.client.0.smithi070.stdout:7/398: sync 2024-03-23T08:23:54.641 INFO:tasks.workunit.client.0.smithi070.stdout:8/471: sync 2024-03-23T08:23:54.641 INFO:tasks.workunit.client.0.smithi070.stdout:5/372: sync 2024-03-23T08:23:54.641 INFO:tasks.workunit.client.0.smithi070.stdout:4/288: sync 2024-03-23T08:23:54.641 INFO:tasks.workunit.client.0.smithi070.stdout:2/462: sync 2024-03-23T08:23:54.644 INFO:tasks.workunit.client.0.smithi070.stdout:7/399: fsync f23 0 2024-03-23T08:23:54.645 INFO:tasks.workunit.client.0.smithi070.stdout:4/289: fsync d3/d19/f4c 0 2024-03-23T08:23:54.649 INFO:tasks.workunit.client.0.smithi070.stdout:8/472: rmdir d5/d8/d9a 39 2024-03-23T08:23:54.651 INFO:tasks.workunit.client.0.smithi070.stdout:7/400: chown c15 2849050 1 2024-03-23T08:23:54.652 INFO:tasks.workunit.client.0.smithi070.stdout:5/373: readlink dc/d1b/d1d/d25/d36/l4e 0 2024-03-23T08:23:54.652 INFO:tasks.workunit.client.0.smithi070.stdout:4/290: mknod d3/d6/d35/c68 0 2024-03-23T08:23:54.654 INFO:tasks.workunit.client.0.smithi070.stdout:8/473: creat d5/d8/d16/d3b/d4d/d6f/fb5 x:0 0 0 2024-03-23T08:23:54.658 INFO:tasks.workunit.client.0.smithi070.stdout:7/401: mkdir d25/d27/d6a/d44/d85/d95 0 2024-03-23T08:23:54.662 INFO:tasks.workunit.client.0.smithi070.stdout:4/291: symlink d3/d6/d9/l69 0 2024-03-23T08:23:54.663 INFO:tasks.workunit.client.0.smithi070.stdout:7/402: dread d25/d27/f6d [0,4194304] 0 2024-03-23T08:23:54.667 INFO:tasks.workunit.client.0.smithi070.stdout:5/374: fsync dc/d1b/d1d/d25/f33 0 2024-03-23T08:23:54.672 INFO:tasks.workunit.client.0.smithi070.stdout:8/474: truncate d5/d8/d16/f2b 463646 0 2024-03-23T08:23:54.677 INFO:tasks.workunit.client.0.smithi070.stdout:5/375: creat dc/d1b/d1d/d25/d38/d53/f7a x:0 0 0 2024-03-23T08:23:54.679 INFO:tasks.workunit.client.0.smithi070.stdout:7/403: mknod d25/d27/d6a/d44/c96 0 2024-03-23T08:23:54.681 INFO:tasks.workunit.client.0.smithi070.stdout:8/475: mknod d5/d8/d16/d3b/d4d/d6f/cb6 0 2024-03-23T08:23:54.682 INFO:tasks.workunit.client.0.smithi070.stdout:5/376: dwrite f1 [0,4194304] 0 2024-03-23T08:23:54.687 INFO:tasks.workunit.client.0.smithi070.stdout:5/377: truncate dc/d1b/d1d/d25/d38/d53/f5f 1638464 0 2024-03-23T08:23:54.687 INFO:tasks.workunit.client.0.smithi070.stdout:4/292: rename d3/d19 to d3/d6/d6a 0 2024-03-23T08:23:54.688 INFO:tasks.workunit.client.0.smithi070.stdout:7/404: dwrite d25/d27/d6a/d44/f56 [0,4194304] 0 2024-03-23T08:23:54.696 INFO:tasks.workunit.client.0.smithi070.stdout:4/293: dwrite d3/f25 [0,4194304] 0 2024-03-23T08:23:54.697 INFO:tasks.workunit.client.0.smithi070.stdout:4/294: write d3/d6/f3b [815676,5352] 0 2024-03-23T08:23:54.698 INFO:tasks.workunit.client.0.smithi070.stdout:4/295: dread - d3/d6/d6a/f58 zero size 2024-03-23T08:23:54.699 INFO:tasks.workunit.client.0.smithi070.stdout:5/378: mknod dc/d19/c7b 0 2024-03-23T08:23:54.707 INFO:tasks.workunit.client.0.smithi070.stdout:4/296: creat d3/d6/d9/f6b x:0 0 0 2024-03-23T08:23:54.708 INFO:tasks.workunit.client.0.smithi070.stdout:5/379: symlink dc/d1b/d1d/d25/d38/d39/l7c 0 2024-03-23T08:23:54.709 INFO:tasks.workunit.client.0.smithi070.stdout:5/380: write dc/d1b/d1d/f44 [181651,126908] 0 2024-03-23T08:23:54.711 INFO:tasks.workunit.client.0.smithi070.stdout:5/381: write dc/d1b/d1d/d25/d38/d48/f71 [2080495,50338] 0 2024-03-23T08:23:54.713 INFO:tasks.workunit.client.0.smithi070.stdout:5/382: fdatasync dc/d1b/d1d/d25/d36/d32/f5a 0 2024-03-23T08:23:54.719 INFO:tasks.workunit.client.0.smithi070.stdout:5/383: mknod dc/d1b/d1d/d25/d38/d53/c7d 0 2024-03-23T08:23:54.725 INFO:tasks.workunit.client.0.smithi070.stdout:5/384: dwrite dc/d1b/d1d/d25/d38/d53/f60 [0,4194304] 0 2024-03-23T08:23:54.726 INFO:tasks.workunit.client.0.smithi070.stdout:4/297: creat d3/d6/f6c x:0 0 0 2024-03-23T08:23:54.726 INFO:tasks.workunit.client.0.smithi070.stdout:7/405: link d25/d27/d6a/d44/l89 d25/d27/d6a/d44/d6f/l97 0 2024-03-23T08:23:54.729 INFO:tasks.workunit.client.0.smithi070.stdout:4/298: creat d3/f6d x:0 0 0 2024-03-23T08:23:54.731 INFO:tasks.workunit.client.0.smithi070.stdout:5/385: symlink dc/d1b/l7e 0 2024-03-23T08:23:54.731 INFO:tasks.workunit.client.0.smithi070.stdout:4/299: truncate d3/d6/d35/d65/d1b/f66 11530 0 2024-03-23T08:23:54.733 INFO:tasks.workunit.client.0.smithi070.stdout:4/300: fsync d3/d6/d35/d65/d1b/f1c 0 2024-03-23T08:23:54.738 INFO:tasks.workunit.client.0.smithi070.stdout:5/386: dwrite dc/f58 [0,4194304] 0 2024-03-23T08:23:54.740 INFO:tasks.workunit.client.0.smithi070.stdout:5/387: read - dc/d1b/d1d/d25/d38/d39/f6b zero size 2024-03-23T08:23:54.741 INFO:tasks.workunit.client.0.smithi070.stdout:4/301: rename d3/d6/d35/d65/d27/d4b/d5c to d3/d6/d35/d3f/d51/d6e 0 2024-03-23T08:23:54.742 INFO:tasks.workunit.client.0.smithi070.stdout:4/302: chown d3/d6/f2f 8053 1 2024-03-23T08:23:54.749 INFO:tasks.workunit.client.0.smithi070.stdout:5/388: dwrite dc/d1b/d1d/d25/d36/f27 [4194304,4194304] 0 2024-03-23T08:23:54.751 INFO:tasks.workunit.client.0.smithi070.stdout:4/303: dwrite d3/f25 [0,4194304] 0 2024-03-23T08:23:54.755 INFO:tasks.workunit.client.0.smithi070.stdout:4/304: mkdir d3/d6/d35/d65/d6f 0 2024-03-23T08:23:54.757 INFO:tasks.workunit.client.0.smithi070.stdout:5/389: symlink dc/d1b/d1d/d25/d38/l7f 0 2024-03-23T08:23:54.763 INFO:tasks.workunit.client.0.smithi070.stdout:2/463: truncate d1/d7/df/f4b 2532593 0 2024-03-23T08:23:54.763 INFO:tasks.workunit.client.0.smithi070.stdout:4/305: dwrite d3/d6/f7 [0,4194304] 0 2024-03-23T08:23:54.768 INFO:tasks.workunit.client.0.smithi070.stdout:5/390: dwrite dc/f3d [0,4194304] 0 2024-03-23T08:23:54.773 INFO:tasks.workunit.client.0.smithi070.stdout:2/464: creat d1/d7/d39/f9c x:0 0 0 2024-03-23T08:23:54.773 INFO:tasks.workunit.client.0.smithi070.stdout:4/306: truncate d3/d6/d11/f55 1020626 0 2024-03-23T08:23:54.774 INFO:tasks.workunit.client.0.smithi070.stdout:5/391: unlink dc/d1b/d1d/d25/d38/d48/f71 0 2024-03-23T08:23:54.775 INFO:tasks.workunit.client.0.smithi070.stdout:8/476: truncate d5/d8/d16/d32/f54 544740 0 2024-03-23T08:23:54.782 INFO:tasks.workunit.client.0.smithi070.stdout:8/477: symlink d5/d8/d16/d8d/da3/lb7 0 2024-03-23T08:23:54.786 INFO:tasks.workunit.client.0.smithi070.stdout:5/392: dwrite dc/d1b/d1d/d25/d36/d32/d5c/f5d [0,4194304] 0 2024-03-23T08:23:54.786 INFO:tasks.workunit.client.0.smithi070.stdout:8/478: rmdir d5/d8/d16/d30/d38/d6b 39 2024-03-23T08:23:54.790 INFO:tasks.workunit.client.0.smithi070.stdout:8/479: unlink d5/d8/d16/d30/f68 0 2024-03-23T08:23:54.791 INFO:tasks.workunit.client.0.smithi070.stdout:5/393: dwrite dc/d1b/d1d/d25/d38/f69 [0,4194304] 0 2024-03-23T08:23:54.796 INFO:tasks.workunit.client.0.smithi070.stdout:8/480: unlink d5/d8/d16/d30/d38/d6b/d72/l8b 0 2024-03-23T08:23:54.799 INFO:tasks.workunit.client.0.smithi070.stdout:1/386: sync 2024-03-23T08:23:54.800 INFO:tasks.workunit.client.0.smithi070.stdout:5/394: creat dc/d1b/d1d/d25/d36/d32/f80 x:0 0 0 2024-03-23T08:23:54.802 INFO:tasks.workunit.client.0.smithi070.stdout:1/387: truncate d8/db/d2c/d3f/d48/f58 1003756 0 2024-03-23T08:23:54.803 INFO:tasks.workunit.client.0.smithi070.stdout:8/481: creat d5/d8/d16/d8d/da3/fb8 x:0 0 0 2024-03-23T08:23:54.805 INFO:tasks.workunit.client.0.smithi070.stdout:5/395: unlink dc/d1b/d1d/d25/d38/d39/l7c 0 2024-03-23T08:23:54.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/396: write dc/d1b/d1d/d25/d38/d53/f60 [1718297,99638] 0 2024-03-23T08:23:54.807 INFO:tasks.workunit.client.0.smithi070.stdout:5/397: chown dc/d1b/d1d/d25/d36/f2a 1001443 1 2024-03-23T08:23:54.835 INFO:tasks.workunit.client.0.smithi070.stdout:2/465: write d1/d7/df/d94/f12 [4473868,32032] 0 2024-03-23T08:23:54.837 INFO:tasks.workunit.client.0.smithi070.stdout:5/398: chown dc/d1b/d1d/c30 0 1 2024-03-23T08:23:54.841 INFO:tasks.workunit.client.0.smithi070.stdout:2/466: symlink d1/d7/d48/l9d 0 2024-03-23T08:23:54.842 INFO:tasks.workunit.client.0.smithi070.stdout:8/482: write d5/d8/f11 [568077,40191] 0 2024-03-23T08:23:54.843 INFO:tasks.workunit.client.0.smithi070.stdout:1/388: dread d8/d1b/f25 [0,4194304] 0 2024-03-23T08:23:54.848 INFO:tasks.workunit.client.0.smithi070.stdout:5/399: mknod dc/d1b/d1d/d25/d36/d32/c81 0 2024-03-23T08:23:54.849 INFO:tasks.workunit.client.0.smithi070.stdout:1/389: mkdir d8/db/d8e 0 2024-03-23T08:23:54.876 INFO:tasks.workunit.client.0.smithi070.stdout:2/467: creat d1/f9e x:0 0 0 2024-03-23T08:23:54.877 INFO:tasks.workunit.client.0.smithi070.stdout:5/400: unlink dc/d1b/d1d/d25/d38/d39/c6a 0 2024-03-23T08:23:54.877 INFO:tasks.workunit.client.0.smithi070.stdout:2/468: fsync d1/d7/d39/d47/f9b 0 2024-03-23T08:23:54.878 INFO:tasks.workunit.client.0.smithi070.stdout:1/390: mkdir d8/d1a/d8f 0 2024-03-23T08:23:54.882 INFO:tasks.workunit.client.0.smithi070.stdout:2/469: link d1/d7/df/f91 d1/d29/d52/f9f 0 2024-03-23T08:23:54.883 INFO:tasks.workunit.client.0.smithi070.stdout:1/391: creat d8/f90 x:0 0 0 2024-03-23T08:23:54.884 INFO:tasks.workunit.client.0.smithi070.stdout:2/470: write d1/d6/d4f/f74 [225906,42874] 0 2024-03-23T08:23:54.885 INFO:tasks.workunit.client.0.smithi070.stdout:5/401: dwrite dc/d1b/d1d/d25/d38/d53/f7a [0,4194304] 0 2024-03-23T08:23:54.887 INFO:tasks.workunit.client.0.smithi070.stdout:1/392: chown d8/d1b/f86 2219087 1 2024-03-23T08:23:54.889 INFO:tasks.workunit.client.0.smithi070.stdout:1/393: chown d8/d11/d61/d74/d5d/d70/l76 1 1 2024-03-23T08:23:54.890 INFO:tasks.workunit.client.0.smithi070.stdout:2/471: creat d1/d7/d48/d69/fa0 x:0 0 0 2024-03-23T08:23:54.895 INFO:tasks.workunit.client.0.smithi070.stdout:5/402: mknod dc/d1b/d1d/d25/d38/d39/c82 0 2024-03-23T08:23:54.896 INFO:tasks.workunit.client.0.smithi070.stdout:1/394: dwrite d8/d26/f7f [0,4194304] 0 2024-03-23T08:23:54.897 INFO:tasks.workunit.client.0.smithi070.stdout:5/403: chown dc/d1b/d1d/d25/d36/d32/d5c 24900779 1 2024-03-23T08:23:54.902 INFO:tasks.workunit.client.0.smithi070.stdout:2/472: link d1/l80 d1/d7/df/la1 0 2024-03-23T08:23:54.904 INFO:tasks.workunit.client.0.smithi070.stdout:8/483: truncate d5/f2a 157384 0 2024-03-23T08:23:54.909 INFO:tasks.workunit.client.0.smithi070.stdout:5/404: dwrite dc/d1b/d1d/d25/d38/d53/f7a [0,4194304] 0 2024-03-23T08:23:54.911 INFO:tasks.workunit.client.0.smithi070.stdout:8/484: rmdir d5/d8/d16/d3b/d8c 39 2024-03-23T08:23:54.913 INFO:tasks.workunit.client.0.smithi070.stdout:2/473: dwrite d1/fd [0,4194304] 0 2024-03-23T08:23:54.923 INFO:tasks.workunit.client.0.smithi070.stdout:1/395: dwrite d8/db/f32 [0,4194304] 0 2024-03-23T08:23:54.926 INFO:tasks.workunit.client.0.smithi070.stdout:5/405: dread dc/d1b/d1d/d25/d38/d39/f3c [0,4194304] 0 2024-03-23T08:23:54.947 INFO:tasks.workunit.client.0.smithi070.stdout:2/474: dread d1/d21/d5f/f67 [0,4194304] 0 2024-03-23T08:23:54.955 INFO:tasks.workunit.client.0.smithi070.stdout:5/406: dread dc/d1b/d1d/d25/d38/d39/f66 [0,4194304] 0 2024-03-23T08:23:54.964 INFO:tasks.workunit.client.0.smithi070.stdout:0/709: sync 2024-03-23T08:23:54.964 INFO:tasks.workunit.client.0.smithi070.stdout:6/542: sync 2024-03-23T08:23:54.964 INFO:tasks.workunit.client.0.smithi070.stdout:3/614: sync 2024-03-23T08:23:54.965 INFO:tasks.workunit.client.0.smithi070.stdout:3/615: write db/df/d1a/f58 [2377819,65039] 0 2024-03-23T08:23:54.966 INFO:tasks.workunit.client.0.smithi070.stdout:6/543: chown d0/d23/d74/d84/db3/d2f/f39 946243040 1 2024-03-23T08:23:54.966 INFO:tasks.workunit.client.0.smithi070.stdout:6/544: fsync d0/f50 0 2024-03-23T08:23:54.971 INFO:tasks.workunit.client.0.smithi070.stdout:6/545: dwrite d0/d23/d74/d84/faa [0,4194304] 0 2024-03-23T08:23:55.017 INFO:tasks.workunit.client.0.smithi070.stdout:8/485: mkdir d5/d8/d16/d3b/db9 0 2024-03-23T08:23:55.021 INFO:tasks.workunit.client.0.smithi070.stdout:1/396: creat d8/d42/d51/f91 x:0 0 0 2024-03-23T08:23:55.023 INFO:tasks.workunit.client.0.smithi070.stdout:2/475: mkdir d1/d6/d34/da2 0 2024-03-23T08:23:55.031 INFO:tasks.workunit.client.0.smithi070.stdout:1/397: creat d8/d26/d41/f92 x:0 0 0 2024-03-23T08:23:55.031 INFO:tasks.workunit.client.0.smithi070.stdout:1/398: chown d8/d1b/f4b 50 1 2024-03-23T08:23:55.033 INFO:tasks.workunit.client.0.smithi070.stdout:1/399: read d8/d11/d61/d74/f71 [398336,124125] 0 2024-03-23T08:23:55.034 INFO:tasks.workunit.client.0.smithi070.stdout:5/407: link dc/d1b/d1d/f44 dc/d1b/d1d/d25/d68/f83 0 2024-03-23T08:23:55.037 INFO:tasks.workunit.client.0.smithi070.stdout:3/616: symlink db/le3 0 2024-03-23T08:23:55.037 INFO:tasks.workunit.client.0.smithi070.stdout:6/546: symlink d0/d23/d74/d84/db3/lbe 0 2024-03-23T08:23:55.039 INFO:tasks.workunit.client.0.smithi070.stdout:1/400: creat d8/d11/d61/d74/d5d/d70/f93 x:0 0 0 2024-03-23T08:23:55.040 INFO:tasks.workunit.client.0.smithi070.stdout:2/476: getdents d1/d6/d34/da2 0 2024-03-23T08:23:55.041 INFO:tasks.workunit.client.0.smithi070.stdout:2/477: fdatasync d1/d6/f72 0 2024-03-23T08:23:55.043 INFO:tasks.workunit.client.0.smithi070.stdout:5/408: dwrite f1 [0,4194304] 0 2024-03-23T08:23:55.043 INFO:tasks.workunit.client.0.smithi070.stdout:2/478: fsync d1/d29/f2d 0 2024-03-23T08:23:55.044 INFO:tasks.workunit.client.0.smithi070.stdout:8/486: dread d5/d8/f17 [0,4194304] 0 2024-03-23T08:23:55.046 INFO:tasks.workunit.client.0.smithi070.stdout:8/487: chown d5/d8/d16/d30/d38/d6b/lb1 6268 1 2024-03-23T08:23:55.049 INFO:tasks.workunit.client.0.smithi070.stdout:0/710: getdents d16/d26/d27/d86/d67/d7d 0 2024-03-23T08:23:55.055 INFO:tasks.workunit.client.0.smithi070.stdout:3/617: write db/df/d15/d3d/f56 [867380,105287] 0 2024-03-23T08:23:55.057 INFO:tasks.workunit.client.0.smithi070.stdout:6/547: truncate d0/f50 2999767 0 2024-03-23T08:23:55.058 INFO:tasks.workunit.client.0.smithi070.stdout:1/401: symlink d8/d11/d61/d74/l94 0 2024-03-23T08:23:55.059 INFO:tasks.workunit.client.0.smithi070.stdout:2/479: dwrite d1/d7/f17 [0,4194304] 0 2024-03-23T08:23:55.060 INFO:tasks.workunit.client.0.smithi070.stdout:8/488: mknod d5/d8/d16/d32/cba 0 2024-03-23T08:23:55.062 INFO:tasks.workunit.client.0.smithi070.stdout:2/480: chown d1/d7/df/d94/l6f 137324 1 2024-03-23T08:23:55.062 INFO:tasks.workunit.client.0.smithi070.stdout:0/711: mknod d16/d26/d27/d4c/d60/d75/cfd 0 2024-03-23T08:23:55.063 INFO:tasks.workunit.client.0.smithi070.stdout:6/548: dread d0/d23/d74/d84/db3/f5d [0,4194304] 0 2024-03-23T08:23:55.063 INFO:tasks.workunit.client.0.smithi070.stdout:0/712: readlink d16/d22/l38 0 2024-03-23T08:23:55.063 INFO:tasks.workunit.client.0.smithi070.stdout:5/409: symlink dc/d1b/d1d/d25/d38/l84 0 2024-03-23T08:23:55.065 INFO:tasks.workunit.client.0.smithi070.stdout:2/481: truncate d1/d7/d48/d69/fa0 387282 0 2024-03-23T08:23:55.069 INFO:tasks.workunit.client.0.smithi070.stdout:1/402: rename d8/db/d2c/d3f/d48/f81 to d8/d11/d61/f95 0 2024-03-23T08:23:55.072 INFO:tasks.workunit.client.0.smithi070.stdout:5/410: creat dc/d1b/d1d/d25/d38/d48/f85 x:0 0 0 2024-03-23T08:23:55.078 INFO:tasks.workunit.client.0.smithi070.stdout:3/618: dwrite db/df/f55 [0,4194304] 0 2024-03-23T08:23:55.079 INFO:tasks.workunit.client.0.smithi070.stdout:3/619: stat db/d14/d65/dae 0 2024-03-23T08:23:55.080 INFO:tasks.workunit.client.0.smithi070.stdout:6/549: mknod d0/d23/d28/cbf 0 2024-03-23T08:23:55.083 INFO:tasks.workunit.client.0.smithi070.stdout:6/550: write d0/d23/d28/d35/d6a/fb1 [714733,14756] 0 2024-03-23T08:23:55.086 INFO:tasks.workunit.client.0.smithi070.stdout:1/403: rename d8/db/c2e to d8/d26/d49/c96 0 2024-03-23T08:23:55.089 INFO:tasks.workunit.client.0.smithi070.stdout:8/489: dwrite d5/d8/f19 [0,4194304] 0 2024-03-23T08:23:55.091 INFO:tasks.workunit.client.0.smithi070.stdout:0/713: creat d16/d26/d27/ffe x:0 0 0 2024-03-23T08:23:55.093 INFO:tasks.workunit.client.0.smithi070.stdout:2/482: write d1/d7/df/d94/d38/d4c/f6a [1410487,10121] 0 2024-03-23T08:23:55.095 INFO:tasks.workunit.client.0.smithi070.stdout:6/551: symlink d0/d23/d4d/da5/lc0 0 2024-03-23T08:23:55.095 INFO:tasks.workunit.client.0.smithi070.stdout:8/490: readlink d5/d8/d16/d30/d38/d6b/d72/l95 0 2024-03-23T08:23:55.097 INFO:tasks.workunit.client.0.smithi070.stdout:1/404: read d8/d11/f29 [2063741,46894] 0 2024-03-23T08:23:55.098 INFO:tasks.workunit.client.0.smithi070.stdout:3/620: mknod db/d14/d6f/ce4 0 2024-03-23T08:23:55.099 INFO:tasks.workunit.client.0.smithi070.stdout:5/411: dwrite dc/d1b/d1d/f44 [0,4194304] 0 2024-03-23T08:23:55.100 INFO:tasks.workunit.client.0.smithi070.stdout:3/621: stat db/d7f/fbe 0 2024-03-23T08:23:55.101 INFO:tasks.workunit.client.0.smithi070.stdout:2/483: creat d1/d7/df/d94/d38/fa3 x:0 0 0 2024-03-23T08:23:55.102 INFO:tasks.workunit.client.0.smithi070.stdout:0/714: rmdir d16/d26/d27/d3d 39 2024-03-23T08:23:55.102 INFO:tasks.workunit.client.0.smithi070.stdout:5/412: chown dc/d1b/d1d/d25/d38/d39/f79 4896 1 2024-03-23T08:23:55.109 INFO:tasks.workunit.client.0.smithi070.stdout:8/491: truncate d5/d8/d16/d3b/d4d/d6f/fb5 418654 0 2024-03-23T08:23:55.109 INFO:tasks.workunit.client.0.smithi070.stdout:1/405: unlink d8/d42/d51/d6f/f8a 0 2024-03-23T08:23:55.109 INFO:tasks.workunit.client.0.smithi070.stdout:3/622: mknod db/d7f/ce5 0 2024-03-23T08:23:55.109 INFO:tasks.workunit.client.0.smithi070.stdout:0/715: write d16/d26/fab [1239916,110387] 0 2024-03-23T08:23:55.109 INFO:tasks.workunit.client.0.smithi070.stdout:3/623: readlink db/d14/d43/lc8 0 2024-03-23T08:23:55.111 INFO:tasks.workunit.client.0.smithi070.stdout:2/484: mkdir d1/d21/d5f/da4 0 2024-03-23T08:23:55.114 INFO:tasks.workunit.client.0.smithi070.stdout:2/485: write d1/d7/f40 [3588225,98903] 0 2024-03-23T08:23:55.115 INFO:tasks.workunit.client.0.smithi070.stdout:1/406: mkdir d8/db/d40/d43/d64/d85/d97 0 2024-03-23T08:23:55.117 INFO:tasks.workunit.client.0.smithi070.stdout:3/624: mkdir db/de6 0 2024-03-23T08:23:55.117 INFO:tasks.workunit.client.0.smithi070.stdout:8/492: dwrite d5/d8/d16/d3b/d4d/d6f/f89 [0,4194304] 0 2024-03-23T08:23:55.118 INFO:tasks.workunit.client.0.smithi070.stdout:2/486: readlink d1/d6/l15 0 2024-03-23T08:23:55.119 INFO:tasks.workunit.client.0.smithi070.stdout:9/553: sync 2024-03-23T08:23:55.122 INFO:tasks.workunit.client.0.smithi070.stdout:1/407: chown d8/db/f15 33497802 1 2024-03-23T08:23:55.122 INFO:tasks.workunit.client.0.smithi070.stdout:5/413: rename dc/f14 to dc/d1b/d1d/d25/d38/f86 0 2024-03-23T08:23:55.122 INFO:tasks.workunit.client.0.smithi070.stdout:8/493: chown d5/l3d 1 1 2024-03-23T08:23:55.122 INFO:tasks.workunit.client.0.smithi070.stdout:0/716: dwrite d16/d22/d43/fe1 [0,4194304] 0 2024-03-23T08:23:55.123 INFO:tasks.workunit.client.0.smithi070.stdout:3/625: symlink db/d14/d43/d7d/le7 0 2024-03-23T08:23:55.130 INFO:tasks.workunit.client.0.smithi070.stdout:6/552: dread d0/d23/d74/d84/f3b [0,4194304] 0 2024-03-23T08:23:55.132 INFO:tasks.workunit.client.0.smithi070.stdout:8/494: chown d5/ld 462174 1 2024-03-23T08:23:55.133 INFO:tasks.workunit.client.0.smithi070.stdout:2/487: creat d1/d7/d48/d69/d7a/fa5 x:0 0 0 2024-03-23T08:23:55.135 INFO:tasks.workunit.client.0.smithi070.stdout:0/717: write d16/f6f [1530527,109495] 0 2024-03-23T08:23:55.137 INFO:tasks.workunit.client.0.smithi070.stdout:9/554: readlink db/d12/d22/d85/l68 0 2024-03-23T08:23:55.139 INFO:tasks.workunit.client.0.smithi070.stdout:9/555: dread - db/d12/d34/d8c/da2/d26/d5c/f8a zero size 2024-03-23T08:23:55.141 INFO:tasks.workunit.client.0.smithi070.stdout:6/553: mknod d0/d23/d74/d84/db3/d2f/cc1 0 2024-03-23T08:23:55.143 INFO:tasks.workunit.client.0.smithi070.stdout:2/488: creat d1/d7/d48/d69/fa6 x:0 0 0 2024-03-23T08:23:55.147 INFO:tasks.workunit.client.0.smithi070.stdout:8/495: write d5/d8/d16/d32/f3c [984644,75324] 0 2024-03-23T08:23:55.147 INFO:tasks.workunit.client.0.smithi070.stdout:8/496: fdatasync d5/d8/f19 0 2024-03-23T08:23:55.150 INFO:tasks.workunit.client.0.smithi070.stdout:3/626: write db/d20/d63/f75 [438108,78536] 0 2024-03-23T08:23:55.151 INFO:tasks.workunit.client.0.smithi070.stdout:9/556: symlink db/d12/d34/d8c/da2/d4f/lb3 0 2024-03-23T08:23:55.153 INFO:tasks.workunit.client.0.smithi070.stdout:2/489: mkdir d1/d29/da7 0 2024-03-23T08:23:55.153 INFO:tasks.workunit.client.0.smithi070.stdout:2/490: dread - d1/d29/d52/f92 zero size 2024-03-23T08:23:55.154 INFO:tasks.workunit.client.0.smithi070.stdout:5/414: creat dc/d1b/d1d/d25/d36/d32/f87 x:0 0 0 2024-03-23T08:23:55.156 INFO:tasks.workunit.client.0.smithi070.stdout:1/408: link f3 d8/d26/d49/d65/f98 0 2024-03-23T08:23:55.158 INFO:tasks.workunit.client.0.smithi070.stdout:3/627: mkdir db/d4e/dd6/dd7/dd9/de8 0 2024-03-23T08:23:55.160 INFO:tasks.workunit.client.0.smithi070.stdout:9/557: mknod db/d12/d34/d8c/da2/d4f/d5a/cb4 0 2024-03-23T08:23:55.161 INFO:tasks.workunit.client.0.smithi070.stdout:0/718: symlink d16/d26/d27/d3d/d4d/d9a/df8/lff 0 2024-03-23T08:23:55.162 INFO:tasks.workunit.client.0.smithi070.stdout:9/558: fdatasync db/d12/d34/d8c/da2/d26/da7/fac 0 2024-03-23T08:23:55.165 INFO:tasks.workunit.client.0.smithi070.stdout:1/409: mkdir d8/d42/d4e/d99 0 2024-03-23T08:23:55.166 INFO:tasks.workunit.client.0.smithi070.stdout:1/410: truncate d8/d1b/f86 935845 0 2024-03-23T08:23:55.167 INFO:tasks.workunit.client.0.smithi070.stdout:3/628: mkdir db/df/d1a/de9 0 2024-03-23T08:23:55.168 INFO:tasks.workunit.client.0.smithi070.stdout:7/406: sync 2024-03-23T08:23:55.168 INFO:tasks.workunit.client.0.smithi070.stdout:4/307: sync 2024-03-23T08:23:55.168 INFO:tasks.workunit.client.0.smithi070.stdout:6/554: link d0/l3d d0/d23/d74/d84/db3/d38/d9a/lc2 0 2024-03-23T08:23:55.169 INFO:tasks.workunit.client.0.smithi070.stdout:6/555: stat d0/d23/d74/d84 0 2024-03-23T08:23:55.170 INFO:tasks.workunit.client.0.smithi070.stdout:7/407: write d25/d27/d6a/d44/f79 [297561,125121] 0 2024-03-23T08:23:55.171 INFO:tasks.workunit.client.0.smithi070.stdout:0/719: creat d16/f100 x:0 0 0 2024-03-23T08:23:55.171 INFO:tasks.workunit.client.0.smithi070.stdout:8/497: creat d5/d8/d16/d8d/da3/fbb x:0 0 0 2024-03-23T08:23:55.177 INFO:tasks.workunit.client.0.smithi070.stdout:2/491: dwrite d1/d7/df/f91 [0,4194304] 0 2024-03-23T08:23:55.178 INFO:tasks.workunit.client.0.smithi070.stdout:2/492: fdatasync d1/d29/d52/f92 0 2024-03-23T08:23:55.178 INFO:tasks.workunit.client.0.smithi070.stdout:4/308: mknod d3/d6/d35/d65/d1b/c70 0 2024-03-23T08:23:55.178 INFO:tasks.workunit.client.0.smithi070.stdout:9/559: write db/d12/d14/f63 [2830726,46689] 0 2024-03-23T08:23:55.181 INFO:tasks.workunit.client.0.smithi070.stdout:5/415: link dc/d1b/d1d/d25/d36/c29 dc/d1b/d1d/d25/d36/c88 0 2024-03-23T08:23:55.182 INFO:tasks.workunit.client.0.smithi070.stdout:4/309: read d3/d6/d35/d65/f3e [150458,100929] 0 2024-03-23T08:23:55.183 INFO:tasks.workunit.client.0.smithi070.stdout:4/310: fsync d3/d6/d35/d65/d1b/f1c 0 2024-03-23T08:23:55.184 INFO:tasks.workunit.client.0.smithi070.stdout:4/311: chown d3/d6/l52 0 1 2024-03-23T08:23:55.185 INFO:tasks.workunit.client.0.smithi070.stdout:7/408: creat d25/d27/d6a/d44/d85/f98 x:0 0 0 2024-03-23T08:23:55.185 INFO:tasks.workunit.client.0.smithi070.stdout:3/629: symlink db/d4e/d92/dbb/de2/lea 0 2024-03-23T08:23:55.187 INFO:tasks.workunit.client.0.smithi070.stdout:3/630: write db/d20/d63/f75 [756133,22462] 0 2024-03-23T08:23:55.187 INFO:tasks.workunit.client.0.smithi070.stdout:0/720: mknod d16/d26/d27/d4c/c101 0 2024-03-23T08:23:55.190 INFO:tasks.workunit.client.0.smithi070.stdout:9/560: dread - db/d12/d34/d8c/da2/f7c zero size 2024-03-23T08:23:55.190 INFO:tasks.workunit.client.0.smithi070.stdout:5/416: symlink dc/d1b/d1d/d25/d36/l89 0 2024-03-23T08:23:55.191 INFO:tasks.workunit.client.0.smithi070.stdout:4/312: symlink d3/d6/d35/d65/d41/d56/l71 0 2024-03-23T08:23:55.194 INFO:tasks.workunit.client.0.smithi070.stdout:6/556: rename d0/d23/l6c to d0/lc3 0 2024-03-23T08:23:55.194 INFO:tasks.workunit.client.0.smithi070.stdout:5/417: fdatasync dc/d1b/d1d/d25/d36/d32/f5a 0 2024-03-23T08:23:55.198 INFO:tasks.workunit.client.0.smithi070.stdout:7/409: dwrite d25/f87 [0,4194304] 0 2024-03-23T08:23:55.198 INFO:tasks.workunit.client.0.smithi070.stdout:2/493: getdents d1/d7/d39/d58/d8f 0 2024-03-23T08:23:55.200 INFO:tasks.workunit.client.0.smithi070.stdout:3/631: mkdir db/d20/d49/deb 0 2024-03-23T08:23:55.200 INFO:tasks.workunit.client.0.smithi070.stdout:8/498: creat d5/d8/d16/d30/fbc x:0 0 0 2024-03-23T08:23:55.201 INFO:tasks.workunit.client.0.smithi070.stdout:1/411: getdents d8/d42 0 2024-03-23T08:23:55.202 INFO:tasks.workunit.client.0.smithi070.stdout:0/721: truncate d16/d22/d43/d47/f70 4592298 0 2024-03-23T08:23:55.202 INFO:tasks.workunit.client.0.smithi070.stdout:8/499: fsync d5/d8/d16/d32/f5c 0 2024-03-23T08:23:55.203 INFO:tasks.workunit.client.0.smithi070.stdout:9/561: mkdir db/d12/d34/d8c/da2/d58/db5 0 2024-03-23T08:23:55.204 INFO:tasks.workunit.client.0.smithi070.stdout:7/410: mknod d25/d27/d6a/c99 0 2024-03-23T08:23:55.206 INFO:tasks.workunit.client.0.smithi070.stdout:9/562: write db/d12/d34/d8c/da2/d26/d5c/d7e/f74 [872815,82057] 0 2024-03-23T08:23:55.207 INFO:tasks.workunit.client.0.smithi070.stdout:3/632: chown db/d20/d49/cdc 1 1 2024-03-23T08:23:55.207 INFO:tasks.workunit.client.0.smithi070.stdout:1/412: read d8/d11/d61/d74/f71 [220804,34723] 0 2024-03-23T08:23:55.209 INFO:tasks.workunit.client.0.smithi070.stdout:0/722: unlink d16/f18 0 2024-03-23T08:23:55.211 INFO:tasks.workunit.client.0.smithi070.stdout:5/418: mkdir dc/d1b/d1d/d25/d68/d8a 0 2024-03-23T08:23:55.211 INFO:tasks.workunit.client.0.smithi070.stdout:5/419: chown dc/l24 1287 1 2024-03-23T08:23:55.212 INFO:tasks.workunit.client.0.smithi070.stdout:8/500: rmdir d5/d8/d16/d30/d38/d6b/d72 39 2024-03-23T08:23:55.214 INFO:tasks.workunit.client.0.smithi070.stdout:5/420: chown dc/d1b/d1d/d25/d36/f27 445 1 2024-03-23T08:23:55.214 INFO:tasks.workunit.client.0.smithi070.stdout:2/494: dwrite d1/fe [0,4194304] 0 2024-03-23T08:23:55.218 INFO:tasks.workunit.client.0.smithi070.stdout:7/411: creat d25/d27/d6a/d44/d7c/f9a x:0 0 0 2024-03-23T08:23:55.218 INFO:tasks.workunit.client.0.smithi070.stdout:3/633: truncate db/fbc 238430 0 2024-03-23T08:23:55.220 INFO:tasks.workunit.client.0.smithi070.stdout:5/421: read dc/d1b/d1d/d25/f33 [79436,87968] 0 2024-03-23T08:23:55.220 INFO:tasks.workunit.client.0.smithi070.stdout:4/313: rename d3/d6/d9/l32 to d3/d6/l72 0 2024-03-23T08:23:55.220 INFO:tasks.workunit.client.0.smithi070.stdout:2/495: dread d1/d6/f3b [0,4194304] 0 2024-03-23T08:23:55.221 INFO:tasks.workunit.client.0.smithi070.stdout:2/496: dread - d1/d29/f7f zero size 2024-03-23T08:23:55.224 INFO:tasks.workunit.client.0.smithi070.stdout:9/563: truncate db/d12/d34/d8c/da2/f45 2275186 0 2024-03-23T08:23:55.227 INFO:tasks.workunit.client.0.smithi070.stdout:8/501: write d5/d8/d16/d30/f39 [1244351,116993] 0 2024-03-23T08:23:55.228 INFO:tasks.workunit.client.0.smithi070.stdout:8/502: chown d5/d8/d16/d8d/da3/lb7 2805450 1 2024-03-23T08:23:55.231 INFO:tasks.workunit.client.0.smithi070.stdout:7/412: write d25/d27/d6a/d44/f7e [205717,111533] 0 2024-03-23T08:23:55.233 INFO:tasks.workunit.client.0.smithi070.stdout:2/497: rmdir d1/d7/d48 39 2024-03-23T08:23:55.233 INFO:tasks.workunit.client.0.smithi070.stdout:9/564: dread db/d12/d34/d8c/da2/f24 [4194304,4194304] 0 2024-03-23T08:23:55.235 INFO:tasks.workunit.client.0.smithi070.stdout:7/413: write d25/d27/d6a/f73 [4523724,51495] 0 2024-03-23T08:23:55.237 INFO:tasks.workunit.client.0.smithi070.stdout:0/723: truncate d16/d22/f39 771956 0 2024-03-23T08:23:55.238 INFO:tasks.workunit.client.0.smithi070.stdout:6/557: truncate d0/d23/d74/d84/faa 3128155 0 2024-03-23T08:23:55.245 INFO:tasks.workunit.client.0.smithi070.stdout:4/314: getdents d3/d6/d35/d65/d6f 0 2024-03-23T08:23:55.246 INFO:tasks.workunit.client.0.smithi070.stdout:2/498: dwrite d1/d7/d39/f95 [0,4194304] 0 2024-03-23T08:23:55.248 INFO:tasks.workunit.client.0.smithi070.stdout:1/413: rmdir d8/d42/d51/d6f 0 2024-03-23T08:23:55.253 INFO:tasks.workunit.client.0.smithi070.stdout:6/558: dread d0/d23/d28/d35/d85/f89 [0,4194304] 0 2024-03-23T08:23:55.253 INFO:tasks.workunit.client.0.smithi070.stdout:2/499: truncate d1/d7/d39/f82 614656 0 2024-03-23T08:23:55.254 INFO:tasks.workunit.client.0.smithi070.stdout:7/414: mknod d25/d27/d6a/d44/d7c/c9b 0 2024-03-23T08:23:55.255 INFO:tasks.workunit.client.0.smithi070.stdout:2/500: fdatasync d1/d7/f40 0 2024-03-23T08:23:55.257 INFO:tasks.workunit.client.0.smithi070.stdout:0/724: dwrite d16/d26/d27/f91 [0,4194304] 0 2024-03-23T08:23:55.264 INFO:tasks.workunit.client.0.smithi070.stdout:9/565: rename db/d12/d34/d8c/da2/d26/d5c/d7e/d6c to db/d12/d34/d8c/da2/d4f/db6 0 2024-03-23T08:23:55.265 INFO:tasks.workunit.client.0.smithi070.stdout:3/634: creat db/df/fec x:0 0 0 2024-03-23T08:23:55.267 INFO:tasks.workunit.client.0.smithi070.stdout:4/315: mkdir d3/d6/d35/d65/d27/d4b/d73 0 2024-03-23T08:23:55.272 INFO:tasks.workunit.client.0.smithi070.stdout:8/503: truncate d5/d8/d16/d32/f5c 7951161 0 2024-03-23T08:23:55.274 INFO:tasks.workunit.client.0.smithi070.stdout:6/559: dread d0/d23/d74/d84/d3c/f51 [0,4194304] 0 2024-03-23T08:23:55.279 INFO:tasks.workunit.client.0.smithi070.stdout:7/415: dread - d25/d27/f57 zero size 2024-03-23T08:23:55.279 INFO:tasks.workunit.client.0.smithi070.stdout:4/316: dread d3/d6/f2f [0,4194304] 0 2024-03-23T08:23:55.281 INFO:tasks.workunit.client.0.smithi070.stdout:2/501: creat d1/d21/d5f/fa8 x:0 0 0 2024-03-23T08:23:55.286 INFO:tasks.workunit.client.0.smithi070.stdout:7/416: rmdir d25/d27/d6a/d44/d6f 39 2024-03-23T08:23:55.288 INFO:tasks.workunit.client.0.smithi070.stdout:0/725: stat d16/d26/d27/d86/d67/d7d 0 2024-03-23T08:23:55.288 INFO:tasks.workunit.client.0.smithi070.stdout:7/417: truncate d25/d27/d6a/d44/d85/f93 33421 0 2024-03-23T08:23:55.289 INFO:tasks.workunit.client.0.smithi070.stdout:4/317: chown d3/d6/d11/f2d 590 1 2024-03-23T08:23:55.290 INFO:tasks.workunit.client.0.smithi070.stdout:9/566: mkdir db/d12/db7 0 2024-03-23T08:23:55.294 INFO:tasks.workunit.client.0.smithi070.stdout:2/502: rename d1/d6/d34/f5a to d1/d21/fa9 0 2024-03-23T08:23:55.295 INFO:tasks.workunit.client.0.smithi070.stdout:3/635: mkdir db/d20/d7a/d80/ded 0 2024-03-23T08:23:55.295 INFO:tasks.workunit.client.0.smithi070.stdout:4/318: dread d3/d6/d45/f47 [0,4194304] 0 2024-03-23T08:23:55.300 INFO:tasks.workunit.client.0.smithi070.stdout:7/418: unlink d25/d27/d6a/d44/f7e 0 2024-03-23T08:23:55.302 INFO:tasks.workunit.client.0.smithi070.stdout:3/636: dwrite db/df/d5d/fda [0,4194304] 0 2024-03-23T08:23:55.308 INFO:tasks.workunit.client.0.smithi070.stdout:8/504: truncate d5/d8/f25 664495 0 2024-03-23T08:23:55.313 INFO:tasks.workunit.client.0.smithi070.stdout:9/567: mkdir db/db8 0 2024-03-23T08:23:55.314 INFO:tasks.workunit.client.0.smithi070.stdout:7/419: creat d25/d27/d6a/f9c x:0 0 0 2024-03-23T08:23:55.316 INFO:tasks.workunit.client.0.smithi070.stdout:9/568: write db/d12/d34/d8c/da2/d26/d5c/d7e/f43 [4496244,29031] 0 2024-03-23T08:23:55.317 INFO:tasks.workunit.client.0.smithi070.stdout:4/319: unlink d3/d6/d35/d65/f40 0 2024-03-23T08:23:55.319 INFO:tasks.workunit.client.0.smithi070.stdout:2/503: rename d1/d7/d48/d69/f77 to d1/d8b/faa 0 2024-03-23T08:23:55.320 INFO:tasks.workunit.client.0.smithi070.stdout:9/569: write db/d12/d14/f63 [1357284,13050] 0 2024-03-23T08:23:55.321 INFO:tasks.workunit.client.0.smithi070.stdout:3/637: dread db/df/d6a/fa1 [0,4194304] 0 2024-03-23T08:23:55.322 INFO:tasks.workunit.client.0.smithi070.stdout:8/505: creat d5/d8/d16/d3b/d4d/d6f/fbd x:0 0 0 2024-03-23T08:23:55.323 INFO:tasks.workunit.client.0.smithi070.stdout:7/420: symlink d25/d27/d6a/l9d 0 2024-03-23T08:23:55.325 INFO:tasks.workunit.client.0.smithi070.stdout:3/638: write db/df/d15/d22/fc5 [3015653,31480] 0 2024-03-23T08:23:55.328 INFO:tasks.workunit.client.0.smithi070.stdout:2/504: dwrite d1/d7/d39/f9c [0,4194304] 0 2024-03-23T08:23:55.339 INFO:tasks.workunit.client.0.smithi070.stdout:4/320: mkdir d3/d6/d35/d65/d27/d4b/d73/d74 0 2024-03-23T08:23:55.339 INFO:tasks.workunit.client.0.smithi070.stdout:4/321: readlink d3/d6/d35/l44 0 2024-03-23T08:23:55.341 INFO:tasks.workunit.client.0.smithi070.stdout:4/322: write d3/d6/d35/d65/d27/f4e [998683,54479] 0 2024-03-23T08:23:55.342 INFO:tasks.workunit.client.0.smithi070.stdout:9/570: write db/d12/d34/d8c/da2/d4f/f72 [967040,129682] 0 2024-03-23T08:23:55.346 INFO:tasks.workunit.client.0.smithi070.stdout:8/506: mkdir d5/d8/d9a/dbe 0 2024-03-23T08:23:55.350 INFO:tasks.workunit.client.0.smithi070.stdout:4/323: dwrite d3/f6d [0,4194304] 0 2024-03-23T08:23:55.354 INFO:tasks.workunit.client.0.smithi070.stdout:7/421: dwrite d25/f34 [0,4194304] 0 2024-03-23T08:23:55.354 INFO:tasks.workunit.client.0.smithi070.stdout:3/639: truncate db/d14/f3f 2143924 0 2024-03-23T08:23:55.361 INFO:tasks.workunit.client.0.smithi070.stdout:4/324: getdents d3/d6/d35/d65/d6f 0 2024-03-23T08:23:55.363 INFO:tasks.workunit.client.0.smithi070.stdout:4/325: chown d3/d6/f7 5 1 2024-03-23T08:23:55.364 INFO:tasks.workunit.client.0.smithi070.stdout:4/326: chown d3/d6/f6c 1 1 2024-03-23T08:23:55.364 INFO:tasks.workunit.client.0.smithi070.stdout:2/505: dwrite d1/d7/df/f4b [0,4194304] 0 2024-03-23T08:23:55.364 INFO:tasks.workunit.client.0.smithi070.stdout:3/640: unlink db/fbc 0 2024-03-23T08:23:55.369 INFO:tasks.workunit.client.0.smithi070.stdout:7/422: mkdir d25/d27/d6a/d44/d85/d95/d9e 0 2024-03-23T08:23:55.373 INFO:tasks.workunit.client.0.smithi070.stdout:4/327: creat d3/d6/d35/d65/f75 x:0 0 0 2024-03-23T08:23:55.374 INFO:tasks.workunit.client.0.smithi070.stdout:2/506: mknod d1/d29/d52/cab 0 2024-03-23T08:23:55.375 INFO:tasks.workunit.client.0.smithi070.stdout:4/328: dread - d3/d6/d9/f6b zero size 2024-03-23T08:23:55.375 INFO:tasks.workunit.client.0.smithi070.stdout:2/507: fsync d1/d6/f72 0 2024-03-23T08:23:55.377 INFO:tasks.workunit.client.0.smithi070.stdout:7/423: mknod d25/d27/d6a/d44/d7c/c9f 0 2024-03-23T08:23:55.381 INFO:tasks.workunit.client.0.smithi070.stdout:2/508: creat d1/d7/df/d99/fac x:0 0 0 2024-03-23T08:23:55.387 INFO:tasks.workunit.client.0.smithi070.stdout:2/509: rename d1/d7/df/d94/d38/d4c/f5c to d1/d6/d34/fad 0 2024-03-23T08:23:55.389 INFO:tasks.workunit.client.0.smithi070.stdout:4/329: dread d3/fd [0,4194304] 0 2024-03-23T08:23:55.390 INFO:tasks.workunit.client.0.smithi070.stdout:7/424: dread d25/f38 [0,4194304] 0 2024-03-23T08:23:55.395 INFO:tasks.workunit.client.0.smithi070.stdout:2/510: unlink d1/d7/f61 0 2024-03-23T08:23:55.398 INFO:tasks.workunit.client.0.smithi070.stdout:7/425: mkdir d25/d27/d6a/d44/d85/d95/da0 0 2024-03-23T08:23:55.401 INFO:tasks.workunit.client.0.smithi070.stdout:4/330: dread d3/d6/d11/f13 [0,4194304] 0 2024-03-23T08:23:55.402 INFO:tasks.workunit.client.0.smithi070.stdout:7/426: dread d25/f43 [0,4194304] 0 2024-03-23T08:23:55.455 INFO:tasks.workunit.client.0.smithi070.stdout:2/511: symlink d1/lae 0 2024-03-23T08:23:55.457 INFO:tasks.workunit.client.0.smithi070.stdout:4/331: dwrite d3/d6/d35/d65/d41/d56/f5a [0,4194304] 0 2024-03-23T08:23:55.459 INFO:tasks.workunit.client.0.smithi070.stdout:4/332: read d3/d6/d6a/d2a/f2b [576735,27493] 0 2024-03-23T08:23:55.461 INFO:tasks.workunit.client.0.smithi070.stdout:7/427: creat d25/d27/d6a/d44/d6f/fa1 x:0 0 0 2024-03-23T08:23:55.464 INFO:tasks.workunit.client.0.smithi070.stdout:4/333: creat d3/d6/d35/f76 x:0 0 0 2024-03-23T08:23:55.464 INFO:tasks.workunit.client.0.smithi070.stdout:3/641: dread db/d14/f3f [0,4194304] 0 2024-03-23T08:23:55.466 INFO:tasks.workunit.client.0.smithi070.stdout:7/428: chown d25/l2d 31 1 2024-03-23T08:23:55.468 INFO:tasks.workunit.client.0.smithi070.stdout:2/512: dwrite d1/f9e [0,4194304] 0 2024-03-23T08:23:55.469 INFO:tasks.workunit.client.0.smithi070.stdout:7/429: truncate d25/d27/d6a/f5f 436736 0 2024-03-23T08:23:55.470 INFO:tasks.workunit.client.0.smithi070.stdout:2/513: write d1/d7/df/d94/f28 [817824,104958] 0 2024-03-23T08:23:55.476 INFO:tasks.workunit.client.0.smithi070.stdout:2/514: mknod d1/d7/df/d99/caf 0 2024-03-23T08:23:55.478 INFO:tasks.workunit.client.0.smithi070.stdout:7/430: read d25/d40/f4b [361868,76644] 0 2024-03-23T08:23:55.479 INFO:tasks.workunit.client.0.smithi070.stdout:3/642: rename db/d14/d43/f5a to db/d20/fee 0 2024-03-23T08:23:55.480 INFO:tasks.workunit.client.0.smithi070.stdout:7/431: dread d25/d40/f76 [0,4194304] 0 2024-03-23T08:23:55.482 INFO:tasks.workunit.client.0.smithi070.stdout:2/515: creat d1/d7/df/d94/fb0 x:0 0 0 2024-03-23T08:23:55.491 INFO:tasks.workunit.client.0.smithi070.stdout:4/334: rename d3/d6/d35/d65/d41/d56/l71 to d3/d6/d11/l77 0 2024-03-23T08:23:55.492 INFO:tasks.workunit.client.0.smithi070.stdout:7/432: dwrite d25/f83 [0,4194304] 0 2024-03-23T08:23:55.493 INFO:tasks.workunit.client.0.smithi070.stdout:7/433: read d25/f87 [3071706,75863] 0 2024-03-23T08:23:55.494 INFO:tasks.workunit.client.0.smithi070.stdout:7/434: chown d25/d27/f48 5546066 1 2024-03-23T08:23:55.495 INFO:tasks.workunit.client.0.smithi070.stdout:2/516: rename d1/d7/df/d94/f2b to d1/d7/d39/d47/fb1 0 2024-03-23T08:23:55.497 INFO:tasks.workunit.client.0.smithi070.stdout:2/517: write d1/d6/d4f/f3e [748332,82956] 0 2024-03-23T08:23:55.500 INFO:tasks.workunit.client.0.smithi070.stdout:3/643: dread db/df/d15/d22/db2/fc7 [0,4194304] 0 2024-03-23T08:23:55.501 INFO:tasks.workunit.client.0.smithi070.stdout:7/435: dread f12 [0,4194304] 0 2024-03-23T08:23:55.503 INFO:tasks.workunit.client.0.smithi070.stdout:4/335: rename d3/d6/d35/c49 to d3/d6/d35/d65/d27/d4b/d73/d74/c78 0 2024-03-23T08:23:55.509 INFO:tasks.workunit.client.0.smithi070.stdout:2/518: mknod d1/d7/d39/cb2 0 2024-03-23T08:23:55.511 INFO:tasks.workunit.client.0.smithi070.stdout:4/336: readlink d3/d6/d11/l77 0 2024-03-23T08:23:55.513 INFO:tasks.workunit.client.0.smithi070.stdout:4/337: truncate d3/d6/d35/f76 607407 0 2024-03-23T08:23:55.515 INFO:tasks.workunit.client.0.smithi070.stdout:3/644: rename db/le to db/df/d15/d22/db2/dba/lef 0 2024-03-23T08:23:55.519 INFO:tasks.workunit.client.0.smithi070.stdout:7/436: mkdir d25/d27/d6a/d44/da2 0 2024-03-23T08:23:55.519 INFO:tasks.workunit.client.0.smithi070.stdout:7/437: readlink d25/d27/d6a/l9d 0 2024-03-23T08:23:55.523 INFO:tasks.workunit.client.0.smithi070.stdout:2/519: rename d1/d6/d4f/l50 to d1/d7/d39/d47/d90/lb3 0 2024-03-23T08:23:55.526 INFO:tasks.workunit.client.0.smithi070.stdout:2/520: read - d1/d7/df/d94/fb0 zero size 2024-03-23T08:23:55.527 INFO:tasks.workunit.client.0.smithi070.stdout:7/438: dwrite f23 [0,4194304] 0 2024-03-23T08:23:55.529 INFO:tasks.workunit.client.0.smithi070.stdout:2/521: truncate d1/d29/d52/f9f 4312429 0 2024-03-23T08:23:55.529 INFO:tasks.workunit.client.0.smithi070.stdout:4/338: write d3/d6/d11/f13 [869909,95493] 0 2024-03-23T08:23:55.529 INFO:tasks.workunit.client.0.smithi070.stdout:7/439: truncate d25/d40/f8f 1012410 0 2024-03-23T08:23:55.531 INFO:tasks.workunit.client.0.smithi070.stdout:4/339: write d3/d6/d9/f4a [234983,42307] 0 2024-03-23T08:23:55.538 INFO:tasks.workunit.client.0.smithi070.stdout:4/340: creat d3/d6/d35/d65/d41/f79 x:0 0 0 2024-03-23T08:23:55.540 INFO:tasks.workunit.client.0.smithi070.stdout:3/645: dwrite db/d14/d65/f78 [0,4194304] 0 2024-03-23T08:23:55.543 INFO:tasks.workunit.client.0.smithi070.stdout:2/522: creat d1/d7/d39/d47/d7d/fb4 x:0 0 0 2024-03-23T08:23:55.543 INFO:tasks.workunit.client.0.smithi070.stdout:4/341: mknod d3/c7a 0 2024-03-23T08:23:55.546 INFO:tasks.workunit.client.0.smithi070.stdout:2/523: chown d1/d7/c9 107 1 2024-03-23T08:23:55.546 INFO:tasks.workunit.client.0.smithi070.stdout:3/646: getdents db/d20/d7a/d80/ded 0 2024-03-23T08:23:55.546 INFO:tasks.workunit.client.0.smithi070.stdout:7/440: dwrite d25/f84 [0,4194304] 0 2024-03-23T08:23:55.548 INFO:tasks.workunit.client.0.smithi070.stdout:2/524: chown d1/d7/d48/d5b/c6c 201 1 2024-03-23T08:23:55.550 INFO:tasks.workunit.client.0.smithi070.stdout:3/647: creat db/d7f/dcd/ff0 x:0 0 0 2024-03-23T08:23:55.553 INFO:tasks.workunit.client.0.smithi070.stdout:4/342: rename d3/d6/d35/d65/d41/d56/f5a to d3/d6/d35/f7b 0 2024-03-23T08:23:55.553 INFO:tasks.workunit.client.0.smithi070.stdout:3/648: rename db/d4e/dd6 to db/d4e/dd6/dd7/dd9/df1 22 2024-03-23T08:23:55.554 INFO:tasks.workunit.client.0.smithi070.stdout:7/441: mkdir d25/d27/d6a/d44/d6f/da3 0 2024-03-23T08:23:55.556 INFO:tasks.workunit.client.0.smithi070.stdout:7/442: write d25/d27/d6a/d44/d85/f93 [647999,126491] 0 2024-03-23T08:23:55.557 INFO:tasks.workunit.client.0.smithi070.stdout:4/343: mkdir d3/d6/d6a/d7c 0 2024-03-23T08:23:55.563 INFO:tasks.workunit.client.0.smithi070.stdout:7/443: dread d25/d27/f4c [0,4194304] 0 2024-03-23T08:23:55.564 INFO:tasks.workunit.client.0.smithi070.stdout:3/649: mknod db/df/d4d/cf2 0 2024-03-23T08:23:55.566 INFO:tasks.workunit.client.0.smithi070.stdout:4/344: dwrite d3/d6/d35/d65/d27/f46 [0,4194304] 0 2024-03-23T08:23:55.570 INFO:tasks.workunit.client.0.smithi070.stdout:7/444: mkdir d25/d27/d6a/d44/da4 0 2024-03-23T08:23:55.572 INFO:tasks.workunit.client.0.smithi070.stdout:3/650: dwrite db/d14/d65/daa/fad [4194304,4194304] 0 2024-03-23T08:23:55.574 INFO:tasks.workunit.client.0.smithi070.stdout:3/651: stat db/d14/f3f 0 2024-03-23T08:23:55.575 INFO:tasks.workunit.client.0.smithi070.stdout:4/345: creat d3/d6/d35/d65/d27/d4b/f7d x:0 0 0 2024-03-23T08:23:55.577 INFO:tasks.workunit.client.0.smithi070.stdout:4/346: write d3/d6/d35/f67 [438668,18833] 0 2024-03-23T08:23:55.581 INFO:tasks.workunit.client.0.smithi070.stdout:2/525: truncate d1/f9e 3084976 0 2024-03-23T08:23:55.582 INFO:tasks.workunit.client.0.smithi070.stdout:2/526: write d1/d6/d4f/f3e [1656972,46313] 0 2024-03-23T08:23:55.584 INFO:tasks.workunit.client.0.smithi070.stdout:3/652: rename db/d14/d65/f77 to db/d14/d9c/ff3 0 2024-03-23T08:23:55.586 INFO:tasks.workunit.client.0.smithi070.stdout:4/347: mkdir d3/d6/d35/d65/d27/d4b/d73/d74/d7e 0 2024-03-23T08:23:55.589 INFO:tasks.workunit.client.0.smithi070.stdout:2/527: creat d1/d21/fb5 x:0 0 0 2024-03-23T08:23:55.590 INFO:tasks.workunit.client.0.smithi070.stdout:3/653: creat db/df/d1a/ff4 x:0 0 0 2024-03-23T08:23:55.591 INFO:tasks.workunit.client.0.smithi070.stdout:2/528: fsync d1/d7/df/d94/d38/d4c/f6a 0 2024-03-23T08:23:55.591 INFO:tasks.workunit.client.0.smithi070.stdout:3/654: write db/df/d1a/f3c [1406552,20297] 0 2024-03-23T08:23:55.593 INFO:tasks.workunit.client.0.smithi070.stdout:4/348: rmdir d3/d6/d35/d65/d41 39 2024-03-23T08:23:55.595 INFO:tasks.workunit.client.0.smithi070.stdout:4/349: write d3/d6/d9/f4a [1863236,30384] 0 2024-03-23T08:23:55.596 INFO:tasks.workunit.client.0.smithi070.stdout:4/350: write d3/f25 [5197101,26356] 0 2024-03-23T08:23:55.597 INFO:tasks.workunit.client.0.smithi070.stdout:2/529: mknod d1/d7/d39/d47/d7d/cb6 0 2024-03-23T08:23:55.598 INFO:tasks.workunit.client.0.smithi070.stdout:4/351: write d3/f6d [3615945,1639] 0 2024-03-23T08:23:55.598 INFO:tasks.workunit.client.0.smithi070.stdout:2/530: chown d1/d21/d5f 184796 1 2024-03-23T08:23:55.600 INFO:tasks.workunit.client.0.smithi070.stdout:3/655: rename db/df/d15/c6e to db/df/d5d/cf5 0 2024-03-23T08:23:55.603 INFO:tasks.workunit.client.0.smithi070.stdout:2/531: fsync d1/d7/f20 0 2024-03-23T08:23:55.604 INFO:tasks.workunit.client.0.smithi070.stdout:4/352: dread d3/d6/d6a/d2a/f2b [0,4194304] 0 2024-03-23T08:23:55.606 INFO:tasks.workunit.client.0.smithi070.stdout:2/532: mknod d1/d7/df/cb7 0 2024-03-23T08:23:55.607 INFO:tasks.workunit.client.0.smithi070.stdout:3/656: dwrite db/d14/d9c/f9f [0,4194304] 0 2024-03-23T08:23:55.608 INFO:tasks.workunit.client.0.smithi070.stdout:3/657: readlink db/df/d4d/l4f 0 2024-03-23T08:23:55.614 INFO:tasks.workunit.client.0.smithi070.stdout:3/658: rename db/c6d to db/df/d15/d22/db2/dba/cf6 0 2024-03-23T08:23:55.615 INFO:tasks.workunit.client.0.smithi070.stdout:4/353: dwrite d3/d6/d35/d65/d1b/f1c [0,4194304] 0 2024-03-23T08:23:55.615 INFO:tasks.workunit.client.0.smithi070.stdout:2/533: dwrite d1/fe [0,4194304] 0 2024-03-23T08:23:55.618 INFO:tasks.workunit.client.0.smithi070.stdout:4/354: creat d3/d6/d9/f7f x:0 0 0 2024-03-23T08:23:55.622 INFO:tasks.workunit.client.0.smithi070.stdout:3/659: dread db/df/f55 [0,4194304] 0 2024-03-23T08:23:55.624 INFO:tasks.workunit.client.0.smithi070.stdout:2/534: dwrite d1/d29/d52/f9f [0,4194304] 0 2024-03-23T08:23:55.631 INFO:tasks.workunit.client.0.smithi070.stdout:2/535: mkdir d1/d7/d48/d69/db8 0 2024-03-23T08:23:55.633 INFO:tasks.workunit.client.0.smithi070.stdout:3/660: write db/df/f8d [4744560,82215] 0 2024-03-23T08:23:55.633 INFO:tasks.workunit.client.0.smithi070.stdout:2/536: readlink d1/d6/d34/l8c 0 2024-03-23T08:23:55.637 INFO:tasks.workunit.client.0.smithi070.stdout:2/537: symlink d1/d7/d48/d69/d7a/lb9 0 2024-03-23T08:23:55.638 INFO:tasks.workunit.client.0.smithi070.stdout:3/661: creat db/df/d1a/de9/ff7 x:0 0 0 2024-03-23T08:23:55.639 INFO:tasks.workunit.client.0.smithi070.stdout:2/538: mknod d1/d7/df/cba 0 2024-03-23T08:23:55.645 INFO:tasks.workunit.client.0.smithi070.stdout:2/539: getdents d1/d7/df/d94/d38/d4c/d76 0 2024-03-23T08:23:55.646 INFO:tasks.workunit.client.0.smithi070.stdout:2/540: rename d1 to d1/d7/df/d94/dbb 22 2024-03-23T08:23:55.647 INFO:tasks.workunit.client.0.smithi070.stdout:3/662: getdents db/df/d15/d3d 0 2024-03-23T08:23:55.650 INFO:tasks.workunit.client.0.smithi070.stdout:7/445: truncate f23 3237998 0 2024-03-23T08:23:55.653 INFO:tasks.workunit.client.0.smithi070.stdout:7/446: creat d25/d40/fa5 x:0 0 0 2024-03-23T08:23:55.657 INFO:tasks.workunit.client.0.smithi070.stdout:7/447: dread d25/f43 [0,4194304] 0 2024-03-23T08:23:55.657 INFO:tasks.workunit.client.0.smithi070.stdout:7/448: readlink d25/l2d 0 2024-03-23T08:23:55.659 INFO:tasks.workunit.client.0.smithi070.stdout:7/449: symlink d25/d40/la6 0 2024-03-23T08:23:55.661 INFO:tasks.workunit.client.0.smithi070.stdout:7/450: fdatasync d25/d27/f57 0 2024-03-23T08:23:55.663 INFO:tasks.workunit.client.0.smithi070.stdout:7/451: dread - d25/d27/d6a/f5b zero size 2024-03-23T08:23:55.670 INFO:tasks.workunit.client.0.smithi070.stdout:7/452: dwrite f12 [0,4194304] 0 2024-03-23T08:23:55.677 INFO:tasks.workunit.client.0.smithi070.stdout:4/355: write d3/f23 [462623,43265] 0 2024-03-23T08:23:55.678 INFO:tasks.workunit.client.0.smithi070.stdout:2/541: rmdir d1/d7/d48/d69/d7a 39 2024-03-23T08:23:55.682 INFO:tasks.workunit.client.0.smithi070.stdout:4/356: mknod d3/d6/d35/d65/d27/d4b/d73/d74/c80 0 2024-03-23T08:23:55.683 INFO:tasks.workunit.client.0.smithi070.stdout:4/357: write d3/f25 [6066632,44898] 0 2024-03-23T08:23:55.684 INFO:tasks.workunit.client.0.smithi070.stdout:4/358: write d3/f6d [2716669,46980] 0 2024-03-23T08:23:55.685 INFO:tasks.workunit.client.0.smithi070.stdout:4/359: write d3/d6/d35/d65/d27/d4b/f7d [754246,21398] 0 2024-03-23T08:23:55.687 INFO:tasks.workunit.client.0.smithi070.stdout:4/360: truncate d3/d6/f43 584250 0 2024-03-23T08:23:55.688 INFO:tasks.workunit.client.0.smithi070.stdout:4/361: write d3/d6/d35/f67 [1493471,94370] 0 2024-03-23T08:23:55.690 INFO:tasks.workunit.client.0.smithi070.stdout:4/362: write d3/d6/d35/f76 [925799,57912] 0 2024-03-23T08:23:55.701 INFO:tasks.workunit.client.0.smithi070.stdout:3/663: dwrite db/df/d15/d22/f2c [0,4194304] 0 2024-03-23T08:23:55.702 INFO:tasks.workunit.client.0.smithi070.stdout:3/664: write db/f21 [1909336,54717] 0 2024-03-23T08:23:55.706 INFO:tasks.workunit.client.0.smithi070.stdout:7/453: write d25/d40/f76 [766130,109049] 0 2024-03-23T08:23:55.709 INFO:tasks.workunit.client.0.smithi070.stdout:7/454: read d25/d27/d6a/d44/f56 [153130,33624] 0 2024-03-23T08:23:55.713 INFO:tasks.workunit.client.0.smithi070.stdout:7/455: write d25/d27/d6a/d44/f56 [368624,121204] 0 2024-03-23T08:23:55.717 INFO:tasks.workunit.client.0.smithi070.stdout:4/363: dread d3/d6/d11/f18 [0,4194304] 0 2024-03-23T08:23:55.718 INFO:tasks.workunit.client.0.smithi070.stdout:7/456: creat d25/d27/d6a/d44/d6f/da3/fa7 x:0 0 0 2024-03-23T08:23:55.718 INFO:tasks.workunit.client.0.smithi070.stdout:3/665: dread db/d20/fee [0,4194304] 0 2024-03-23T08:23:55.721 INFO:tasks.workunit.client.0.smithi070.stdout:4/364: symlink d3/d6/d35/d65/d27/d4b/d73/l81 0 2024-03-23T08:23:55.722 INFO:tasks.workunit.client.0.smithi070.stdout:7/457: dread d25/d27/d6a/d44/d7c/f8b [0,4194304] 0 2024-03-23T08:23:55.725 INFO:tasks.workunit.client.0.smithi070.stdout:4/365: rename d3/fd to d3/d6/d35/d65/d6f/f82 0 2024-03-23T08:23:55.729 INFO:tasks.workunit.client.0.smithi070.stdout:7/458: creat d25/d27/d6a/d44/fa8 x:0 0 0 2024-03-23T08:23:55.732 INFO:tasks.workunit.client.0.smithi070.stdout:4/366: dwrite d3/d6/d11/f55 [0,4194304] 0 2024-03-23T08:23:55.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:55 smithi070 ceph-mon[39699]: pgmap v37: 65 pgs: 65 active+clean; 1.1 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 23 MiB/s rd, 118 MiB/s wr, 257 op/s 2024-03-23T08:23:55.737 INFO:tasks.workunit.client.0.smithi070.stdout:7/459: link d25/l42 d25/d40/la9 0 2024-03-23T08:23:55.741 INFO:tasks.workunit.client.0.smithi070.stdout:7/460: mknod d25/d27/d6a/d44/d6f/caa 0 2024-03-23T08:23:55.743 INFO:tasks.workunit.client.0.smithi070.stdout:7/461: mkdir d25/d27/d6a/d44/d6f/dab 0 2024-03-23T08:23:55.745 INFO:tasks.workunit.client.0.smithi070.stdout:7/462: dread - d25/d27/f5c zero size 2024-03-23T08:23:55.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:55 smithi193 ceph-mon[48370]: pgmap v37: 65 pgs: 65 active+clean; 1.1 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 23 MiB/s rd, 118 MiB/s wr, 257 op/s 2024-03-23T08:23:55.877 INFO:tasks.workunit.client.0.smithi070.stdout:5/422: sync 2024-03-23T08:23:55.877 INFO:tasks.workunit.client.0.smithi070.stdout:1/414: sync 2024-03-23T08:23:55.877 INFO:tasks.workunit.client.0.smithi070.stdout:1/415: readlink d8/d1b/l3a 0 2024-03-23T08:23:55.878 INFO:tasks.workunit.client.0.smithi070.stdout:1/416: dread - d8/d11/f5c zero size 2024-03-23T08:23:55.885 INFO:tasks.workunit.client.0.smithi070.stdout:3/666: write db/d20/fee [1614265,99034] 0 2024-03-23T08:23:55.910 INFO:tasks.workunit.client.0.smithi070.stdout:5/423: dread dc/f58 [0,4194304] 0 2024-03-23T08:23:55.912 INFO:tasks.workunit.client.0.smithi070.stdout:5/424: write dc/f2b [3697447,89399] 0 2024-03-23T08:23:55.914 INFO:tasks.workunit.client.0.smithi070.stdout:5/425: fdatasync dc/d1b/d1d/d25/d38/d39/f6b 0 2024-03-23T08:23:55.922 INFO:tasks.workunit.client.0.smithi070.stdout:4/367: truncate d3/f25 4513260 0 2024-03-23T08:23:55.926 INFO:tasks.workunit.client.0.smithi070.stdout:5/426: dwrite dc/d1b/d1d/d25/d38/d48/f57 [0,4194304] 0 2024-03-23T08:23:55.938 INFO:tasks.workunit.client.0.smithi070.stdout:5/427: dwrite dc/d1b/d1d/d25/d36/d32/d5c/f5d [0,4194304] 0 2024-03-23T08:23:55.941 INFO:tasks.workunit.client.0.smithi070.stdout:4/368: rename d3/d6/d35/d65/d41/d56/c64 to d3/d6/d35/c83 0 2024-03-23T08:23:55.943 INFO:tasks.workunit.client.0.smithi070.stdout:1/417: getdents d8/db/d2c/d3f/d48 0 2024-03-23T08:23:55.944 INFO:tasks.workunit.client.0.smithi070.stdout:3/667: getdents db/d20 0 2024-03-23T08:23:55.948 INFO:tasks.workunit.client.0.smithi070.stdout:5/428: dwrite dc/d1b/d1d/d25/d38/f73 [0,4194304] 0 2024-03-23T08:23:55.948 INFO:tasks.workunit.client.0.smithi070.stdout:4/369: rename d3/d6/d6a/d2a to d3/d6/d35/d65/d27/d4b/d73/d84 0 2024-03-23T08:23:55.953 INFO:tasks.workunit.client.0.smithi070.stdout:4/370: mknod d3/d6/d35/d65/d27/d4b/d73/d84/c85 0 2024-03-23T08:23:55.954 INFO:tasks.workunit.client.0.smithi070.stdout:5/429: getdents dc/d1b/d1d/d25/d68/d8a 0 2024-03-23T08:23:55.957 INFO:tasks.workunit.client.0.smithi070.stdout:3/668: getdents db/df/d15/d22 0 2024-03-23T08:23:55.958 INFO:tasks.workunit.client.0.smithi070.stdout:5/430: unlink dc/d1b/d1d/d25/f33 0 2024-03-23T08:23:55.962 INFO:tasks.workunit.client.0.smithi070.stdout:4/371: dwrite d3/d6/f38 [0,4194304] 0 2024-03-23T08:23:55.967 INFO:tasks.workunit.client.0.smithi070.stdout:5/431: dread dc/f58 [0,4194304] 0 2024-03-23T08:23:55.968 INFO:tasks.workunit.client.0.smithi070.stdout:5/432: fsync dc/d1b/d1d/d25/d38/d53/f60 0 2024-03-23T08:23:55.969 INFO:tasks.workunit.client.0.smithi070.stdout:5/433: chown dc/d19/c7b 31 1 2024-03-23T08:23:55.970 INFO:tasks.workunit.client.0.smithi070.stdout:4/372: read d3/f17 [436697,47419] 0 2024-03-23T08:23:55.970 INFO:tasks.workunit.client.0.smithi070.stdout:6/560: sync 2024-03-23T08:23:55.971 INFO:tasks.workunit.client.0.smithi070.stdout:0/726: sync 2024-03-23T08:23:55.972 INFO:tasks.workunit.client.0.smithi070.stdout:5/434: symlink dc/d19/l8b 0 2024-03-23T08:23:55.972 INFO:tasks.workunit.client.0.smithi070.stdout:5/435: fdatasync dc/d1b/f54 0 2024-03-23T08:23:55.976 INFO:tasks.workunit.client.0.smithi070.stdout:4/373: symlink d3/d6/d35/d65/d41/d56/l86 0 2024-03-23T08:23:55.976 INFO:tasks.workunit.client.0.smithi070.stdout:6/561: rmdir d0/d23/d74/d84/d3c 39 2024-03-23T08:23:55.977 INFO:tasks.workunit.client.0.smithi070.stdout:6/562: fsync d0/d23/d28/d35/d85/f9e 0 2024-03-23T08:23:55.979 INFO:tasks.workunit.client.0.smithi070.stdout:5/436: symlink dc/d1b/d1d/d61/l8c 0 2024-03-23T08:23:55.983 INFO:tasks.workunit.client.0.smithi070.stdout:4/374: dwrite d3/d6/d35/f67 [0,4194304] 0 2024-03-23T08:23:55.990 INFO:tasks.workunit.client.0.smithi070.stdout:4/375: dwrite d3/d6/f38 [0,4194304] 0 2024-03-23T08:23:55.993 INFO:tasks.workunit.client.0.smithi070.stdout:4/376: chown d3/d6/d11/f2d 1620 1 2024-03-23T08:23:55.998 INFO:tasks.workunit.client.0.smithi070.stdout:5/437: rename dc/c13 to dc/d1b/d1d/d25/d36/c8d 0 2024-03-23T08:23:55.998 INFO:tasks.workunit.client.0.smithi070.stdout:6/563: link d0/d23/c14 d0/d23/d74/d84/db3/d38/cc4 0 2024-03-23T08:23:55.999 INFO:tasks.workunit.client.0.smithi070.stdout:5/438: write dc/d1b/d1d/d25/d38/d39/f79 [403339,688] 0 2024-03-23T08:23:56.012 INFO:tasks.workunit.client.0.smithi070.stdout:1/418: dwrite d8/d11/d61/d74/f71 [0,4194304] 0 2024-03-23T08:23:56.012 INFO:tasks.workunit.client.0.smithi070.stdout:5/439: dread dc/d1b/d1d/d25/d38/d39/f66 [0,4194304] 0 2024-03-23T08:23:56.013 INFO:tasks.workunit.client.0.smithi070.stdout:5/440: chown dc/d1b/d1d/d25/d38/f40 3510 1 2024-03-23T08:23:56.020 INFO:tasks.workunit.client.0.smithi070.stdout:3/669: dwrite db/d4e/fb7 [0,4194304] 0 2024-03-23T08:23:56.027 INFO:tasks.workunit.client.0.smithi070.stdout:4/377: dread d3/d6/d35/d65/d1b/f28 [0,4194304] 0 2024-03-23T08:23:56.027 INFO:tasks.workunit.client.0.smithi070.stdout:7/463: sync 2024-03-23T08:23:56.027 INFO:tasks.workunit.client.0.smithi070.stdout:2/542: sync 2024-03-23T08:23:56.028 INFO:tasks.workunit.client.0.smithi070.stdout:8/507: sync 2024-03-23T08:23:56.028 INFO:tasks.workunit.client.0.smithi070.stdout:9/571: sync 2024-03-23T08:23:56.029 INFO:tasks.workunit.client.0.smithi070.stdout:0/727: write d16/f51 [1280477,109069] 0 2024-03-23T08:23:56.037 INFO:tasks.workunit.client.0.smithi070.stdout:6/564: write d0/d23/d74/d84/db3/d2f/f30 [248235,83098] 0 2024-03-23T08:23:56.041 INFO:tasks.workunit.client.0.smithi070.stdout:0/728: fsync d16/d22/fd3 0 2024-03-23T08:23:56.046 INFO:tasks.workunit.client.0.smithi070.stdout:4/378: creat d3/d6/d35/d3f/f87 x:0 0 0 2024-03-23T08:23:56.048 INFO:tasks.workunit.client.0.smithi070.stdout:4/379: stat d3/d6/d35/d65/d1b/f59 0 2024-03-23T08:23:56.052 INFO:tasks.workunit.client.0.smithi070.stdout:1/419: dwrite d8/d42/d4e/f66 [0,4194304] 0 2024-03-23T08:23:56.053 INFO:tasks.workunit.client.0.smithi070.stdout:7/464: dread d25/d27/f48 [0,4194304] 0 2024-03-23T08:23:56.053 INFO:tasks.workunit.client.0.smithi070.stdout:3/670: write db/d14/d43/f54 [3574822,123041] 0 2024-03-23T08:23:56.053 INFO:tasks.workunit.client.0.smithi070.stdout:5/441: truncate dc/f3d 568498 0 2024-03-23T08:23:56.055 INFO:tasks.workunit.client.0.smithi070.stdout:9/572: symlink db/d12/d34/d8c/da2/d4f/db6/d79/lb9 0 2024-03-23T08:23:56.056 INFO:tasks.workunit.client.0.smithi070.stdout:5/442: write f8 [204400,47626] 0 2024-03-23T08:23:56.058 INFO:tasks.workunit.client.0.smithi070.stdout:9/573: dread - db/d12/d22/fb0 zero size 2024-03-23T08:23:56.058 INFO:tasks.workunit.client.0.smithi070.stdout:5/443: chown dc/d19/f2c 241 1 2024-03-23T08:23:56.063 INFO:tasks.workunit.client.0.smithi070.stdout:8/508: write d5/d8/f4a [882747,104813] 0 2024-03-23T08:23:56.063 INFO:tasks.workunit.client.0.smithi070.stdout:2/543: dwrite d1/d7/d48/d69/d7a/fa5 [0,4194304] 0 2024-03-23T08:23:56.066 INFO:tasks.workunit.client.0.smithi070.stdout:8/509: write d5/d8/d16/d3b/faa [949725,118650] 0 2024-03-23T08:23:56.068 INFO:tasks.workunit.client.0.smithi070.stdout:2/544: chown d1/d7/d48/d69 15 1 2024-03-23T08:23:56.070 INFO:tasks.workunit.client.0.smithi070.stdout:7/465: fdatasync d25/f87 0 2024-03-23T08:23:56.078 INFO:tasks.workunit.client.0.smithi070.stdout:3/671: dread db/df/d15/d22/fc5 [0,4194304] 0 2024-03-23T08:23:56.079 INFO:tasks.workunit.client.0.smithi070.stdout:5/444: dwrite dc/d1b/f54 [0,4194304] 0 2024-03-23T08:23:56.083 INFO:tasks.workunit.client.0.smithi070.stdout:2/545: dwrite d1/fc [0,4194304] 0 2024-03-23T08:23:56.087 INFO:tasks.workunit.client.0.smithi070.stdout:9/574: creat db/d12/dab/fba x:0 0 0 2024-03-23T08:23:56.093 INFO:tasks.workunit.client.0.smithi070.stdout:7/466: creat d25/d27/fac x:0 0 0 2024-03-23T08:23:56.093 INFO:tasks.workunit.client.0.smithi070.stdout:1/420: write d8/f9 [1528003,127071] 0 2024-03-23T08:23:56.095 INFO:tasks.workunit.client.0.smithi070.stdout:6/565: dwrite d0/d23/d28/d35/d85/f89 [0,4194304] 0 2024-03-23T08:23:56.098 INFO:tasks.workunit.client.0.smithi070.stdout:8/510: unlink d5/d8/d16/d32/c70 0 2024-03-23T08:23:56.098 INFO:tasks.workunit.client.0.smithi070.stdout:0/729: dwrite d16/d26/d27/d4c/d60/d75/fa7 [0,4194304] 0 2024-03-23T08:23:56.106 INFO:tasks.workunit.client.0.smithi070.stdout:3/672: unlink db/d4e/ca6 0 2024-03-23T08:23:56.108 INFO:tasks.workunit.client.0.smithi070.stdout:0/730: write d16/d26/d27/d4c/d60/d7c/daf/fe9 [1014815,90703] 0 2024-03-23T08:23:56.112 INFO:tasks.workunit.client.0.smithi070.stdout:7/467: dwrite d25/d27/d6a/f80 [0,4194304] 0 2024-03-23T08:23:56.119 INFO:tasks.workunit.client.0.smithi070.stdout:2/546: dread d1/fd [0,4194304] 0 2024-03-23T08:23:56.120 INFO:tasks.workunit.client.0.smithi070.stdout:4/380: getdents d3/d6/d35/d65/d6f 0 2024-03-23T08:23:56.132 INFO:tasks.workunit.client.0.smithi070.stdout:5/445: dwrite dc/d1b/d1d/d25/f43 [0,4194304] 0 2024-03-23T08:23:56.136 INFO:tasks.workunit.client.0.smithi070.stdout:9/575: symlink db/db8/lbb 0 2024-03-23T08:23:56.137 INFO:tasks.workunit.client.0.smithi070.stdout:9/576: write db/d12/d34/d8c/da2/d4f/fa9 [985953,3852] 0 2024-03-23T08:23:56.141 INFO:tasks.workunit.client.0.smithi070.stdout:3/673: rename db/df/d15 to db/d4e/df8 0 2024-03-23T08:23:56.144 INFO:tasks.workunit.client.0.smithi070.stdout:7/468: rmdir d25/d27/d6a/d44/d6f 39 2024-03-23T08:23:56.145 INFO:tasks.workunit.client.0.smithi070.stdout:2/547: mknod d1/d6/cbc 0 2024-03-23T08:23:56.146 INFO:tasks.workunit.client.0.smithi070.stdout:4/381: mkdir d3/d6/d35/d65/d27/d4b/d73/d84/d88 0 2024-03-23T08:23:56.147 INFO:tasks.workunit.client.0.smithi070.stdout:4/382: chown d3/d6/f43 0 1 2024-03-23T08:23:56.150 INFO:tasks.workunit.client.0.smithi070.stdout:9/577: creat db/d12/d34/d8c/da2/d26/da7/fbc x:0 0 0 2024-03-23T08:23:56.152 INFO:tasks.workunit.client.0.smithi070.stdout:3/674: creat db/d20/d63/ff9 x:0 0 0 2024-03-23T08:23:56.152 INFO:tasks.workunit.client.0.smithi070.stdout:7/469: creat d25/d27/d6a/d44/d85/fad x:0 0 0 2024-03-23T08:23:56.154 INFO:tasks.workunit.client.0.smithi070.stdout:2/548: dwrite d1/d7/d39/f95 [0,4194304] 0 2024-03-23T08:23:56.156 INFO:tasks.workunit.client.0.smithi070.stdout:7/470: dread d25/d40/f8f [0,4194304] 0 2024-03-23T08:23:56.157 INFO:tasks.workunit.client.0.smithi070.stdout:5/446: creat dc/d1b/d1d/d25/d68/d8a/f8e x:0 0 0 2024-03-23T08:23:56.162 INFO:tasks.workunit.client.0.smithi070.stdout:6/566: getdents d0/d23/d97/d98 0 2024-03-23T08:23:56.163 INFO:tasks.workunit.client.0.smithi070.stdout:9/578: mknod db/d12/d34/d8c/da2/d4f/db6/cbd 0 2024-03-23T08:23:56.164 INFO:tasks.workunit.client.0.smithi070.stdout:8/511: truncate d5/d8/d16/d32/f74 943939 0 2024-03-23T08:23:56.165 INFO:tasks.workunit.client.0.smithi070.stdout:9/579: chown db/d12/d34/c37 110286764 1 2024-03-23T08:23:56.166 INFO:tasks.workunit.client.0.smithi070.stdout:1/421: mkdir d8/db/d40/d43/d64/d85/d97/d9a 0 2024-03-23T08:23:56.169 INFO:tasks.workunit.client.0.smithi070.stdout:4/383: dwrite d3/d6/d35/d65/d6f/f82 [0,4194304] 0 2024-03-23T08:23:56.174 INFO:tasks.workunit.client.0.smithi070.stdout:4/384: write d3/d6/d35/f67 [576704,110431] 0 2024-03-23T08:23:56.179 INFO:tasks.workunit.client.0.smithi070.stdout:8/512: fsync d5/d8/f57 0 2024-03-23T08:23:56.181 INFO:tasks.workunit.client.0.smithi070.stdout:0/731: dwrite d16/f68 [0,4194304] 0 2024-03-23T08:23:56.182 INFO:tasks.workunit.client.0.smithi070.stdout:1/422: fsync d8/db/d2c/d3f/f56 0 2024-03-23T08:23:56.187 INFO:tasks.workunit.client.0.smithi070.stdout:6/567: write d0/d3/d2c/f49 [13482,39760] 0 2024-03-23T08:23:56.193 INFO:tasks.workunit.client.0.smithi070.stdout:2/549: dwrite d1/d7/d39/d47/d7b/f9a [0,4194304] 0 2024-03-23T08:23:56.194 INFO:tasks.workunit.client.0.smithi070.stdout:4/385: mkdir d3/d6/d35/d65/d27/d4b/d73/d89 0 2024-03-23T08:23:56.196 INFO:tasks.workunit.client.0.smithi070.stdout:2/550: read d1/d7/f7e [67995,19639] 0 2024-03-23T08:23:56.197 INFO:tasks.workunit.client.0.smithi070.stdout:2/551: fdatasync d1/d29/f35 0 2024-03-23T08:23:56.199 INFO:tasks.workunit.client.0.smithi070.stdout:1/423: mknod d8/db/d40/d43/d64/d85/d97/c9b 0 2024-03-23T08:23:56.201 INFO:tasks.workunit.client.0.smithi070.stdout:9/580: link db/d12/d14/f63 db/d12/da1/fbe 0 2024-03-23T08:23:56.202 INFO:tasks.workunit.client.0.smithi070.stdout:5/447: dread dc/d1b/d1d/f3a [0,4194304] 0 2024-03-23T08:23:56.202 INFO:tasks.workunit.client.0.smithi070.stdout:9/581: fsync db/d12/d34/d8c/da2/f6a 0 2024-03-23T08:23:56.209 INFO:tasks.workunit.client.0.smithi070.stdout:0/732: rename d16/d26/d27/d4c/d60/d75/de2 to d16/d26/d27/d3d/d4d/dc9/d102 0 2024-03-23T08:23:56.213 INFO:tasks.workunit.client.0.smithi070.stdout:8/513: mkdir d5/d8/d16/d30/d38/d6b/d72/d88/dbf 0 2024-03-23T08:23:56.220 INFO:tasks.workunit.client.0.smithi070.stdout:1/424: truncate d8/db/d2c/f3c 1244526 0 2024-03-23T08:23:56.221 INFO:tasks.workunit.client.0.smithi070.stdout:7/471: getdents d25/d40 0 2024-03-23T08:23:56.223 INFO:tasks.workunit.client.0.smithi070.stdout:5/448: read dc/d1b/d1d/d25/d38/d48/f56 [32360,14683] 0 2024-03-23T08:23:56.226 INFO:tasks.workunit.client.0.smithi070.stdout:4/386: mknod d3/d6/d35/d65/d27/d4b/d73/d84/d88/c8a 0 2024-03-23T08:23:56.228 INFO:tasks.workunit.client.0.smithi070.stdout:2/552: mknod d1/d7/cbd 0 2024-03-23T08:23:56.228 INFO:tasks.workunit.client.0.smithi070.stdout:1/425: mkdir d8/d26/d7b/d9c 0 2024-03-23T08:23:56.232 INFO:tasks.workunit.client.0.smithi070.stdout:0/733: unlink d16/cba 0 2024-03-23T08:23:56.233 INFO:tasks.workunit.client.0.smithi070.stdout:0/734: dread - d16/d26/d27/d3d/d4d/dad/ff7 zero size 2024-03-23T08:23:56.234 INFO:tasks.workunit.client.0.smithi070.stdout:5/449: stat dc/d1b/d1d/d25/d36/d32/l3e 0 2024-03-23T08:23:56.236 INFO:tasks.workunit.client.0.smithi070.stdout:8/514: getdents d5/d8/d9a/dbe 0 2024-03-23T08:23:56.236 INFO:tasks.workunit.client.0.smithi070.stdout:4/387: mkdir d3/d6/d35/d65/d27/d4b/d8b 0 2024-03-23T08:23:56.240 INFO:tasks.workunit.client.0.smithi070.stdout:1/426: mknod d8/d26/d41/c9d 0 2024-03-23T08:23:56.242 INFO:tasks.workunit.client.0.smithi070.stdout:9/582: link db/d12/d22/d85/f66 db/fbf 0 2024-03-23T08:23:56.244 INFO:tasks.workunit.client.0.smithi070.stdout:6/568: getdents d0/d23/d74/d84/db3/d38/d9a 0 2024-03-23T08:23:56.245 INFO:tasks.workunit.client.0.smithi070.stdout:0/735: symlink d16/d22/d43/d47/d87/l103 0 2024-03-23T08:23:56.245 INFO:tasks.workunit.client.0.smithi070.stdout:2/553: dwrite d1/d7/d39/d58/f89 [0,4194304] 0 2024-03-23T08:23:56.245 INFO:tasks.workunit.client.0.smithi070.stdout:2/554: readlink d1/d7/l1a 0 2024-03-23T08:23:56.248 INFO:tasks.workunit.client.0.smithi070.stdout:5/450: mknod dc/c8f 0 2024-03-23T08:23:56.248 INFO:tasks.workunit.client.0.smithi070.stdout:8/515: mknod d5/d8/d16/d32/cc0 0 2024-03-23T08:23:56.251 INFO:tasks.workunit.client.0.smithi070.stdout:5/451: fdatasync dc/d1b/d1d/d25/d36/d32/d5c/f5d 0 2024-03-23T08:23:56.256 INFO:tasks.workunit.client.0.smithi070.stdout:6/569: rename d0/d23/f17 to d0/d3/d2c/fc5 0 2024-03-23T08:23:56.258 INFO:tasks.workunit.client.0.smithi070.stdout:0/736: symlink d16/d26/d27/d3d/d4d/dc9/d102/l104 0 2024-03-23T08:23:56.260 INFO:tasks.workunit.client.0.smithi070.stdout:4/388: mknod d3/d6/d35/d65/d27/d4b/d73/d74/d7e/c8c 0 2024-03-23T08:23:56.262 INFO:tasks.workunit.client.0.smithi070.stdout:9/583: dread db/d12/da1/fbe [0,4194304] 0 2024-03-23T08:23:56.264 INFO:tasks.workunit.client.0.smithi070.stdout:5/452: dread dc/f58 [0,4194304] 0 2024-03-23T08:23:56.266 INFO:tasks.workunit.client.0.smithi070.stdout:7/472: getdents d25/d27/d6a/d44 0 2024-03-23T08:23:56.268 INFO:tasks.workunit.client.0.smithi070.stdout:2/555: fsync d1/f9e 0 2024-03-23T08:23:56.269 INFO:tasks.workunit.client.0.smithi070.stdout:7/473: write d25/d27/d6a/d44/d7c/f9a [80889,93132] 0 2024-03-23T08:23:56.270 INFO:tasks.workunit.client.0.smithi070.stdout:2/556: fdatasync d1/d6/f72 0 2024-03-23T08:23:56.275 INFO:tasks.workunit.client.0.smithi070.stdout:0/737: rmdir d16/d26/d27/d86/d67/d7d 39 2024-03-23T08:23:56.276 INFO:tasks.workunit.client.0.smithi070.stdout:9/584: dread - db/d12/d34/d8c/da2/d4f/d5a/f71 zero size 2024-03-23T08:23:56.279 INFO:tasks.workunit.client.0.smithi070.stdout:2/557: dwrite d1/d21/fb5 [0,4194304] 0 2024-03-23T08:23:56.288 INFO:tasks.workunit.client.0.smithi070.stdout:4/389: link d3/d6/d35/d65/d1b/f26 d3/d6/d35/d65/d41/d56/f8d 0 2024-03-23T08:23:56.292 INFO:tasks.workunit.client.0.smithi070.stdout:2/558: dread d1/d6/d4f/f3e [0,4194304] 0 2024-03-23T08:23:56.292 INFO:tasks.workunit.client.0.smithi070.stdout:5/453: symlink dc/d1b/l90 0 2024-03-23T08:23:56.292 INFO:tasks.workunit.client.0.smithi070.stdout:0/738: creat d16/d26/d27/d4c/d60/f105 x:0 0 0 2024-03-23T08:23:56.294 INFO:tasks.workunit.client.0.smithi070.stdout:9/585: unlink db/d12/d34/d8c/da2/d26/d80/fb2 0 2024-03-23T08:23:56.297 INFO:tasks.workunit.client.0.smithi070.stdout:4/390: dread d3/d6/d35/d65/f3e [0,4194304] 0 2024-03-23T08:23:56.298 INFO:tasks.workunit.client.0.smithi070.stdout:8/516: rename d5/d8/d16/d8d/da3/lb7 to d5/d8/lc1 0 2024-03-23T08:23:56.301 INFO:tasks.workunit.client.0.smithi070.stdout:6/570: link d0/d23/d28/l66 d0/d23/d4d/lc6 0 2024-03-23T08:23:56.302 INFO:tasks.workunit.client.0.smithi070.stdout:9/586: dwrite db/d12/d22/fb0 [0,4194304] 0 2024-03-23T08:23:56.304 INFO:tasks.workunit.client.0.smithi070.stdout:2/559: rmdir d1/d29/d52 39 2024-03-23T08:23:56.308 INFO:tasks.workunit.client.0.smithi070.stdout:9/587: write db/d12/f39 [1286430,107118] 0 2024-03-23T08:23:56.308 INFO:tasks.workunit.client.0.smithi070.stdout:2/560: chown d1/d7/df/l63 9204 1 2024-03-23T08:23:56.312 INFO:tasks.workunit.client.0.smithi070.stdout:9/588: chown db/d12/d34/d8c/da2/d26/d5c/d7e/c7b 15628825 1 2024-03-23T08:23:56.312 INFO:tasks.workunit.client.0.smithi070.stdout:5/454: creat dc/d1b/d1d/d25/d38/d39/f91 x:0 0 0 2024-03-23T08:23:56.312 INFO:tasks.workunit.client.0.smithi070.stdout:9/589: chown db/f65 30 1 2024-03-23T08:23:56.317 INFO:tasks.workunit.client.0.smithi070.stdout:5/455: dread - dc/d1b/d1d/d25/d38/d39/f6b zero size 2024-03-23T08:23:56.317 INFO:tasks.workunit.client.0.smithi070.stdout:1/427: truncate d8/db/d40/f84 2707210 0 2024-03-23T08:23:56.317 INFO:tasks.workunit.client.0.smithi070.stdout:5/456: chown dc/d1b/d1d/d25/d36/d32 405658615 1 2024-03-23T08:23:56.318 INFO:tasks.workunit.client.0.smithi070.stdout:7/474: rmdir d25/d27/d6a/d44/d85/d95/d9e 0 2024-03-23T08:23:56.329 INFO:tasks.workunit.client.0.smithi070.stdout:5/457: mkdir dc/d1b/d1d/d25/d38/d48/d92 0 2024-03-23T08:23:56.335 INFO:tasks.workunit.client.0.smithi070.stdout:4/391: write d3/d6/d11/f18 [2991928,42815] 0 2024-03-23T08:23:56.339 INFO:tasks.workunit.client.0.smithi070.stdout:2/561: symlink d1/d29/d52/lbe 0 2024-03-23T08:23:56.340 INFO:tasks.workunit.client.0.smithi070.stdout:2/562: write d1/fe [2167863,1771] 0 2024-03-23T08:23:56.341 INFO:tasks.workunit.client.0.smithi070.stdout:2/563: chown d1/d6/d34/fad 6 1 2024-03-23T08:23:56.343 INFO:tasks.workunit.client.0.smithi070.stdout:0/739: write d16/d22/f39 [278180,33829] 0 2024-03-23T08:23:56.346 INFO:tasks.workunit.client.0.smithi070.stdout:0/740: fdatasync d16/d26/d27/d3d/d4d/dad/fd9 0 2024-03-23T08:23:56.349 INFO:tasks.workunit.client.0.smithi070.stdout:7/475: symlink d25/d27/d6a/d44/da2/lae 0 2024-03-23T08:23:56.349 INFO:tasks.workunit.client.0.smithi070.stdout:8/517: creat d5/d8/d16/d30/d38/fc2 x:0 0 0 2024-03-23T08:23:56.353 INFO:tasks.workunit.client.0.smithi070.stdout:9/590: truncate db/d12/f17 4050943 0 2024-03-23T08:23:56.354 INFO:tasks.workunit.client.0.smithi070.stdout:2/564: chown d1/d7/df/la1 8881 1 2024-03-23T08:23:56.357 INFO:tasks.workunit.client.0.smithi070.stdout:0/741: creat d16/d26/d27/d4c/d60/f106 x:0 0 0 2024-03-23T08:23:56.359 INFO:tasks.workunit.client.0.smithi070.stdout:6/571: getdents d0/d23/d74 0 2024-03-23T08:23:56.359 INFO:tasks.workunit.client.0.smithi070.stdout:9/591: dwrite db/d12/f9a [0,4194304] 0 2024-03-23T08:23:56.361 INFO:tasks.workunit.client.0.smithi070.stdout:9/592: fdatasync db/d12/dab/fba 0 2024-03-23T08:23:56.362 INFO:tasks.workunit.client.0.smithi070.stdout:4/392: symlink d3/d6/d35/d65/l8e 0 2024-03-23T08:23:56.362 INFO:tasks.workunit.client.0.smithi070.stdout:7/476: creat d25/d27/d6a/d44/d85/faf x:0 0 0 2024-03-23T08:23:56.366 INFO:tasks.workunit.client.0.smithi070.stdout:5/458: rename c6 to dc/d1b/c93 0 2024-03-23T08:23:56.368 INFO:tasks.workunit.client.0.smithi070.stdout:9/593: rmdir db/d12/d22/d85/d42 39 2024-03-23T08:23:56.370 INFO:tasks.workunit.client.0.smithi070.stdout:8/518: symlink d5/d8/d16/d30/d38/lc3 0 2024-03-23T08:23:56.370 INFO:tasks.workunit.client.0.smithi070.stdout:4/393: dwrite d3/d6/d35/f76 [0,4194304] 0 2024-03-23T08:23:56.373 INFO:tasks.workunit.client.0.smithi070.stdout:4/394: read d3/d6/d35/d65/f3e [1536389,67090] 0 2024-03-23T08:23:56.373 INFO:tasks.workunit.client.0.smithi070.stdout:2/565: creat d1/d98/fbf x:0 0 0 2024-03-23T08:23:56.376 INFO:tasks.workunit.client.0.smithi070.stdout:7/477: rename d25/d27/d6a/d44/f56 to d25/d40/fb0 0 2024-03-23T08:23:56.377 INFO:tasks.workunit.client.0.smithi070.stdout:8/519: truncate d5/d8/d16/d32/d84/f96 363482 0 2024-03-23T08:23:56.378 INFO:tasks.workunit.client.0.smithi070.stdout:0/742: mkdir d16/d107 0 2024-03-23T08:23:56.378 INFO:tasks.workunit.client.0.smithi070.stdout:7/478: read d25/f83 [1259528,34407] 0 2024-03-23T08:23:56.382 INFO:tasks.workunit.client.0.smithi070.stdout:0/743: fdatasync d16/d26/d27/d3d/d4d/ffb 0 2024-03-23T08:23:56.385 INFO:tasks.workunit.client.0.smithi070.stdout:2/566: dwrite d1/d7/f17 [0,4194304] 0 2024-03-23T08:23:56.386 INFO:tasks.workunit.client.0.smithi070.stdout:7/479: dread d25/f84 [0,4194304] 0 2024-03-23T08:23:56.388 INFO:tasks.workunit.client.0.smithi070.stdout:4/395: rename d3/d6/d35/f7b to d3/f8f 0 2024-03-23T08:23:56.388 INFO:tasks.workunit.client.0.smithi070.stdout:2/567: truncate d1/d6/d4f/f74 673492 0 2024-03-23T08:23:56.393 INFO:tasks.workunit.client.0.smithi070.stdout:5/459: creat dc/d1b/d1d/d61/f94 x:0 0 0 2024-03-23T08:23:56.396 INFO:tasks.workunit.client.0.smithi070.stdout:9/594: dwrite db/d12/d34/d8c/da2/d26/d5c/d7e/f75 [0,4194304] 0 2024-03-23T08:23:56.396 INFO:tasks.workunit.client.0.smithi070.stdout:4/396: dread d3/d6/f38 [0,4194304] 0 2024-03-23T08:23:56.397 INFO:tasks.workunit.client.0.smithi070.stdout:5/460: fdatasync dc/d1b/d1d/d25/d36/d32/f5a 0 2024-03-23T08:23:56.397 INFO:tasks.workunit.client.0.smithi070.stdout:8/520: creat d5/d8/d16/d30/d38/d6b/d72/fc4 x:0 0 0 2024-03-23T08:23:56.399 INFO:tasks.workunit.client.0.smithi070.stdout:4/397: write d3/d6/d35/d65/d1b/f66 [835478,126102] 0 2024-03-23T08:23:56.399 INFO:tasks.workunit.client.0.smithi070.stdout:9/595: chown db/d12/d34/d8c/da2/d26/d80/c96 1093973 1 2024-03-23T08:23:56.401 INFO:tasks.workunit.client.0.smithi070.stdout:9/596: write db/d12/d34/d8c/da2/d4f/fa9 [1282588,26415] 0 2024-03-23T08:23:56.409 INFO:tasks.workunit.client.0.smithi070.stdout:4/398: rename d3/d6/d35/d65/d27/d4b/d73/d84/f2b to d3/d6/d35/d65/d1b/f90 0 2024-03-23T08:23:56.412 INFO:tasks.workunit.client.0.smithi070.stdout:5/461: unlink dc/d1b/d1d/f62 0 2024-03-23T08:23:56.414 INFO:tasks.workunit.client.0.smithi070.stdout:9/597: mknod db/d12/d34/d8c/da2/d58/db5/cc0 0 2024-03-23T08:23:56.415 INFO:tasks.workunit.client.0.smithi070.stdout:4/399: mkdir d3/d6/d35/d65/d27/d4b/d73/d84/d91 0 2024-03-23T08:23:56.415 INFO:tasks.workunit.client.0.smithi070.stdout:9/598: chown db/d12/d34/d8c/da2/d26/c2e 228695545 1 2024-03-23T08:23:56.415 INFO:tasks.workunit.client.0.smithi070.stdout:5/462: symlink dc/l95 0 2024-03-23T08:23:56.417 INFO:tasks.workunit.client.0.smithi070.stdout:4/400: readlink d3/d6/l72 0 2024-03-23T08:23:56.418 INFO:tasks.workunit.client.0.smithi070.stdout:9/599: creat db/d12/d67/fc1 x:0 0 0 2024-03-23T08:23:56.419 INFO:tasks.workunit.client.0.smithi070.stdout:5/463: creat dc/d1b/d1d/d25/d68/f96 x:0 0 0 2024-03-23T08:23:56.421 INFO:tasks.workunit.client.0.smithi070.stdout:4/401: mkdir d3/d6/d35/d65/d27/d4b/d73/d89/d92 0 2024-03-23T08:23:56.422 INFO:tasks.workunit.client.0.smithi070.stdout:9/600: mkdir db/d12/d34/d47/d90/dc2 0 2024-03-23T08:23:56.424 INFO:tasks.workunit.client.0.smithi070.stdout:4/402: mkdir d3/d6/d35/d65/d6f/d93 0 2024-03-23T08:23:56.429 INFO:tasks.workunit.client.0.smithi070.stdout:4/403: link d3/d6/d35/d3f/c5f d3/d6/d6a/d7c/c94 0 2024-03-23T08:23:56.499 INFO:tasks.workunit.client.0.smithi070.stdout:7/480: dread - d25/d27/d6a/d44/d85/f98 zero size 2024-03-23T08:23:56.501 INFO:tasks.workunit.client.0.smithi070.stdout:7/481: chown d25/d27/d6a/d44/f67 56093896 1 2024-03-23T08:23:56.504 INFO:tasks.workunit.client.0.smithi070.stdout:0/744: write d16/d26/d27/d4c/d60/d7c/fb8 [1019827,33700] 0 2024-03-23T08:23:56.509 INFO:tasks.workunit.client.0.smithi070.stdout:7/482: dread d25/f83 [0,4194304] 0 2024-03-23T08:23:56.511 INFO:tasks.workunit.client.0.smithi070.stdout:2/568: dwrite d1/d7/f7e [0,4194304] 0 2024-03-23T08:23:56.514 INFO:tasks.workunit.client.0.smithi070.stdout:0/745: dwrite d16/d22/d43/f56 [0,4194304] 0 2024-03-23T08:23:56.517 INFO:tasks.workunit.client.0.smithi070.stdout:0/746: write d16/d26/d27/d3d/d4d/dad/fe8 [396581,31680] 0 2024-03-23T08:23:56.528 INFO:tasks.workunit.client.0.smithi070.stdout:4/404: truncate d3/d6/d11/f13 8255 0 2024-03-23T08:23:56.532 INFO:tasks.workunit.client.0.smithi070.stdout:9/601: dwrite db/d12/d34/d8c/da2/f7c [0,4194304] 0 2024-03-23T08:23:56.538 INFO:tasks.workunit.client.0.smithi070.stdout:0/747: dwrite d16/f1a [0,4194304] 0 2024-03-23T08:23:56.539 INFO:tasks.workunit.client.0.smithi070.stdout:2/569: dread d1/d7/df/d94/f56 [0,4194304] 0 2024-03-23T08:23:56.542 INFO:tasks.workunit.client.0.smithi070.stdout:4/405: getdents d3/d6/d35 0 2024-03-23T08:23:56.543 INFO:tasks.workunit.client.0.smithi070.stdout:4/406: chown d3 31990100 1 2024-03-23T08:23:56.544 INFO:tasks.workunit.client.0.smithi070.stdout:4/407: write d3/d6/d11/f2d [272804,49224] 0 2024-03-23T08:23:56.548 INFO:tasks.workunit.client.0.smithi070.stdout:2/570: mkdir d1/d6/dc0 0 2024-03-23T08:23:56.550 INFO:tasks.workunit.client.0.smithi070.stdout:2/571: write d1/d7/d39/f82 [884055,117069] 0 2024-03-23T08:23:56.551 INFO:tasks.workunit.client.0.smithi070.stdout:4/408: mknod d3/d6/d35/c95 0 2024-03-23T08:23:56.555 INFO:tasks.workunit.client.0.smithi070.stdout:9/602: dwrite f4 [0,4194304] 0 2024-03-23T08:23:56.557 INFO:tasks.workunit.client.0.smithi070.stdout:4/409: mkdir d3/d6/d96 0 2024-03-23T08:23:56.559 INFO:tasks.workunit.client.0.smithi070.stdout:4/410: chown d3/d6/d35/d65/d1b/f1c 32392 1 2024-03-23T08:23:56.561 INFO:tasks.workunit.client.0.smithi070.stdout:9/603: read db/d12/d14/f7a [2333295,47605] 0 2024-03-23T08:23:56.565 INFO:tasks.workunit.client.0.smithi070.stdout:4/411: mkdir d3/d6/d35/d3f/d51/d6e/d97 0 2024-03-23T08:23:56.569 INFO:tasks.workunit.client.0.smithi070.stdout:2/572: dread d1/d7/d39/f9c [0,4194304] 0 2024-03-23T08:23:56.574 INFO:tasks.workunit.client.0.smithi070.stdout:9/604: dwrite f8 [0,4194304] 0 2024-03-23T08:23:56.576 INFO:tasks.workunit.client.0.smithi070.stdout:0/748: dread d16/d22/d43/f5a [0,4194304] 0 2024-03-23T08:23:56.581 INFO:tasks.workunit.client.0.smithi070.stdout:9/605: creat db/d12/d34/d8c/da2/d58/db5/fc3 x:0 0 0 2024-03-23T08:23:56.583 INFO:tasks.workunit.client.0.smithi070.stdout:0/749: mknod d16/d26/d27/d86/def/c108 0 2024-03-23T08:23:56.585 INFO:tasks.workunit.client.0.smithi070.stdout:9/606: fdatasync db/d12/f89 0 2024-03-23T08:23:56.586 INFO:tasks.workunit.client.0.smithi070.stdout:4/412: dwrite d3/d6/d35/d65/f30 [0,4194304] 0 2024-03-23T08:23:56.588 INFO:tasks.workunit.client.0.smithi070.stdout:0/750: write d16/f30 [3368860,21086] 0 2024-03-23T08:23:56.592 INFO:tasks.workunit.client.0.smithi070.stdout:9/607: creat db/d12/d34/d8c/da2/d4f/db6/fc4 x:0 0 0 2024-03-23T08:23:56.592 INFO:tasks.workunit.client.0.smithi070.stdout:4/413: fdatasync d3/d6/d35/d65/d27/f46 0 2024-03-23T08:23:56.594 INFO:tasks.workunit.client.0.smithi070.stdout:2/573: dread d1/d6/d34/fad [0,4194304] 0 2024-03-23T08:23:56.595 INFO:tasks.workunit.client.0.smithi070.stdout:2/574: dread - d1/d21/d5f/fa8 zero size 2024-03-23T08:23:56.623 INFO:tasks.workunit.client.0.smithi070.stdout:9/608: creat db/d12/d34/d8c/da2/d26/d5c/d7e/fc5 x:0 0 0 2024-03-23T08:23:56.625 INFO:tasks.workunit.client.0.smithi070.stdout:2/575: dwrite d1/d6/d30/f46 [0,4194304] 0 2024-03-23T08:23:56.628 INFO:tasks.workunit.client.0.smithi070.stdout:4/414: mknod d3/d6/d35/d65/d6f/d93/c98 0 2024-03-23T08:23:56.638 INFO:tasks.workunit.client.0.smithi070.stdout:9/609: unlink db/db8/lbb 0 2024-03-23T08:23:56.640 INFO:tasks.workunit.client.0.smithi070.stdout:2/576: chown d1/d21/d5f/c6b 3247 1 2024-03-23T08:23:56.641 INFO:tasks.workunit.client.0.smithi070.stdout:4/415: creat d3/d6/d35/d3f/d51/d6e/f99 x:0 0 0 2024-03-23T08:23:56.642 INFO:tasks.workunit.client.0.smithi070.stdout:0/751: write d16/d26/d27/d4c/d60/d75/fd2 [1034505,94259] 0 2024-03-23T08:23:56.646 INFO:tasks.workunit.client.0.smithi070.stdout:0/752: read d16/d26/d27/d4c/d60/d7c/fb8 [425687,38632] 0 2024-03-23T08:23:56.646 INFO:tasks.workunit.client.0.smithi070.stdout:4/416: dread d3/d6/d9/f2e [0,4194304] 0 2024-03-23T08:23:56.649 INFO:tasks.workunit.client.0.smithi070.stdout:2/577: dwrite d1/d7/d39/d47/d7d/fb4 [0,4194304] 0 2024-03-23T08:23:56.650 INFO:tasks.workunit.client.0.smithi070.stdout:9/610: rename db/d12/d34/d8c/da2/d26/d5c to db/d12/d34/d8c/da2/dc6 0 2024-03-23T08:23:56.650 INFO:tasks.workunit.client.0.smithi070.stdout:4/417: chown d3/d6/d35/d65/d41/d56/f8d 240 1 2024-03-23T08:23:56.652 INFO:tasks.workunit.client.0.smithi070.stdout:9/611: chown db/d12/d34/c40 1374523 1 2024-03-23T08:23:56.654 INFO:tasks.workunit.client.0.smithi070.stdout:9/612: chown db/d12/d14/f35 24900457 1 2024-03-23T08:23:56.656 INFO:tasks.workunit.client.0.smithi070.stdout:2/578: fdatasync d1/d7/df/d94/f56 0 2024-03-23T08:23:56.657 INFO:tasks.workunit.client.0.smithi070.stdout:2/579: chown d1/d7/l1a 4 1 2024-03-23T08:23:56.657 INFO:tasks.workunit.client.0.smithi070.stdout:4/418: mkdir d3/d6/d35/d65/d27/d4b/d73/d74/d7e/d9a 0 2024-03-23T08:23:56.660 INFO:tasks.workunit.client.0.smithi070.stdout:0/753: mkdir d16/d26/d27/d86/d67/d109 0 2024-03-23T08:23:56.663 INFO:tasks.workunit.client.0.smithi070.stdout:2/580: dread d1/d6/d30/f46 [0,4194304] 0 2024-03-23T08:23:56.670 INFO:tasks.workunit.client.0.smithi070.stdout:4/419: dread d3/d6/f2f [0,4194304] 0 2024-03-23T08:23:56.670 INFO:tasks.workunit.client.0.smithi070.stdout:0/754: rmdir d16/d26/d27/d86/d67/d7d 39 2024-03-23T08:23:56.671 INFO:tasks.workunit.client.0.smithi070.stdout:9/613: dread db/d12/d34/d8c/da2/d26/f30 [0,4194304] 0 2024-03-23T08:23:56.673 INFO:tasks.workunit.client.0.smithi070.stdout:2/581: mkdir d1/d7/d48/d69/dc1 0 2024-03-23T08:23:56.674 INFO:tasks.workunit.client.0.smithi070.stdout:2/582: chown d1/d29/d52/f9f 1 1 2024-03-23T08:23:56.677 INFO:tasks.workunit.client.0.smithi070.stdout:0/755: dwrite d16/d26/d27/f91 [0,4194304] 0 2024-03-23T08:23:56.685 INFO:tasks.workunit.client.0.smithi070.stdout:4/420: link d3/d6/d35/c95 d3/c9b 0 2024-03-23T08:23:56.687 INFO:tasks.workunit.client.0.smithi070.stdout:2/583: getdents d1/d7/df/d94/d38/d4c/d76 0 2024-03-23T08:23:56.692 INFO:tasks.workunit.client.0.smithi070.stdout:2/584: mknod d1/d29/d52/cc2 0 2024-03-23T08:23:56.694 INFO:tasks.workunit.client.0.smithi070.stdout:9/614: rename db/d12/d22/d85/d42/f46 to db/d12/d22/fc7 0 2024-03-23T08:23:56.695 INFO:tasks.workunit.client.0.smithi070.stdout:4/421: link d3/d6/f6c d3/d6/d6a/d7c/f9c 0 2024-03-23T08:23:56.698 INFO:tasks.workunit.client.0.smithi070.stdout:2/585: rename d1/d6/d4f to d1/d21/d5f/da4/dc3 0 2024-03-23T08:23:56.701 INFO:tasks.workunit.client.0.smithi070.stdout:9/615: mkdir db/d12/d22/d85/d42/dc8 0 2024-03-23T08:23:56.704 INFO:tasks.workunit.client.0.smithi070.stdout:4/422: dwrite d3/d6/d35/d65/d6f/f82 [4194304,4194304] 0 2024-03-23T08:23:56.706 INFO:tasks.workunit.client.0.smithi070.stdout:4/423: write d3/d6/d35/d65/d41/f79 [524056,96308] 0 2024-03-23T08:23:56.707 INFO:tasks.workunit.client.0.smithi070.stdout:2/586: dread d1/d7/d39/d47/d7b/f9a [0,4194304] 0 2024-03-23T08:23:56.709 INFO:tasks.workunit.client.0.smithi070.stdout:2/587: chown d1/d7/d39/d58/c64 93262 1 2024-03-23T08:23:56.710 INFO:tasks.workunit.client.0.smithi070.stdout:4/424: mkdir d3/d6/d35/d3f/d51/d6e/d97/d9d 0 2024-03-23T08:23:56.714 INFO:tasks.workunit.client.0.smithi070.stdout:2/588: link d1/d21/fa9 d1/d21/d5f/da4/fc4 0 2024-03-23T08:23:56.716 INFO:tasks.workunit.client.0.smithi070.stdout:0/756: truncate d16/d26/d27/d3d/ff6 1052839 0 2024-03-23T08:23:56.716 INFO:tasks.workunit.client.0.smithi070.stdout:2/589: fsync d1/d7/d39/d47/f9b 0 2024-03-23T08:23:56.718 INFO:tasks.workunit.client.0.smithi070.stdout:2/590: chown d1/d29/f70 29788146 1 2024-03-23T08:23:56.719 INFO:tasks.workunit.client.0.smithi070.stdout:4/425: dwrite d3/d6/d11/f2d [0,4194304] 0 2024-03-23T08:23:56.719 INFO:tasks.workunit.client.0.smithi070.stdout:0/757: fdatasync d16/d26/d27/d86/db6/fce 0 2024-03-23T08:23:56.721 INFO:tasks.workunit.client.0.smithi070.stdout:2/591: read d1/d7/d48/d69/d7a/fa5 [174545,45463] 0 2024-03-23T08:23:56.727 INFO:tasks.workunit.client.0.smithi070.stdout:0/758: dread - d16/d26/d27/d4c/d60/d7c/fd0 zero size 2024-03-23T08:23:56.730 INFO:tasks.workunit.client.0.smithi070.stdout:0/759: chown d16/d26/d27/d4c/d60/d75/fd2 65679 1 2024-03-23T08:23:56.730 INFO:tasks.workunit.client.0.smithi070.stdout:9/616: dread db/d12/d14/f23 [0,4194304] 0 2024-03-23T08:23:56.731 INFO:tasks.workunit.client.0.smithi070.stdout:2/592: fsync d1/d7/d39/d47/fb1 0 2024-03-23T08:23:56.732 INFO:tasks.workunit.client.0.smithi070.stdout:0/760: truncate d16/d22/fd3 594318 0 2024-03-23T08:23:56.732 INFO:tasks.workunit.client.0.smithi070.stdout:2/593: fsync d1/d7/df/d94/f12 0 2024-03-23T08:23:56.743 INFO:tasks.workunit.client.0.smithi070.stdout:0/761: rename d16/d22/d43/l6a to d16/d26/d27/d3d/l10a 0 2024-03-23T08:23:56.745 INFO:tasks.workunit.client.0.smithi070.stdout:9/617: dread db/d12/d34/d8c/da2/f24 [0,4194304] 0 2024-03-23T08:23:56.746 INFO:tasks.workunit.client.0.smithi070.stdout:0/762: mknod d16/d26/d27/d3d/d4d/d9a/df8/c10b 0 2024-03-23T08:23:56.747 INFO:tasks.workunit.client.0.smithi070.stdout:9/618: write db/d12/d34/d8c/da2/d4f/db6/d79/f9d [573408,102758] 0 2024-03-23T08:23:56.757 INFO:tasks.workunit.client.0.smithi070.stdout:9/619: getdents db/db8 0 2024-03-23T08:23:56.764 INFO:tasks.workunit.client.0.smithi070.stdout:0/763: dread d16/f3e [0,4194304] 0 2024-03-23T08:23:56.768 INFO:tasks.workunit.client.0.smithi070.stdout:0/764: creat d16/d26/d27/f10c x:0 0 0 2024-03-23T08:23:56.768 INFO:tasks.workunit.client.0.smithi070.stdout:4/426: write d3/d6/d35/d3f/d51/f63 [3078924,106164] 0 2024-03-23T08:23:56.769 INFO:tasks.workunit.client.0.smithi070.stdout:4/427: chown d3/d6/f6c 20722 1 2024-03-23T08:23:56.772 INFO:tasks.workunit.client.0.smithi070.stdout:4/428: read d3/d6/d35/d65/d1b/f66 [131423,5784] 0 2024-03-23T08:23:56.773 INFO:tasks.workunit.client.0.smithi070.stdout:4/429: chown d3/d6/d35/d65/d27/f4e 97165296 1 2024-03-23T08:23:56.775 INFO:tasks.workunit.client.0.smithi070.stdout:4/430: dread d3/d6/d35/d65/d1b/f26 [0,4194304] 0 2024-03-23T08:23:56.776 INFO:tasks.workunit.client.0.smithi070.stdout:4/431: chown d3/d6/d35/d65/d1b/f90 342 1 2024-03-23T08:23:56.777 INFO:tasks.workunit.client.0.smithi070.stdout:2/594: truncate d1/d6/d30/f3a 569699 0 2024-03-23T08:23:56.777 INFO:tasks.workunit.client.0.smithi070.stdout:4/432: write d3/d6/d35/f67 [2563148,66889] 0 2024-03-23T08:23:56.779 INFO:tasks.workunit.client.0.smithi070.stdout:2/595: chown d1/d6/cbc 2356869 1 2024-03-23T08:23:56.781 INFO:tasks.workunit.client.0.smithi070.stdout:2/596: mknod d1/d7/d39/d47/d7b/cc5 0 2024-03-23T08:23:56.784 INFO:tasks.workunit.client.0.smithi070.stdout:4/433: dwrite d3/d6/d35/d65/d1b/f90 [0,4194304] 0 2024-03-23T08:23:56.787 INFO:tasks.workunit.client.0.smithi070.stdout:2/597: rmdir d1/d29/da7 0 2024-03-23T08:23:56.791 INFO:tasks.workunit.client.0.smithi070.stdout:4/434: mknod d3/d6/d35/d65/d27/d4b/c9e 0 2024-03-23T08:23:56.795 INFO:tasks.workunit.client.0.smithi070.stdout:2/598: dwrite d1/d7/f66 [0,4194304] 0 2024-03-23T08:23:56.797 INFO:tasks.workunit.client.0.smithi070.stdout:2/599: write d1/fc [3982098,99821] 0 2024-03-23T08:23:56.797 INFO:tasks.workunit.client.0.smithi070.stdout:4/435: write d3/d6/d35/d65/d1b/f66 [626056,85281] 0 2024-03-23T08:23:56.799 INFO:tasks.workunit.client.0.smithi070.stdout:2/600: stat d1/d7/d48/d69/dc1 0 2024-03-23T08:23:56.799 INFO:tasks.workunit.client.0.smithi070.stdout:4/436: dread d3/d6/d9/f2e [0,4194304] 0 2024-03-23T08:23:56.805 INFO:tasks.workunit.client.0.smithi070.stdout:9/620: write db/d12/f2b [3897400,122641] 0 2024-03-23T08:23:56.807 INFO:tasks.workunit.client.0.smithi070.stdout:2/601: dwrite d1/d29/f35 [0,4194304] 0 2024-03-23T08:23:56.849 INFO:tasks.workunit.client.0.smithi070.stdout:6/572: sync 2024-03-23T08:23:56.849 INFO:tasks.workunit.client.0.smithi070.stdout:3/675: sync 2024-03-23T08:23:56.849 INFO:tasks.workunit.client.0.smithi070.stdout:1/428: sync 2024-03-23T08:23:56.853 INFO:tasks.workunit.client.0.smithi070.stdout:1/429: stat d8/d1a/c1d 0 2024-03-23T08:23:56.863 INFO:tasks.workunit.client.0.smithi070.stdout:6/573: chown d0/d23/d74/d84/db3/d38/d9a/cb5 321 1 2024-03-23T08:23:56.864 INFO:tasks.workunit.client.0.smithi070.stdout:3/676: mknod db/d4e/d92/cfa 0 2024-03-23T08:23:56.864 INFO:tasks.workunit.client.0.smithi070.stdout:2/602: fsync d1/d7/d39/f9c 0 2024-03-23T08:23:56.877 INFO:tasks.workunit.client.0.smithi070.stdout:0/765: write d16/d26/d27/d4c/d60/d75/fcc [2415166,104160] 0 2024-03-23T08:23:56.890 INFO:tasks.workunit.client.0.smithi070.stdout:2/603: dread d1/d7/d39/d47/fb1 [0,4194304] 0 2024-03-23T08:23:56.893 INFO:tasks.workunit.client.0.smithi070.stdout:2/604: read d1/d7/d39/d47/d7d/fb4 [2769591,38703] 0 2024-03-23T08:23:56.902 INFO:tasks.workunit.client.0.smithi070.stdout:9/621: creat db/d12/d22/d85/fc9 x:0 0 0 2024-03-23T08:23:56.906 INFO:tasks.workunit.client.0.smithi070.stdout:1/430: mknod d8/d42/d4e/c9e 0 2024-03-23T08:23:56.915 INFO:tasks.workunit.client.0.smithi070.stdout:9/622: dread db/d12/d14/f4e [4194304,4194304] 0 2024-03-23T08:23:56.919 INFO:tasks.workunit.client.0.smithi070.stdout:1/431: mknod d8/d42/d51/c9f 0 2024-03-23T08:23:56.924 INFO:tasks.workunit.client.0.smithi070.stdout:3/677: mkdir db/d14/d65/da8/dfb 0 2024-03-23T08:23:56.924 INFO:tasks.workunit.client.0.smithi070.stdout:0/766: dwrite d16/d26/d27/d86/db6/dde/fbf [0,4194304] 0 2024-03-23T08:23:56.926 INFO:tasks.workunit.client.0.smithi070.stdout:0/767: chown d16/d26/d27/d86/d9e/f8a 3686 1 2024-03-23T08:23:56.927 INFO:tasks.workunit.client.0.smithi070.stdout:0/768: fsync d16/f30 0 2024-03-23T08:23:56.929 INFO:tasks.workunit.client.0.smithi070.stdout:0/769: chown d16/d26/d27/d86/d67/fbe 197433465 1 2024-03-23T08:23:56.932 INFO:tasks.workunit.client.0.smithi070.stdout:2/605: creat d1/d6/dc0/fc6 x:0 0 0 2024-03-23T08:23:56.942 INFO:tasks.workunit.client.0.smithi070.stdout:3/678: mkdir db/d4e/dfc 0 2024-03-23T08:23:56.942 INFO:tasks.workunit.client.0.smithi070.stdout:1/432: read f3 [650902,14144] 0 2024-03-23T08:23:56.943 INFO:tasks.workunit.client.0.smithi070.stdout:3/679: chown db/d20/d49/l8f 25 1 2024-03-23T08:23:56.945 INFO:tasks.workunit.client.0.smithi070.stdout:0/770: symlink d16/d26/l10d 0 2024-03-23T08:23:56.949 INFO:tasks.workunit.client.0.smithi070.stdout:6/574: getdents d0/d23/d4d 0 2024-03-23T08:23:56.951 INFO:tasks.workunit.client.0.smithi070.stdout:3/680: rename db/d14/c26 to db/d4e/dd6/dd7/cfd 0 2024-03-23T08:23:56.953 INFO:tasks.workunit.client.0.smithi070.stdout:4/437: truncate d3/d6/f43 8997 0 2024-03-23T08:23:56.955 INFO:tasks.workunit.client.0.smithi070.stdout:0/771: symlink d16/d26/d27/l10e 0 2024-03-23T08:23:56.957 INFO:tasks.workunit.client.0.smithi070.stdout:3/681: mkdir db/d20/d63/d83/dfe 0 2024-03-23T08:23:56.958 INFO:tasks.workunit.client.0.smithi070.stdout:6/575: creat d0/d23/d74/d84/fc7 x:0 0 0 2024-03-23T08:23:56.961 INFO:tasks.workunit.client.0.smithi070.stdout:6/576: write d0/d23/d74/d84/d3c/f60 [195124,119458] 0 2024-03-23T08:23:56.961 INFO:tasks.workunit.client.0.smithi070.stdout:0/772: symlink d16/d26/d27/d86/d67/d7d/l10f 0 2024-03-23T08:23:56.964 INFO:tasks.workunit.client.0.smithi070.stdout:4/438: dread d3/d6/d11/f18 [0,4194304] 0 2024-03-23T08:23:56.965 INFO:tasks.workunit.client.0.smithi070.stdout:6/577: fdatasync d0/d23/d28/d35/d85/f8b 0 2024-03-23T08:23:56.968 INFO:tasks.workunit.client.0.smithi070.stdout:6/578: rename d0/d23/d28/f8e to d0/d3/fc8 0 2024-03-23T08:23:56.970 INFO:tasks.workunit.client.0.smithi070.stdout:0/773: truncate d16/d22/d43/d47/da9/fe6 835891 0 2024-03-23T08:23:56.975 INFO:tasks.workunit.client.0.smithi070.stdout:6/579: dwrite d0/d23/d74/d84/db3/f5d [0,4194304] 0 2024-03-23T08:23:56.976 INFO:tasks.workunit.client.0.smithi070.stdout:6/580: chown d0/d23/d74/d84/d3c/f53 941480394 1 2024-03-23T08:23:56.980 INFO:tasks.workunit.client.0.smithi070.stdout:6/581: unlink d0/d23/d4d/l96 0 2024-03-23T08:23:56.983 INFO:tasks.workunit.client.0.smithi070.stdout:6/582: symlink d0/d23/d74/d84/lc9 0 2024-03-23T08:23:56.989 INFO:tasks.workunit.client.0.smithi070.stdout:2/606: dread d1/d6/d30/f3a [0,4194304] 0 2024-03-23T08:23:56.989 INFO:tasks.workunit.client.0.smithi070.stdout:6/583: dread d0/d23/d74/d84/d3c/f51 [0,4194304] 0 2024-03-23T08:23:56.992 INFO:tasks.workunit.client.0.smithi070.stdout:6/584: fsync d0/d23/d28/f36 0 2024-03-23T08:23:56.999 INFO:tasks.workunit.client.0.smithi070.stdout:9/623: dwrite db/d12/d34/d8c/da2/dc6/d7e/f56 [0,4194304] 0 2024-03-23T08:23:57.009 INFO:tasks.workunit.client.0.smithi070.stdout:1/433: dwrite d8/d11/d61/f78 [0,4194304] 0 2024-03-23T08:23:57.010 INFO:tasks.workunit.client.0.smithi070.stdout:6/585: dread d0/d23/d28/f4a [0,4194304] 0 2024-03-23T08:23:57.012 INFO:tasks.workunit.client.0.smithi070.stdout:6/586: dread - d0/fa7 zero size 2024-03-23T08:23:57.015 INFO:tasks.workunit.client.0.smithi070.stdout:1/434: mknod d8/ca0 0 2024-03-23T08:23:57.022 INFO:tasks.workunit.client.0.smithi070.stdout:1/435: rename d8/l36 to d8/db/d2c/la1 0 2024-03-23T08:23:57.025 INFO:tasks.workunit.client.0.smithi070.stdout:1/436: creat d8/db/d40/fa2 x:0 0 0 2024-03-23T08:23:57.026 INFO:tasks.workunit.client.0.smithi070.stdout:2/607: truncate d1/d7/f66 3375551 0 2024-03-23T08:23:57.028 INFO:tasks.workunit.client.0.smithi070.stdout:6/587: dread d0/d23/d74/d84/db3/f8f [0,4194304] 0 2024-03-23T08:23:57.033 INFO:tasks.workunit.client.0.smithi070.stdout:9/624: dwrite db/d12/d22/d85/d42/fa4 [0,4194304] 0 2024-03-23T08:23:57.040 INFO:tasks.workunit.client.0.smithi070.stdout:2/608: mkdir d1/d7/d48/d69/db8/dc7 0 2024-03-23T08:23:57.042 INFO:tasks.workunit.client.0.smithi070.stdout:2/609: chown d1/d7/d39/d47/d7b/cc5 81187 1 2024-03-23T08:23:57.046 INFO:tasks.workunit.client.0.smithi070.stdout:2/610: symlink d1/d29/d52/lc8 0 2024-03-23T08:23:57.047 INFO:tasks.workunit.client.0.smithi070.stdout:1/437: dwrite d8/d1a/f39 [0,4194304] 0 2024-03-23T08:23:57.047 INFO:tasks.workunit.client.0.smithi070.stdout:6/588: link d0/d23/d74/d84/d3c/f51 d0/d23/d74/d84/db3/d2f/dab/db8/fca 0 2024-03-23T08:23:57.049 INFO:tasks.workunit.client.0.smithi070.stdout:1/438: write d8/db/f1f [1485594,59999] 0 2024-03-23T08:23:57.051 INFO:tasks.workunit.client.0.smithi070.stdout:2/611: write d1/d7/d39/f9c [84368,62591] 0 2024-03-23T08:23:57.053 INFO:tasks.workunit.client.0.smithi070.stdout:9/625: creat db/d12/d34/d8c/da2/fca x:0 0 0 2024-03-23T08:23:57.057 INFO:tasks.workunit.client.0.smithi070.stdout:6/589: dread d0/d23/d28/d35/f77 [0,4194304] 0 2024-03-23T08:23:57.058 INFO:tasks.workunit.client.0.smithi070.stdout:2/612: unlink d1/d29/d52/lc8 0 2024-03-23T08:23:57.069 INFO:tasks.workunit.client.0.smithi070.stdout:2/613: mknod d1/d7/d48/d69/cc9 0 2024-03-23T08:23:57.069 INFO:tasks.workunit.client.0.smithi070.stdout:1/439: write d8/d11/f5c [435702,77438] 0 2024-03-23T08:23:57.070 INFO:tasks.workunit.client.0.smithi070.stdout:9/626: rmdir db/d12/d22/d85/d42/dc8 0 2024-03-23T08:23:57.073 INFO:tasks.workunit.client.0.smithi070.stdout:9/627: dread db/d12/d14/f63 [0,4194304] 0 2024-03-23T08:23:57.074 INFO:tasks.workunit.client.0.smithi070.stdout:2/614: read d1/d7/d48/d69/d7a/fa5 [2719235,111102] 0 2024-03-23T08:23:57.074 INFO:tasks.workunit.client.0.smithi070.stdout:9/628: read db/d12/d22/fb0 [3795221,122498] 0 2024-03-23T08:23:57.075 INFO:tasks.workunit.client.0.smithi070.stdout:2/615: write d1/d7/f7e [4925563,49213] 0 2024-03-23T08:23:57.075 INFO:tasks.workunit.client.0.smithi070.stdout:1/440: rename d8/db/d40/f4c to d8/d1a/d8f/fa3 0 2024-03-23T08:23:57.084 INFO:tasks.workunit.client.0.smithi070.stdout:6/590: dwrite d0/d23/d28/f37 [0,4194304] 0 2024-03-23T08:23:57.086 INFO:tasks.workunit.client.0.smithi070.stdout:9/629: rename db/d12/d22/d85/l68 to db/d12/d34/d8c/da2/d26/d80/lcb 0 2024-03-23T08:23:57.089 INFO:tasks.workunit.client.0.smithi070.stdout:2/616: dwrite d1/d6/f4d [0,4194304] 0 2024-03-23T08:23:57.089 INFO:tasks.workunit.client.0.smithi070.stdout:2/617: chown d1/d7/d39/d58/f75 4528 1 2024-03-23T08:23:57.091 INFO:tasks.workunit.client.0.smithi070.stdout:1/441: symlink d8/db/d8e/la4 0 2024-03-23T08:23:57.095 INFO:tasks.workunit.client.0.smithi070.stdout:2/618: readlink d1/d8b/l93 0 2024-03-23T08:23:57.099 INFO:tasks.workunit.client.0.smithi070.stdout:1/442: rmdir d8/d26/d49 39 2024-03-23T08:23:57.100 INFO:tasks.workunit.client.0.smithi070.stdout:1/443: chown d8/d26/f38 114423 1 2024-03-23T08:23:57.102 INFO:tasks.workunit.client.0.smithi070.stdout:1/444: write d8/d11/d61/d74/d5d/f87 [4527883,96138] 0 2024-03-23T08:23:57.107 INFO:tasks.workunit.client.0.smithi070.stdout:2/619: dread d1/d7/f66 [0,4194304] 0 2024-03-23T08:23:57.108 INFO:tasks.workunit.client.0.smithi070.stdout:9/630: link db/d12/d67/cae db/d12/d22/d85/d42/ccc 0 2024-03-23T08:23:57.110 INFO:tasks.workunit.client.0.smithi070.stdout:6/591: getdents d0/d23/d4d/da5 0 2024-03-23T08:23:57.111 INFO:tasks.workunit.client.0.smithi070.stdout:9/631: fsync db/d12/d14/f23 0 2024-03-23T08:23:57.112 INFO:tasks.workunit.client.0.smithi070.stdout:2/620: creat d1/d7/d39/d58/d8f/fca x:0 0 0 2024-03-23T08:23:57.113 INFO:tasks.workunit.client.0.smithi070.stdout:2/621: fsync d1/d7/f17 0 2024-03-23T08:23:57.167 INFO:tasks.workunit.client.0.smithi070.stdout:2/622: unlink d1/fe 0 2024-03-23T08:23:57.195 INFO:tasks.workunit.client.0.smithi070.stdout:1/445: getdents d8/db/d8e 0 2024-03-23T08:23:57.196 INFO:tasks.workunit.client.0.smithi070.stdout:6/592: write d0/d3/d2c/f76 [338387,81648] 0 2024-03-23T08:23:57.197 INFO:tasks.workunit.client.0.smithi070.stdout:9/632: mkdir db/dcd 0 2024-03-23T08:23:57.197 INFO:tasks.workunit.client.0.smithi070.stdout:9/633: read db/d12/da1/fbe [2177447,19463] 0 2024-03-23T08:23:57.200 INFO:tasks.workunit.client.0.smithi070.stdout:2/623: rename d1/d7/d48/d69/d7a/lb9 to d1/d8b/lcb 0 2024-03-23T08:23:57.201 INFO:tasks.workunit.client.0.smithi070.stdout:1/446: dread - d8/db/d40/d43/f7a zero size 2024-03-23T08:23:57.205 INFO:tasks.workunit.client.0.smithi070.stdout:1/447: read - d8/d26/d49/d65/f77 zero size 2024-03-23T08:23:57.206 INFO:tasks.workunit.client.0.smithi070.stdout:1/448: mknod d8/d42/ca5 0 2024-03-23T08:23:57.207 INFO:tasks.workunit.client.0.smithi070.stdout:1/449: rmdir d8/db/d2c/d3f 39 2024-03-23T08:23:57.209 INFO:tasks.workunit.client.0.smithi070.stdout:1/450: readlink d8/db/d2c/d3f/d48/l52 0 2024-03-23T08:23:57.211 INFO:tasks.workunit.client.0.smithi070.stdout:6/593: write d0/d3/f7a [772394,60499] 0 2024-03-23T08:23:57.213 INFO:tasks.workunit.client.0.smithi070.stdout:9/634: write db/f65 [1468952,112393] 0 2024-03-23T08:23:57.216 INFO:tasks.workunit.client.0.smithi070.stdout:9/635: write db/d12/d34/d8c/da2/dc6/d7e/f75 [1410375,62287] 0 2024-03-23T08:23:57.226 INFO:tasks.workunit.client.0.smithi070.stdout:9/636: rename db/d12/d22 to db/d12/d34/d8c/da2/dc6/dce 0 2024-03-23T08:23:57.228 INFO:tasks.workunit.client.0.smithi070.stdout:6/594: write d0/d3/fc8 [776306,119138] 0 2024-03-23T08:23:57.228 INFO:tasks.workunit.client.0.smithi070.stdout:1/451: dread d8/f9 [4194304,4194304] 0 2024-03-23T08:23:57.229 INFO:tasks.workunit.client.0.smithi070.stdout:9/637: creat db/d12/db7/fcf x:0 0 0 2024-03-23T08:23:57.233 INFO:tasks.workunit.client.0.smithi070.stdout:6/595: symlink d0/d23/d4d/da5/lcb 0 2024-03-23T08:23:57.234 INFO:tasks.workunit.client.0.smithi070.stdout:1/452: symlink d8/db/d2c/d3f/la6 0 2024-03-23T08:23:57.236 INFO:tasks.workunit.client.0.smithi070.stdout:9/638: mknod db/d12/d14/cd0 0 2024-03-23T08:23:57.237 INFO:tasks.workunit.client.0.smithi070.stdout:6/596: creat d0/d23/d74/d84/db3/d38/d9a/fcc x:0 0 0 2024-03-23T08:23:57.238 INFO:tasks.workunit.client.0.smithi070.stdout:6/597: readlink d0/d23/d74/d84/db3/d38/d52/l99 0 2024-03-23T08:23:57.240 INFO:tasks.workunit.client.0.smithi070.stdout:1/453: link d8/d1a/f1e d8/db/d40/d43/d64/d85/d97/fa7 0 2024-03-23T08:23:57.241 INFO:tasks.workunit.client.0.smithi070.stdout:1/454: stat d8/db/l17 0 2024-03-23T08:23:57.242 INFO:tasks.workunit.client.0.smithi070.stdout:6/598: dread d0/d23/d74/d84/d3c/f53 [0,4194304] 0 2024-03-23T08:23:57.243 INFO:tasks.workunit.client.0.smithi070.stdout:6/599: chown d0/d23/d88/db6 1102066026 1 2024-03-23T08:23:57.244 INFO:tasks.workunit.client.0.smithi070.stdout:6/600: write d0/d3/d2c/f58 [1699543,91087] 0 2024-03-23T08:23:57.247 INFO:tasks.workunit.client.0.smithi070.stdout:6/601: mkdir d0/d23/d74/dcd 0 2024-03-23T08:23:57.254 INFO:tasks.workunit.client.0.smithi070.stdout:1/455: getdents d8/db/d2c/d3f 0 2024-03-23T08:23:57.256 INFO:tasks.workunit.client.0.smithi070.stdout:1/456: fsync d8/d11/f4d 0 2024-03-23T08:23:57.265 INFO:tasks.workunit.client.0.smithi070.stdout:1/457: link d8/d1a/f23 d8/d11/d61/d74/d5d/d70/fa8 0 2024-03-23T08:23:57.265 INFO:tasks.workunit.client.0.smithi070.stdout:1/458: fsync d8/d11/d61/f75 0 2024-03-23T08:23:57.282 INFO:tasks.workunit.client.0.smithi070.stdout:6/602: dwrite d0/d23/d74/d84/db3/d38/d9a/fcc [0,4194304] 0 2024-03-23T08:23:57.284 INFO:tasks.workunit.client.0.smithi070.stdout:6/603: write d0/d23/d74/d84/f3b [1999262,108001] 0 2024-03-23T08:23:57.286 INFO:tasks.workunit.client.0.smithi070.stdout:1/459: dwrite d8/d11/f4d [0,4194304] 0 2024-03-23T08:23:57.286 INFO:tasks.workunit.client.0.smithi070.stdout:6/604: chown d0/d3/f4 27 1 2024-03-23T08:23:57.294 INFO:tasks.workunit.client.0.smithi070.stdout:1/460: fdatasync d8/d1a/d8f/fa3 0 2024-03-23T08:23:57.295 INFO:tasks.workunit.client.0.smithi070.stdout:1/461: chown d8/d26/d49/d65/f77 28 1 2024-03-23T08:23:57.296 INFO:tasks.workunit.client.0.smithi070.stdout:1/462: chown d8/d11/f4d 24550 1 2024-03-23T08:23:57.298 INFO:tasks.workunit.client.0.smithi070.stdout:5/464: sync 2024-03-23T08:23:57.299 INFO:tasks.workunit.client.0.smithi070.stdout:8/521: sync 2024-03-23T08:23:57.302 INFO:tasks.workunit.client.0.smithi070.stdout:5/465: write dc/d1b/d1d/d25/d36/d32/d5c/f5d [454208,83268] 0 2024-03-23T08:23:57.304 INFO:tasks.workunit.client.0.smithi070.stdout:8/522: creat d5/d8/d16/d32/d84/d98/fc5 x:0 0 0 2024-03-23T08:23:57.306 INFO:tasks.workunit.client.0.smithi070.stdout:8/523: readlink d5/d8/d16/d32/l86 0 2024-03-23T08:23:57.306 INFO:tasks.workunit.client.0.smithi070.stdout:6/605: dread d0/d23/d28/d35/d85/f89 [0,4194304] 0 2024-03-23T08:23:57.308 INFO:tasks.workunit.client.0.smithi070.stdout:1/463: rename d8/d42/d4e/l53 to d8/d11/d61/d74/la9 0 2024-03-23T08:23:57.309 INFO:tasks.workunit.client.0.smithi070.stdout:1/464: dread - d8/d26/d41/f92 zero size 2024-03-23T08:23:57.311 INFO:tasks.workunit.client.0.smithi070.stdout:8/524: chown d5/d8/d16/d30/d38/d6b/l78 109272 1 2024-03-23T08:23:57.313 INFO:tasks.workunit.client.0.smithi070.stdout:6/606: fdatasync d0/d23/d28/d35/d85/f89 0 2024-03-23T08:23:57.313 INFO:tasks.workunit.client.0.smithi070.stdout:6/607: readlink d0/d23/d74/d84/db3/d38/l4e 0 2024-03-23T08:23:57.318 INFO:tasks.workunit.client.0.smithi070.stdout:5/466: getdents dc/d1b/d1d/d25/d38 0 2024-03-23T08:23:57.321 INFO:tasks.workunit.client.0.smithi070.stdout:1/465: link d8/d11/d61/d74/f71 d8/db/d2c/d3f/faa 0 2024-03-23T08:23:57.324 INFO:tasks.workunit.client.0.smithi070.stdout:6/608: fdatasync d0/d23/d74/d84/d3c/f75 0 2024-03-23T08:23:57.325 INFO:tasks.workunit.client.0.smithi070.stdout:6/609: chown d0/d23/d88/l9b 1847 1 2024-03-23T08:23:57.329 INFO:tasks.workunit.client.0.smithi070.stdout:6/610: dread d0/f9c [0,4194304] 0 2024-03-23T08:23:57.329 INFO:tasks.workunit.client.0.smithi070.stdout:6/611: chown d0/d23/d74/d84/db3/d38/l4e 90583 1 2024-03-23T08:23:57.334 INFO:tasks.workunit.client.0.smithi070.stdout:1/466: unlink d8/d26/d49/c96 0 2024-03-23T08:23:57.336 INFO:tasks.workunit.client.0.smithi070.stdout:6/612: creat d0/d23/d28/d35/d6a/fce x:0 0 0 2024-03-23T08:23:57.340 INFO:tasks.workunit.client.0.smithi070.stdout:1/467: dwrite d8/d42/d4e/f66 [4194304,4194304] 0 2024-03-23T08:23:57.343 INFO:tasks.workunit.client.0.smithi070.stdout:5/467: getdents dc 0 2024-03-23T08:23:57.350 INFO:tasks.workunit.client.0.smithi070.stdout:5/468: dwrite dc/d1b/d1d/d25/d38/d48/f85 [0,4194304] 0 2024-03-23T08:23:57.353 INFO:tasks.workunit.client.0.smithi070.stdout:5/469: write dc/d1b/d1d/d25/d38/d53/f60 [5206184,131020] 0 2024-03-23T08:23:57.354 INFO:tasks.workunit.client.0.smithi070.stdout:1/468: dread d8/d26/d49/d65/f98 [0,4194304] 0 2024-03-23T08:23:57.354 INFO:tasks.workunit.client.0.smithi070.stdout:6/613: link d0/d23/d74/d84/d3c/f75 d0/d23/d4d/da5/fcf 0 2024-03-23T08:23:57.361 INFO:tasks.workunit.client.0.smithi070.stdout:5/470: symlink dc/d1b/d1d/d25/l97 0 2024-03-23T08:23:57.362 INFO:tasks.workunit.client.0.smithi070.stdout:6/614: dwrite d0/d23/d28/d35/d85/f8b [0,4194304] 0 2024-03-23T08:23:57.367 INFO:tasks.workunit.client.0.smithi070.stdout:5/471: rename dc/d1b/f2f to dc/d1b/d1d/d25/d38/d48/d92/f98 0 2024-03-23T08:23:57.375 INFO:tasks.workunit.client.0.smithi070.stdout:1/469: dwrite d8/d1b/f25 [0,4194304] 0 2024-03-23T08:23:57.379 INFO:tasks.workunit.client.0.smithi070.stdout:6/615: dread d0/d23/d74/d84/db3/d38/d52/f7f [0,4194304] 0 2024-03-23T08:23:57.380 INFO:tasks.workunit.client.0.smithi070.stdout:1/470: mkdir d8/db/d2c/d3f/d48/dab 0 2024-03-23T08:23:57.383 INFO:tasks.workunit.client.0.smithi070.stdout:1/471: fdatasync d8/db/f16 0 2024-03-23T08:23:57.386 INFO:tasks.workunit.client.0.smithi070.stdout:5/472: dread dc/d1b/d1d/d25/d36/f27 [0,4194304] 0 2024-03-23T08:23:57.390 INFO:tasks.workunit.client.0.smithi070.stdout:1/472: rename d8/d26/d7b/l80 to d8/d11/lac 0 2024-03-23T08:23:57.395 INFO:tasks.workunit.client.0.smithi070.stdout:6/616: dread d0/d23/d28/f36 [0,4194304] 0 2024-03-23T08:23:57.397 INFO:tasks.workunit.client.0.smithi070.stdout:1/473: creat d8/d26/d41/fad x:0 0 0 2024-03-23T08:23:57.401 INFO:tasks.workunit.client.0.smithi070.stdout:5/473: getdents dc/d1b/d1d/d25/d38/d48 0 2024-03-23T08:23:57.402 INFO:tasks.workunit.client.0.smithi070.stdout:1/474: creat d8/d26/d41/fae x:0 0 0 2024-03-23T08:23:57.406 INFO:tasks.workunit.client.0.smithi070.stdout:5/474: mkdir dc/d1b/d1d/d25/d68/d8a/d99 0 2024-03-23T08:23:57.408 INFO:tasks.workunit.client.0.smithi070.stdout:5/475: creat dc/d1b/d1d/d25/d38/d53/f9a x:0 0 0 2024-03-23T08:23:57.409 INFO:tasks.workunit.client.0.smithi070.stdout:6/617: dread d0/d23/d74/d84/faa [0,4194304] 0 2024-03-23T08:23:57.412 INFO:tasks.workunit.client.0.smithi070.stdout:1/475: getdents d8/db 0 2024-03-23T08:23:57.412 INFO:tasks.workunit.client.0.smithi070.stdout:6/618: creat d0/d3/fd0 x:0 0 0 2024-03-23T08:23:57.413 INFO:tasks.workunit.client.0.smithi070.stdout:1/476: write d8/d26/d41/fad [369420,121021] 0 2024-03-23T08:23:57.418 INFO:tasks.workunit.client.0.smithi070.stdout:5/476: write dc/d1b/d1d/d25/d38/d39/f66 [483696,41743] 0 2024-03-23T08:23:57.427 INFO:tasks.workunit.client.0.smithi070.stdout:1/477: dwrite d8/db/f16 [0,4194304] 0 2024-03-23T08:23:57.428 INFO:tasks.workunit.client.0.smithi070.stdout:5/477: dwrite dc/f2b [0,4194304] 0 2024-03-23T08:23:57.429 INFO:tasks.workunit.client.0.smithi070.stdout:1/478: write d8/d1b/f86 [1699557,93786] 0 2024-03-23T08:23:57.437 INFO:tasks.workunit.client.0.smithi070.stdout:1/479: dread d8/d1b/f4b [0,4194304] 0 2024-03-23T08:23:57.438 INFO:tasks.workunit.client.0.smithi070.stdout:6/619: dread d0/d23/d74/d84/db3/d38/d52/f6b [0,4194304] 0 2024-03-23T08:23:57.439 INFO:tasks.workunit.client.0.smithi070.stdout:6/620: stat d0/d23/d74/d84/db3/d38/d52/c95 0 2024-03-23T08:23:57.445 INFO:tasks.workunit.client.0.smithi070.stdout:6/621: mkdir d0/d3/d2c/dd1 0 2024-03-23T08:23:57.447 INFO:tasks.workunit.client.0.smithi070.stdout:5/478: truncate dc/d1b/d1d/d25/d36/f2a 2704330 0 2024-03-23T08:23:57.448 INFO:tasks.workunit.client.0.smithi070.stdout:1/480: link d8/db/f27 d8/db/d2c/d3f/faf 0 2024-03-23T08:23:57.449 INFO:tasks.workunit.client.0.smithi070.stdout:1/481: fsync d8/db/f1f 0 2024-03-23T08:23:57.449 INFO:tasks.workunit.client.0.smithi070.stdout:5/479: write dc/d1b/d1d/d25/d38/d39/f79 [1553666,49577] 0 2024-03-23T08:23:57.450 INFO:tasks.workunit.client.0.smithi070.stdout:6/622: symlink d0/d23/d74/d84/db3/d2f/dab/ld2 0 2024-03-23T08:23:57.453 INFO:tasks.workunit.client.0.smithi070.stdout:1/482: creat d8/d1a/fb0 x:0 0 0 2024-03-23T08:23:57.454 INFO:tasks.workunit.client.0.smithi070.stdout:5/480: creat dc/d1b/d1d/d25/d36/d32/d5c/f9b x:0 0 0 2024-03-23T08:23:57.456 INFO:tasks.workunit.client.0.smithi070.stdout:6/623: truncate d0/d23/d28/d35/f77 151542 0 2024-03-23T08:23:57.456 INFO:tasks.workunit.client.0.smithi070.stdout:5/481: write dc/d1b/d1d/d25/d38/d39/f6b [326917,67875] 0 2024-03-23T08:23:57.457 INFO:tasks.workunit.client.0.smithi070.stdout:6/624: chown d0/d23/c90 516321 1 2024-03-23T08:23:57.462 INFO:tasks.workunit.client.0.smithi070.stdout:5/482: rename dc/d1b/d1d/d25/d38/d39/f3c to dc/d1b/d1d/d25/d68/d8a/f9c 0 2024-03-23T08:23:57.466 INFO:tasks.workunit.client.0.smithi070.stdout:1/483: dread d8/db/d2c/d3f/f56 [0,4194304] 0 2024-03-23T08:23:57.469 INFO:tasks.workunit.client.0.smithi070.stdout:5/483: getdents dc/d1b/d1d 0 2024-03-23T08:23:57.470 INFO:tasks.workunit.client.0.smithi070.stdout:5/484: readlink dc/d1b/d1d/d25/l47 0 2024-03-23T08:23:57.473 INFO:tasks.workunit.client.0.smithi070.stdout:6/625: dread d0/d3/f80 [0,4194304] 0 2024-03-23T08:23:57.475 INFO:tasks.workunit.client.0.smithi070.stdout:5/485: mknod dc/d1b/d1d/d25/d36/d32/c9d 0 2024-03-23T08:23:57.476 INFO:tasks.workunit.client.0.smithi070.stdout:6/626: rmdir d0/d23/d74/d84/db3/d38/d52 39 2024-03-23T08:23:57.481 INFO:tasks.workunit.client.0.smithi070.stdout:5/486: dwrite f1 [0,4194304] 0 2024-03-23T08:23:57.482 INFO:tasks.workunit.client.0.smithi070.stdout:6/627: mknod d0/cd3 0 2024-03-23T08:23:57.483 INFO:tasks.workunit.client.0.smithi070.stdout:5/487: chown dc/d1b/d1d/d25/d38/d39/f79 25017 1 2024-03-23T08:23:57.491 INFO:tasks.workunit.client.0.smithi070.stdout:5/488: dwrite dc/d1b/d1d/d25/d36/d32/d5c/f65 [0,4194304] 0 2024-03-23T08:23:57.492 INFO:tasks.workunit.client.0.smithi070.stdout:5/489: stat dc/d19/f2c 0 2024-03-23T08:23:57.500 INFO:tasks.workunit.client.0.smithi070.stdout:1/484: dwrite d8/db/d2c/d3f/d48/f58 [0,4194304] 0 2024-03-23T08:23:57.546 INFO:tasks.workunit.client.0.smithi070.stdout:1/485: fsync d8/d26/d49/d65/f77 0 2024-03-23T08:23:57.546 INFO:tasks.workunit.client.0.smithi070.stdout:7/483: sync 2024-03-23T08:23:57.549 INFO:tasks.workunit.client.0.smithi070.stdout:7/484: chown d25/d27/d6a/d44/d85/l8e 0 1 2024-03-23T08:23:57.552 INFO:tasks.workunit.client.0.smithi070.stdout:6/628: dwrite d0/d23/d4d/da5/fcf [0,4194304] 0 2024-03-23T08:23:57.552 INFO:tasks.workunit.client.0.smithi070.stdout:7/485: dread - d25/d27/d6a/d44/d85/faf zero size 2024-03-23T08:23:57.558 INFO:tasks.workunit.client.0.smithi070.stdout:1/486: mknod d8/d11/d61/d74/d5d/cb1 0 2024-03-23T08:23:57.562 INFO:tasks.workunit.client.0.smithi070.stdout:5/490: rename dc/c3f to dc/d1b/d1d/d25/c9e 0 2024-03-23T08:23:57.571 INFO:tasks.workunit.client.0.smithi070.stdout:5/491: creat dc/f9f x:0 0 0 2024-03-23T08:23:57.596 INFO:tasks.workunit.client.0.smithi070.stdout:1/487: creat d8/d11/fb2 x:0 0 0 2024-03-23T08:23:57.600 INFO:tasks.workunit.client.0.smithi070.stdout:1/488: link d8/d1a/l57 d8/db/d2c/d3f/lb3 0 2024-03-23T08:23:57.602 INFO:tasks.workunit.client.0.smithi070.stdout:1/489: fdatasync d8/d11/f5c 0 2024-03-23T08:23:57.606 INFO:tasks.workunit.client.0.smithi070.stdout:6/629: dwrite d0/d23/d88/f94 [0,4194304] 0 2024-03-23T08:23:57.607 INFO:tasks.workunit.client.0.smithi070.stdout:4/439: sync 2024-03-23T08:23:57.607 INFO:tasks.workunit.client.0.smithi070.stdout:3/682: sync 2024-03-23T08:23:57.607 INFO:tasks.workunit.client.0.smithi070.stdout:0/774: sync 2024-03-23T08:23:57.607 INFO:tasks.workunit.client.0.smithi070.stdout:8/525: sync 2024-03-23T08:23:57.607 INFO:tasks.workunit.client.0.smithi070.stdout:2/624: sync 2024-03-23T08:23:57.607 INFO:tasks.workunit.client.0.smithi070.stdout:9/639: sync 2024-03-23T08:23:57.610 INFO:tasks.workunit.client.0.smithi070.stdout:3/683: chown db/d4e/df8/d3d/cdf 4090 1 2024-03-23T08:23:57.613 INFO:tasks.workunit.client.0.smithi070.stdout:1/490: mknod d8/db/d40/d43/d64/d85/d97/cb4 0 2024-03-23T08:23:57.615 INFO:tasks.workunit.client.0.smithi070.stdout:5/492: truncate dc/d1b/d1d/d25/d38/d48/f57 602062 0 2024-03-23T08:23:57.618 INFO:tasks.workunit.client.0.smithi070.stdout:5/493: readlink dc/l95 0 2024-03-23T08:23:57.620 INFO:tasks.workunit.client.0.smithi070.stdout:4/440: dread d3/d6/f43 [0,4194304] 0 2024-03-23T08:23:57.621 INFO:tasks.workunit.client.0.smithi070.stdout:7/486: rename d25/d27/f60 to d25/d27/d6a/fb1 0 2024-03-23T08:23:57.622 INFO:tasks.workunit.client.0.smithi070.stdout:0/775: read d16/f4b [3256571,121048] 0 2024-03-23T08:23:57.622 INFO:tasks.workunit.client.0.smithi070.stdout:6/630: creat d0/d23/d4d/fd4 x:0 0 0 2024-03-23T08:23:57.623 INFO:tasks.workunit.client.0.smithi070.stdout:2/625: dwrite d1/d7/df/d94/d38/d4c/f6a [0,4194304] 0 2024-03-23T08:23:57.629 INFO:tasks.workunit.client.0.smithi070.stdout:8/526: rename d5/d8/d16/d32/l93 to d5/d8/d9a/lc6 0 2024-03-23T08:23:57.641 INFO:tasks.workunit.client.0.smithi070.stdout:0/776: dread d16/d26/f37 [0,4194304] 0 2024-03-23T08:23:57.647 INFO:tasks.workunit.client.0.smithi070.stdout:4/441: symlink d3/d6/d35/d65/d6f/l9f 0 2024-03-23T08:23:57.651 INFO:tasks.workunit.client.0.smithi070.stdout:1/491: truncate d8/d11/d61/f78 1917217 0 2024-03-23T08:23:57.654 INFO:tasks.workunit.client.0.smithi070.stdout:9/640: write db/d12/d34/d8c/da2/d4f/db6/faf [3342517,34499] 0 2024-03-23T08:23:57.659 INFO:tasks.workunit.client.0.smithi070.stdout:4/442: dwrite d3/d6/f3b [0,4194304] 0 2024-03-23T08:23:57.659 INFO:tasks.workunit.client.0.smithi070.stdout:8/527: dread d5/d8/d16/d3b/d4d/d6f/f89 [0,4194304] 0 2024-03-23T08:23:57.659 INFO:tasks.workunit.client.0.smithi070.stdout:5/494: truncate dc/d1b/d1d/d25/d38/d39/f4f 2427361 0 2024-03-23T08:23:57.666 INFO:tasks.workunit.client.0.smithi070.stdout:1/492: dread d8/f9 [0,4194304] 0 2024-03-23T08:23:57.666 INFO:tasks.workunit.client.0.smithi070.stdout:7/487: creat d25/d27/d6a/d44/d85/d95/da0/fb2 x:0 0 0 2024-03-23T08:23:57.667 INFO:tasks.workunit.client.0.smithi070.stdout:3/684: rename db/d14/d9c/ff3 to db/df/d1a/de9/fff 0 2024-03-23T08:23:57.677 INFO:tasks.workunit.client.0.smithi070.stdout:2/626: rmdir d1/d7/df/d94/d38/d4c/d76 0 2024-03-23T08:23:57.678 INFO:tasks.workunit.client.0.smithi070.stdout:2/627: truncate d1/d7/d39/f82 1747677 0 2024-03-23T08:23:57.685 INFO:tasks.workunit.client.0.smithi070.stdout:6/631: rename d0/d23/d74/d84/db3/d2f/cbb to d0/d23/d74/d84/d3c/d86/cd5 0 2024-03-23T08:23:57.686 INFO:tasks.workunit.client.0.smithi070.stdout:3/685: chown db/df/d4d/c84 0 1 2024-03-23T08:23:57.688 INFO:tasks.workunit.client.0.smithi070.stdout:6/632: truncate d0/d23/d4d/fd4 903657 0 2024-03-23T08:23:57.688 INFO:tasks.workunit.client.0.smithi070.stdout:4/443: mknod d3/d6/d35/d65/ca0 0 2024-03-23T08:23:57.690 INFO:tasks.workunit.client.0.smithi070.stdout:2/628: symlink d1/d6/d30/lcc 0 2024-03-23T08:23:57.691 INFO:tasks.workunit.client.0.smithi070.stdout:8/528: link d5/d8/d16/d30/f2f d5/d8/d16/d30/d38/d6b/d72/d88/fc7 0 2024-03-23T08:23:57.693 INFO:tasks.workunit.client.0.smithi070.stdout:7/488: symlink d25/d27/d6a/d44/d6f/da3/lb3 0 2024-03-23T08:23:57.695 INFO:tasks.workunit.client.0.smithi070.stdout:3/686: mkdir db/d7f/dcd/d100 0 2024-03-23T08:23:57.695 INFO:tasks.workunit.client.0.smithi070.stdout:3/687: write db/d14/d65/dae/fb8 [290134,58892] 0 2024-03-23T08:23:57.696 INFO:tasks.workunit.client.0.smithi070.stdout:6/633: mkdir d0/d23/d28/d35/dd6 0 2024-03-23T08:23:57.699 INFO:tasks.workunit.client.0.smithi070.stdout:4/444: symlink d3/d6/d35/d65/d27/d4b/d73/d84/la1 0 2024-03-23T08:23:57.700 INFO:tasks.workunit.client.0.smithi070.stdout:2/629: creat d1/d7/d48/d69/d7a/fcd x:0 0 0 2024-03-23T08:23:57.700 INFO:tasks.workunit.client.0.smithi070.stdout:4/445: write d3/d6/d35/d65/f3e [918269,3815] 0 2024-03-23T08:23:57.704 INFO:tasks.workunit.client.0.smithi070.stdout:3/688: creat db/d4e/d92/f101 x:0 0 0 2024-03-23T08:23:57.705 INFO:tasks.workunit.client.0.smithi070.stdout:1/493: getdents d8/d11/d61/d74 0 2024-03-23T08:23:57.706 INFO:tasks.workunit.client.0.smithi070.stdout:8/529: symlink d5/d8/d16/d32/d84/da7/lc8 0 2024-03-23T08:23:57.707 INFO:tasks.workunit.client.0.smithi070.stdout:1/494: read - d8/db/d2c/d3f/f5f zero size 2024-03-23T08:23:57.708 INFO:tasks.workunit.client.0.smithi070.stdout:2/630: dwrite d1/d7/df/d94/d38/f87 [0,4194304] 0 2024-03-23T08:23:57.708 INFO:tasks.workunit.client.0.smithi070.stdout:4/446: symlink d3/d6/d6a/la2 0 2024-03-23T08:23:57.709 INFO:tasks.workunit.client.0.smithi070.stdout:2/631: stat d1/d6/d34/da2 0 2024-03-23T08:23:57.710 INFO:tasks.workunit.client.0.smithi070.stdout:6/634: mknod d0/d23/d74/d84/cd7 0 2024-03-23T08:23:57.711 INFO:tasks.workunit.client.0.smithi070.stdout:3/689: mknod db/df/d1a/c102 0 2024-03-23T08:23:57.716 INFO:tasks.workunit.client.0.smithi070.stdout:2/632: fsync d1/d21/fa9 0 2024-03-23T08:23:57.719 INFO:tasks.workunit.client.0.smithi070.stdout:2/633: readlink d1/d7/df/la1 0 2024-03-23T08:23:57.720 INFO:tasks.workunit.client.0.smithi070.stdout:3/690: creat db/d7f/dcd/f103 x:0 0 0 2024-03-23T08:23:57.721 INFO:tasks.workunit.client.0.smithi070.stdout:6/635: dwrite d0/d23/d88/f94 [0,4194304] 0 2024-03-23T08:23:57.724 INFO:tasks.workunit.client.0.smithi070.stdout:4/447: rename d3/d6/d35/d65/d27/d4b/d73/d89/d92 to d3/d6/d35/d65/d27/d4b/d73/d74/da3 0 2024-03-23T08:23:57.726 INFO:tasks.workunit.client.0.smithi070.stdout:7/489: getdents d25/d27 0 2024-03-23T08:23:57.728 INFO:tasks.workunit.client.0.smithi070.stdout:9/641: write db/d12/d34/d8c/da2/d4f/d5a/f9c [904436,120626] 0 2024-03-23T08:23:57.732 INFO:tasks.workunit.client.0.smithi070.stdout:3/691: creat db/d4e/dd6/dd7/dd9/f104 x:0 0 0 2024-03-23T08:23:57.733 INFO:tasks.workunit.client.0.smithi070.stdout:5/495: write dc/d1b/d1d/d25/d38/f86 [5231270,131018] 0 2024-03-23T08:23:57.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:57 smithi070 ceph-mon[39699]: pgmap v38: 65 pgs: 65 active+clean; 1.1 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 23 MiB/s rd, 115 MiB/s wr, 243 op/s 2024-03-23T08:23:57.737 INFO:tasks.workunit.client.0.smithi070.stdout:4/448: creat d3/d6/d35/d65/d27/d4b/d73/fa4 x:0 0 0 2024-03-23T08:23:57.738 INFO:tasks.workunit.client.0.smithi070.stdout:2/634: dwrite d1/d29/f35 [0,4194304] 0 2024-03-23T08:23:57.738 INFO:tasks.workunit.client.0.smithi070.stdout:3/692: stat db/d7f/c82 0 2024-03-23T08:23:57.738 INFO:tasks.workunit.client.0.smithi070.stdout:3/693: readlink db/l5b 0 2024-03-23T08:23:57.744 INFO:tasks.workunit.client.0.smithi070.stdout:9/642: mknod db/d12/d34/d8c/da2/d58/db5/cd1 0 2024-03-23T08:23:57.744 INFO:tasks.workunit.client.0.smithi070.stdout:5/496: symlink dc/d1b/d1d/la0 0 2024-03-23T08:23:57.752 INFO:tasks.workunit.client.0.smithi070.stdout:2/635: mknod d1/d7/d39/d47/cce 0 2024-03-23T08:23:57.752 INFO:tasks.workunit.client.0.smithi070.stdout:7/490: dread f24 [0,4194304] 0 2024-03-23T08:23:57.753 INFO:tasks.workunit.client.0.smithi070.stdout:0/777: truncate d16/d26/d27/f45 2797008 0 2024-03-23T08:23:57.755 INFO:tasks.workunit.client.0.smithi070.stdout:2/636: chown d1/d7/df/d94/d38/f87 30 1 2024-03-23T08:23:57.771 INFO:tasks.workunit.client.0.smithi070.stdout:7/491: mkdir d25/d27/d6a/d44/d7c/db4 0 2024-03-23T08:23:57.773 INFO:tasks.workunit.client.0.smithi070.stdout:2/637: dwrite d1/d21/d5f/f67 [0,4194304] 0 2024-03-23T08:23:57.773 INFO:tasks.workunit.client.0.smithi070.stdout:4/449: link d3/d6/d35/d65/d6f/d93/c98 d3/d6/d35/d65/d41/d56/ca5 0 2024-03-23T08:23:57.774 INFO:tasks.workunit.client.0.smithi070.stdout:4/450: fdatasync d3/d6/d9/f6b 0 2024-03-23T08:23:57.776 INFO:tasks.workunit.client.0.smithi070.stdout:0/778: rename d16/l19 to d16/d26/d27/dee/l110 0 2024-03-23T08:23:57.777 INFO:tasks.workunit.client.0.smithi070.stdout:3/694: getdents db/d14/d6f 0 2024-03-23T08:23:57.782 INFO:tasks.workunit.client.0.smithi070.stdout:4/451: chown d3/d6/d35/c95 36176 1 2024-03-23T08:23:57.787 INFO:tasks.workunit.client.0.smithi070.stdout:7/492: mkdir d25/d27/d6a/d44/da2/db5 0 2024-03-23T08:23:57.790 INFO:tasks.workunit.client.0.smithi070.stdout:4/452: creat d3/d6/d35/d65/d27/d4b/d73/d84/d88/fa6 x:0 0 0 2024-03-23T08:23:57.790 INFO:tasks.workunit.client.0.smithi070.stdout:7/493: rmdir d25/d40 39 2024-03-23T08:23:57.792 INFO:tasks.workunit.client.0.smithi070.stdout:3/695: dread db/df/d6a/fa1 [0,4194304] 0 2024-03-23T08:23:57.794 INFO:tasks.workunit.client.0.smithi070.stdout:7/494: creat d25/d27/d6a/d44/d7c/db4/fb6 x:0 0 0 2024-03-23T08:23:57.794 INFO:tasks.workunit.client.0.smithi070.stdout:3/696: write db/d4e/d92/f101 [693517,47166] 0 2024-03-23T08:23:57.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:57 smithi193 ceph-mon[48370]: pgmap v38: 65 pgs: 65 active+clean; 1.1 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 23 MiB/s rd, 115 MiB/s wr, 243 op/s 2024-03-23T08:23:57.831 INFO:tasks.workunit.client.0.smithi070.stdout:8/530: write d5/d8/d16/d32/f56 [632012,122668] 0 2024-03-23T08:23:57.835 INFO:tasks.workunit.client.0.smithi070.stdout:2/638: write d1/d7/d48/d69/d7a/fa5 [157451,11601] 0 2024-03-23T08:23:57.838 INFO:tasks.workunit.client.0.smithi070.stdout:4/453: dwrite d3/d6/d35/d65/d27/d4b/d73/fa4 [0,4194304] 0 2024-03-23T08:23:57.844 INFO:tasks.workunit.client.0.smithi070.stdout:9/643: dwrite db/d12/d34/d8c/da2/d26/f30 [0,4194304] 0 2024-03-23T08:23:57.844 INFO:tasks.workunit.client.0.smithi070.stdout:5/497: rmdir dc/d1b/d1d/d25/d36 39 2024-03-23T08:23:57.844 INFO:tasks.workunit.client.0.smithi070.stdout:3/697: write db/d14/f3f [505619,30029] 0 2024-03-23T08:23:57.844 INFO:tasks.workunit.client.0.smithi070.stdout:4/454: truncate d3/d6/d6a/d7c/f9c 725840 0 2024-03-23T08:23:57.846 INFO:tasks.workunit.client.0.smithi070.stdout:3/698: readlink db/d20/d49/l8f 0 2024-03-23T08:23:57.848 INFO:tasks.workunit.client.0.smithi070.stdout:0/779: dread d16/f34 [0,4194304] 0 2024-03-23T08:23:57.849 INFO:tasks.workunit.client.0.smithi070.stdout:4/455: dread d3/d6/d9/f2e [0,4194304] 0 2024-03-23T08:23:57.850 INFO:tasks.workunit.client.0.smithi070.stdout:8/531: unlink d5/f2a 0 2024-03-23T08:23:57.851 INFO:tasks.workunit.client.0.smithi070.stdout:7/495: creat d25/d27/d6a/d44/d7c/db4/fb7 x:0 0 0 2024-03-23T08:23:57.851 INFO:tasks.workunit.client.0.smithi070.stdout:3/699: mkdir db/d4e/d105 0 2024-03-23T08:23:57.851 INFO:tasks.workunit.client.0.smithi070.stdout:4/456: write d3/d6/d11/f2d [2140849,123343] 0 2024-03-23T08:23:57.854 INFO:tasks.workunit.client.0.smithi070.stdout:4/457: write d3/d6/d6a/f58 [222261,5962] 0 2024-03-23T08:23:57.857 INFO:tasks.workunit.client.0.smithi070.stdout:4/458: read d3/d6/d35/d65/d41/d56/f8d [89884,74183] 0 2024-03-23T08:23:57.857 INFO:tasks.workunit.client.0.smithi070.stdout:5/498: rename dc/d1b/d1d/d25/d68/f96 to dc/d1b/d1d/d25/d38/d53/fa1 0 2024-03-23T08:23:57.867 INFO:tasks.workunit.client.0.smithi070.stdout:3/700: mknod db/df/d6a/c106 0 2024-03-23T08:23:57.868 INFO:tasks.workunit.client.0.smithi070.stdout:0/780: unlink d16/fda 0 2024-03-23T08:23:57.872 INFO:tasks.workunit.client.0.smithi070.stdout:9/644: dwrite db/d12/da1/fbe [0,4194304] 0 2024-03-23T08:23:57.873 INFO:tasks.workunit.client.0.smithi070.stdout:8/532: write d5/d8/d16/d30/f2f [3722716,77636] 0 2024-03-23T08:23:57.876 INFO:tasks.workunit.client.0.smithi070.stdout:7/496: write f12 [3098988,68191] 0 2024-03-23T08:23:57.876 INFO:tasks.workunit.client.0.smithi070.stdout:2/639: link d1/d29/d52/cab d1/d7/df/ccf 0 2024-03-23T08:23:57.898 INFO:tasks.workunit.client.0.smithi070.stdout:3/701: unlink db/d14/d9c/f9f 0 2024-03-23T08:23:57.899 INFO:tasks.workunit.client.0.smithi070.stdout:0/781: mknod d16/d22/d43/c111 0 2024-03-23T08:23:57.899 INFO:tasks.workunit.client.0.smithi070.stdout:3/702: chown db/df 23246999 1 2024-03-23T08:23:57.902 INFO:tasks.workunit.client.0.smithi070.stdout:5/499: creat dc/d1b/d1d/d25/d36/d32/d5c/fa2 x:0 0 0 2024-03-23T08:23:57.903 INFO:tasks.workunit.client.0.smithi070.stdout:8/533: unlink d5/d8/f19 0 2024-03-23T08:23:57.904 INFO:tasks.workunit.client.0.smithi070.stdout:4/459: getdents d3/d6/d96 0 2024-03-23T08:23:57.905 INFO:tasks.workunit.client.0.smithi070.stdout:4/460: fsync d3/d6/d6a/f58 0 2024-03-23T08:23:57.911 INFO:tasks.workunit.client.0.smithi070.stdout:3/703: rmdir db/d4e 39 2024-03-23T08:23:57.914 INFO:tasks.workunit.client.0.smithi070.stdout:8/534: dread d5/f23 [0,4194304] 0 2024-03-23T08:23:57.921 INFO:tasks.workunit.client.0.smithi070.stdout:4/461: creat d3/d6/d35/d65/d27/d4b/d73/d84/fa7 x:0 0 0 2024-03-23T08:23:57.925 INFO:tasks.workunit.client.0.smithi070.stdout:3/704: truncate db/df/d5d/fa9 249879 0 2024-03-23T08:23:57.926 INFO:tasks.workunit.client.0.smithi070.stdout:8/535: mkdir d5/d8/d16/d3b/d4d/dc9 0 2024-03-23T08:23:57.928 INFO:tasks.workunit.client.0.smithi070.stdout:4/462: dwrite d3/d6/d11/f55 [0,4194304] 0 2024-03-23T08:23:57.931 INFO:tasks.workunit.client.0.smithi070.stdout:7/497: creat d25/fb8 x:0 0 0 2024-03-23T08:23:57.933 INFO:tasks.workunit.client.0.smithi070.stdout:9/645: link db/d12/d67/cae db/d12/db7/cd2 0 2024-03-23T08:23:57.935 INFO:tasks.workunit.client.0.smithi070.stdout:0/782: link d16/d26/d27/d4c/c101 d16/d26/d27/d86/d67/d109/c112 0 2024-03-23T08:23:57.936 INFO:tasks.workunit.client.0.smithi070.stdout:0/783: chown d16/d26/d27/d4c/d60/c7f 10146872 1 2024-03-23T08:23:57.937 INFO:tasks.workunit.client.0.smithi070.stdout:4/463: creat d3/d6/d35/d65/d41/d56/fa8 x:0 0 0 2024-03-23T08:23:57.937 INFO:tasks.workunit.client.0.smithi070.stdout:2/640: link d1/d7/df/c2a d1/d29/cd0 0 2024-03-23T08:23:57.938 INFO:tasks.workunit.client.0.smithi070.stdout:7/498: creat d25/d27/d6a/d44/d85/d95/fb9 x:0 0 0 2024-03-23T08:23:57.940 INFO:tasks.workunit.client.0.smithi070.stdout:4/464: chown d3/d6/d35/d65/d27/d4b/d73/fa4 938692 1 2024-03-23T08:23:57.940 INFO:tasks.workunit.client.0.smithi070.stdout:2/641: read d1/d8b/faa [1725382,83092] 0 2024-03-23T08:23:57.945 INFO:tasks.workunit.client.0.smithi070.stdout:8/536: link d5/d8/d16/d30/d38/l42 d5/d8/d16/d8d/da3/lca 0 2024-03-23T08:23:57.948 INFO:tasks.workunit.client.0.smithi070.stdout:7/499: dwrite d25/d27/fac [0,4194304] 0 2024-03-23T08:23:57.950 INFO:tasks.workunit.client.0.smithi070.stdout:2/642: unlink d1/d7/d48/d5b/c6c 0 2024-03-23T08:23:57.952 INFO:tasks.workunit.client.0.smithi070.stdout:8/537: dread d5/d8/d9a/da2/dae/f94 [0,4194304] 0 2024-03-23T08:23:57.957 INFO:tasks.workunit.client.0.smithi070.stdout:4/465: dread d3/d6/d35/d65/d1b/f90 [0,4194304] 0 2024-03-23T08:23:57.958 INFO:tasks.workunit.client.0.smithi070.stdout:8/538: creat d5/d8/d16/d3b/d4d/fcb x:0 0 0 2024-03-23T08:23:57.959 INFO:tasks.workunit.client.0.smithi070.stdout:2/643: dwrite d1/d7/df/d94/d38/fa3 [0,4194304] 0 2024-03-23T08:23:57.959 INFO:tasks.workunit.client.0.smithi070.stdout:4/466: chown d3/d6/d35/d65/d27/d4b/c54 0 1 2024-03-23T08:23:57.965 INFO:tasks.workunit.client.0.smithi070.stdout:2/644: mkdir d1/d7/d48/d69/db8/dd1 0 2024-03-23T08:23:57.966 INFO:tasks.workunit.client.0.smithi070.stdout:0/784: dread d16/f51 [0,4194304] 0 2024-03-23T08:23:57.966 INFO:tasks.workunit.client.0.smithi070.stdout:3/705: write db/df/d5d/fcf [810762,10015] 0 2024-03-23T08:23:57.970 INFO:tasks.workunit.client.0.smithi070.stdout:4/467: mkdir d3/d6/d35/d3f/da9 0 2024-03-23T08:23:57.972 INFO:tasks.workunit.client.0.smithi070.stdout:8/539: mkdir d5/d8/dcc 0 2024-03-23T08:23:57.976 INFO:tasks.workunit.client.0.smithi070.stdout:3/706: truncate db/df/d6a/fa2 321571 0 2024-03-23T08:23:57.978 INFO:tasks.workunit.client.0.smithi070.stdout:9/646: dwrite db/d12/f17 [4194304,4194304] 0 2024-03-23T08:23:57.981 INFO:tasks.workunit.client.0.smithi070.stdout:9/647: dread - db/d12/d34/d8c/da2/d26/da7/fac zero size 2024-03-23T08:23:57.985 INFO:tasks.workunit.client.0.smithi070.stdout:0/785: dwrite d16/f34 [0,4194304] 0 2024-03-23T08:23:57.988 INFO:tasks.workunit.client.0.smithi070.stdout:3/707: chown db/d4e/d92 16066 1 2024-03-23T08:23:57.990 INFO:tasks.workunit.client.0.smithi070.stdout:3/708: chown db/d14/d43/d62/d95/l96 201720306 1 2024-03-23T08:23:57.991 INFO:tasks.workunit.client.0.smithi070.stdout:2/645: write d1/d21/d5f/da4/dc3/f3c [2029118,7256] 0 2024-03-23T08:23:57.993 INFO:tasks.workunit.client.0.smithi070.stdout:8/540: dwrite d5/d8/f9d [0,4194304] 0 2024-03-23T08:23:57.994 INFO:tasks.workunit.client.0.smithi070.stdout:3/709: truncate db/d20/fee 4423427 0 2024-03-23T08:23:57.996 INFO:tasks.workunit.client.0.smithi070.stdout:3/710: chown db/d7f/ce5 155 1 2024-03-23T08:23:58.003 INFO:tasks.workunit.client.0.smithi070.stdout:9/648: mknod db/d12/d34/d8c/da2/dc6/dce/cd3 0 2024-03-23T08:23:58.003 INFO:tasks.workunit.client.0.smithi070.stdout:4/468: creat d3/d6/d35/d65/faa x:0 0 0 2024-03-23T08:23:58.004 INFO:tasks.workunit.client.0.smithi070.stdout:2/646: creat d1/d6/d30/fd2 x:0 0 0 2024-03-23T08:23:58.005 INFO:tasks.workunit.client.0.smithi070.stdout:0/786: mkdir d16/d26/d27/d3d/d4d/dca/d113 0 2024-03-23T08:23:58.005 INFO:tasks.workunit.client.0.smithi070.stdout:8/541: mknod d5/d8/d16/d32/d84/ccd 0 2024-03-23T08:23:58.007 INFO:tasks.workunit.client.0.smithi070.stdout:2/647: symlink d1/d7/d48/d69/d7a/ld3 0 2024-03-23T08:23:58.011 INFO:tasks.workunit.client.0.smithi070.stdout:2/648: mknod d1/d7/d39/d47/d7b/cd4 0 2024-03-23T08:23:58.013 INFO:tasks.workunit.client.0.smithi070.stdout:0/787: truncate d16/d22/d43/d47/d87/f92 2867380 0 2024-03-23T08:23:58.013 INFO:tasks.workunit.client.0.smithi070.stdout:4/469: dwrite d3/d6/d9/f6b [0,4194304] 0 2024-03-23T08:23:58.014 INFO:tasks.workunit.client.0.smithi070.stdout:4/470: chown d3/f17 1 1 2024-03-23T08:23:58.015 INFO:tasks.workunit.client.0.smithi070.stdout:8/542: truncate d5/d8/f57 1117425 0 2024-03-23T08:23:58.020 INFO:tasks.workunit.client.0.smithi070.stdout:2/649: symlink d1/d7/d39/d47/d7d/ld5 0 2024-03-23T08:23:58.021 INFO:tasks.workunit.client.0.smithi070.stdout:0/788: symlink d16/d22/d43/d47/l114 0 2024-03-23T08:23:58.023 INFO:tasks.workunit.client.0.smithi070.stdout:3/711: truncate db/d7f/fbe 3205868 0 2024-03-23T08:23:58.025 INFO:tasks.workunit.client.0.smithi070.stdout:4/471: chown d3/d6/d35/d65/d41/d56/ca5 14529 1 2024-03-23T08:23:58.026 INFO:tasks.workunit.client.0.smithi070.stdout:8/543: stat d5/d8/d16/d8d/da3/lca 0 2024-03-23T08:23:58.027 INFO:tasks.workunit.client.0.smithi070.stdout:4/472: dread - d3/d6/d35/d65/faa zero size 2024-03-23T08:23:58.029 INFO:tasks.workunit.client.0.smithi070.stdout:2/650: dwrite d1/d7/d48/d69/d7a/fa5 [0,4194304] 0 2024-03-23T08:23:58.033 INFO:tasks.workunit.client.0.smithi070.stdout:8/544: fsync d5/d8/f79 0 2024-03-23T08:23:58.033 INFO:tasks.workunit.client.0.smithi070.stdout:4/473: symlink d3/d6/d9/lab 0 2024-03-23T08:23:58.035 INFO:tasks.workunit.client.0.smithi070.stdout:4/474: write d3/f6d [1050304,63419] 0 2024-03-23T08:23:58.035 INFO:tasks.workunit.client.0.smithi070.stdout:0/789: mkdir d16/d22/df0/df5/d115 0 2024-03-23T08:23:58.037 INFO:tasks.workunit.client.0.smithi070.stdout:4/475: readlink d3/l24 0 2024-03-23T08:23:58.037 INFO:tasks.workunit.client.0.smithi070.stdout:3/712: symlink db/d4e/dd6/dd7/dd9/de8/l107 0 2024-03-23T08:23:58.039 INFO:tasks.workunit.client.0.smithi070.stdout:2/651: dwrite d1/d21/d5f/fa8 [0,4194304] 0 2024-03-23T08:23:58.046 INFO:tasks.workunit.client.0.smithi070.stdout:4/476: chown d3/d6/d35/d65/d27/d4b/d73/d84/fa7 7126578 1 2024-03-23T08:23:58.046 INFO:tasks.workunit.client.0.smithi070.stdout:8/545: chown d5/d8/d16/f2b 232 1 2024-03-23T08:23:58.048 INFO:tasks.workunit.client.0.smithi070.stdout:0/790: mknod d16/d22/df0/df5/c116 0 2024-03-23T08:23:58.053 INFO:tasks.workunit.client.0.smithi070.stdout:3/713: dwrite db/df/d1a/f50 [0,4194304] 0 2024-03-23T08:23:58.053 INFO:tasks.workunit.client.0.smithi070.stdout:8/546: truncate d5/d8/d16/d3b/f5f 159110 0 2024-03-23T08:23:58.054 INFO:tasks.workunit.client.0.smithi070.stdout:2/652: dread d1/d6/d30/f46 [0,4194304] 0 2024-03-23T08:23:58.056 INFO:tasks.workunit.client.0.smithi070.stdout:4/477: mknod d3/d6/d35/d65/d41/cac 0 2024-03-23T08:23:58.057 INFO:tasks.workunit.client.0.smithi070.stdout:4/478: write d3/d6/d35/d65/d27/d4b/d73/fa4 [4214103,49029] 0 2024-03-23T08:23:58.058 INFO:tasks.workunit.client.0.smithi070.stdout:0/791: truncate d16/d22/fa6 2377646 0 2024-03-23T08:23:58.059 INFO:tasks.workunit.client.0.smithi070.stdout:0/792: stat d16/d26/d27/d3d/d4d/dca/d113 0 2024-03-23T08:23:58.060 INFO:tasks.workunit.client.0.smithi070.stdout:8/547: fsync d5/d8/f4e 0 2024-03-23T08:23:58.061 INFO:tasks.workunit.client.0.smithi070.stdout:2/653: symlink d1/d21/ld6 0 2024-03-23T08:23:58.061 INFO:tasks.workunit.client.0.smithi070.stdout:0/793: dread - d16/d26/d27/d3d/d4d/ffb zero size 2024-03-23T08:23:58.069 INFO:tasks.workunit.client.0.smithi070.stdout:0/794: unlink d16/d26/d27/f10c 0 2024-03-23T08:23:58.070 INFO:tasks.workunit.client.0.smithi070.stdout:8/548: rename d5/d8/d16/d3b/d4d/d6f/fbd to d5/d8/d9a/da2/dae/fce 0 2024-03-23T08:23:58.072 INFO:tasks.workunit.client.0.smithi070.stdout:3/714: mknod db/d14/d65/da8/dfb/c108 0 2024-03-23T08:23:58.073 INFO:tasks.workunit.client.0.smithi070.stdout:3/715: dread - db/d4e/dd6/dd7/dd9/f104 zero size 2024-03-23T08:23:58.074 INFO:tasks.workunit.client.0.smithi070.stdout:3/716: fdatasync db/d14/d43/f54 0 2024-03-23T08:23:58.074 INFO:tasks.workunit.client.0.smithi070.stdout:0/795: dread d16/d26/d27/d86/d67/d7d/fc2 [0,4194304] 0 2024-03-23T08:23:58.083 INFO:tasks.workunit.client.0.smithi070.stdout:8/549: write d5/d8/d16/d3b/d4d/d6f/f89 [3530463,114102] 0 2024-03-23T08:23:58.086 INFO:tasks.workunit.client.0.smithi070.stdout:0/796: fdatasync d16/d26/f5d 0 2024-03-23T08:23:58.092 INFO:tasks.workunit.client.0.smithi070.stdout:4/479: dwrite d3/f17 [0,4194304] 0 2024-03-23T08:23:58.092 INFO:tasks.workunit.client.0.smithi070.stdout:2/654: rename d1/d7/df/c2a to d1/d7/d48/d5b/cd7 0 2024-03-23T08:23:58.103 INFO:tasks.workunit.client.0.smithi070.stdout:0/797: symlink d16/d26/d27/d3d/d4d/dca/l117 0 2024-03-23T08:23:58.112 INFO:tasks.workunit.client.0.smithi070.stdout:8/550: dread d5/d8/d16/f22 [4194304,4194304] 0 2024-03-23T08:23:58.112 INFO:tasks.workunit.client.0.smithi070.stdout:2/655: symlink d1/ld8 0 2024-03-23T08:23:58.114 INFO:tasks.workunit.client.0.smithi070.stdout:2/656: write d1/d7/d39/d47/fb1 [891913,106282] 0 2024-03-23T08:23:58.117 INFO:tasks.workunit.client.0.smithi070.stdout:2/657: chown d1/d6/d30/lcc 26 1 2024-03-23T08:23:58.118 INFO:tasks.workunit.client.0.smithi070.stdout:0/798: dwrite d16/d26/d27/d86/d67/d7d/fc2 [0,4194304] 0 2024-03-23T08:23:58.119 INFO:tasks.workunit.client.0.smithi070.stdout:0/799: chown d16/d22/d43/c55 249673581 1 2024-03-23T08:23:58.157 INFO:tasks.workunit.client.0.smithi070.stdout:3/717: getdents db/d14 0 2024-03-23T08:23:58.161 INFO:tasks.workunit.client.0.smithi070.stdout:2/658: rmdir d1/d6 39 2024-03-23T08:23:58.162 INFO:tasks.workunit.client.0.smithi070.stdout:0/800: mknod d16/d26/d27/d86/d9e/ddd/c118 0 2024-03-23T08:23:58.164 INFO:tasks.workunit.client.0.smithi070.stdout:4/480: rmdir d3/d6/d96 0 2024-03-23T08:23:58.166 INFO:tasks.workunit.client.0.smithi070.stdout:3/718: dread db/df/d5d/fa9 [0,4194304] 0 2024-03-23T08:23:58.168 INFO:tasks.workunit.client.0.smithi070.stdout:3/719: write db/df/d1a/de9/ff7 [790701,126822] 0 2024-03-23T08:23:58.170 INFO:tasks.workunit.client.0.smithi070.stdout:4/481: unlink d3/d6/d35/d65/d27/d4b/d73/d74/d7e/c8c 0 2024-03-23T08:23:58.172 INFO:tasks.workunit.client.0.smithi070.stdout:8/551: write d5/d8/d16/f2b [1458319,118607] 0 2024-03-23T08:23:58.176 INFO:tasks.workunit.client.0.smithi070.stdout:8/552: chown d5/d8/d9a/da2/dae/l92 449 1 2024-03-23T08:23:58.176 INFO:tasks.workunit.client.0.smithi070.stdout:0/801: dwrite d16/f100 [0,4194304] 0 2024-03-23T08:23:58.178 INFO:tasks.workunit.client.0.smithi070.stdout:4/482: symlink d3/d6/d35/d65/d27/d4b/d73/d89/lad 0 2024-03-23T08:23:58.178 INFO:tasks.workunit.client.0.smithi070.stdout:3/720: link db/d20/c94 db/d20/d49/c109 0 2024-03-23T08:23:58.179 INFO:tasks.workunit.client.0.smithi070.stdout:2/659: dread d1/d7/f20 [0,4194304] 0 2024-03-23T08:23:58.182 INFO:tasks.workunit.client.0.smithi070.stdout:8/553: creat d5/d8/d16/d3b/d4d/d6f/fcf x:0 0 0 2024-03-23T08:23:58.183 INFO:tasks.workunit.client.0.smithi070.stdout:8/554: readlink d5/d8/d16/d8d/da3/lca 0 2024-03-23T08:23:58.186 INFO:tasks.workunit.client.0.smithi070.stdout:0/802: mknod d16/c119 0 2024-03-23T08:23:58.187 INFO:tasks.workunit.client.0.smithi070.stdout:0/803: chown d16/d22/f2f 317973684 1 2024-03-23T08:23:58.188 INFO:tasks.workunit.client.0.smithi070.stdout:2/660: fsync d1/d7/d39/d47/d7d/fb4 0 2024-03-23T08:23:58.193 INFO:tasks.workunit.client.0.smithi070.stdout:0/804: mkdir d16/d26/d27/d3d/d4d/d9a/df8/d11a 0 2024-03-23T08:23:58.195 INFO:tasks.workunit.client.0.smithi070.stdout:8/555: dread d5/f23 [0,4194304] 0 2024-03-23T08:23:58.195 INFO:tasks.workunit.client.0.smithi070.stdout:2/661: dwrite d1/d7/d39/d58/f89 [0,4194304] 0 2024-03-23T08:23:58.203 INFO:tasks.workunit.client.0.smithi070.stdout:2/662: creat d1/d21/d5f/da4/dc3/fd9 x:0 0 0 2024-03-23T08:23:58.204 INFO:tasks.workunit.client.0.smithi070.stdout:2/663: write d1/d29/f7f [261502,62128] 0 2024-03-23T08:23:58.204 INFO:tasks.workunit.client.0.smithi070.stdout:3/721: link db/df/l1d db/d14/d65/l10a 0 2024-03-23T08:23:58.206 INFO:tasks.workunit.client.0.smithi070.stdout:8/556: unlink d5/d8/d16/d30/c40 0 2024-03-23T08:23:58.207 INFO:tasks.workunit.client.0.smithi070.stdout:3/722: dread db/f36 [0,4194304] 0 2024-03-23T08:23:58.210 INFO:tasks.workunit.client.0.smithi070.stdout:2/664: rename d1/d8b to d1/d7/d39/d47/dda 0 2024-03-23T08:23:58.211 INFO:tasks.workunit.client.0.smithi070.stdout:2/665: read d1/d7/df/f91 [1598433,97842] 0 2024-03-23T08:23:58.212 INFO:tasks.workunit.client.0.smithi070.stdout:0/805: getdents d16/d26/d27/dee 0 2024-03-23T08:23:58.213 INFO:tasks.workunit.client.0.smithi070.stdout:3/723: symlink db/d14/d43/d7d/l10b 0 2024-03-23T08:23:58.215 INFO:tasks.workunit.client.0.smithi070.stdout:8/557: symlink d5/d8/d16/d8d/da3/dab/ld0 0 2024-03-23T08:23:58.216 INFO:tasks.workunit.client.0.smithi070.stdout:8/558: fdatasync d5/d8/d16/d3b/f5d 0 2024-03-23T08:23:58.218 INFO:tasks.workunit.client.0.smithi070.stdout:0/806: write d16/d26/d27/d86/d9e/ddd/ff3 [2358825,101392] 0 2024-03-23T08:23:58.222 INFO:tasks.workunit.client.0.smithi070.stdout:0/807: read d16/d26/d27/d4c/d60/d75/fd2 [237693,93626] 0 2024-03-23T08:23:58.222 INFO:tasks.workunit.client.0.smithi070.stdout:2/666: dwrite d1/d6/f65 [0,4194304] 0 2024-03-23T08:23:58.223 INFO:tasks.workunit.client.0.smithi070.stdout:3/724: mkdir db/d4e/dc4/d10c 0 2024-03-23T08:23:58.223 INFO:tasks.workunit.client.0.smithi070.stdout:0/808: readlink d16/d26/d27/d86/d67/d7d/l10f 0 2024-03-23T08:23:58.227 INFO:tasks.workunit.client.0.smithi070.stdout:8/559: unlink d5/d8/d16/d32/d84/da7/lc8 0 2024-03-23T08:23:58.232 INFO:tasks.workunit.client.0.smithi070.stdout:2/667: creat d1/d29/d52/fdb x:0 0 0 2024-03-23T08:23:58.233 INFO:tasks.workunit.client.0.smithi070.stdout:2/668: chown d1/d7/f17 7387 1 2024-03-23T08:23:58.233 INFO:tasks.workunit.client.0.smithi070.stdout:0/809: read d16/d26/d27/d4c/d60/d7c/f83 [621686,48817] 0 2024-03-23T08:23:58.243 INFO:tasks.workunit.client.0.smithi070.stdout:0/810: fsync d16/d26/d27/fc8 0 2024-03-23T08:23:58.246 INFO:tasks.workunit.client.0.smithi070.stdout:2/669: rename d1/d29/c44 to d1/d7/d48/d69/d7a/cdc 0 2024-03-23T08:23:58.246 INFO:tasks.workunit.client.0.smithi070.stdout:8/560: dread d5/d8/f1d [0,4194304] 0 2024-03-23T08:23:58.246 INFO:tasks.workunit.client.0.smithi070.stdout:2/670: chown d1/d7/c24 105 1 2024-03-23T08:23:58.247 INFO:tasks.workunit.client.0.smithi070.stdout:2/671: chown d1/d21/fb5 12562645 1 2024-03-23T08:23:58.248 INFO:tasks.workunit.client.0.smithi070.stdout:2/672: write d1/d21/d5f/fa8 [1903195,48256] 0 2024-03-23T08:23:58.250 INFO:tasks.workunit.client.0.smithi070.stdout:3/725: creat db/df/f10d x:0 0 0 2024-03-23T08:23:58.251 INFO:tasks.workunit.client.0.smithi070.stdout:3/726: readlink db/d4e/df8/d3d/dac/le0 0 2024-03-23T08:23:58.253 INFO:tasks.workunit.client.0.smithi070.stdout:0/811: creat d16/d22/df0/f11b x:0 0 0 2024-03-23T08:23:58.255 INFO:tasks.workunit.client.0.smithi070.stdout:2/673: dwrite d1/d7/df/d94/f12 [0,4194304] 0 2024-03-23T08:23:58.256 INFO:tasks.workunit.client.0.smithi070.stdout:8/561: link d5/d8/d16/f2b d5/d8/d16/d3b/db9/fd1 0 2024-03-23T08:23:58.257 INFO:tasks.workunit.client.0.smithi070.stdout:3/727: symlink db/d4e/dd6/l10e 0 2024-03-23T08:23:58.260 INFO:tasks.workunit.client.0.smithi070.stdout:2/674: rmdir d1/d7/d48/d5b 39 2024-03-23T08:23:58.265 INFO:tasks.workunit.client.0.smithi070.stdout:3/728: dwrite db/df/fec [0,4194304] 0 2024-03-23T08:23:58.268 INFO:tasks.workunit.client.0.smithi070.stdout:2/675: unlink d1/d7/d48/d69/fa6 0 2024-03-23T08:23:58.269 INFO:tasks.workunit.client.0.smithi070.stdout:0/812: creat d16/d26/d27/d3d/f11c x:0 0 0 2024-03-23T08:23:58.273 INFO:tasks.workunit.client.0.smithi070.stdout:3/729: creat db/d14/d43/d62/d95/f10f x:0 0 0 2024-03-23T08:23:58.273 INFO:tasks.workunit.client.0.smithi070.stdout:0/813: read d16/d26/d27/d3d/d4d/dad/fe8 [283210,116028] 0 2024-03-23T08:23:58.274 INFO:tasks.workunit.client.0.smithi070.stdout:0/814: stat d16/d26/d27/d3d/c93 0 2024-03-23T08:23:58.281 INFO:tasks.workunit.client.0.smithi070.stdout:2/676: fsync d1/d6/d34/f3d 0 2024-03-23T08:23:58.284 INFO:tasks.workunit.client.0.smithi070.stdout:3/730: rename db/d14/d9c/lde to db/d14/d43/d62/d95/l110 0 2024-03-23T08:23:58.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/677: mkdir d1/d7/d39/d47/d90/ddd 0 2024-03-23T08:23:58.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/678: readlink d1/d7/d39/l68 0 2024-03-23T08:23:58.290 INFO:tasks.workunit.client.0.smithi070.stdout:8/562: getdents d5/d8/d16/d3b/d4d/d6f 0 2024-03-23T08:23:58.291 INFO:tasks.workunit.client.0.smithi070.stdout:3/731: creat db/d20/d49/de1/f111 x:0 0 0 2024-03-23T08:23:58.294 INFO:tasks.workunit.client.0.smithi070.stdout:2/679: creat d1/d7/df/d94/d38/d4c/fde x:0 0 0 2024-03-23T08:23:58.296 INFO:tasks.workunit.client.0.smithi070.stdout:8/563: creat d5/d8/d9a/da2/fd2 x:0 0 0 2024-03-23T08:23:58.310 INFO:tasks.workunit.client.0.smithi070.stdout:2/680: getdents d1/d21/d5f/da4 0 2024-03-23T08:23:58.312 INFO:tasks.workunit.client.0.smithi070.stdout:2/681: creat d1/d6/fdf x:0 0 0 2024-03-23T08:23:58.316 INFO:tasks.workunit.client.0.smithi070.stdout:8/564: dread d5/d8/d16/d32/f46 [0,4194304] 0 2024-03-23T08:23:58.319 INFO:tasks.workunit.client.0.smithi070.stdout:3/732: dwrite db/df/d6a/fa1 [0,4194304] 0 2024-03-23T08:23:58.320 INFO:tasks.workunit.client.0.smithi070.stdout:8/565: rmdir d5/d8/d16/d8d/da3/dab 39 2024-03-23T08:23:58.321 INFO:tasks.workunit.client.0.smithi070.stdout:8/566: read d5/d8/d16/d32/f56 [710470,124417] 0 2024-03-23T08:23:58.322 INFO:tasks.workunit.client.0.smithi070.stdout:2/682: write d1/d7/f1e [113126,59381] 0 2024-03-23T08:23:58.385 INFO:tasks.workunit.client.0.smithi070.stdout:8/567: unlink d5/d8/d9a/da2/dae/fce 0 2024-03-23T08:23:58.386 INFO:tasks.workunit.client.0.smithi070.stdout:8/568: stat d5/d8/d16/d32/d84/ccd 0 2024-03-23T08:23:58.387 INFO:tasks.workunit.client.0.smithi070.stdout:8/569: dread - d5/d8/d16/d30/d38/fc2 zero size 2024-03-23T08:23:58.392 INFO:tasks.workunit.client.0.smithi070.stdout:2/683: truncate d1/d7/df/f4b 4024545 0 2024-03-23T08:23:58.394 INFO:tasks.workunit.client.0.smithi070.stdout:2/684: creat d1/d7/d39/d47/fe0 x:0 0 0 2024-03-23T08:23:58.399 INFO:tasks.workunit.client.0.smithi070.stdout:2/685: rename d1/d29/c4a to d1/d7/df/ce1 0 2024-03-23T08:23:58.399 INFO:tasks.workunit.client.0.smithi070.stdout:8/570: dwrite d5/d8/d16/f22 [8388608,4194304] 0 2024-03-23T08:23:58.407 INFO:tasks.workunit.client.0.smithi070.stdout:8/571: unlink d5/d8/d16/d30/d38/d6b/d72/fc4 0 2024-03-23T08:23:58.408 INFO:tasks.workunit.client.0.smithi070.stdout:8/572: chown d5/d8/d16/d30/d38/d6b 0 1 2024-03-23T08:23:58.409 INFO:tasks.workunit.client.0.smithi070.stdout:8/573: write d5/d8/f11 [615929,109433] 0 2024-03-23T08:23:58.411 INFO:tasks.workunit.client.0.smithi070.stdout:8/574: write d5/d8/d16/d32/f5c [9558822,112916] 0 2024-03-23T08:23:58.425 INFO:tasks.workunit.client.0.smithi070.stdout:8/575: dwrite d5/d8/f17 [0,4194304] 0 2024-03-23T08:23:58.426 INFO:tasks.workunit.client.0.smithi070.stdout:2/686: dread d1/d7/df/f91 [0,4194304] 0 2024-03-23T08:23:58.430 INFO:tasks.workunit.client.0.smithi070.stdout:8/576: creat d5/fd3 x:0 0 0 2024-03-23T08:23:58.431 INFO:tasks.workunit.client.0.smithi070.stdout:2/687: rename d1/d7/d39/d58/d8f to d1/d7/d39/d47/d90/ddd/de2 0 2024-03-23T08:23:58.434 INFO:tasks.workunit.client.0.smithi070.stdout:2/688: creat d1/d7/d39/fe3 x:0 0 0 2024-03-23T08:23:58.436 INFO:tasks.workunit.client.0.smithi070.stdout:2/689: truncate d1/d7/df/d99/fac 950097 0 2024-03-23T08:23:58.439 INFO:tasks.workunit.client.0.smithi070.stdout:2/690: mkdir d1/d29/d52/de4 0 2024-03-23T08:23:58.443 INFO:tasks.workunit.client.0.smithi070.stdout:8/577: dread d5/d8/f2c [0,4194304] 0 2024-03-23T08:23:58.457 INFO:tasks.workunit.client.0.smithi070.stdout:2/691: write d1/d7/f36 [6717360,49668] 0 2024-03-23T08:23:58.457 INFO:tasks.workunit.client.0.smithi070.stdout:2/692: stat d1/d6/d30 0 2024-03-23T08:23:58.461 INFO:tasks.workunit.client.0.smithi070.stdout:2/693: dread d1/d7/df/d94/f56 [4194304,4194304] 0 2024-03-23T08:23:58.462 INFO:tasks.workunit.client.0.smithi070.stdout:8/578: dwrite d5/d8/f1d [0,4194304] 0 2024-03-23T08:23:58.469 INFO:tasks.workunit.client.0.smithi070.stdout:2/694: rmdir d1/d7/d48/d69/dc1 0 2024-03-23T08:23:58.470 INFO:tasks.workunit.client.0.smithi070.stdout:2/695: write d1/d7/df/d99/fac [1265109,100968] 0 2024-03-23T08:23:58.471 INFO:tasks.workunit.client.0.smithi070.stdout:2/696: fsync d1/d21/d5f/da4/dc3/fd9 0 2024-03-23T08:23:58.473 INFO:tasks.workunit.client.0.smithi070.stdout:8/579: rename d5/d8/d16/d3b/d4d to d5/d8/d16/dd4 0 2024-03-23T08:23:58.474 INFO:tasks.workunit.client.0.smithi070.stdout:2/697: readlink d1/d7/d39/d47/dda/lcb 0 2024-03-23T08:23:58.479 INFO:tasks.workunit.client.0.smithi070.stdout:8/580: truncate d5/d8/d16/d30/d38/fa4 214614 0 2024-03-23T08:23:58.479 INFO:tasks.workunit.client.0.smithi070.stdout:8/581: symlink d5/d8/d16/d32/d84/d9f/ld5 0 2024-03-23T08:23:58.481 INFO:tasks.workunit.client.0.smithi070.stdout:2/698: link d1/d21/d5f/da4/dc3/f74 d1/d29/fe5 0 2024-03-23T08:23:58.483 INFO:tasks.workunit.client.0.smithi070.stdout:2/699: creat d1/d6/dc0/fe6 x:0 0 0 2024-03-23T08:23:58.485 INFO:tasks.workunit.client.0.smithi070.stdout:8/582: creat d5/d8/d16/d3b/d8c/fd6 x:0 0 0 2024-03-23T08:23:58.487 INFO:tasks.workunit.client.0.smithi070.stdout:2/700: stat d1/d7/d48/d5b/cd7 0 2024-03-23T08:23:58.490 INFO:tasks.workunit.client.0.smithi070.stdout:8/583: mknod d5/d8/d16/d8d/da3/dab/cd7 0 2024-03-23T08:23:58.490 INFO:tasks.workunit.client.0.smithi070.stdout:2/701: mknod d1/d29/ce7 0 2024-03-23T08:23:58.493 INFO:tasks.workunit.client.0.smithi070.stdout:2/702: symlink d1/d29/le8 0 2024-03-23T08:23:58.495 INFO:tasks.workunit.client.0.smithi070.stdout:8/584: rename d5/d8/d16/d30/d38/d6b/lb1 to d5/d8/d16/d32/d84/d9f/ld8 0 2024-03-23T08:23:58.496 INFO:tasks.workunit.client.0.smithi070.stdout:2/703: creat d1/d7/df/d94/fe9 x:0 0 0 2024-03-23T08:23:58.496 INFO:tasks.workunit.client.0.smithi070.stdout:8/585: write d5/d8/d16/d32/d84/f96 [903472,3472] 0 2024-03-23T08:23:58.498 INFO:tasks.workunit.client.0.smithi070.stdout:2/704: unlink d1/d21/l31 0 2024-03-23T08:23:58.499 INFO:tasks.workunit.client.0.smithi070.stdout:2/705: stat d1/d7/d39/c5e 0 2024-03-23T08:23:58.499 INFO:tasks.workunit.client.0.smithi070.stdout:2/706: chown d1/d7/d48/d5b/cd7 25 1 2024-03-23T08:23:58.507 INFO:tasks.workunit.client.0.smithi070.stdout:2/707: rename d1/d21/d5f/da4 to d1/d7/df/d99/dea 0 2024-03-23T08:23:58.512 INFO:tasks.workunit.client.0.smithi070.stdout:2/708: dwrite d1/d7/d48/d5b/f96 [0,4194304] 0 2024-03-23T08:23:58.525 INFO:tasks.workunit.client.0.smithi070.stdout:1/495: sync 2024-03-23T08:23:58.531 INFO:tasks.workunit.client.0.smithi070.stdout:2/709: dwrite d1/d7/df/d94/f25 [0,4194304] 0 2024-03-23T08:23:58.534 INFO:tasks.workunit.client.0.smithi070.stdout:8/586: dwrite d5/d8/d16/f2b [0,4194304] 0 2024-03-23T08:23:58.542 INFO:tasks.workunit.client.0.smithi070.stdout:1/496: dwrite d8/d26/d41/fae [0,4194304] 0 2024-03-23T08:23:58.542 INFO:tasks.workunit.client.0.smithi070.stdout:8/587: write d5/d8/f79 [754849,121441] 0 2024-03-23T08:23:58.546 INFO:tasks.workunit.client.0.smithi070.stdout:8/588: unlink d5/d8/d16/d32/f5c 0 2024-03-23T08:23:58.548 INFO:tasks.workunit.client.0.smithi070.stdout:2/710: link d1/d7/df/d94/fb0 d1/d7/d39/d47/d7b/feb 0 2024-03-23T08:23:58.548 INFO:tasks.workunit.client.0.smithi070.stdout:2/711: chown d1/fc 7358540 1 2024-03-23T08:23:58.551 INFO:tasks.workunit.client.0.smithi070.stdout:1/497: unlink d8/db/d2c/d3f/faf 0 2024-03-23T08:23:58.555 INFO:tasks.workunit.client.0.smithi070.stdout:8/589: mknod d5/d8/d16/d8d/da3/cd9 0 2024-03-23T08:23:58.555 INFO:tasks.workunit.client.0.smithi070.stdout:2/712: dwrite d1/d7/d39/f82 [0,4194304] 0 2024-03-23T08:23:58.556 INFO:tasks.workunit.client.0.smithi070.stdout:1/498: creat d8/db/d8e/fb5 x:0 0 0 2024-03-23T08:23:58.561 INFO:tasks.workunit.client.0.smithi070.stdout:2/713: creat d1/d7/d48/d69/d7a/fec x:0 0 0 2024-03-23T08:23:58.564 INFO:tasks.workunit.client.0.smithi070.stdout:8/590: unlink d5/d8/d16/d30/d38/c52 0 2024-03-23T08:23:58.565 INFO:tasks.workunit.client.0.smithi070.stdout:8/591: fsync d5/d8/d16/d32/d84/f96 0 2024-03-23T08:23:58.565 INFO:tasks.workunit.client.0.smithi070.stdout:2/714: symlink d1/d6/dc0/led 0 2024-03-23T08:23:58.569 INFO:tasks.workunit.client.0.smithi070.stdout:2/715: mknod d1/d7/d48/d69/db8/dc7/cee 0 2024-03-23T08:23:58.573 INFO:tasks.workunit.client.0.smithi070.stdout:2/716: symlink d1/d7/d39/d47/d7d/lef 0 2024-03-23T08:23:58.576 INFO:tasks.workunit.client.0.smithi070.stdout:2/717: rename d1/d7/d39/d47/d7d/ld5 to d1/d7/d39/d47/lf0 0 2024-03-23T08:23:58.579 INFO:tasks.workunit.client.0.smithi070.stdout:2/718: mknod d1/d29/d52/de4/cf1 0 2024-03-23T08:23:58.580 INFO:tasks.workunit.client.0.smithi070.stdout:2/719: dread - d1/d7/d48/d69/d7a/fec zero size 2024-03-23T08:23:58.583 INFO:tasks.workunit.client.0.smithi070.stdout:1/499: write d8/d11/f3b [242990,35738] 0 2024-03-23T08:23:58.584 INFO:tasks.workunit.client.0.smithi070.stdout:2/720: write d1/d7/d39/d47/dda/faa [451779,128106] 0 2024-03-23T08:23:58.586 INFO:tasks.workunit.client.0.smithi070.stdout:2/721: symlink d1/d7/df/lf2 0 2024-03-23T08:23:58.589 INFO:tasks.workunit.client.0.smithi070.stdout:1/500: rename d8/d26/d41/fad to d8/db/d40/d43/d64/d85/fb6 0 2024-03-23T08:23:58.593 INFO:tasks.workunit.client.0.smithi070.stdout:8/592: dwrite d5/d8/d16/d8d/da3/fb2 [0,4194304] 0 2024-03-23T08:23:58.595 INFO:tasks.workunit.client.0.smithi070.stdout:8/593: write d5/d8/d16/d32/f3c [93406,14122] 0 2024-03-23T08:23:58.603 INFO:tasks.workunit.client.0.smithi070.stdout:2/722: link d1/d7/d48/d69/db8/dc7/cee d1/d7/d48/d69/db8/cf3 0 2024-03-23T08:23:58.603 INFO:tasks.workunit.client.0.smithi070.stdout:8/594: dwrite d5/d8/d16/dd4/d6f/fb5 [0,4194304] 0 2024-03-23T08:23:58.604 INFO:tasks.workunit.client.0.smithi070.stdout:1/501: getdents d8/d26/d7b 0 2024-03-23T08:23:58.605 INFO:tasks.workunit.client.0.smithi070.stdout:2/723: read d1/d7/df/d94/d38/d4c/f6a [3753071,561] 0 2024-03-23T08:23:58.609 INFO:tasks.workunit.client.0.smithi070.stdout:1/502: creat d8/db/d40/d43/d64/d85/d97/d9a/fb7 x:0 0 0 2024-03-23T08:23:58.610 INFO:tasks.workunit.client.0.smithi070.stdout:2/724: dread d1/d7/f20 [0,4194304] 0 2024-03-23T08:23:58.613 INFO:tasks.workunit.client.0.smithi070.stdout:1/503: mkdir d8/d42/d51/db8 0 2024-03-23T08:23:58.618 INFO:tasks.workunit.client.0.smithi070.stdout:2/725: getdents d1/d7/d48/d69 0 2024-03-23T08:23:58.620 INFO:tasks.workunit.client.0.smithi070.stdout:1/504: dwrite d8/d42/d51/f91 [0,4194304] 0 2024-03-23T08:23:58.623 INFO:tasks.workunit.client.0.smithi070.stdout:1/505: mknod d8/d42/d51/cb9 0 2024-03-23T08:23:58.623 INFO:tasks.workunit.client.0.smithi070.stdout:2/726: dread d1/d29/fe5 [0,4194304] 0 2024-03-23T08:23:58.624 INFO:tasks.workunit.client.0.smithi070.stdout:1/506: truncate d8/d26/d41/f92 981752 0 2024-03-23T08:23:58.626 INFO:tasks.workunit.client.0.smithi070.stdout:2/727: creat d1/d7/d39/d58/ff4 x:0 0 0 2024-03-23T08:23:58.629 INFO:tasks.workunit.client.0.smithi070.stdout:2/728: mknod d1/d6/d34/da2/cf5 0 2024-03-23T08:23:58.630 INFO:tasks.workunit.client.0.smithi070.stdout:2/729: chown d1/d7/df/d94/fe9 0 1 2024-03-23T08:23:58.631 INFO:tasks.workunit.client.0.smithi070.stdout:1/507: getdents d8/db/d40/d43/d64/d85/d97 0 2024-03-23T08:23:58.632 INFO:tasks.workunit.client.0.smithi070.stdout:1/508: chown d8/d11 6744 1 2024-03-23T08:23:58.635 INFO:tasks.workunit.client.0.smithi070.stdout:1/509: rename d8/db/f21 to d8/d11/d61/d74/d5d/fba 0 2024-03-23T08:23:58.636 INFO:tasks.workunit.client.0.smithi070.stdout:1/510: chown d8/db/f1f 3982825 1 2024-03-23T08:23:58.638 INFO:tasks.workunit.client.0.smithi070.stdout:2/730: getdents d1/d7/d39 0 2024-03-23T08:23:58.644 INFO:tasks.workunit.client.0.smithi070.stdout:8/595: dwrite d5/d8/d9a/da2/dae/f94 [0,4194304] 0 2024-03-23T08:23:58.649 INFO:tasks.workunit.client.0.smithi070.stdout:8/596: dread d5/d8/d16/d32/f54 [0,4194304] 0 2024-03-23T08:23:58.649 INFO:tasks.workunit.client.0.smithi070.stdout:2/731: creat d1/d7/d39/d47/d7d/ff6 x:0 0 0 2024-03-23T08:23:58.654 INFO:tasks.workunit.client.0.smithi070.stdout:1/511: getdents d8/d42 0 2024-03-23T08:23:58.657 INFO:tasks.workunit.client.0.smithi070.stdout:8/597: dwrite d5/d8/d16/d30/fbc [0,4194304] 0 2024-03-23T08:23:58.659 INFO:tasks.workunit.client.0.smithi070.stdout:8/598: truncate d5/d8/d16/dd4/d6f/fa0 221068 0 2024-03-23T08:23:58.660 INFO:tasks.workunit.client.0.smithi070.stdout:2/732: mknod d1/cf7 0 2024-03-23T08:23:58.662 INFO:tasks.workunit.client.0.smithi070.stdout:1/512: symlink d8/d1a/lbb 0 2024-03-23T08:23:58.665 INFO:tasks.workunit.client.0.smithi070.stdout:8/599: dread d5/d8/d16/dd4/d6f/f89 [0,4194304] 0 2024-03-23T08:23:58.666 INFO:tasks.workunit.client.0.smithi070.stdout:2/733: dread d1/d6/f3b [0,4194304] 0 2024-03-23T08:23:58.668 INFO:tasks.workunit.client.0.smithi070.stdout:2/734: rmdir d1/d6/d30 39 2024-03-23T08:23:58.671 INFO:tasks.workunit.client.0.smithi070.stdout:2/735: truncate d1/d29/fe5 788428 0 2024-03-23T08:23:58.672 INFO:tasks.workunit.client.0.smithi070.stdout:2/736: truncate d1/d21/fb5 5095249 0 2024-03-23T08:23:58.674 INFO:tasks.workunit.client.0.smithi070.stdout:2/737: symlink d1/d7/d39/d47/dda/lf8 0 2024-03-23T08:23:58.675 INFO:tasks.workunit.client.0.smithi070.stdout:2/738: truncate d1/d7/f66 3035097 0 2024-03-23T08:23:58.678 INFO:tasks.workunit.client.0.smithi070.stdout:2/739: rename d1/d29/f2d to d1/d21/ff9 0 2024-03-23T08:23:58.680 INFO:tasks.workunit.client.0.smithi070.stdout:2/740: creat d1/d7/d48/d5b/ffa x:0 0 0 2024-03-23T08:23:58.681 INFO:tasks.workunit.client.0.smithi070.stdout:2/741: mknod d1/d7/cfb 0 2024-03-23T08:23:58.682 INFO:tasks.workunit.client.0.smithi070.stdout:2/742: readlink d1/d7/d48/d69/d7a/ld3 0 2024-03-23T08:23:58.682 INFO:tasks.workunit.client.0.smithi070.stdout:6/636: sync 2024-03-23T08:23:58.686 INFO:tasks.workunit.client.0.smithi070.stdout:5/500: sync 2024-03-23T08:23:58.687 INFO:tasks.workunit.client.0.smithi070.stdout:2/743: rmdir d1/d7/d48/d69 39 2024-03-23T08:23:58.689 INFO:tasks.workunit.client.0.smithi070.stdout:5/501: rmdir dc/d1b/d1d/d25/d68/d8a 39 2024-03-23T08:23:58.698 INFO:tasks.workunit.client.0.smithi070.stdout:2/744: truncate d1/d7/d39/d47/d7d/fb4 1938189 0 2024-03-23T08:23:58.701 INFO:tasks.workunit.client.0.smithi070.stdout:1/513: write d8/db/f15 [797059,36826] 0 2024-03-23T08:23:58.702 INFO:tasks.workunit.client.0.smithi070.stdout:2/745: symlink d1/d29/d52/de4/lfc 0 2024-03-23T08:23:58.706 INFO:tasks.workunit.client.0.smithi070.stdout:2/746: chown d1/d7/d39/d47/d7d/ff6 0 1 2024-03-23T08:23:58.707 INFO:tasks.workunit.client.0.smithi070.stdout:2/747: chown d1/d7/d39/d47/d90 341 1 2024-03-23T08:23:58.709 INFO:tasks.workunit.client.0.smithi070.stdout:5/502: unlink dc/d1b/d1d/d25/d68/d8a/f9c 0 2024-03-23T08:23:58.709 INFO:tasks.workunit.client.0.smithi070.stdout:6/637: link d0/l33 d0/ld8 0 2024-03-23T08:23:58.709 INFO:tasks.workunit.client.0.smithi070.stdout:2/748: write d1/d98/fbf [931774,57999] 0 2024-03-23T08:23:58.711 INFO:tasks.workunit.client.0.smithi070.stdout:1/514: creat d8/db/d40/d43/d64/d85/fbc x:0 0 0 2024-03-23T08:23:58.714 INFO:tasks.workunit.client.0.smithi070.stdout:8/600: write d5/d8/f25 [1063256,38995] 0 2024-03-23T08:23:58.724 INFO:tasks.workunit.client.0.smithi070.stdout:2/749: mknod d1/d7/df/cfd 0 2024-03-23T08:23:58.726 INFO:tasks.workunit.client.0.smithi070.stdout:6/638: mkdir d0/d23/d88/dd9 0 2024-03-23T08:23:58.726 INFO:tasks.workunit.client.0.smithi070.stdout:2/750: chown d1/d7/df/d94/c32 12255800 1 2024-03-23T08:23:58.729 INFO:tasks.workunit.client.0.smithi070.stdout:5/503: read dc/d1b/d1d/d25/d38/d39/f4f [159161,22433] 0 2024-03-23T08:23:58.731 INFO:tasks.workunit.client.0.smithi070.stdout:8/601: mknod d5/d8/d16/d30/d38/d6b/da9/cda 0 2024-03-23T08:23:58.736 INFO:tasks.workunit.client.0.smithi070.stdout:5/504: creat dc/d1b/d1d/d25/d68/d8a/fa3 x:0 0 0 2024-03-23T08:23:58.739 INFO:tasks.workunit.client.0.smithi070.stdout:6/639: dread d0/d23/d74/d84/db3/f5d [0,4194304] 0 2024-03-23T08:23:58.739 INFO:tasks.workunit.client.0.smithi070.stdout:8/602: rmdir d5/d8/d16/d30/d38/d6b/d72 39 2024-03-23T08:23:58.741 INFO:tasks.workunit.client.0.smithi070.stdout:5/505: dread dc/f3d [0,4194304] 0 2024-03-23T08:23:58.742 INFO:tasks.workunit.client.0.smithi070.stdout:2/751: truncate d1/d7/df/d94/f56 3131888 0 2024-03-23T08:23:58.744 INFO:tasks.workunit.client.0.smithi070.stdout:6/640: fsync d0/d23/d74/d84/db3/f2b 0 2024-03-23T08:23:58.796 INFO:tasks.workunit.client.0.smithi070.stdout:2/752: dwrite d1/d7/d39/d58/f89 [0,4194304] 0 2024-03-23T08:23:58.797 INFO:tasks.workunit.client.0.smithi070.stdout:8/603: rmdir d5/d8/d16/d30/d38/d6b/d72/d88 39 2024-03-23T08:23:58.802 INFO:tasks.workunit.client.0.smithi070.stdout:6/641: write d0/d23/d74/d84/db3/d2f/f39 [5193931,71971] 0 2024-03-23T08:23:58.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/506: dwrite dc/d1b/d1d/d25/d38/d53/fa1 [0,4194304] 0 2024-03-23T08:23:58.810 INFO:tasks.workunit.client.0.smithi070.stdout:1/515: dwrite d8/db/f27 [0,4194304] 0 2024-03-23T08:23:58.813 INFO:tasks.workunit.client.0.smithi070.stdout:1/516: fdatasync d8/d1a/f1e 0 2024-03-23T08:23:58.813 INFO:tasks.workunit.client.0.smithi070.stdout:2/753: dwrite d1/d6/f3b [0,4194304] 0 2024-03-23T08:23:58.814 INFO:tasks.workunit.client.0.smithi070.stdout:8/604: link d5/d8/d16/d30/l29 d5/d8/d16/dd4/ldb 0 2024-03-23T08:23:58.823 INFO:tasks.workunit.client.0.smithi070.stdout:8/605: dread d5/d8/d16/d32/f46 [0,4194304] 0 2024-03-23T08:23:58.824 INFO:tasks.workunit.client.0.smithi070.stdout:2/754: dwrite d1/d6/dc0/fc6 [0,4194304] 0 2024-03-23T08:23:58.825 INFO:tasks.workunit.client.0.smithi070.stdout:1/517: unlink d8/d11/d61/d74/d5d/cb1 0 2024-03-23T08:23:58.825 INFO:tasks.workunit.client.0.smithi070.stdout:1/518: chown d8/d11/l13 11879 1 2024-03-23T08:23:58.831 INFO:tasks.workunit.client.0.smithi070.stdout:2/755: stat d1/d6/d34/l88 0 2024-03-23T08:23:58.832 INFO:tasks.workunit.client.0.smithi070.stdout:1/519: dread - d8/d42/d51/f67 zero size 2024-03-23T08:23:58.832 INFO:tasks.workunit.client.0.smithi070.stdout:6/642: write d0/d3/f4 [1424020,7010] 0 2024-03-23T08:23:58.834 INFO:tasks.workunit.client.0.smithi070.stdout:2/756: write d1/d7/d39/d47/f9b [345415,127152] 0 2024-03-23T08:23:58.834 INFO:tasks.workunit.client.0.smithi070.stdout:1/520: fsync d8/d42/d51/f91 0 2024-03-23T08:23:58.834 INFO:tasks.workunit.client.0.smithi070.stdout:8/606: fdatasync d5/d8/d16/f22 0 2024-03-23T08:23:58.839 INFO:tasks.workunit.client.0.smithi070.stdout:6/643: mknod d0/d23/d74/d84/d3c/d86/cda 0 2024-03-23T08:23:58.840 INFO:tasks.workunit.client.0.smithi070.stdout:2/757: rename d1/l6e to d1/d7/d39/d47/lfe 0 2024-03-23T08:23:58.842 INFO:tasks.workunit.client.0.smithi070.stdout:8/607: creat d5/d8/d16/d32/d84/fdc x:0 0 0 2024-03-23T08:23:58.845 INFO:tasks.workunit.client.0.smithi070.stdout:6/644: dread d0/d3/d2c/f76 [0,4194304] 0 2024-03-23T08:23:58.847 INFO:tasks.workunit.client.0.smithi070.stdout:8/608: dread d5/d8/f25 [0,4194304] 0 2024-03-23T08:23:58.847 INFO:tasks.workunit.client.0.smithi070.stdout:1/521: rename d8/db/c31 to d8/db/d2c/d3f/d48/dab/cbd 0 2024-03-23T08:23:58.853 INFO:tasks.workunit.client.0.smithi070.stdout:6/645: dread d0/d23/d97/fa2 [0,4194304] 0 2024-03-23T08:23:58.853 INFO:tasks.workunit.client.0.smithi070.stdout:8/609: creat d5/d8/d9a/fdd x:0 0 0 2024-03-23T08:23:58.854 INFO:tasks.workunit.client.0.smithi070.stdout:1/522: mknod d8/db/d2c/d3f/d48/dab/cbe 0 2024-03-23T08:23:58.855 INFO:tasks.workunit.client.0.smithi070.stdout:8/610: chown d5/d8/d16/d30/l1f 4 1 2024-03-23T08:23:58.858 INFO:tasks.workunit.client.0.smithi070.stdout:6/646: creat d0/d23/d88/fdb x:0 0 0 2024-03-23T08:23:58.858 INFO:tasks.workunit.client.0.smithi070.stdout:1/523: fdatasync d8/f9 0 2024-03-23T08:23:58.859 INFO:tasks.workunit.client.0.smithi070.stdout:8/611: rmdir d5/d8/d16/dd4 39 2024-03-23T08:23:58.860 INFO:tasks.workunit.client.0.smithi070.stdout:8/612: chown d5/d8/d16/d3b/f5d 0 1 2024-03-23T08:23:58.865 INFO:tasks.workunit.client.0.smithi070.stdout:6/647: dwrite d0/d3/d2c/f58 [0,4194304] 0 2024-03-23T08:23:58.865 INFO:tasks.workunit.client.0.smithi070.stdout:1/524: truncate d8/d1a/d8f/fa3 131548 0 2024-03-23T08:23:58.871 INFO:tasks.workunit.client.0.smithi070.stdout:2/758: getdents d1/d7/d48/d5b 0 2024-03-23T08:23:58.873 INFO:tasks.workunit.client.0.smithi070.stdout:1/525: read d8/d1b/f25 [2902349,27885] 0 2024-03-23T08:23:58.884 INFO:tasks.workunit.client.0.smithi070.stdout:6/648: dread d0/d3/d2c/f49 [0,4194304] 0 2024-03-23T08:23:58.899 INFO:tasks.workunit.client.0.smithi070.stdout:7/500: sync 2024-03-23T08:23:58.904 INFO:tasks.workunit.client.0.smithi070.stdout:2/759: symlink d1/d7/d39/d58/lff 0 2024-03-23T08:23:58.907 INFO:tasks.workunit.client.0.smithi070.stdout:1/526: creat d8/db/d2c/d3f/fbf x:0 0 0 2024-03-23T08:23:58.913 INFO:tasks.workunit.client.0.smithi070.stdout:2/760: chown d1/d7/d48/d69/db8/dc7 158178059 1 2024-03-23T08:23:58.916 INFO:tasks.workunit.client.0.smithi070.stdout:1/527: dread - d8/db/d40/d43/d64/f8b zero size 2024-03-23T08:23:58.916 INFO:tasks.workunit.client.0.smithi070.stdout:6/649: mknod d0/d23/d88/dd9/cdc 0 2024-03-23T08:23:58.920 INFO:tasks.workunit.client.0.smithi070.stdout:2/761: dwrite d1/d7/d39/d47/d90/ddd/de2/fca [0,4194304] 0 2024-03-23T08:23:58.922 INFO:tasks.workunit.client.0.smithi070.stdout:2/762: truncate d1/d6/dc0/fe6 325689 0 2024-03-23T08:23:58.926 INFO:tasks.workunit.client.0.smithi070.stdout:1/528: rename d8/d11/d61/d74/d5d/d70/l76 to d8/d42/d51/lc0 0 2024-03-23T08:23:58.927 INFO:tasks.workunit.client.0.smithi070.stdout:1/529: stat d8/d1b/l3a 0 2024-03-23T08:23:58.929 INFO:tasks.workunit.client.0.smithi070.stdout:1/530: write d8/d42/d4e/f66 [7918770,44286] 0 2024-03-23T08:23:58.934 INFO:tasks.workunit.client.0.smithi070.stdout:8/613: write d5/d8/d16/dd4/d6f/d7c/f91 [767241,52462] 0 2024-03-23T08:23:58.940 INFO:tasks.workunit.client.0.smithi070.stdout:7/501: dwrite d25/d27/d6a/d44/f67 [0,4194304] 0 2024-03-23T08:23:58.941 INFO:tasks.workunit.client.0.smithi070.stdout:8/614: chown d5/d8/d16/dd4/l64 800 1 2024-03-23T08:23:58.943 INFO:tasks.workunit.client.0.smithi070.stdout:8/615: truncate d5/d8/d16/d30/d38/fc2 644461 0 2024-03-23T08:23:58.947 INFO:tasks.workunit.client.0.smithi070.stdout:2/763: getdents d1/d6/d30 0 2024-03-23T08:23:58.948 INFO:tasks.workunit.client.0.smithi070.stdout:2/764: fdatasync d1/d6/fdf 0 2024-03-23T08:23:58.950 INFO:tasks.workunit.client.0.smithi070.stdout:8/616: rename d5/d8/d16/d8d to d5/d8/d16/d32/d84/d98/dde 0 2024-03-23T08:23:58.950 INFO:tasks.workunit.client.0.smithi070.stdout:2/765: read d1/d7/df/d99/dea/dc3/f3c [940285,76735] 0 2024-03-23T08:23:58.955 INFO:tasks.workunit.client.0.smithi070.stdout:8/617: creat d5/d8/d16/d30/d38/d6b/da9/fdf x:0 0 0 2024-03-23T08:23:58.958 INFO:tasks.workunit.client.0.smithi070.stdout:6/650: dwrite d0/d23/d74/d84/db3/d38/d52/f6b [0,4194304] 0 2024-03-23T08:23:58.958 INFO:tasks.workunit.client.0.smithi070.stdout:8/618: fdatasync d5/d8/d16/d30/fbc 0 2024-03-23T08:23:58.958 INFO:tasks.workunit.client.0.smithi070.stdout:1/531: truncate d8/d26/f7f 3362876 0 2024-03-23T08:23:58.962 INFO:tasks.workunit.client.0.smithi070.stdout:2/766: dwrite d1/d7/df/d99/dea/dc3/fd9 [0,4194304] 0 2024-03-23T08:23:58.965 INFO:tasks.workunit.client.0.smithi070.stdout:1/532: creat d8/d26/d7b/fc1 x:0 0 0 2024-03-23T08:23:58.967 INFO:tasks.workunit.client.0.smithi070.stdout:8/619: dread - d5/d8/d16/f51 zero size 2024-03-23T08:23:58.968 INFO:tasks.workunit.client.0.smithi070.stdout:6/651: write d0/d23/d74/d84/db3/d2f/f4c [2994155,98548] 0 2024-03-23T08:23:58.974 INFO:tasks.workunit.client.0.smithi070.stdout:2/767: truncate d1/d29/f70 124215 0 2024-03-23T08:23:58.974 INFO:tasks.workunit.client.0.smithi070.stdout:1/533: mkdir d8/db/d40/d43/d64/dc2 0 2024-03-23T08:23:58.976 INFO:tasks.workunit.client.0.smithi070.stdout:7/502: dwrite d25/d27/d6a/d44/f8c [0,4194304] 0 2024-03-23T08:23:58.977 INFO:tasks.workunit.client.0.smithi070.stdout:7/503: stat d25/d27/f5c 0 2024-03-23T08:23:58.978 INFO:tasks.workunit.client.0.smithi070.stdout:2/768: rmdir d1/d7/d48/d5b 39 2024-03-23T08:23:58.980 INFO:tasks.workunit.client.0.smithi070.stdout:1/534: creat d8/d26/d49/d65/fc3 x:0 0 0 2024-03-23T08:23:58.981 INFO:tasks.workunit.client.0.smithi070.stdout:2/769: rmdir d1/d7/d39/d47/dda 39 2024-03-23T08:23:58.982 INFO:tasks.workunit.client.0.smithi070.stdout:2/770: fsync d1/d7/d39/f82 0 2024-03-23T08:23:58.984 INFO:tasks.workunit.client.0.smithi070.stdout:1/535: chown d8/db/d40/d43/f6c 9835 1 2024-03-23T08:23:58.990 INFO:tasks.workunit.client.0.smithi070.stdout:1/536: symlink d8/db/d2c/d3f/d48/dab/lc4 0 2024-03-23T08:23:58.993 INFO:tasks.workunit.client.0.smithi070.stdout:7/504: getdents d25/d27/d6a 0 2024-03-23T08:23:58.994 INFO:tasks.workunit.client.0.smithi070.stdout:8/620: write d5/d8/d16/d30/f61 [4398395,117207] 0 2024-03-23T08:23:59.000 INFO:tasks.workunit.client.0.smithi070.stdout:2/771: getdents d1/d7/d48/d5b 0 2024-03-23T08:23:59.002 INFO:tasks.workunit.client.0.smithi070.stdout:8/621: dwrite d5/d8/d16/d32/d84/fdc [0,4194304] 0 2024-03-23T08:23:59.007 INFO:tasks.workunit.client.0.smithi070.stdout:8/622: mkdir d5/d8/d16/d32/de0 0 2024-03-23T08:23:59.008 INFO:tasks.workunit.client.0.smithi070.stdout:1/537: write d8/db/d40/d43/f6c [746338,53339] 0 2024-03-23T08:23:59.011 INFO:tasks.workunit.client.0.smithi070.stdout:2/772: dwrite d1/d7/df/d94/f84 [4194304,4194304] 0 2024-03-23T08:23:59.012 INFO:tasks.workunit.client.0.smithi070.stdout:6/652: dread d0/d23/d74/d84/db3/d2f/f39 [0,4194304] 0 2024-03-23T08:23:59.013 INFO:tasks.workunit.client.0.smithi070.stdout:1/538: rmdir d8/db/d2c 39 2024-03-23T08:23:59.013 INFO:tasks.workunit.client.0.smithi070.stdout:6/653: stat d0/d23/d74/d84/d3c/l82 0 2024-03-23T08:23:59.014 INFO:tasks.workunit.client.0.smithi070.stdout:8/623: mknod d5/d8/d16/d32/d84/d98/dde/ce1 0 2024-03-23T08:23:59.015 INFO:tasks.workunit.client.0.smithi070.stdout:8/624: chown d5/d8/d16/d3b/db9/fd1 816 1 2024-03-23T08:23:59.015 INFO:tasks.workunit.client.0.smithi070.stdout:2/773: creat d1/d6/d34/f100 x:0 0 0 2024-03-23T08:23:59.016 INFO:tasks.workunit.client.0.smithi070.stdout:1/539: stat d8/d1a/l57 0 2024-03-23T08:23:59.017 INFO:tasks.workunit.client.0.smithi070.stdout:8/625: stat d5/d8/d16/d32/d84/d98/dde 0 2024-03-23T08:23:59.017 INFO:tasks.workunit.client.0.smithi070.stdout:2/774: chown d1/d7/cfb 71 1 2024-03-23T08:23:59.018 INFO:tasks.workunit.client.0.smithi070.stdout:1/540: dread - d8/db/d2c/d3f/f5f zero size 2024-03-23T08:23:59.019 INFO:tasks.workunit.client.0.smithi070.stdout:1/541: fsync d8/d11/f3b 0 2024-03-23T08:23:59.021 INFO:tasks.workunit.client.0.smithi070.stdout:6/654: creat d0/d23/d28/d35/dd6/fdd x:0 0 0 2024-03-23T08:23:59.022 INFO:tasks.workunit.client.0.smithi070.stdout:8/626: readlink d5/d8/d16/d32/d84/d9f/ld8 0 2024-03-23T08:23:59.026 INFO:tasks.workunit.client.0.smithi070.stdout:2/775: fsync d1/d7/d39/d47/d7b/f9a 0 2024-03-23T08:23:59.027 INFO:tasks.workunit.client.0.smithi070.stdout:1/542: creat d8/d11/fc5 x:0 0 0 2024-03-23T08:23:59.029 INFO:tasks.workunit.client.0.smithi070.stdout:8/627: dread - d5/d8/d16/dd4/fa6 zero size 2024-03-23T08:23:59.033 INFO:tasks.workunit.client.0.smithi070.stdout:6/655: write d0/d23/f55 [24178,88690] 0 2024-03-23T08:23:59.033 INFO:tasks.workunit.client.0.smithi070.stdout:1/543: write d8/d26/d49/d65/f98 [2494815,67882] 0 2024-03-23T08:23:59.034 INFO:tasks.workunit.client.0.smithi070.stdout:8/628: creat d5/d8/d16/d32/fe2 x:0 0 0 2024-03-23T08:23:59.038 INFO:tasks.workunit.client.0.smithi070.stdout:6/656: rename d0/d23/d28/d93 to d0/d23/d74/d84/d3c/d5b/dde 0 2024-03-23T08:23:59.038 INFO:tasks.workunit.client.0.smithi070.stdout:6/657: stat d0/d23/d74/d84/db3/d2f/f30 0 2024-03-23T08:23:59.044 INFO:tasks.workunit.client.0.smithi070.stdout:2/776: link d1/d7/c24 d1/d7/d39/d47/d90/ddd/c101 0 2024-03-23T08:23:59.047 INFO:tasks.workunit.client.0.smithi070.stdout:6/658: mknod d0/d23/d74/cdf 0 2024-03-23T08:23:59.051 INFO:tasks.workunit.client.0.smithi070.stdout:1/544: truncate d8/db/f32 897221 0 2024-03-23T08:23:59.052 INFO:tasks.workunit.client.0.smithi070.stdout:1/545: truncate d8/d11/f3b 1013061 0 2024-03-23T08:23:59.055 INFO:tasks.workunit.client.0.smithi070.stdout:2/777: symlink d1/d6/l102 0 2024-03-23T08:23:59.055 INFO:tasks.workunit.client.0.smithi070.stdout:6/659: stat d0/d23/d74/d84/d3c/d5b/l81 0 2024-03-23T08:23:59.060 INFO:tasks.workunit.client.0.smithi070.stdout:8/629: dwrite d5/d8/d16/d30/d38/d6b/d72/d88/f8a [0,4194304] 0 2024-03-23T08:23:59.061 INFO:tasks.workunit.client.0.smithi070.stdout:8/630: chown d5/d8/d16/d30/d38/c66 0 1 2024-03-23T08:23:59.062 INFO:tasks.workunit.client.0.smithi070.stdout:6/660: mknod d0/d23/d74/d84/d3c/d5b/ce0 0 2024-03-23T08:23:59.066 INFO:tasks.workunit.client.0.smithi070.stdout:6/661: write d0/d23/d28/d35/d6a/fb1 [1621456,2482] 0 2024-03-23T08:23:59.066 INFO:tasks.workunit.client.0.smithi070.stdout:6/662: chown d0/d3/f80 101636 1 2024-03-23T08:23:59.168 INFO:tasks.workunit.client.0.smithi070.stdout:4/483: sync 2024-03-23T08:23:59.169 INFO:tasks.workunit.client.0.smithi070.stdout:9/649: sync 2024-03-23T08:23:59.170 INFO:tasks.workunit.client.0.smithi070.stdout:1/546: link d8/cf d8/d42/d51/cc6 0 2024-03-23T08:23:59.171 INFO:tasks.workunit.client.0.smithi070.stdout:1/547: fdatasync d8/d11/f4d 0 2024-03-23T08:23:59.171 INFO:tasks.workunit.client.0.smithi070.stdout:9/650: read db/d12/d34/d8c/da2/f7c [3207820,85804] 0 2024-03-23T08:23:59.172 INFO:tasks.workunit.client.0.smithi070.stdout:8/631: dread d5/d8/d16/d3b/d8c/f99 [0,4194304] 0 2024-03-23T08:23:59.175 INFO:tasks.workunit.client.0.smithi070.stdout:4/484: creat d3/d6/d35/d65/d1b/fae x:0 0 0 2024-03-23T08:23:59.176 INFO:tasks.workunit.client.0.smithi070.stdout:9/651: read - db/d12/d34/d8c/da2/fca zero size 2024-03-23T08:23:59.178 INFO:tasks.workunit.client.0.smithi070.stdout:8/632: chown d5/d8/d9a/da2/dae/c8e 46721731 1 2024-03-23T08:23:59.178 INFO:tasks.workunit.client.0.smithi070.stdout:2/778: dwrite d1/d7/df/d94/fb0 [0,4194304] 0 2024-03-23T08:23:59.179 INFO:tasks.workunit.client.0.smithi070.stdout:4/485: fsync d3/d6/d6a/f4c 0 2024-03-23T08:23:59.184 INFO:tasks.workunit.client.0.smithi070.stdout:1/548: dread d8/d1a/f23 [0,4194304] 0 2024-03-23T08:23:59.186 INFO:tasks.workunit.client.0.smithi070.stdout:8/633: chown d5/d8/d16/d30/l90 15227036 1 2024-03-23T08:23:59.187 INFO:tasks.workunit.client.0.smithi070.stdout:1/549: chown d8/f90 1 1 2024-03-23T08:23:59.187 INFO:tasks.workunit.client.0.smithi070.stdout:6/663: dwrite d0/d3/d2c/fc5 [4194304,4194304] 0 2024-03-23T08:23:59.188 INFO:tasks.workunit.client.0.smithi070.stdout:8/634: dread - d5/d8/d16/f51 zero size 2024-03-23T08:23:59.190 INFO:tasks.workunit.client.0.smithi070.stdout:4/486: chown d3/d6/d35/d3f/c5f 8529611 1 2024-03-23T08:23:59.193 INFO:tasks.workunit.client.0.smithi070.stdout:1/550: stat d8/db/d40/d43/l88 0 2024-03-23T08:23:59.194 INFO:tasks.workunit.client.0.smithi070.stdout:6/664: mknod d0/d23/d28/d35/ce1 0 2024-03-23T08:23:59.197 INFO:tasks.workunit.client.0.smithi070.stdout:9/652: creat db/d12/d34/d8c/da2/dc6/dce/d85/fd4 x:0 0 0 2024-03-23T08:23:59.200 INFO:tasks.workunit.client.0.smithi070.stdout:1/551: mkdir d8/db/d40/d43/d64/d85/dc7 0 2024-03-23T08:23:59.201 INFO:tasks.workunit.client.0.smithi070.stdout:1/552: chown d8/d11/d61/d74/d5d/d70/f93 781539009 1 2024-03-23T08:23:59.201 INFO:tasks.workunit.client.0.smithi070.stdout:4/487: rename d3/d6/d11/f13 to d3/d6/d35/d65/d27/d4b/d73/d74/faf 0 2024-03-23T08:23:59.202 INFO:tasks.workunit.client.0.smithi070.stdout:1/553: chown d8/d42 9994 1 2024-03-23T08:23:59.202 INFO:tasks.workunit.client.0.smithi070.stdout:2/779: getdents d1/d7/df/d99 0 2024-03-23T08:23:59.205 INFO:tasks.workunit.client.0.smithi070.stdout:4/488: mknod d3/d6/d35/d65/d41/d56/cb0 0 2024-03-23T08:23:59.206 INFO:tasks.workunit.client.0.smithi070.stdout:1/554: dread d8/db/f3e [0,4194304] 0 2024-03-23T08:23:59.208 INFO:tasks.workunit.client.0.smithi070.stdout:2/780: mkdir d1/d6/d103 0 2024-03-23T08:23:59.208 INFO:tasks.workunit.client.0.smithi070.stdout:2/781: readlink d1/d7/d48/d69/d7a/ld3 0 2024-03-23T08:23:59.213 INFO:tasks.workunit.client.0.smithi070.stdout:1/555: creat d8/d42/d4e/fc8 x:0 0 0 2024-03-23T08:23:59.215 INFO:tasks.workunit.client.0.smithi070.stdout:4/489: dwrite d3/d6/d45/f47 [0,4194304] 0 2024-03-23T08:23:59.222 INFO:tasks.workunit.client.0.smithi070.stdout:1/556: write d8/d1a/f47 [520278,126854] 0 2024-03-23T08:23:59.229 INFO:tasks.workunit.client.0.smithi070.stdout:2/782: dwrite d1/d7/df/d94/f56 [0,4194304] 0 2024-03-23T08:23:59.229 INFO:tasks.workunit.client.0.smithi070.stdout:4/490: dwrite d3/d6/d35/d65/d1b/fae [0,4194304] 0 2024-03-23T08:23:59.232 INFO:tasks.workunit.client.0.smithi070.stdout:1/557: truncate d8/d11/d61/d74/d5d/d70/fa8 486518 0 2024-03-23T08:23:59.236 INFO:tasks.workunit.client.0.smithi070.stdout:4/491: creat d3/d6/d35/d65/d6f/fb1 x:0 0 0 2024-03-23T08:23:59.237 INFO:tasks.workunit.client.0.smithi070.stdout:1/558: unlink d8/db/f6a 0 2024-03-23T08:23:59.240 INFO:tasks.workunit.client.0.smithi070.stdout:1/559: dread - d8/d42/d4e/fc8 zero size 2024-03-23T08:23:59.241 INFO:tasks.workunit.client.0.smithi070.stdout:1/560: stat d8/db/d2c/d3f/lb3 0 2024-03-23T08:23:59.243 INFO:tasks.workunit.client.0.smithi070.stdout:1/561: stat d8/d26/d49/d65/fc3 0 2024-03-23T08:23:59.243 INFO:tasks.workunit.client.0.smithi070.stdout:2/783: dwrite d1/d29/f7f [0,4194304] 0 2024-03-23T08:23:59.245 INFO:tasks.workunit.client.0.smithi070.stdout:4/492: mknod d3/d6/d35/d3f/da9/cb2 0 2024-03-23T08:23:59.248 INFO:tasks.workunit.client.0.smithi070.stdout:1/562: symlink d8/db/d40/d43/d64/lc9 0 2024-03-23T08:23:59.249 INFO:tasks.workunit.client.0.smithi070.stdout:1/563: truncate d8/d42/d4e/fc8 963245 0 2024-03-23T08:23:59.250 INFO:tasks.workunit.client.0.smithi070.stdout:2/784: fdatasync d1/d7/d39/d47/d7d/fb4 0 2024-03-23T08:23:59.251 INFO:tasks.workunit.client.0.smithi070.stdout:4/493: mkdir d3/d6/d35/d65/d27/d4b/d73/d74/da3/db3 0 2024-03-23T08:23:59.253 INFO:tasks.workunit.client.0.smithi070.stdout:1/564: creat d8/d26/d41/fca x:0 0 0 2024-03-23T08:23:59.254 INFO:tasks.workunit.client.0.smithi070.stdout:2/785: fsync d1/d29/d52/f9f 0 2024-03-23T08:23:59.255 INFO:tasks.workunit.client.0.smithi070.stdout:1/565: rename d8/d11/d61 to d8/d11/d61/d74/d5d/dcb 22 2024-03-23T08:23:59.259 INFO:tasks.workunit.client.0.smithi070.stdout:4/494: rmdir d3/d6/d35/d65/d27/d4b/d73/d74/da3/db3 0 2024-03-23T08:23:59.262 INFO:tasks.workunit.client.0.smithi070.stdout:4/495: dread - d3/f62 zero size 2024-03-23T08:23:59.263 INFO:tasks.workunit.client.0.smithi070.stdout:4/496: write d3/d6/d35/d65/f30 [3269335,112426] 0 2024-03-23T08:23:59.264 INFO:tasks.workunit.client.0.smithi070.stdout:1/566: link d8/db/d2c/d3f/fbf d8/db/d40/d43/d64/d85/dc7/fcc 0 2024-03-23T08:23:59.265 INFO:tasks.workunit.client.0.smithi070.stdout:1/567: write d8/d11/fb2 [966134,122918] 0 2024-03-23T08:23:59.266 INFO:tasks.workunit.client.0.smithi070.stdout:4/497: creat d3/d6/d9/fb4 x:0 0 0 2024-03-23T08:23:59.268 INFO:tasks.workunit.client.0.smithi070.stdout:4/498: write d3/d6/d35/d3f/d51/f63 [4912286,53547] 0 2024-03-23T08:23:59.273 INFO:tasks.workunit.client.0.smithi070.stdout:1/568: getdents d8 0 2024-03-23T08:23:59.274 INFO:tasks.workunit.client.0.smithi070.stdout:2/786: write d1/d7/df/d94/d38/d4c/f6a [5180996,4013] 0 2024-03-23T08:23:59.277 INFO:tasks.workunit.client.0.smithi070.stdout:2/787: mkdir d1/d6/d30/d104 0 2024-03-23T08:23:59.279 INFO:tasks.workunit.client.0.smithi070.stdout:1/569: link d8/d42/d4e/f59 d8/db/d8e/fcd 0 2024-03-23T08:23:59.280 INFO:tasks.workunit.client.0.smithi070.stdout:1/570: stat d8/d11/d61 0 2024-03-23T08:23:59.280 INFO:tasks.workunit.client.0.smithi070.stdout:4/499: write d3/d6/d35/d65/d27/d4b/d73/d84/f37 [511580,11972] 0 2024-03-23T08:23:59.287 INFO:tasks.workunit.client.0.smithi070.stdout:1/571: rename d8/db/d2c/d3f/d48/dab/cbe to d8/db/d40/d43/cce 0 2024-03-23T08:23:59.288 INFO:tasks.workunit.client.0.smithi070.stdout:2/788: truncate d1/d21/d5f/f67 3989736 0 2024-03-23T08:23:59.289 INFO:tasks.workunit.client.0.smithi070.stdout:2/789: dread - d1/d7/d48/d69/d7a/fcd zero size 2024-03-23T08:23:59.291 INFO:tasks.workunit.client.0.smithi070.stdout:4/500: rename d3/d6/d35/d65/d41/d56 to d3/d6/d35/d65/d27/d4b/d73/d74/db5 0 2024-03-23T08:23:59.298 INFO:tasks.workunit.client.0.smithi070.stdout:4/501: dwrite d3/d6/d35/d3f/d51/d6e/f99 [0,4194304] 0 2024-03-23T08:23:59.302 INFO:tasks.workunit.client.0.smithi070.stdout:4/502: write d3/d6/d35/d3f/d51/f63 [1921676,92806] 0 2024-03-23T08:23:59.304 INFO:tasks.workunit.client.0.smithi070.stdout:1/572: creat d8/db/fcf x:0 0 0 2024-03-23T08:23:59.308 INFO:tasks.workunit.client.0.smithi070.stdout:4/503: mknod d3/d6/d35/d3f/d51/cb6 0 2024-03-23T08:23:59.309 INFO:tasks.workunit.client.0.smithi070.stdout:4/504: fdatasync d3/d6/d35/d65/d1b/f90 0 2024-03-23T08:23:59.310 INFO:tasks.workunit.client.0.smithi070.stdout:4/505: write d3/d6/d35/d65/d27/d4b/d73/d84/f37 [1069768,61532] 0 2024-03-23T08:23:59.311 INFO:tasks.workunit.client.0.smithi070.stdout:1/573: mknod d8/d11/cd0 0 2024-03-23T08:23:59.312 INFO:tasks.workunit.client.0.smithi070.stdout:2/790: truncate d1/d7/d39/d47/d7b/feb 1849644 0 2024-03-23T08:23:59.315 INFO:tasks.workunit.client.0.smithi070.stdout:2/791: creat d1/d7/d39/d47/d7b/f105 x:0 0 0 2024-03-23T08:23:59.388 INFO:tasks.workunit.client.0.smithi070.stdout:4/506: mkdir d3/d6/db7 0 2024-03-23T08:23:59.389 INFO:tasks.workunit.client.0.smithi070.stdout:1/574: symlink d8/d42/d51/db8/ld1 0 2024-03-23T08:23:59.389 INFO:tasks.workunit.client.0.smithi070.stdout:0/815: sync 2024-03-23T08:23:59.389 INFO:tasks.workunit.client.0.smithi070.stdout:3/733: sync 2024-03-23T08:23:59.389 INFO:tasks.workunit.client.0.smithi070.stdout:5/507: sync 2024-03-23T08:23:59.391 INFO:tasks.workunit.client.0.smithi070.stdout:3/734: chown db/d14/d43/f54 325587 1 2024-03-23T08:23:59.391 INFO:tasks.workunit.client.0.smithi070.stdout:5/508: stat dc/d1b/d1d/d25/d68/f83 0 2024-03-23T08:23:59.395 INFO:tasks.workunit.client.0.smithi070.stdout:4/507: chown d3/d6/d35/d65/d27/d4b/d73/d74/faf 519082100 1 2024-03-23T08:23:59.396 INFO:tasks.workunit.client.0.smithi070.stdout:1/575: dread d8/db/d40/d43/f54 [0,4194304] 0 2024-03-23T08:23:59.397 INFO:tasks.workunit.client.0.smithi070.stdout:1/576: chown d8/d26/d49 8480394 1 2024-03-23T08:23:59.406 INFO:tasks.workunit.client.0.smithi070.stdout:1/577: rmdir d8/db/d40/d43/d64/d85 39 2024-03-23T08:23:59.408 INFO:tasks.workunit.client.0.smithi070.stdout:1/578: truncate d8/d26/d41/fca 299920 0 2024-03-23T08:23:59.409 INFO:tasks.workunit.client.0.smithi070.stdout:1/579: stat d8/d26/d7b/fc1 0 2024-03-23T08:23:59.409 INFO:tasks.workunit.client.0.smithi070.stdout:5/509: dread dc/f58 [0,4194304] 0 2024-03-23T08:23:59.410 INFO:tasks.workunit.client.0.smithi070.stdout:2/792: link d1/d29/d52/de4/lfc d1/d21/d5f/l106 0 2024-03-23T08:23:59.412 INFO:tasks.workunit.client.0.smithi070.stdout:0/816: dread d16/d22/d43/d47/f70 [0,4194304] 0 2024-03-23T08:23:59.418 INFO:tasks.workunit.client.0.smithi070.stdout:5/510: dwrite dc/d1b/d1d/d25/f43 [4194304,4194304] 0 2024-03-23T08:23:59.426 INFO:tasks.workunit.client.0.smithi070.stdout:3/735: dwrite db/d14/f1b [0,4194304] 0 2024-03-23T08:23:59.429 INFO:tasks.workunit.client.0.smithi070.stdout:3/736: chown db/d7f/dcd 4 1 2024-03-23T08:23:59.432 INFO:tasks.workunit.client.0.smithi070.stdout:2/793: symlink d1/d7/d39/d47/l107 0 2024-03-23T08:23:59.433 INFO:tasks.workunit.client.0.smithi070.stdout:3/737: write db/df/d1a/f58 [1178211,117492] 0 2024-03-23T08:23:59.443 INFO:tasks.workunit.client.0.smithi070.stdout:4/508: dwrite d3/d6/d6a/d7c/f9c [0,4194304] 0 2024-03-23T08:23:59.444 INFO:tasks.workunit.client.0.smithi070.stdout:1/580: dwrite d8/d1a/f39 [4194304,4194304] 0 2024-03-23T08:23:59.454 INFO:tasks.workunit.client.0.smithi070.stdout:5/511: dread dc/d1b/d1d/d25/d38/f86 [0,4194304] 0 2024-03-23T08:23:59.455 INFO:tasks.workunit.client.0.smithi070.stdout:1/581: rename d8/d26/d49/d65/f98 to d8/db/d2c/fd2 0 2024-03-23T08:23:59.459 INFO:tasks.workunit.client.0.smithi070.stdout:5/512: mknod dc/d1b/d1d/d25/d36/ca4 0 2024-03-23T08:23:59.459 INFO:tasks.workunit.client.0.smithi070.stdout:2/794: truncate d1/d7/d39/d58/f89 3787730 0 2024-03-23T08:23:59.461 INFO:tasks.workunit.client.0.smithi070.stdout:4/509: write d3/d6/d35/d65/d27/d4b/d73/d74/db5/f8d [934272,116288] 0 2024-03-23T08:23:59.462 INFO:tasks.workunit.client.0.smithi070.stdout:3/738: dwrite db/f17 [0,4194304] 0 2024-03-23T08:23:59.462 INFO:tasks.workunit.client.0.smithi070.stdout:5/513: fsync dc/d1b/d1d/d25/d38/d53/fa1 0 2024-03-23T08:23:59.462 INFO:tasks.workunit.client.0.smithi070.stdout:2/795: write d1/d7/f36 [2681533,30994] 0 2024-03-23T08:23:59.468 INFO:tasks.workunit.client.0.smithi070.stdout:5/514: fsync dc/d1b/d1d/d25/d36/d32/d5c/f65 0 2024-03-23T08:23:59.472 INFO:tasks.workunit.client.0.smithi070.stdout:5/515: write dc/d1b/d1d/d25/d38/d39/f91 [236110,71955] 0 2024-03-23T08:23:59.472 INFO:tasks.workunit.client.0.smithi070.stdout:3/739: creat db/d14/d43/d62/f112 x:0 0 0 2024-03-23T08:23:59.475 INFO:tasks.workunit.client.0.smithi070.stdout:3/740: read db/d4e/d92/f101 [148422,12459] 0 2024-03-23T08:23:59.478 INFO:tasks.workunit.client.0.smithi070.stdout:4/510: rename d3/d6/d35/d65/d27/d4b/d73/d84 to d3/d6/d35/d65/d6f/d93/db8 0 2024-03-23T08:23:59.480 INFO:tasks.workunit.client.0.smithi070.stdout:5/516: dwrite dc/d1b/d1d/d25/d38/d39/f79 [0,4194304] 0 2024-03-23T08:23:59.482 INFO:tasks.workunit.client.0.smithi070.stdout:2/796: getdents d1/d98 0 2024-03-23T08:23:59.487 INFO:tasks.workunit.client.0.smithi070.stdout:4/511: creat d3/d6/d35/d65/d41/fb9 x:0 0 0 2024-03-23T08:23:59.490 INFO:tasks.workunit.client.0.smithi070.stdout:5/517: dwrite dc/d1b/d1d/d25/d68/d8a/f8e [0,4194304] 0 2024-03-23T08:23:59.498 INFO:tasks.workunit.client.0.smithi070.stdout:4/512: creat d3/d6/d35/d65/d27/d4b/fba x:0 0 0 2024-03-23T08:23:59.499 INFO:tasks.workunit.client.0.smithi070.stdout:1/582: dwrite d8/f1c [0,4194304] 0 2024-03-23T08:23:59.500 INFO:tasks.workunit.client.0.smithi070.stdout:5/518: dread - dc/d1b/d1d/d25/d38/d48/f72 zero size 2024-03-23T08:23:59.507 INFO:tasks.workunit.client.0.smithi070.stdout:1/583: unlink d8/d1b/f33 0 2024-03-23T08:23:59.509 INFO:tasks.workunit.client.0.smithi070.stdout:5/519: symlink dc/d1b/d1d/d25/d38/la5 0 2024-03-23T08:23:59.513 INFO:tasks.workunit.client.0.smithi070.stdout:1/584: mkdir d8/db/d2c/d3f/dd3 0 2024-03-23T08:23:59.516 INFO:tasks.workunit.client.0.smithi070.stdout:5/520: rename dc/d1b/d1d/d25/d36/d32/f80 to dc/fa6 0 2024-03-23T08:23:59.517 INFO:tasks.workunit.client.0.smithi070.stdout:4/513: getdents d3/d6/d9 0 2024-03-23T08:23:59.519 INFO:tasks.workunit.client.0.smithi070.stdout:1/585: mknod d8/db/d40/cd4 0 2024-03-23T08:23:59.523 INFO:tasks.workunit.client.0.smithi070.stdout:4/514: mkdir d3/d6/d35/d65/d27/d4b/d73/d74/db5/dbb 0 2024-03-23T08:23:59.523 INFO:tasks.workunit.client.0.smithi070.stdout:2/797: write d1/d7/df/f4b [3700590,74664] 0 2024-03-23T08:23:59.523 INFO:tasks.workunit.client.0.smithi070.stdout:5/521: creat dc/d1b/d1d/d25/d38/d53/fa7 x:0 0 0 2024-03-23T08:23:59.529 INFO:tasks.workunit.client.0.smithi070.stdout:1/586: fdatasync d8/d11/d61/d74/d5d/fba 0 2024-03-23T08:23:59.533 INFO:tasks.workunit.client.0.smithi070.stdout:5/522: dread dc/d1b/d1d/d25/d36/d32/f5a [0,4194304] 0 2024-03-23T08:23:59.536 INFO:tasks.workunit.client.0.smithi070.stdout:1/587: mknod d8/d11/d61/d74/d5d/d70/cd5 0 2024-03-23T08:23:59.536 INFO:tasks.workunit.client.0.smithi070.stdout:5/523: read - dc/d1b/d1d/d25/d36/d32/d5c/fa2 zero size 2024-03-23T08:23:59.538 INFO:tasks.workunit.client.0.smithi070.stdout:2/798: dread d1/fd [0,4194304] 0 2024-03-23T08:23:59.538 INFO:tasks.workunit.client.0.smithi070.stdout:4/515: rmdir d3/d6/d35/d3f/d51/d6e/d97/d9d 0 2024-03-23T08:23:59.538 INFO:tasks.workunit.client.0.smithi070.stdout:5/524: chown dc/d1b/d1d/d25/d36/d32/d5c/l64 57 1 2024-03-23T08:23:59.541 INFO:tasks.workunit.client.0.smithi070.stdout:2/799: fdatasync d1/d6/d30/f46 0 2024-03-23T08:23:59.543 INFO:tasks.workunit.client.0.smithi070.stdout:4/516: mknod d3/d6/db7/cbc 0 2024-03-23T08:23:59.545 INFO:tasks.workunit.client.0.smithi070.stdout:4/517: write d3/d6/d35/f76 [922835,8009] 0 2024-03-23T08:23:59.546 INFO:tasks.workunit.client.0.smithi070.stdout:1/588: dwrite d8/d11/f3b [0,4194304] 0 2024-03-23T08:23:59.559 INFO:tasks.workunit.client.0.smithi070.stdout:2/800: mkdir d1/d7/d48/d69/d7a/d108 0 2024-03-23T08:23:59.560 INFO:tasks.workunit.client.0.smithi070.stdout:1/589: dread - d8/d11/f73 zero size 2024-03-23T08:23:59.561 INFO:tasks.workunit.client.0.smithi070.stdout:7/505: sync 2024-03-23T08:23:59.563 INFO:tasks.workunit.client.0.smithi070.stdout:2/801: write d1/d7/d48/d69/fa0 [474645,101439] 0 2024-03-23T08:23:59.564 INFO:tasks.workunit.client.0.smithi070.stdout:5/525: dwrite dc/d1b/d1d/d25/d36/d32/d5c/f5d [0,4194304] 0 2024-03-23T08:23:59.568 INFO:tasks.workunit.client.0.smithi070.stdout:5/526: write dc/d1b/f54 [3817092,13699] 0 2024-03-23T08:23:59.570 INFO:tasks.workunit.client.0.smithi070.stdout:7/506: creat d25/d27/d6a/fba x:0 0 0 2024-03-23T08:23:59.571 INFO:tasks.workunit.client.0.smithi070.stdout:4/518: dwrite d3/d6/d35/d3f/d51/d6e/f99 [0,4194304] 0 2024-03-23T08:23:59.575 INFO:tasks.workunit.client.0.smithi070.stdout:2/802: symlink d1/d7/d39/d47/l109 0 2024-03-23T08:23:59.576 INFO:tasks.workunit.client.0.smithi070.stdout:7/507: mknod d25/d27/d6a/d44/d7c/cbb 0 2024-03-23T08:23:59.580 INFO:tasks.workunit.client.0.smithi070.stdout:5/527: rename dc/c8f to dc/d1b/d1d/d25/d38/ca8 0 2024-03-23T08:23:59.580 INFO:tasks.workunit.client.0.smithi070.stdout:2/803: mkdir d1/d29/d52/de4/d10a 0 2024-03-23T08:23:59.582 INFO:tasks.workunit.client.0.smithi070.stdout:1/590: dread d8/d26/f7f [0,4194304] 0 2024-03-23T08:23:59.585 INFO:tasks.workunit.client.0.smithi070.stdout:4/519: creat d3/d6/d35/d65/d27/d4b/d73/d74/db5/dbb/fbd x:0 0 0 2024-03-23T08:23:59.588 INFO:tasks.workunit.client.0.smithi070.stdout:2/804: symlink d1/d7/d48/d69/l10b 0 2024-03-23T08:23:59.593 INFO:tasks.workunit.client.0.smithi070.stdout:7/508: rename d25/d27/d6a/l3e to d25/d27/d6a/d44/da2/db5/lbc 0 2024-03-23T08:23:59.593 INFO:tasks.workunit.client.0.smithi070.stdout:4/520: symlink d3/d6/d35/d65/d41/lbe 0 2024-03-23T08:23:59.599 INFO:tasks.workunit.client.0.smithi070.stdout:5/528: mkdir dc/d1b/d1d/d25/da9 0 2024-03-23T08:23:59.601 INFO:tasks.workunit.client.0.smithi070.stdout:7/509: mknod d25/d27/d6a/d44/d6f/da3/cbd 0 2024-03-23T08:23:59.604 INFO:tasks.workunit.client.0.smithi070.stdout:2/805: dwrite d1/d7/d39/d47/d7d/ff6 [0,4194304] 0 2024-03-23T08:23:59.605 INFO:tasks.workunit.client.0.smithi070.stdout:4/521: dwrite d3/d6/d35/d3f/d51/d6e/f99 [0,4194304] 0 2024-03-23T08:23:59.612 INFO:tasks.workunit.client.0.smithi070.stdout:7/510: creat d25/d27/d6a/fbe x:0 0 0 2024-03-23T08:23:59.613 INFO:tasks.workunit.client.0.smithi070.stdout:5/529: unlink dc/d1b/d1d/d25/d38/ca8 0 2024-03-23T08:23:59.619 INFO:tasks.workunit.client.0.smithi070.stdout:4/522: rmdir d3/d6/d35/d65/d27/d4b 39 2024-03-23T08:23:59.621 INFO:tasks.workunit.client.0.smithi070.stdout:7/511: creat d25/d27/d6a/d44/da2/fbf x:0 0 0 2024-03-23T08:23:59.622 INFO:tasks.workunit.client.0.smithi070.stdout:1/591: rename d8/cf to d8/db/cd6 0 2024-03-23T08:23:59.624 INFO:tasks.workunit.client.0.smithi070.stdout:5/530: chown dc/d1b/d1d/d25/d36/f2a 18 1 2024-03-23T08:23:59.627 INFO:tasks.workunit.client.0.smithi070.stdout:7/512: stat d25/d27/d6a/d44/f78 0 2024-03-23T08:23:59.627 INFO:tasks.workunit.client.0.smithi070.stdout:4/523: dwrite d3/d6/d35/d65/d6f/d93/db8/f37 [0,4194304] 0 2024-03-23T08:23:59.631 INFO:tasks.workunit.client.0.smithi070.stdout:2/806: creat d1/d7/d48/d69/d7a/f10c x:0 0 0 2024-03-23T08:23:59.635 INFO:tasks.workunit.client.0.smithi070.stdout:4/524: symlink d3/d6/d35/d3f/d51/d6e/lbf 0 2024-03-23T08:23:59.636 INFO:tasks.workunit.client.0.smithi070.stdout:1/592: write d8/d26/f7f [1716553,23499] 0 2024-03-23T08:23:59.639 INFO:tasks.workunit.client.0.smithi070.stdout:7/513: dwrite d25/d27/d6a/d44/f54 [0,4194304] 0 2024-03-23T08:23:59.641 INFO:tasks.workunit.client.0.smithi070.stdout:2/807: rename d1/d21/ff9 to d1/d7/d39/d47/d7d/f10d 0 2024-03-23T08:23:59.643 INFO:tasks.workunit.client.0.smithi070.stdout:4/525: chown d3/d6/d35/d65/d27/d4b/d73/d74/db5/l86 0 1 2024-03-23T08:23:59.644 INFO:tasks.workunit.client.0.smithi070.stdout:1/593: creat d8/d26/d41/fd7 x:0 0 0 2024-03-23T08:23:59.654 INFO:tasks.workunit.client.0.smithi070.stdout:4/526: mkdir d3/d6/d35/d65/d27/d4b/dc0 0 2024-03-23T08:23:59.655 INFO:tasks.workunit.client.0.smithi070.stdout:4/527: fsync d3/f6d 0 2024-03-23T08:23:59.655 INFO:tasks.workunit.client.0.smithi070.stdout:1/594: symlink d8/d1b/ld8 0 2024-03-23T08:23:59.655 INFO:tasks.workunit.client.0.smithi070.stdout:1/595: chown d8/d11/f73 78290 1 2024-03-23T08:23:59.660 INFO:tasks.workunit.client.0.smithi070.stdout:1/596: dread d8/d42/d51/f91 [0,4194304] 0 2024-03-23T08:23:59.663 INFO:tasks.workunit.client.0.smithi070.stdout:4/528: creat d3/d6/d35/d3f/da9/fc1 x:0 0 0 2024-03-23T08:23:59.665 INFO:tasks.workunit.client.0.smithi070.stdout:2/808: link d1/d7/df/cfd d1/d7/df/d99/c10e 0 2024-03-23T08:23:59.667 INFO:tasks.workunit.client.0.smithi070.stdout:1/597: write d8/db/d40/d43/d64/d85/d97/fa7 [5217309,120039] 0 2024-03-23T08:23:59.668 INFO:tasks.workunit.client.0.smithi070.stdout:4/529: creat d3/d6/d35/d65/d41/fc2 x:0 0 0 2024-03-23T08:23:59.670 INFO:tasks.workunit.client.0.smithi070.stdout:1/598: mkdir d8/d26/d41/dd9 0 2024-03-23T08:23:59.675 INFO:tasks.workunit.client.0.smithi070.stdout:4/530: dwrite d3/d6/d35/d65/d41/fb9 [0,4194304] 0 2024-03-23T08:23:59.677 INFO:tasks.workunit.client.0.smithi070.stdout:4/531: fsync d3/d6/f6c 0 2024-03-23T08:23:59.680 INFO:tasks.workunit.client.0.smithi070.stdout:7/514: dwrite f23 [0,4194304] 0 2024-03-23T08:23:59.684 INFO:tasks.workunit.client.0.smithi070.stdout:1/599: link d8/d1b/f22 d8/d42/d4e/d99/fda 0 2024-03-23T08:23:59.687 INFO:tasks.workunit.client.0.smithi070.stdout:4/532: unlink d3/d6/d11/f18 0 2024-03-23T08:23:59.687 INFO:tasks.workunit.client.0.smithi070.stdout:1/600: dread d8/d26/d41/fca [0,4194304] 0 2024-03-23T08:23:59.692 INFO:tasks.workunit.client.0.smithi070.stdout:4/533: chown d3/d6/d35/d65/d27/l4f 16 1 2024-03-23T08:23:59.692 INFO:tasks.workunit.client.0.smithi070.stdout:7/515: dwrite d25/d27/d6a/f9c [0,4194304] 0 2024-03-23T08:23:59.698 INFO:tasks.workunit.client.0.smithi070.stdout:1/601: rmdir d8/d11 39 2024-03-23T08:23:59.698 INFO:tasks.workunit.client.0.smithi070.stdout:1/602: readlink d8/d26/d7b/l7d 0 2024-03-23T08:23:59.700 INFO:tasks.workunit.client.0.smithi070.stdout:4/534: symlink d3/d6/d35/d65/d6f/d93/lc3 0 2024-03-23T08:23:59.705 INFO:tasks.workunit.client.0.smithi070.stdout:7/516: rename d25/d27/d6a/d44/d6f/c88 to d25/cc0 0 2024-03-23T08:23:59.707 INFO:tasks.workunit.client.0.smithi070.stdout:7/517: dread - d25/d27/d6a/d44/d85/d95/da0/fb2 zero size 2024-03-23T08:23:59.707 INFO:tasks.workunit.client.0.smithi070.stdout:1/603: link d8/db/d40/d43/d64/d85/fb6 d8/d26/d49/d65/fdb 0 2024-03-23T08:23:59.709 INFO:tasks.workunit.client.0.smithi070.stdout:1/604: mknod d8/d42/d51/cdc 0 2024-03-23T08:23:59.710 INFO:tasks.workunit.client.0.smithi070.stdout:7/518: creat d25/d27/d6a/d44/d7c/db4/fc1 x:0 0 0 2024-03-23T08:23:59.715 INFO:tasks.workunit.client.0.smithi070.stdout:1/605: rename d8/d42/d51/db8 to d8/db/d2c/d3f/ddd 0 2024-03-23T08:23:59.718 INFO:tasks.workunit.client.0.smithi070.stdout:7/519: dwrite d25/f38 [4194304,4194304] 0 2024-03-23T08:23:59.720 INFO:tasks.workunit.client.0.smithi070.stdout:4/535: dread d3/f8f [0,4194304] 0 2024-03-23T08:23:59.720 INFO:tasks.workunit.client.0.smithi070.stdout:4/536: chown d3/d6/d35/d3f/d51/d6e/f99 0 1 2024-03-23T08:23:59.726 INFO:tasks.workunit.client.0.smithi070.stdout:7/520: mkdir d25/d27/dc2 0 2024-03-23T08:23:59.727 INFO:tasks.workunit.client.0.smithi070.stdout:7/521: chown d25/d27/d6a/f5b 2135422599 1 2024-03-23T08:23:59.728 INFO:tasks.workunit.client.0.smithi070.stdout:4/537: symlink d3/d6/d6a/lc4 0 2024-03-23T08:23:59.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:23:59 smithi070 ceph-mon[39699]: pgmap v39: 65 pgs: 65 active+clean; 1.4 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 41 MiB/s rd, 171 MiB/s wr, 409 op/s 2024-03-23T08:23:59.736 INFO:tasks.workunit.client.0.smithi070.stdout:7/522: mknod d25/d27/d6a/cc3 0 2024-03-23T08:23:59.737 INFO:tasks.workunit.client.0.smithi070.stdout:7/523: fdatasync d25/d27/d6a/d44/da2/fbf 0 2024-03-23T08:23:59.738 INFO:tasks.workunit.client.0.smithi070.stdout:4/538: rename d3/d6/d35/d65/d27/d4b/d73/d89/lad to d3/d6/d35/d65/d6f/lc5 0 2024-03-23T08:23:59.738 INFO:tasks.workunit.client.0.smithi070.stdout:7/524: fsync d25/f38 0 2024-03-23T08:23:59.740 INFO:tasks.workunit.client.0.smithi070.stdout:1/606: link d8/db/d40/d43/d64/d85/dc7/fcc d8/d26/d7b/d9c/fde 0 2024-03-23T08:23:59.743 INFO:tasks.workunit.client.0.smithi070.stdout:4/539: chown d3/d6/d11/l2c 330 1 2024-03-23T08:23:59.743 INFO:tasks.workunit.client.0.smithi070.stdout:4/540: readlink d3/d6/l72 0 2024-03-23T08:23:59.745 INFO:tasks.workunit.client.0.smithi070.stdout:4/541: write d3/d6/d9/f6b [1461148,76425] 0 2024-03-23T08:23:59.746 INFO:tasks.workunit.client.0.smithi070.stdout:7/525: link d25/f38 d25/d27/dc2/fc4 0 2024-03-23T08:23:59.749 INFO:tasks.workunit.client.0.smithi070.stdout:4/542: creat d3/d6/db7/fc6 x:0 0 0 2024-03-23T08:23:59.750 INFO:tasks.workunit.client.0.smithi070.stdout:7/526: dread d25/d27/d6a/d44/d7c/f8b [0,4194304] 0 2024-03-23T08:23:59.750 INFO:tasks.workunit.client.0.smithi070.stdout:4/543: fsync d3/d6/d35/d65/f30 0 2024-03-23T08:23:59.752 INFO:tasks.workunit.client.0.smithi070.stdout:1/607: dread d8/d1a/f28 [0,4194304] 0 2024-03-23T08:23:59.753 INFO:tasks.workunit.client.0.smithi070.stdout:4/544: mkdir d3/d6/d35/d65/d6f/d93/dc7 0 2024-03-23T08:23:59.754 INFO:tasks.workunit.client.0.smithi070.stdout:7/527: truncate d25/d40/f4b 1390615 0 2024-03-23T08:23:59.755 INFO:tasks.workunit.client.0.smithi070.stdout:7/528: chown d25/c74 1832 1 2024-03-23T08:23:59.760 INFO:tasks.workunit.client.0.smithi070.stdout:1/608: dwrite d8/f37 [0,4194304] 0 2024-03-23T08:23:59.763 INFO:tasks.workunit.client.0.smithi070.stdout:4/545: dwrite d3/d6/d9/fb4 [0,4194304] 0 2024-03-23T08:23:59.766 INFO:tasks.workunit.client.0.smithi070.stdout:4/546: chown d3/d6/d35/d65/d27/d4b/d73/d74/db5/l86 17 1 2024-03-23T08:23:59.769 INFO:tasks.workunit.client.0.smithi070.stdout:7/529: dwrite d25/d27/d6a/d44/d7c/f8b [0,4194304] 0 2024-03-23T08:23:59.774 INFO:tasks.workunit.client.0.smithi070.stdout:4/547: mknod d3/d6/d35/d65/d27/d4b/d73/d74/da3/cc8 0 2024-03-23T08:23:59.774 INFO:tasks.workunit.client.0.smithi070.stdout:7/530: rmdir d25/d27 39 2024-03-23T08:23:59.780 INFO:tasks.workunit.client.0.smithi070.stdout:7/531: creat d25/d27/d6a/d44/d6f/fc5 x:0 0 0 2024-03-23T08:23:59.782 INFO:tasks.workunit.client.0.smithi070.stdout:4/548: creat d3/d6/d35/d65/d6f/d93/dc7/fc9 x:0 0 0 2024-03-23T08:23:59.783 INFO:tasks.workunit.client.0.smithi070.stdout:8/635: sync 2024-03-23T08:23:59.784 INFO:tasks.workunit.client.0.smithi070.stdout:8/636: write d5/d8/d16/d30/fbc [4740610,12860] 0 2024-03-23T08:23:59.786 INFO:tasks.workunit.client.0.smithi070.stdout:8/637: write d5/d8/d16/d32/d84/d98/dde/da3/fbb [716883,84639] 0 2024-03-23T08:23:59.787 INFO:tasks.workunit.client.0.smithi070.stdout:4/549: read d3/d6/d35/f76 [1558126,66120] 0 2024-03-23T08:23:59.788 INFO:tasks.workunit.client.0.smithi070.stdout:8/638: chown d5/d8/d16/d30/d38/d6b/l78 1691 1 2024-03-23T08:23:59.788 INFO:tasks.workunit.client.0.smithi070.stdout:4/550: fdatasync d3/d6/d35/d65/d41/fc2 0 2024-03-23T08:23:59.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:23:59 smithi193 ceph-mon[48370]: pgmap v39: 65 pgs: 65 active+clean; 1.4 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 41 MiB/s rd, 171 MiB/s wr, 409 op/s 2024-03-23T08:23:59.896 INFO:tasks.workunit.client.0.smithi070.stdout:8/639: creat d5/d8/d16/d32/d84/d9f/fe3 x:0 0 0 2024-03-23T08:23:59.896 INFO:tasks.workunit.client.0.smithi070.stdout:4/551: mkdir d3/d6/dca 0 2024-03-23T08:23:59.898 INFO:tasks.workunit.client.0.smithi070.stdout:4/552: stat d3/d6/d35/d65/d27/d4b/fba 0 2024-03-23T08:23:59.960 INFO:tasks.workunit.client.0.smithi070.stdout:6/665: sync 2024-03-23T08:24:00.002 INFO:tasks.workunit.client.0.smithi070.stdout:7/532: read d25/d40/f4b [994481,86238] 0 2024-03-23T08:24:00.003 INFO:tasks.workunit.client.0.smithi070.stdout:7/533: chown d25/d27/d6a/cc3 12 1 2024-03-23T08:24:00.006 INFO:tasks.workunit.client.0.smithi070.stdout:7/534: read - d25/d27/d6a/d44/d7c/db4/fb6 zero size 2024-03-23T08:24:00.007 INFO:tasks.workunit.client.0.smithi070.stdout:7/535: read d25/d27/d6a/d44/d7c/f8b [4128916,116221] 0 2024-03-23T08:24:00.011 INFO:tasks.workunit.client.0.smithi070.stdout:1/609: write d8/db/d40/d43/f54 [1626280,9523] 0 2024-03-23T08:24:00.012 INFO:tasks.workunit.client.0.smithi070.stdout:1/610: readlink d8/d42/d4e/l6d 0 2024-03-23T08:24:00.014 INFO:tasks.workunit.client.0.smithi070.stdout:6/666: fdatasync d0/d3/f7e 0 2024-03-23T08:24:00.020 INFO:tasks.workunit.client.0.smithi070.stdout:8/640: truncate d5/d8/d16/d32/d84/f96 215597 0 2024-03-23T08:24:00.020 INFO:tasks.workunit.client.0.smithi070.stdout:7/536: symlink d25/d27/d6a/d44/d85/lc6 0 2024-03-23T08:24:00.021 INFO:tasks.workunit.client.0.smithi070.stdout:7/537: stat d25/d27/d6a/d44/f54 0 2024-03-23T08:24:00.022 INFO:tasks.workunit.client.0.smithi070.stdout:7/538: truncate d25/d27/d6a/f80 5143554 0 2024-03-23T08:24:00.026 INFO:tasks.workunit.client.0.smithi070.stdout:4/553: dread d3/d6/d35/d65/d1b/fae [0,4194304] 0 2024-03-23T08:24:00.026 INFO:tasks.workunit.client.0.smithi070.stdout:4/554: chown d3/l3a 228862 1 2024-03-23T08:24:00.029 INFO:tasks.workunit.client.0.smithi070.stdout:6/667: rmdir d0/d23/d4d 39 2024-03-23T08:24:00.034 INFO:tasks.workunit.client.0.smithi070.stdout:8/641: creat d5/d8/d16/dd4/d6f/d7c/d7d/fe4 x:0 0 0 2024-03-23T08:24:00.038 INFO:tasks.workunit.client.0.smithi070.stdout:7/539: dread d25/d27/d6a/d44/d6f/f71 [0,4194304] 0 2024-03-23T08:24:00.038 INFO:tasks.workunit.client.0.smithi070.stdout:1/611: rename d8/db/cd6 to d8/d42/d51/cdf 0 2024-03-23T08:24:00.041 INFO:tasks.workunit.client.0.smithi070.stdout:4/555: creat d3/d6/d35/d65/d27/d4b/fcb x:0 0 0 2024-03-23T08:24:00.042 INFO:tasks.workunit.client.0.smithi070.stdout:6/668: mknod d0/d3/d2c/ce2 0 2024-03-23T08:24:00.048 INFO:tasks.workunit.client.0.smithi070.stdout:1/612: creat d8/d26/d7b/fe0 x:0 0 0 2024-03-23T08:24:00.049 INFO:tasks.workunit.client.0.smithi070.stdout:7/540: unlink d25/d27/d6a/d44/f78 0 2024-03-23T08:24:00.052 INFO:tasks.workunit.client.0.smithi070.stdout:6/669: chown d0/c5c 920 1 2024-03-23T08:24:00.054 INFO:tasks.workunit.client.0.smithi070.stdout:1/613: dwrite d8/f1c [0,4194304] 0 2024-03-23T08:24:00.055 INFO:tasks.workunit.client.0.smithi070.stdout:8/642: mknod d5/d8/d16/dd4/ce5 0 2024-03-23T08:24:00.057 INFO:tasks.workunit.client.0.smithi070.stdout:8/643: readlink d5/d8/d16/d32/d84/d98/dde/l9e 0 2024-03-23T08:24:00.061 INFO:tasks.workunit.client.0.smithi070.stdout:7/541: truncate d25/d27/d6a/f5e 126938 0 2024-03-23T08:24:00.063 INFO:tasks.workunit.client.0.smithi070.stdout:7/542: write d25/d27/d6a/f73 [2674816,113953] 0 2024-03-23T08:24:00.065 INFO:tasks.workunit.client.0.smithi070.stdout:6/670: fdatasync d0/d23/d4d/da5/fcf 0 2024-03-23T08:24:00.068 INFO:tasks.workunit.client.0.smithi070.stdout:4/556: creat d3/d6/d11/fcc x:0 0 0 2024-03-23T08:24:00.075 INFO:tasks.workunit.client.0.smithi070.stdout:1/614: symlink d8/db/d2c/d3f/dd3/le1 0 2024-03-23T08:24:00.078 INFO:tasks.workunit.client.0.smithi070.stdout:7/543: truncate d25/d27/d6a/d44/f77 284046 0 2024-03-23T08:24:00.081 INFO:tasks.workunit.client.0.smithi070.stdout:6/671: fdatasync d0/d23/d28/d35/f77 0 2024-03-23T08:24:00.086 INFO:tasks.workunit.client.0.smithi070.stdout:6/672: dread d0/d23/f55 [0,4194304] 0 2024-03-23T08:24:00.091 INFO:tasks.workunit.client.0.smithi070.stdout:6/673: dwrite d0/d23/d28/d35/d6a/fb1 [0,4194304] 0 2024-03-23T08:24:00.183 INFO:tasks.workunit.client.0.smithi070.stdout:0/817: sync 2024-03-23T08:24:00.183 INFO:tasks.workunit.client.0.smithi070.stdout:9/653: sync 2024-03-23T08:24:00.183 INFO:tasks.workunit.client.0.smithi070.stdout:1/615: creat d8/d11/d61/d74/d5d/fe2 x:0 0 0 2024-03-23T08:24:00.188 INFO:tasks.workunit.client.0.smithi070.stdout:9/654: unlink db/d12/fa6 0 2024-03-23T08:24:00.193 INFO:tasks.workunit.client.0.smithi070.stdout:6/674: link d0/l33 d0/d23/d97/le3 0 2024-03-23T08:24:00.195 INFO:tasks.workunit.client.0.smithi070.stdout:9/655: creat db/fd5 x:0 0 0 2024-03-23T08:24:00.198 INFO:tasks.workunit.client.0.smithi070.stdout:9/656: mkdir db/d12/d34/d8c/da2/d26/dd6 0 2024-03-23T08:24:00.199 INFO:tasks.workunit.client.0.smithi070.stdout:6/675: chown d0/d23/d97/le3 122 1 2024-03-23T08:24:00.200 INFO:tasks.workunit.client.0.smithi070.stdout:9/657: creat db/d12/d34/d47/fd7 x:0 0 0 2024-03-23T08:24:00.200 INFO:tasks.workunit.client.0.smithi070.stdout:6/676: write d0/d23/d74/d84/db3/d2f/f4c [2578953,128200] 0 2024-03-23T08:24:00.208 INFO:tasks.workunit.client.0.smithi070.stdout:9/658: dwrite db/d12/d34/d8c/da2/f78 [0,4194304] 0 2024-03-23T08:24:00.210 INFO:tasks.workunit.client.0.smithi070.stdout:4/557: truncate d3/d6/d35/d65/d27/f4e 834323 0 2024-03-23T08:24:00.210 INFO:tasks.workunit.client.0.smithi070.stdout:7/544: write d25/d27/f5c [266568,33203] 0 2024-03-23T08:24:00.211 INFO:tasks.workunit.client.0.smithi070.stdout:9/659: mkdir db/db8/dd8 0 2024-03-23T08:24:00.213 INFO:tasks.workunit.client.0.smithi070.stdout:4/558: write d3/d6/d35/d65/f30 [4216976,45603] 0 2024-03-23T08:24:00.217 INFO:tasks.workunit.client.0.smithi070.stdout:9/660: rename db/d12/d34/d47/d90 to db/d12/d34/d8c/da2/d4f/dd9 0 2024-03-23T08:24:00.223 INFO:tasks.workunit.client.0.smithi070.stdout:7/545: symlink d25/d27/d6a/d44/da2/db5/lc7 0 2024-03-23T08:24:00.223 INFO:tasks.workunit.client.0.smithi070.stdout:7/546: dread - d25/d27/d6a/fba zero size 2024-03-23T08:24:00.226 INFO:tasks.workunit.client.0.smithi070.stdout:4/559: mknod d3/d6/d35/d65/d27/d4b/d73/d74/d7e/d9a/ccd 0 2024-03-23T08:24:00.227 INFO:tasks.workunit.client.0.smithi070.stdout:9/661: link db/d12/d34/d47/f86 db/dcd/fda 0 2024-03-23T08:24:00.230 INFO:tasks.workunit.client.0.smithi070.stdout:4/560: truncate d3/d6/d35/d65/d6f/d93/db8/f37 4544508 0 2024-03-23T08:24:00.232 INFO:tasks.workunit.client.0.smithi070.stdout:4/561: chown d3/d6/d9/f22 42 1 2024-03-23T08:24:00.234 INFO:tasks.workunit.client.0.smithi070.stdout:4/562: stat d3/d6/d35/d3f 0 2024-03-23T08:24:00.234 INFO:tasks.workunit.client.0.smithi070.stdout:6/677: dwrite d0/d23/d74/d84/d3c/f53 [0,4194304] 0 2024-03-23T08:24:00.235 INFO:tasks.workunit.client.0.smithi070.stdout:9/662: rename db/d12/d34/d8c/cb1 to db/d12/d67/cdb 0 2024-03-23T08:24:00.236 INFO:tasks.workunit.client.0.smithi070.stdout:9/663: fsync db/d12/d34/d8c/da2/f6a 0 2024-03-23T08:24:00.241 INFO:tasks.workunit.client.0.smithi070.stdout:6/678: creat d0/d23/d74/d84/db3/d2f/fe4 x:0 0 0 2024-03-23T08:24:00.242 INFO:tasks.workunit.client.0.smithi070.stdout:4/563: dwrite d3/d6/d11/f55 [0,4194304] 0 2024-03-23T08:24:00.247 INFO:tasks.workunit.client.0.smithi070.stdout:4/564: write d3/d6/f6c [4348585,60306] 0 2024-03-23T08:24:00.249 INFO:tasks.workunit.client.0.smithi070.stdout:6/679: creat d0/d23/d74/d84/db3/d2f/fe5 x:0 0 0 2024-03-23T08:24:00.250 INFO:tasks.workunit.client.0.smithi070.stdout:9/664: dwrite db/d12/d34/d8c/da2/dc6/d7e/fc5 [0,4194304] 0 2024-03-23T08:24:00.252 INFO:tasks.workunit.client.0.smithi070.stdout:6/680: dread - d0/d23/d74/d84/db3/d2f/fe5 zero size 2024-03-23T08:24:00.252 INFO:tasks.workunit.client.0.smithi070.stdout:6/681: chown d0/d3/d2c/ce2 609 1 2024-03-23T08:24:00.253 INFO:tasks.workunit.client.0.smithi070.stdout:9/665: creat db/d12/db7/fdc x:0 0 0 2024-03-23T08:24:00.254 INFO:tasks.workunit.client.0.smithi070.stdout:6/682: chown d0/d23/d28/fb4 1717942699 1 2024-03-23T08:24:00.256 INFO:tasks.workunit.client.0.smithi070.stdout:9/666: creat db/d12/d67/fdd x:0 0 0 2024-03-23T08:24:00.257 INFO:tasks.workunit.client.0.smithi070.stdout:4/565: creat d3/d6/d11/fce x:0 0 0 2024-03-23T08:24:00.258 INFO:tasks.workunit.client.0.smithi070.stdout:6/683: creat d0/d23/d28/d35/d85/fe6 x:0 0 0 2024-03-23T08:24:00.262 INFO:tasks.workunit.client.0.smithi070.stdout:4/566: rename d3/d6/d35/d65/d27/d4b/d73/d74 to d3/d6/d35/dcf 0 2024-03-23T08:24:00.265 INFO:tasks.workunit.client.0.smithi070.stdout:6/684: rename d0/d23/d74/d84/d3c/d5b/l81 to d0/d3/d2c/dd1/le7 0 2024-03-23T08:24:00.266 INFO:tasks.workunit.client.0.smithi070.stdout:9/667: creat db/fde x:0 0 0 2024-03-23T08:24:00.267 INFO:tasks.workunit.client.0.smithi070.stdout:9/668: stat db/d12/d34/d8c/da2/dc6/d7e/fc5 0 2024-03-23T08:24:00.268 INFO:tasks.workunit.client.0.smithi070.stdout:4/567: creat d3/d6/d35/d3f/fd0 x:0 0 0 2024-03-23T08:24:00.268 INFO:tasks.workunit.client.0.smithi070.stdout:4/568: fsync d3/d6/d35/d3f/da9/fc1 0 2024-03-23T08:24:00.270 INFO:tasks.workunit.client.0.smithi070.stdout:9/669: mknod db/cdf 0 2024-03-23T08:24:00.271 INFO:tasks.workunit.client.0.smithi070.stdout:6/685: creat d0/d23/d74/fe8 x:0 0 0 2024-03-23T08:24:00.274 INFO:tasks.workunit.client.0.smithi070.stdout:4/569: fdatasync d3/d6/d35/d65/d1b/f28 0 2024-03-23T08:24:00.276 INFO:tasks.workunit.client.0.smithi070.stdout:6/686: unlink d0/d23/d74/d84/d3c/d5b/ce0 0 2024-03-23T08:24:00.278 INFO:tasks.workunit.client.0.smithi070.stdout:9/670: dwrite db/d12/d34/d47/fd7 [0,4194304] 0 2024-03-23T08:24:00.279 INFO:tasks.workunit.client.0.smithi070.stdout:4/570: write d3/fc [1444452,115806] 0 2024-03-23T08:24:00.279 INFO:tasks.workunit.client.0.smithi070.stdout:6/687: chown d0/d23/d74/d84/d3c/d5b/cb0 236273767 1 2024-03-23T08:24:00.284 INFO:tasks.workunit.client.0.smithi070.stdout:9/671: mkdir db/d12/d34/d8c/da2/d58/db5/de0 0 2024-03-23T08:24:00.285 INFO:tasks.workunit.client.0.smithi070.stdout:9/672: stat db/d12/d34/d8c/da2/d4f/c8d 0 2024-03-23T08:24:00.287 INFO:tasks.workunit.client.0.smithi070.stdout:9/673: write db/d12/d14/f63 [3028445,99631] 0 2024-03-23T08:24:00.295 INFO:tasks.workunit.client.0.smithi070.stdout:9/674: symlink db/d12/d34/d8c/da2/dc6/le1 0 2024-03-23T08:24:00.296 INFO:tasks.workunit.client.0.smithi070.stdout:4/571: dwrite d3/d6/d35/d65/d1b/f1c [0,4194304] 0 2024-03-23T08:24:00.298 INFO:tasks.workunit.client.0.smithi070.stdout:9/675: write db/d12/d34/d8c/da2/d26/da7/fac [502133,13536] 0 2024-03-23T08:24:00.300 INFO:tasks.workunit.client.0.smithi070.stdout:6/688: dread d0/d23/f10 [0,4194304] 0 2024-03-23T08:24:00.300 INFO:tasks.workunit.client.0.smithi070.stdout:6/689: chown d0/d23/d4d/da5/lc0 7 1 2024-03-23T08:24:00.306 INFO:tasks.workunit.client.0.smithi070.stdout:6/690: unlink d0/lf 0 2024-03-23T08:24:00.308 INFO:tasks.workunit.client.0.smithi070.stdout:6/691: truncate d0/d23/d28/d35/d85/f9e 432030 0 2024-03-23T08:24:00.310 INFO:tasks.workunit.client.0.smithi070.stdout:4/572: truncate d3/f17 1479775 0 2024-03-23T08:24:00.310 INFO:tasks.workunit.client.0.smithi070.stdout:9/676: write db/d12/d34/d8c/da2/d26/f64 [1224455,68912] 0 2024-03-23T08:24:00.310 INFO:tasks.workunit.client.0.smithi070.stdout:6/692: truncate d0/d3/d2c/f76 993272 0 2024-03-23T08:24:00.313 INFO:tasks.workunit.client.0.smithi070.stdout:4/573: truncate d3/d6/d35/d65/d1b/f1c 4745135 0 2024-03-23T08:24:00.313 INFO:tasks.workunit.client.0.smithi070.stdout:9/677: mknod db/d12/d34/d8c/da2/d26/da7/ce2 0 2024-03-23T08:24:00.317 INFO:tasks.workunit.client.0.smithi070.stdout:6/693: dread d0/f20 [0,4194304] 0 2024-03-23T08:24:00.320 INFO:tasks.workunit.client.0.smithi070.stdout:6/694: stat d0/d23/d74/d84/db3/d38 0 2024-03-23T08:24:00.321 INFO:tasks.workunit.client.0.smithi070.stdout:9/678: rename db/dcd/fda to db/d12/d34/d8c/da2/d4f/dd9/dc2/fe3 0 2024-03-23T08:24:00.326 INFO:tasks.workunit.client.0.smithi070.stdout:4/574: dwrite d3/d6/d35/dcf/db5/dbb/fbd [0,4194304] 0 2024-03-23T08:24:00.327 INFO:tasks.workunit.client.0.smithi070.stdout:9/679: mknod db/d12/d34/ce4 0 2024-03-23T08:24:00.329 INFO:tasks.workunit.client.0.smithi070.stdout:9/680: read db/d12/f9a [3005525,23069] 0 2024-03-23T08:24:00.332 INFO:tasks.workunit.client.0.smithi070.stdout:4/575: mknod d3/d6/d35/d3f/d51/cd1 0 2024-03-23T08:24:00.332 INFO:tasks.workunit.client.0.smithi070.stdout:4/576: readlink d3/d6/d35/dcf/db5/l86 0 2024-03-23T08:24:00.341 INFO:tasks.workunit.client.0.smithi070.stdout:6/695: truncate d0/d23/d74/d84/f3b 2450852 0 2024-03-23T08:24:00.341 INFO:tasks.workunit.client.0.smithi070.stdout:9/681: write db/f1b [2035355,106120] 0 2024-03-23T08:24:00.344 INFO:tasks.workunit.client.0.smithi070.stdout:6/696: unlink d0/d3/f7e 0 2024-03-23T08:24:00.350 INFO:tasks.workunit.client.0.smithi070.stdout:9/682: rename db/d12/d34/d8c to db/d12/d14/de5 0 2024-03-23T08:24:00.354 INFO:tasks.workunit.client.0.smithi070.stdout:6/697: creat d0/d23/d88/db6/fe9 x:0 0 0 2024-03-23T08:24:00.361 INFO:tasks.workunit.client.0.smithi070.stdout:6/698: creat d0/d23/d74/d84/db3/fea x:0 0 0 2024-03-23T08:24:00.362 INFO:tasks.workunit.client.0.smithi070.stdout:6/699: chown d0/d23/d74/d84/d3c/f75 1 1 2024-03-23T08:24:00.366 INFO:tasks.workunit.client.0.smithi070.stdout:6/700: mknod d0/d23/d74/d84/d3c/d5b/dde/ceb 0 2024-03-23T08:24:00.368 INFO:tasks.workunit.client.0.smithi070.stdout:6/701: symlink d0/d23/d28/d35/dd6/lec 0 2024-03-23T08:24:00.374 INFO:tasks.workunit.client.0.smithi070.stdout:6/702: dwrite d0/d23/d74/d84/db3/d38/d52/f6b [0,4194304] 0 2024-03-23T08:24:00.375 INFO:tasks.workunit.client.0.smithi070.stdout:6/703: stat d0/d23/d74/d84/db3/d38/d52/f7f 0 2024-03-23T08:24:00.378 INFO:tasks.workunit.client.0.smithi070.stdout:6/704: unlink d0/f8c 0 2024-03-23T08:24:00.384 INFO:tasks.workunit.client.0.smithi070.stdout:6/705: dwrite d0/d23/d88/db6/fe9 [0,4194304] 0 2024-03-23T08:24:00.389 INFO:tasks.workunit.client.0.smithi070.stdout:6/706: getdents d0/d23/d28/d35/d85 0 2024-03-23T08:24:00.390 INFO:tasks.workunit.client.0.smithi070.stdout:6/707: rmdir d0/d3/d2c/dd1 39 2024-03-23T08:24:00.396 INFO:tasks.workunit.client.0.smithi070.stdout:6/708: dwrite d0/d3/d2c/f58 [0,4194304] 0 2024-03-23T08:24:00.404 INFO:tasks.workunit.client.0.smithi070.stdout:6/709: dwrite d0/d23/d28/d35/d6a/fb1 [0,4194304] 0 2024-03-23T08:24:00.415 INFO:tasks.workunit.client.0.smithi070.stdout:3/741: sync 2024-03-23T08:24:00.418 INFO:tasks.workunit.client.0.smithi070.stdout:3/742: rename db/df/d4d/cb9 to db/d4e/dd6/dd7/dd9/de8/c113 0 2024-03-23T08:24:00.419 INFO:tasks.workunit.client.0.smithi070.stdout:3/743: stat db/d14/d65/daa/fad 0 2024-03-23T08:24:00.454 INFO:tasks.workunit.client.0.smithi070.stdout:6/710: truncate d0/d23/d74/d84/d3c/f53 2218239 0 2024-03-23T08:24:00.455 INFO:tasks.workunit.client.0.smithi070.stdout:3/744: symlink db/d4e/df8/l114 0 2024-03-23T08:24:00.478 INFO:tasks.workunit.client.0.smithi070.stdout:3/745: symlink db/d20/d63/d83/l115 0 2024-03-23T08:24:00.480 INFO:tasks.workunit.client.0.smithi070.stdout:6/711: fdatasync d0/d23/d74/d84/db3/d2f/dab/db8/fca 0 2024-03-23T08:24:00.485 INFO:tasks.workunit.client.0.smithi070.stdout:3/746: truncate db/d14/d43/f54 1623359 0 2024-03-23T08:24:00.493 INFO:tasks.workunit.client.0.smithi070.stdout:3/747: creat db/d20/d7a/d80/f116 x:0 0 0 2024-03-23T08:24:00.501 INFO:tasks.workunit.client.0.smithi070.stdout:6/712: write d0/d23/d28/f36 [997819,80321] 0 2024-03-23T08:24:00.504 INFO:tasks.workunit.client.0.smithi070.stdout:6/713: fdatasync d0/f50 0 2024-03-23T08:24:00.507 INFO:tasks.workunit.client.0.smithi070.stdout:3/748: truncate db/df/d1a/de9/ff7 139222 0 2024-03-23T08:24:00.513 INFO:tasks.workunit.client.0.smithi070.stdout:3/749: link db/d14/d43/d62/l70 db/d7f/dcd/l117 0 2024-03-23T08:24:00.516 INFO:tasks.workunit.client.0.smithi070.stdout:6/714: dwrite d0/d23/d74/d84/faa [0,4194304] 0 2024-03-23T08:24:00.522 INFO:tasks.workunit.client.0.smithi070.stdout:6/715: creat d0/d23/d74/dcd/fed x:0 0 0 2024-03-23T08:24:00.528 INFO:tasks.workunit.client.0.smithi070.stdout:6/716: dwrite d0/d23/d28/d35/d85/f89 [0,4194304] 0 2024-03-23T08:24:00.529 INFO:tasks.workunit.client.0.smithi070.stdout:6/717: chown d0/d23/d74/d84/lc9 0 1 2024-03-23T08:24:00.552 INFO:tasks.workunit.client.0.smithi070.stdout:6/718: dread d0/d23/d88/f94 [0,4194304] 0 2024-03-23T08:24:00.553 INFO:tasks.workunit.client.0.smithi070.stdout:6/719: read d0/d3/fc8 [611956,53266] 0 2024-03-23T08:24:00.564 INFO:tasks.workunit.client.0.smithi070.stdout:6/720: truncate d0/d23/d74/d84/db3/d38/d52/f6b 2796916 0 2024-03-23T08:24:00.565 INFO:tasks.workunit.client.0.smithi070.stdout:6/721: stat d0/d23/d74/d84/db3/d38/d9a/lc2 0 2024-03-23T08:24:00.566 INFO:tasks.workunit.client.0.smithi070.stdout:6/722: truncate d0/d23/d28/d35/dd6/fdd 373703 0 2024-03-23T08:24:00.568 INFO:tasks.workunit.client.0.smithi070.stdout:6/723: symlink d0/d23/d74/d84/d3c/d86/lee 0 2024-03-23T08:24:00.572 INFO:tasks.workunit.client.0.smithi070.stdout:6/724: dread d0/d23/d88/db6/fe9 [0,4194304] 0 2024-03-23T08:24:00.573 INFO:tasks.workunit.client.0.smithi070.stdout:6/725: fsync d0/d23/d74/d84/faa 0 2024-03-23T08:24:00.576 INFO:tasks.workunit.client.0.smithi070.stdout:6/726: write d0/d23/d97/fa2 [1583993,115913] 0 2024-03-23T08:24:00.577 INFO:tasks.workunit.client.0.smithi070.stdout:6/727: chown d0/d3/d2c/f49 716950 1 2024-03-23T08:24:00.580 INFO:tasks.workunit.client.0.smithi070.stdout:6/728: fdatasync d0/d23/d28/d35/f77 0 2024-03-23T08:24:00.586 INFO:tasks.workunit.client.0.smithi070.stdout:6/729: dwrite d0/d3/d2c/f58 [0,4194304] 0 2024-03-23T08:24:00.587 INFO:tasks.workunit.client.0.smithi070.stdout:6/730: read - d0/d23/d74/d84/fc7 zero size 2024-03-23T08:24:00.598 INFO:tasks.workunit.client.0.smithi070.stdout:6/731: dread d0/d23/d4d/da5/fcf [0,4194304] 0 2024-03-23T08:24:00.599 INFO:tasks.workunit.client.0.smithi070.stdout:6/732: chown d0/d3/d2c/c83 600 1 2024-03-23T08:24:00.600 INFO:tasks.workunit.client.0.smithi070.stdout:6/733: chown d0/d23/d28/f37 2591 1 2024-03-23T08:24:00.601 INFO:tasks.workunit.client.0.smithi070.stdout:6/734: chown d0/d23/d74/d84/db3/d2f/dab/db8/fca 383445 1 2024-03-23T08:24:00.601 INFO:tasks.workunit.client.0.smithi070.stdout:6/735: stat d0/d23/d88/fdb 0 2024-03-23T08:24:00.604 INFO:tasks.workunit.client.0.smithi070.stdout:6/736: dread d0/d23/d4d/fd4 [0,4194304] 0 2024-03-23T08:24:00.613 INFO:tasks.workunit.client.0.smithi070.stdout:6/737: dwrite d0/d23/f19 [4194304,4194304] 0 2024-03-23T08:24:00.616 INFO:tasks.workunit.client.0.smithi070.stdout:6/738: truncate d0/d23/d74/d84/db3/d38/d9a/fad 78662 0 2024-03-23T08:24:00.617 INFO:tasks.workunit.client.0.smithi070.stdout:6/739: fsync d0/d23/d74/d84/db3/d38/d52/f7f 0 2024-03-23T08:24:00.625 INFO:tasks.workunit.client.0.smithi070.stdout:6/740: dread d0/d23/d28/f4a [0,4194304] 0 2024-03-23T08:24:00.673 INFO:tasks.workunit.client.0.smithi070.stdout:5/531: sync 2024-03-23T08:24:00.673 INFO:tasks.workunit.client.0.smithi070.stdout:2/809: sync 2024-03-23T08:24:00.674 INFO:tasks.workunit.client.0.smithi070.stdout:5/532: write dc/d1b/d1d/d25/d36/d32/f87 [202018,33705] 0 2024-03-23T08:24:00.675 INFO:tasks.workunit.client.0.smithi070.stdout:2/810: fdatasync d1/d7/df/d99/fac 0 2024-03-23T08:24:00.691 INFO:tasks.workunit.client.0.smithi070.stdout:2/811: dread d1/d7/df/d94/f28 [0,4194304] 0 2024-03-23T08:24:00.692 INFO:tasks.workunit.client.0.smithi070.stdout:2/812: read d1/fc [3665791,61325] 0 2024-03-23T08:24:00.738 INFO:tasks.workunit.client.0.smithi070.stdout:6/741: rmdir d0/d23/d4d/da5 39 2024-03-23T08:24:00.744 INFO:tasks.workunit.client.0.smithi070.stdout:5/533: fsync dc/d1b/d1d/d25/d38/d48/d92/f98 0 2024-03-23T08:24:00.746 INFO:tasks.workunit.client.0.smithi070.stdout:6/742: dread d0/d3/f4 [0,4194304] 0 2024-03-23T08:24:00.748 INFO:tasks.workunit.client.0.smithi070.stdout:6/743: fdatasync d0/d23/d74/fe8 0 2024-03-23T08:24:00.750 INFO:tasks.workunit.client.0.smithi070.stdout:5/534: dwrite dc/f2b [0,4194304] 0 2024-03-23T08:24:00.766 INFO:tasks.workunit.client.0.smithi070.stdout:5/535: mknod dc/d1b/d1d/d25/d38/d48/d92/caa 0 2024-03-23T08:24:00.768 INFO:tasks.workunit.client.0.smithi070.stdout:6/744: mknod d0/d23/d74/d84/db3/cef 0 2024-03-23T08:24:00.768 INFO:tasks.workunit.client.0.smithi070.stdout:5/536: stat dc/d1b/d1d/f37 0 2024-03-23T08:24:00.769 INFO:tasks.workunit.client.0.smithi070.stdout:5/537: truncate dc/f2b 4245087 0 2024-03-23T08:24:00.777 INFO:tasks.workunit.client.0.smithi070.stdout:2/813: write d1/d7/df/d99/dea/dc3/f3e [1357567,24231] 0 2024-03-23T08:24:00.780 INFO:tasks.workunit.client.0.smithi070.stdout:2/814: creat d1/d21/f10f x:0 0 0 2024-03-23T08:24:00.784 INFO:tasks.workunit.client.0.smithi070.stdout:2/815: creat d1/d7/d48/d69/db8/f110 x:0 0 0 2024-03-23T08:24:00.788 INFO:tasks.workunit.client.0.smithi070.stdout:2/816: rename d1/d6/d30 to d1/d7/d48/d69/db8/dc7/d111 0 2024-03-23T08:24:00.790 INFO:tasks.workunit.client.0.smithi070.stdout:2/817: readlink d1/d7/d39/d47/d90/lb3 0 2024-03-23T08:24:00.870 INFO:tasks.workunit.client.0.smithi070.stdout:2/818: fsync d1/d7/d39/d47/dda/faa 0 2024-03-23T08:24:00.871 INFO:tasks.workunit.client.0.smithi070.stdout:2/819: chown d1/d7/df/d99/dea/dc3 3923093 1 2024-03-23T08:24:00.890 INFO:tasks.workunit.client.0.smithi070.stdout:5/538: write dc/d1b/d1d/d25/d38/f40 [1050331,95382] 0 2024-03-23T08:24:00.891 INFO:tasks.workunit.client.0.smithi070.stdout:6/745: dwrite d0/f20 [0,4194304] 0 2024-03-23T08:24:00.900 INFO:tasks.workunit.client.0.smithi070.stdout:5/539: dwrite dc/d1b/d1d/d25/d38/d39/f66 [0,4194304] 0 2024-03-23T08:24:00.902 INFO:tasks.workunit.client.0.smithi070.stdout:5/540: write dc/d1b/d1d/d25/d38/d53/f9a [463459,25199] 0 2024-03-23T08:24:00.913 INFO:tasks.workunit.client.0.smithi070.stdout:8/644: sync 2024-03-23T08:24:00.920 INFO:tasks.workunit.client.0.smithi070.stdout:8/645: dwrite d5/fe [0,4194304] 0 2024-03-23T08:24:00.929 INFO:tasks.workunit.client.0.smithi070.stdout:2/820: write d1/d7/df/f91 [2204981,128121] 0 2024-03-23T08:24:00.929 INFO:tasks.workunit.client.0.smithi070.stdout:6/746: rmdir d0/d23/d97 39 2024-03-23T08:24:00.932 INFO:tasks.workunit.client.0.smithi070.stdout:8/646: mkdir d5/d8/d16/d32/d84/d98/dde/da3/dab/de6 0 2024-03-23T08:24:00.936 INFO:tasks.workunit.client.0.smithi070.stdout:6/747: dread d0/d23/d74/d84/d3c/f53 [0,4194304] 0 2024-03-23T08:24:00.945 INFO:tasks.workunit.client.0.smithi070.stdout:5/541: dread dc/d1b/d1d/d25/d38/d53/f7a [0,4194304] 0 2024-03-23T08:24:00.966 INFO:tasks.workunit.client.0.smithi070.stdout:6/748: symlink d0/d23/d74/dcd/lf0 0 2024-03-23T08:24:00.967 INFO:tasks.workunit.client.0.smithi070.stdout:5/542: creat dc/d1b/d1d/d25/d36/d32/d5c/fab x:0 0 0 2024-03-23T08:24:00.968 INFO:tasks.workunit.client.0.smithi070.stdout:6/749: symlink d0/d23/d74/d84/d3c/lf1 0 2024-03-23T08:24:00.969 INFO:tasks.workunit.client.0.smithi070.stdout:6/750: readlink d0/d23/d74/d84/db3/d38/l4e 0 2024-03-23T08:24:00.971 INFO:tasks.workunit.client.0.smithi070.stdout:6/751: creat d0/d23/d28/d35/ff2 x:0 0 0 2024-03-23T08:24:00.974 INFO:tasks.workunit.client.0.smithi070.stdout:6/752: read d0/d23/d74/d84/db3/d2f/f30 [224067,96714] 0 2024-03-23T08:24:00.977 INFO:tasks.workunit.client.0.smithi070.stdout:6/753: truncate d0/d23/d74/d84/db3/f8f 3695910 0 2024-03-23T08:24:00.987 INFO:tasks.workunit.client.0.smithi070.stdout:6/754: chown d0/d3/d2c/l42 25047204 1 2024-03-23T08:24:00.993 INFO:tasks.workunit.client.0.smithi070.stdout:6/755: fdatasync d0/d23/d74/d84/db3/d38/d9a/fad 0 2024-03-23T08:24:01.003 INFO:tasks.workunit.client.0.smithi070.stdout:6/756: mknod d0/d23/d74/d84/db3/cf3 0 2024-03-23T08:24:01.011 INFO:tasks.workunit.client.0.smithi070.stdout:6/757: mkdir d0/d23/d28/df4 0 2024-03-23T08:24:01.017 INFO:tasks.workunit.client.0.smithi070.stdout:1/616: sync 2024-03-23T08:24:01.018 INFO:tasks.workunit.client.0.smithi070.stdout:0/818: sync 2024-03-23T08:24:01.018 INFO:tasks.workunit.client.0.smithi070.stdout:1/617: readlink d8/ld 0 2024-03-23T08:24:01.020 INFO:tasks.workunit.client.0.smithi070.stdout:0/819: chown d16/d26/d27/lc7 1012403 1 2024-03-23T08:24:01.020 INFO:tasks.workunit.client.0.smithi070.stdout:8/647: write d5/d8/d16/d3b/d8c/f99 [1293442,79035] 0 2024-03-23T08:24:01.024 INFO:tasks.workunit.client.0.smithi070.stdout:1/618: creat d8/d1a/fe3 x:0 0 0 2024-03-23T08:24:01.029 INFO:tasks.workunit.client.0.smithi070.stdout:6/758: dread d0/d23/d4d/da5/fcf [0,4194304] 0 2024-03-23T08:24:01.031 INFO:tasks.workunit.client.0.smithi070.stdout:1/619: creat d8/d26/d41/fe4 x:0 0 0 2024-03-23T08:24:01.037 INFO:tasks.workunit.client.0.smithi070.stdout:6/759: dread d0/d3/d2c/f58 [0,4194304] 0 2024-03-23T08:24:01.040 INFO:tasks.workunit.client.0.smithi070.stdout:1/620: rename d8/db/d2c/d3f/ddd/ld1 to d8/d11/d61/d74/d5d/d70/le5 0 2024-03-23T08:24:01.044 INFO:tasks.workunit.client.0.smithi070.stdout:6/760: creat d0/d23/d74/d84/db3/d2f/dab/ff5 x:0 0 0 2024-03-23T08:24:01.046 INFO:tasks.workunit.client.0.smithi070.stdout:8/648: link d5/d8/d16/dd4/f69 d5/d8/d16/d30/fe7 0 2024-03-23T08:24:01.047 INFO:tasks.workunit.client.0.smithi070.stdout:8/649: fdatasync d5/d8/d16/d3b/f5f 0 2024-03-23T08:24:01.048 INFO:tasks.workunit.client.0.smithi070.stdout:1/621: mkdir d8/db/d40/d43/d64/d85/dc7/de6 0 2024-03-23T08:24:01.048 INFO:tasks.workunit.client.0.smithi070.stdout:1/622: chown d8/ca0 154118 1 2024-03-23T08:24:01.049 INFO:tasks.workunit.client.0.smithi070.stdout:6/761: read d0/d23/d4d/fd4 [284618,56084] 0 2024-03-23T08:24:01.052 INFO:tasks.workunit.client.0.smithi070.stdout:6/762: symlink d0/d23/d74/dcd/lf6 0 2024-03-23T08:24:01.053 INFO:tasks.workunit.client.0.smithi070.stdout:1/623: mkdir d8/db/d40/d43/d64/d85/d97/de7 0 2024-03-23T08:24:01.055 INFO:tasks.workunit.client.0.smithi070.stdout:8/650: getdents d5/d8/d16/d32/de0 0 2024-03-23T08:24:01.057 INFO:tasks.workunit.client.0.smithi070.stdout:1/624: mkdir d8/d26/d49/d65/de8 0 2024-03-23T08:24:01.059 INFO:tasks.workunit.client.0.smithi070.stdout:8/651: rename d5/d8/d16/d3b/f65 to d5/d8/d16/d32/de0/fe8 0 2024-03-23T08:24:01.060 INFO:tasks.workunit.client.0.smithi070.stdout:1/625: creat d8/db/d40/d43/d64/d85/dc7/fe9 x:0 0 0 2024-03-23T08:24:01.064 INFO:tasks.workunit.client.0.smithi070.stdout:1/626: creat d8/d26/d41/dd9/fea x:0 0 0 2024-03-23T08:24:01.067 INFO:tasks.workunit.client.0.smithi070.stdout:1/627: dread d8/d1a/f28 [0,4194304] 0 2024-03-23T08:24:01.068 INFO:tasks.workunit.client.0.smithi070.stdout:1/628: readlink d8/db/d2c/d3f/d48/dab/lc4 0 2024-03-23T08:24:01.071 INFO:tasks.workunit.client.0.smithi070.stdout:8/652: dread d5/d8/d16/d32/f46 [0,4194304] 0 2024-03-23T08:24:01.074 INFO:tasks.workunit.client.0.smithi070.stdout:8/653: creat d5/d8/d9a/da2/fe9 x:0 0 0 2024-03-23T08:24:01.076 INFO:tasks.workunit.client.0.smithi070.stdout:1/629: link d8/d11/fb2 d8/db/d40/d43/feb 0 2024-03-23T08:24:01.077 INFO:tasks.workunit.client.0.smithi070.stdout:0/820: dwrite d16/d22/d43/d47/fd6 [0,4194304] 0 2024-03-23T08:24:01.084 INFO:tasks.workunit.client.0.smithi070.stdout:1/630: mknod d8/d26/d7b/d9c/cec 0 2024-03-23T08:24:01.090 INFO:tasks.workunit.client.0.smithi070.stdout:8/654: link d5/d8/d16/d32/d84/d98/dde/lb4 d5/d8/d16/dd4/d6f/lea 0 2024-03-23T08:24:01.092 INFO:tasks.workunit.client.0.smithi070.stdout:0/821: rmdir d16/d22/d43/d47/da9/de7/dfa 0 2024-03-23T08:24:01.094 INFO:tasks.workunit.client.0.smithi070.stdout:8/655: mkdir d5/d8/d16/d32/d84/da7/deb 0 2024-03-23T08:24:01.095 INFO:tasks.workunit.client.0.smithi070.stdout:6/763: write d0/d3/f4 [3909393,19630] 0 2024-03-23T08:24:01.096 INFO:tasks.workunit.client.0.smithi070.stdout:6/764: write d0/d23/d74/d84/fc7 [844767,51372] 0 2024-03-23T08:24:01.100 INFO:tasks.workunit.client.0.smithi070.stdout:4/577: sync 2024-03-23T08:24:01.100 INFO:tasks.workunit.client.0.smithi070.stdout:3/750: sync 2024-03-23T08:24:01.100 INFO:tasks.workunit.client.0.smithi070.stdout:7/547: sync 2024-03-23T08:24:01.100 INFO:tasks.workunit.client.0.smithi070.stdout:9/683: sync 2024-03-23T08:24:01.102 INFO:tasks.workunit.client.0.smithi070.stdout:3/751: stat db/d20/d7a/ld2 0 2024-03-23T08:24:01.103 INFO:tasks.workunit.client.0.smithi070.stdout:0/822: link d16/d22/c44 d16/d26/d27/d4c/d60/d75/c11d 0 2024-03-23T08:24:01.105 INFO:tasks.workunit.client.0.smithi070.stdout:6/765: mknod d0/d23/d74/d84/db3/d2f/dab/db8/cf7 0 2024-03-23T08:24:01.108 INFO:tasks.workunit.client.0.smithi070.stdout:6/766: chown d0/fa7 1937 1 2024-03-23T08:24:01.110 INFO:tasks.workunit.client.0.smithi070.stdout:1/631: fsync d8/db/d40/d43/feb 0 2024-03-23T08:24:01.110 INFO:tasks.workunit.client.0.smithi070.stdout:7/548: dread d25/d27/d6a/f9c [0,4194304] 0 2024-03-23T08:24:01.112 INFO:tasks.workunit.client.0.smithi070.stdout:9/684: dwrite db/d12/d14/f63 [0,4194304] 0 2024-03-23T08:24:01.114 INFO:tasks.workunit.client.0.smithi070.stdout:7/549: fsync d25/d27/fac 0 2024-03-23T08:24:01.115 INFO:tasks.workunit.client.0.smithi070.stdout:8/656: dread d5/d8/d16/d30/f39 [0,4194304] 0 2024-03-23T08:24:01.121 INFO:tasks.workunit.client.0.smithi070.stdout:6/767: dwrite d0/d23/d28/d35/d85/f9e [0,4194304] 0 2024-03-23T08:24:01.123 INFO:tasks.workunit.client.0.smithi070.stdout:0/823: rmdir d16/d26/d27/d4c/d60/d75 39 2024-03-23T08:24:01.124 INFO:tasks.workunit.client.0.smithi070.stdout:9/685: dwrite db/f1b [0,4194304] 0 2024-03-23T08:24:01.129 INFO:tasks.workunit.client.0.smithi070.stdout:1/632: fdatasync d8/f90 0 2024-03-23T08:24:01.133 INFO:tasks.workunit.client.0.smithi070.stdout:3/752: getdents db/d14/d6f 0 2024-03-23T08:24:01.135 INFO:tasks.workunit.client.0.smithi070.stdout:3/753: write db/d14/d43/d62/f112 [963321,129239] 0 2024-03-23T08:24:01.137 INFO:tasks.workunit.client.0.smithi070.stdout:3/754: read db/d20/fee [3697648,5479] 0 2024-03-23T08:24:01.138 INFO:tasks.workunit.client.0.smithi070.stdout:8/657: chown d5/d8/d16/d32/f74 1254076 1 2024-03-23T08:24:01.140 INFO:tasks.workunit.client.0.smithi070.stdout:0/824: mknod d16/d22/d43/d47/d87/c11e 0 2024-03-23T08:24:01.140 INFO:tasks.workunit.client.0.smithi070.stdout:8/658: write d5/d8/d16/d3b/f5d [767300,23028] 0 2024-03-23T08:24:01.140 INFO:tasks.workunit.client.0.smithi070.stdout:5/543: sync 2024-03-23T08:24:01.141 INFO:tasks.workunit.client.0.smithi070.stdout:2/821: sync 2024-03-23T08:24:01.143 INFO:tasks.workunit.client.0.smithi070.stdout:9/686: mkdir db/d12/d14/de5/da2/d4f/de6 0 2024-03-23T08:24:01.146 INFO:tasks.workunit.client.0.smithi070.stdout:7/550: write d25/d40/fa5 [292236,58305] 0 2024-03-23T08:24:01.146 INFO:tasks.workunit.client.0.smithi070.stdout:7/551: chown l11 8691936 1 2024-03-23T08:24:01.152 INFO:tasks.workunit.client.0.smithi070.stdout:6/768: truncate d0/d23/d74/d84/d3c/f75 472136 0 2024-03-23T08:24:01.153 INFO:tasks.workunit.client.0.smithi070.stdout:8/659: symlink d5/d8/d16/d3b/d8c/lec 0 2024-03-23T08:24:01.153 INFO:tasks.workunit.client.0.smithi070.stdout:1/633: dread d8/db/f32 [0,4194304] 0 2024-03-23T08:24:01.154 INFO:tasks.workunit.client.0.smithi070.stdout:2/822: symlink d1/d29/d52/de4/l112 0 2024-03-23T08:24:01.155 INFO:tasks.workunit.client.0.smithi070.stdout:5/544: rename dc/d1b/d1d to dc/d1b/d1d/dac 22 2024-03-23T08:24:01.155 INFO:tasks.workunit.client.0.smithi070.stdout:8/660: write d5/d8/d16/d32/d84/d98/dde/da3/fb2 [2299158,83183] 0 2024-03-23T08:24:01.156 INFO:tasks.workunit.client.0.smithi070.stdout:9/687: write db/d12/d14/de5/da2/dc6/dce/fc7 [2508033,26798] 0 2024-03-23T08:24:01.160 INFO:tasks.workunit.client.0.smithi070.stdout:7/552: chown d25/d27/d6a/d44/d7c 81192 1 2024-03-23T08:24:01.161 INFO:tasks.workunit.client.0.smithi070.stdout:3/755: write db/d4e/df8/f3e [3220477,114344] 0 2024-03-23T08:24:01.163 INFO:tasks.workunit.client.0.smithi070.stdout:4/578: link d3/d6/d35/d65/d6f/lc5 d3/d6/d35/d65/d1b/ld2 0 2024-03-23T08:24:01.164 INFO:tasks.workunit.client.0.smithi070.stdout:1/634: dwrite d8/db/d2c/d3f/d48/f58 [0,4194304] 0 2024-03-23T08:24:01.166 INFO:tasks.workunit.client.0.smithi070.stdout:6/769: fdatasync d0/d23/d74/d84/db3/d2f/dab/fac 0 2024-03-23T08:24:01.169 INFO:tasks.workunit.client.0.smithi070.stdout:4/579: dread d3/d6/f43 [0,4194304] 0 2024-03-23T08:24:01.171 INFO:tasks.workunit.client.0.smithi070.stdout:2/823: creat d1/d29/d52/f113 x:0 0 0 2024-03-23T08:24:01.171 INFO:tasks.workunit.client.0.smithi070.stdout:7/553: dwrite d25/f83 [0,4194304] 0 2024-03-23T08:24:01.174 INFO:tasks.workunit.client.0.smithi070.stdout:6/770: dread d0/d23/d74/d84/fc7 [0,4194304] 0 2024-03-23T08:24:01.174 INFO:tasks.workunit.client.0.smithi070.stdout:7/554: dread - d25/d27/d6a/fbe zero size 2024-03-23T08:24:01.174 INFO:tasks.workunit.client.0.smithi070.stdout:5/545: mkdir dc/d1b/d1d/d25/d68/d8a/dad 0 2024-03-23T08:24:01.177 INFO:tasks.workunit.client.0.smithi070.stdout:6/771: write d0/d23/d74/fe8 [521693,109145] 0 2024-03-23T08:24:01.179 INFO:tasks.workunit.client.0.smithi070.stdout:9/688: mkdir db/d12/d14/de5/da2/d4f/dd9/dc2/de7 0 2024-03-23T08:24:01.180 INFO:tasks.workunit.client.0.smithi070.stdout:6/772: chown d0/d3/c18 0 1 2024-03-23T08:24:01.182 INFO:tasks.workunit.client.0.smithi070.stdout:2/824: dwrite d1/d7/d39/d47/d7b/f105 [0,4194304] 0 2024-03-23T08:24:01.183 INFO:tasks.workunit.client.0.smithi070.stdout:4/580: chown d3/d6/d45/c61 36127 1 2024-03-23T08:24:01.184 INFO:tasks.workunit.client.0.smithi070.stdout:4/581: chown d3/d6/d9/f2e 512412180 1 2024-03-23T08:24:01.189 INFO:tasks.workunit.client.0.smithi070.stdout:5/546: fdatasync dc/d1b/d1d/d25/d36/f2a 0 2024-03-23T08:24:01.189 INFO:tasks.workunit.client.0.smithi070.stdout:8/661: rename d5/d8/d16/d32/d84/f96 to d5/d8/d9a/dbe/fed 0 2024-03-23T08:24:01.195 INFO:tasks.workunit.client.0.smithi070.stdout:4/582: dread d3/d6/d35/d65/d41/f79 [0,4194304] 0 2024-03-23T08:24:01.198 INFO:tasks.workunit.client.0.smithi070.stdout:9/689: mkdir db/de8 0 2024-03-23T08:24:01.201 INFO:tasks.workunit.client.0.smithi070.stdout:2/825: mknod d1/d29/d52/de4/d10a/c114 0 2024-03-23T08:24:01.204 INFO:tasks.workunit.client.0.smithi070.stdout:4/583: symlink d3/d6/d35/d65/d27/d4b/d73/ld3 0 2024-03-23T08:24:01.206 INFO:tasks.workunit.client.0.smithi070.stdout:5/547: rename dc/f2b to dc/d1b/d1d/d61/fae 0 2024-03-23T08:24:01.209 INFO:tasks.workunit.client.0.smithi070.stdout:9/690: unlink db/d12/d14/de5/da2/dc6/dce/fb0 0 2024-03-23T08:24:01.211 INFO:tasks.workunit.client.0.smithi070.stdout:7/555: dwrite d25/d27/f92 [0,4194304] 0 2024-03-23T08:24:01.212 INFO:tasks.workunit.client.0.smithi070.stdout:7/556: chown d25/f38 76459 1 2024-03-23T08:24:01.213 INFO:tasks.workunit.client.0.smithi070.stdout:4/584: mknod d3/d6/d35/cd4 0 2024-03-23T08:24:01.216 INFO:tasks.workunit.client.0.smithi070.stdout:5/548: dread - dc/d1b/d1d/f6f zero size 2024-03-23T08:24:01.217 INFO:tasks.workunit.client.0.smithi070.stdout:2/826: creat d1/d7/d48/d69/db8/dd1/f115 x:0 0 0 2024-03-23T08:24:01.221 INFO:tasks.workunit.client.0.smithi070.stdout:8/662: rename d5/d8/d16/l5a to d5/d8/d16/dd4/dc9/lee 0 2024-03-23T08:24:01.222 INFO:tasks.workunit.client.0.smithi070.stdout:8/663: readlink d5/d8/d16/d3b/d8c/lec 0 2024-03-23T08:24:01.228 INFO:tasks.workunit.client.0.smithi070.stdout:7/557: mkdir d25/d27/d6a/d44/dc8 0 2024-03-23T08:24:01.233 INFO:tasks.workunit.client.0.smithi070.stdout:8/664: creat d5/d8/d16/d32/d84/fef x:0 0 0 2024-03-23T08:24:01.236 INFO:tasks.workunit.client.0.smithi070.stdout:2/827: symlink d1/d6/d103/l116 0 2024-03-23T08:24:01.237 INFO:tasks.workunit.client.0.smithi070.stdout:7/558: creat d25/d40/fc9 x:0 0 0 2024-03-23T08:24:01.238 INFO:tasks.workunit.client.0.smithi070.stdout:4/585: creat d3/d6/d11/fd5 x:0 0 0 2024-03-23T08:24:01.240 INFO:tasks.workunit.client.0.smithi070.stdout:8/665: rename d5/d8/d9a/da2/dae/l9c to d5/d8/d16/d32/d84/d98/lf0 0 2024-03-23T08:24:01.244 INFO:tasks.workunit.client.0.smithi070.stdout:2/828: rename d1/d6/f72 to d1/d7/df/d94/d38/f117 0 2024-03-23T08:24:01.244 INFO:tasks.workunit.client.0.smithi070.stdout:8/666: dread d5/d8/d16/d30/f39 [0,4194304] 0 2024-03-23T08:24:01.245 INFO:tasks.workunit.client.0.smithi070.stdout:7/559: symlink d25/d27/d6a/d44/d85/lca 0 2024-03-23T08:24:01.246 INFO:tasks.workunit.client.0.smithi070.stdout:8/667: readlink d5/d8/d16/d32/d84/d98/dde/da3/dab/ld0 0 2024-03-23T08:24:01.246 INFO:tasks.workunit.client.0.smithi070.stdout:8/668: read - d5/fd3 zero size 2024-03-23T08:24:01.248 INFO:tasks.workunit.client.0.smithi070.stdout:4/586: rename d3/d6/d35/d3f/d51/d6e/lbf to d3/d6/d35/dcf/da3/ld6 0 2024-03-23T08:24:01.250 INFO:tasks.workunit.client.0.smithi070.stdout:5/549: truncate dc/d1b/d1d/d25/d38/f86 2418542 0 2024-03-23T08:24:01.251 INFO:tasks.workunit.client.0.smithi070.stdout:8/669: creat d5/d8/d9a/da2/ff1 x:0 0 0 2024-03-23T08:24:01.252 INFO:tasks.workunit.client.0.smithi070.stdout:4/587: creat d3/d6/d35/d65/d27/d4b/fd7 x:0 0 0 2024-03-23T08:24:01.254 INFO:tasks.workunit.client.0.smithi070.stdout:7/560: mknod d25/ccb 0 2024-03-23T08:24:01.255 INFO:tasks.workunit.client.0.smithi070.stdout:8/670: chown d5/d8/f57 159 1 2024-03-23T08:24:01.257 INFO:tasks.workunit.client.0.smithi070.stdout:5/550: symlink dc/d1b/d1d/d25/d36/d32/laf 0 2024-03-23T08:24:01.257 INFO:tasks.workunit.client.0.smithi070.stdout:7/561: dread - d25/d27/d6a/f7f zero size 2024-03-23T08:24:01.258 INFO:tasks.workunit.client.0.smithi070.stdout:4/588: mkdir d3/d6/dd8 0 2024-03-23T08:24:01.259 INFO:tasks.workunit.client.0.smithi070.stdout:4/589: dread - d3/d6/d35/d65/d6f/fb1 zero size 2024-03-23T08:24:01.325 INFO:tasks.workunit.client.0.smithi070.stdout:2/829: getdents d1/d7/d48/d69/d7a 0 2024-03-23T08:24:01.326 INFO:tasks.workunit.client.0.smithi070.stdout:5/551: truncate dc/d1b/d1d/d25/d38/d48/f72 547379 0 2024-03-23T08:24:01.328 INFO:tasks.workunit.client.0.smithi070.stdout:4/590: creat d3/d6/d35/d65/d27/d4b/d73/d89/fd9 x:0 0 0 2024-03-23T08:24:01.328 INFO:tasks.workunit.client.0.smithi070.stdout:4/591: chown d3/d6/d35/d3f/da9/cb2 5736720 1 2024-03-23T08:24:01.329 INFO:tasks.workunit.client.0.smithi070.stdout:7/562: mkdir d25/d27/d6a/d44/d6f/dcc 0 2024-03-23T08:24:01.329 INFO:tasks.workunit.client.0.smithi070.stdout:4/592: chown d3/d6/d35/dcf/db5/dbb/fbd 2764 1 2024-03-23T08:24:01.345 INFO:tasks.workunit.client.0.smithi070.stdout:2/830: unlink d1/d6/cbc 0 2024-03-23T08:24:01.354 INFO:tasks.workunit.client.0.smithi070.stdout:5/552: truncate dc/d1b/d1d/d25/d38/d39/f79 3968279 0 2024-03-23T08:24:01.355 INFO:tasks.workunit.client.0.smithi070.stdout:7/563: mkdir d25/d27/d6a/d44/d6f/dab/dcd 0 2024-03-23T08:24:01.356 INFO:tasks.workunit.client.0.smithi070.stdout:2/831: write d1/d21/fa9 [409478,113616] 0 2024-03-23T08:24:01.358 INFO:tasks.workunit.client.0.smithi070.stdout:4/593: rename d3/d6/d35/d65/d6f/d93 to d3/d6/d35/dcf/db5/dda 0 2024-03-23T08:24:01.361 INFO:tasks.workunit.client.0.smithi070.stdout:5/553: rmdir dc/d1b/d1d/d25/d68 39 2024-03-23T08:24:01.361 INFO:tasks.workunit.client.0.smithi070.stdout:5/554: read - dc/d1b/d1d/d61/f94 zero size 2024-03-23T08:24:01.365 INFO:tasks.workunit.client.0.smithi070.stdout:7/564: rename d25/d40/l62 to d25/d27/d6a/d44/dc8/lce 0 2024-03-23T08:24:01.366 INFO:tasks.workunit.client.0.smithi070.stdout:4/594: rmdir d3/d6/d6a 39 2024-03-23T08:24:01.373 INFO:tasks.workunit.client.0.smithi070.stdout:5/555: dread dc/d1b/d1d/d25/d38/f73 [0,4194304] 0 2024-03-23T08:24:01.374 INFO:tasks.workunit.client.0.smithi070.stdout:5/556: readlink dc/l24 0 2024-03-23T08:24:01.376 INFO:tasks.workunit.client.0.smithi070.stdout:7/565: creat d25/d27/d6a/d44/da2/db5/fcf x:0 0 0 2024-03-23T08:24:01.380 INFO:tasks.workunit.client.0.smithi070.stdout:2/832: rename d1/d6 to d1/d118 0 2024-03-23T08:24:01.380 INFO:tasks.workunit.client.0.smithi070.stdout:2/833: chown d1/d7/d39/f9c 228782 1 2024-03-23T08:24:01.381 INFO:tasks.workunit.client.0.smithi070.stdout:2/834: stat d1/d7/df/d94/f84 0 2024-03-23T08:24:01.387 INFO:tasks.workunit.client.0.smithi070.stdout:2/835: unlink d1/d7/cbd 0 2024-03-23T08:24:01.392 INFO:tasks.workunit.client.0.smithi070.stdout:7/566: write d25/d27/d6a/f5e [607846,128503] 0 2024-03-23T08:24:01.393 INFO:tasks.workunit.client.0.smithi070.stdout:2/836: creat d1/d7/d48/d69/d7a/d108/f119 x:0 0 0 2024-03-23T08:24:01.394 INFO:tasks.workunit.client.0.smithi070.stdout:4/595: dwrite d3/f17 [0,4194304] 0 2024-03-23T08:24:01.400 INFO:tasks.workunit.client.0.smithi070.stdout:7/567: read d25/d27/d6a/d44/f54 [1562211,6697] 0 2024-03-23T08:24:01.401 INFO:tasks.workunit.client.0.smithi070.stdout:5/557: dread dc/d1b/d1d/d25/d38/f69 [0,4194304] 0 2024-03-23T08:24:01.405 INFO:tasks.workunit.client.0.smithi070.stdout:7/568: creat d25/d27/d6a/d44/d7c/fd0 x:0 0 0 2024-03-23T08:24:01.407 INFO:tasks.workunit.client.0.smithi070.stdout:4/596: dwrite d3/d6/d45/f47 [0,4194304] 0 2024-03-23T08:24:01.409 INFO:tasks.workunit.client.0.smithi070.stdout:4/597: stat d3/d6/d35/d3f/d51/cb6 0 2024-03-23T08:24:01.414 INFO:tasks.workunit.client.0.smithi070.stdout:2/837: dread d1/d118/f65 [0,4194304] 0 2024-03-23T08:24:01.415 INFO:tasks.workunit.client.0.smithi070.stdout:7/569: mknod d25/d27/dc2/cd1 0 2024-03-23T08:24:01.416 INFO:tasks.workunit.client.0.smithi070.stdout:4/598: rename d3/d6/d35/c95 to d3/d6/d35/d3f/da9/cdb 0 2024-03-23T08:24:01.418 INFO:tasks.workunit.client.0.smithi070.stdout:2/838: rmdir d1/d7/d48/d5b 39 2024-03-23T08:24:01.420 INFO:tasks.workunit.client.0.smithi070.stdout:2/839: truncate d1/d7/d48/d69/d7a/fec 524052 0 2024-03-23T08:24:01.422 INFO:tasks.workunit.client.0.smithi070.stdout:5/558: link dc/d1b/d1d/d25/d38/c3b dc/d1b/d1d/d25/d38/d48/cb0 0 2024-03-23T08:24:01.422 INFO:tasks.workunit.client.0.smithi070.stdout:2/840: readlink d1/d118/l102 0 2024-03-23T08:24:01.425 INFO:tasks.workunit.client.0.smithi070.stdout:5/559: chown dc/d1b/d1d/d25/d38/d39/c82 1128887692 1 2024-03-23T08:24:01.426 INFO:tasks.workunit.client.0.smithi070.stdout:7/570: dwrite d25/d27/d6a/d44/f67 [0,4194304] 0 2024-03-23T08:24:01.428 INFO:tasks.workunit.client.0.smithi070.stdout:4/599: rename d3/d6/d6a/c34 to d3/d6/d35/d3f/d51/d6e/d97/cdc 0 2024-03-23T08:24:01.429 INFO:tasks.workunit.client.0.smithi070.stdout:4/600: stat d3/d6/d35/d65/d1b/ld2 0 2024-03-23T08:24:01.429 INFO:tasks.workunit.client.0.smithi070.stdout:5/560: write dc/d1b/d1d/d25/d38/d53/f60 [2074281,62537] 0 2024-03-23T08:24:01.437 INFO:tasks.workunit.client.0.smithi070.stdout:4/601: creat d3/d6/d35/dcf/db5/dda/db8/d88/fdd x:0 0 0 2024-03-23T08:24:01.439 INFO:tasks.workunit.client.0.smithi070.stdout:5/561: read dc/d1b/d1d/d25/d38/d48/f57 [53159,41430] 0 2024-03-23T08:24:01.442 INFO:tasks.workunit.client.0.smithi070.stdout:5/562: write dc/d1b/d1d/d25/d38/f40 [3727299,68637] 0 2024-03-23T08:24:01.444 INFO:tasks.workunit.client.0.smithi070.stdout:2/841: dwrite d1/d118/d34/f3d [0,4194304] 0 2024-03-23T08:24:01.446 INFO:tasks.workunit.client.0.smithi070.stdout:7/571: truncate f24 104744 0 2024-03-23T08:24:01.446 INFO:tasks.workunit.client.0.smithi070.stdout:2/842: dread - d1/d7/d48/d69/db8/dc7/d111/fd2 zero size 2024-03-23T08:24:01.447 INFO:tasks.workunit.client.0.smithi070.stdout:7/572: write d25/d40/fa5 [816035,49698] 0 2024-03-23T08:24:01.448 INFO:tasks.workunit.client.0.smithi070.stdout:2/843: write d1/d7/d39/d47/dda/faa [6307028,46689] 0 2024-03-23T08:24:01.448 INFO:tasks.workunit.client.0.smithi070.stdout:4/602: write d3/d6/d35/d65/d1b/f28 [4192555,107912] 0 2024-03-23T08:24:01.449 INFO:tasks.workunit.client.0.smithi070.stdout:5/563: symlink dc/d1b/lb1 0 2024-03-23T08:24:01.450 INFO:tasks.workunit.client.0.smithi070.stdout:7/573: write d25/d27/fac [4779870,33406] 0 2024-03-23T08:24:01.452 INFO:tasks.workunit.client.0.smithi070.stdout:4/603: symlink d3/d6/d35/d3f/lde 0 2024-03-23T08:24:01.452 INFO:tasks.workunit.client.0.smithi070.stdout:5/564: dread - dc/d1b/d1d/d25/d36/d32/d5c/f9b zero size 2024-03-23T08:24:01.454 INFO:tasks.workunit.client.0.smithi070.stdout:2/844: rename d1/d7/df/la1 to d1/d21/d5f/l11a 0 2024-03-23T08:24:01.456 INFO:tasks.workunit.client.0.smithi070.stdout:2/845: dread - d1/d7/df/d94/fe9 zero size 2024-03-23T08:24:01.460 INFO:tasks.workunit.client.0.smithi070.stdout:7/574: rmdir d25/d27/d6a/d44/d7c 39 2024-03-23T08:24:01.462 INFO:tasks.workunit.client.0.smithi070.stdout:4/604: rename d3/d6/db7 to d3/d6/d35/d3f/d51/d6e/d97/ddf 0 2024-03-23T08:24:01.465 INFO:tasks.workunit.client.0.smithi070.stdout:2/846: creat d1/d7/df/d99/f11b x:0 0 0 2024-03-23T08:24:01.467 INFO:tasks.workunit.client.0.smithi070.stdout:4/605: dread d3/d6/d45/f47 [0,4194304] 0 2024-03-23T08:24:01.473 INFO:tasks.workunit.client.0.smithi070.stdout:2/847: stat d1/d7/f66 0 2024-03-23T08:24:01.475 INFO:tasks.workunit.client.0.smithi070.stdout:4/606: dwrite d3/d6/d35/dcf/db5/dda/db8/fa7 [0,4194304] 0 2024-03-23T08:24:01.476 INFO:tasks.workunit.client.0.smithi070.stdout:5/565: creat dc/d1b/d1d/d25/d68/d8a/d99/fb2 x:0 0 0 2024-03-23T08:24:01.478 INFO:tasks.workunit.client.0.smithi070.stdout:4/607: fsync d3/d6/d9/f6b 0 2024-03-23T08:24:01.480 INFO:tasks.workunit.client.0.smithi070.stdout:5/566: fsync dc/d1b/d1d/d25/d38/d53/f9a 0 2024-03-23T08:24:01.481 INFO:tasks.workunit.client.0.smithi070.stdout:7/575: dwrite d25/d40/f68 [0,4194304] 0 2024-03-23T08:24:01.483 INFO:tasks.workunit.client.0.smithi070.stdout:5/567: chown dc/d1b/d1d/d25/d38/l7f 111 1 2024-03-23T08:24:01.484 INFO:tasks.workunit.client.0.smithi070.stdout:7/576: chown d25/d27/d6a/d44/d6f/dab 97199 1 2024-03-23T08:24:01.488 INFO:tasks.workunit.client.0.smithi070.stdout:5/568: symlink dc/d1b/d1d/d25/d38/d39/lb3 0 2024-03-23T08:24:01.494 INFO:tasks.workunit.client.0.smithi070.stdout:5/569: rename dc/d1b/d1d/d61/fae to dc/d1b/d1d/d25/fb4 0 2024-03-23T08:24:01.495 INFO:tasks.workunit.client.0.smithi070.stdout:5/570: fdatasync dc/d1b/d1d/d25/d38/d53/f60 0 2024-03-23T08:24:01.498 INFO:tasks.workunit.client.0.smithi070.stdout:2/848: dread d1/fc [0,4194304] 0 2024-03-23T08:24:01.499 INFO:tasks.workunit.client.0.smithi070.stdout:2/849: fsync d1/d7/f20 0 2024-03-23T08:24:01.500 INFO:tasks.workunit.client.0.smithi070.stdout:4/608: truncate d3/d6/d9/fb4 1825785 0 2024-03-23T08:24:01.502 INFO:tasks.workunit.client.0.smithi070.stdout:2/850: mknod d1/d29/d52/c11c 0 2024-03-23T08:24:01.503 INFO:tasks.workunit.client.0.smithi070.stdout:5/571: dwrite dc/d1b/d1d/d25/d38/d53/f76 [0,4194304] 0 2024-03-23T08:24:01.504 INFO:tasks.workunit.client.0.smithi070.stdout:2/851: readlink d1/d7/d39/d47/dda/l93 0 2024-03-23T08:24:01.515 INFO:tasks.workunit.client.0.smithi070.stdout:5/572: link dc/d1b/d1d/d25/d38/d39/f4f dc/d1b/d1d/d25/d68/d8a/fb5 0 2024-03-23T08:24:01.521 INFO:tasks.workunit.client.0.smithi070.stdout:5/573: dwrite dc/d1b/d1d/d25/d38/d53/f60 [0,4194304] 0 2024-03-23T08:24:01.522 INFO:tasks.workunit.client.0.smithi070.stdout:5/574: stat f1 0 2024-03-23T08:24:01.531 INFO:tasks.workunit.client.0.smithi070.stdout:2/852: dread d1/d7/f40 [0,4194304] 0 2024-03-23T08:24:01.535 INFO:tasks.workunit.client.0.smithi070.stdout:2/853: creat d1/d7/df/d99/dea/f11d x:0 0 0 2024-03-23T08:24:01.536 INFO:tasks.workunit.client.0.smithi070.stdout:2/854: dread - d1/d7/df/d94/fe9 zero size 2024-03-23T08:24:01.539 INFO:tasks.workunit.client.0.smithi070.stdout:2/855: unlink d1/d118/dc0/fe6 0 2024-03-23T08:24:01.540 INFO:tasks.workunit.client.0.smithi070.stdout:2/856: write d1/d7/df/d94/f84 [8218133,80577] 0 2024-03-23T08:24:01.542 INFO:tasks.workunit.client.0.smithi070.stdout:2/857: read - d1/d7/d48/d69/db8/dd1/f115 zero size 2024-03-23T08:24:01.548 INFO:tasks.workunit.client.0.smithi070.stdout:2/858: dwrite d1/d7/f1e [0,4194304] 0 2024-03-23T08:24:01.562 INFO:tasks.workunit.client.0.smithi070.stdout:2/859: dwrite d1/d7/d39/d47/d7d/fb4 [0,4194304] 0 2024-03-23T08:24:01.563 INFO:tasks.workunit.client.0.smithi070.stdout:2/860: creat d1/d29/d52/f11e x:0 0 0 2024-03-23T08:24:01.568 INFO:tasks.workunit.client.0.smithi070.stdout:2/861: dread d1/d21/fa9 [0,4194304] 0 2024-03-23T08:24:01.570 INFO:tasks.workunit.client.0.smithi070.stdout:2/862: mknod d1/d118/d34/c11f 0 2024-03-23T08:24:01.573 INFO:tasks.workunit.client.0.smithi070.stdout:2/863: creat d1/d29/d52/de4/d10a/f120 x:0 0 0 2024-03-23T08:24:01.575 INFO:tasks.workunit.client.0.smithi070.stdout:2/864: mknod d1/d118/c121 0 2024-03-23T08:24:01.577 INFO:tasks.workunit.client.0.smithi070.stdout:2/865: mkdir d1/d7/df/d94/d38/d122 0 2024-03-23T08:24:01.579 INFO:tasks.workunit.client.0.smithi070.stdout:2/866: dread d1/d7/d48/d69/d7a/fec [0,4194304] 0 2024-03-23T08:24:01.583 INFO:tasks.workunit.client.0.smithi070.stdout:2/867: creat d1/d7/d48/d69/db8/dc7/d111/d104/f123 x:0 0 0 2024-03-23T08:24:01.585 INFO:tasks.workunit.client.0.smithi070.stdout:2/868: creat d1/d7/df/d94/d38/f124 x:0 0 0 2024-03-23T08:24:01.586 INFO:tasks.workunit.client.0.smithi070.stdout:2/869: rmdir d1/d118/d34 39 2024-03-23T08:24:01.595 INFO:tasks.workunit.client.0.smithi070.stdout:2/870: dread d1/d7/df/f91 [0,4194304] 0 2024-03-23T08:24:01.596 INFO:tasks.workunit.client.0.smithi070.stdout:2/871: mknod d1/d21/c125 0 2024-03-23T08:24:01.603 INFO:tasks.workunit.client.0.smithi070.stdout:2/872: dwrite d1/d7/d48/d69/d7a/fa5 [4194304,4194304] 0 2024-03-23T08:24:01.603 INFO:tasks.workunit.client.0.smithi070.stdout:2/873: chown d1/d7/d39/f82 337855599 1 2024-03-23T08:24:01.666 INFO:tasks.workunit.client.0.smithi070.stdout:0/825: sync 2024-03-23T08:24:01.666 INFO:tasks.workunit.client.0.smithi070.stdout:0/826: write d16/d26/d27/d3d/d4d/ffb [84907,76135] 0 2024-03-23T08:24:01.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:01 smithi070 ceph-mon[39699]: pgmap v40: 65 pgs: 65 active+clean; 1.4 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 37 MiB/s rd, 114 MiB/s wr, 300 op/s 2024-03-23T08:24:01.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:01 smithi193 ceph-mon[48370]: pgmap v40: 65 pgs: 65 active+clean; 1.4 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 37 MiB/s rd, 114 MiB/s wr, 300 op/s 2024-03-23T08:24:01.863 INFO:tasks.workunit.client.0.smithi070.stdout:1/635: sync 2024-03-23T08:24:01.863 INFO:tasks.workunit.client.0.smithi070.stdout:3/756: sync 2024-03-23T08:24:01.863 INFO:tasks.workunit.client.0.smithi070.stdout:3/757: fdatasync db/f17 0 2024-03-23T08:24:01.902 INFO:tasks.workunit.client.0.smithi070.stdout:1/636: stat d8/db/d8e/fcd 0 2024-03-23T08:24:01.902 INFO:tasks.workunit.client.0.smithi070.stdout:3/758: mknod db/d20/d49/de1/c118 0 2024-03-23T08:24:01.903 INFO:tasks.workunit.client.0.smithi070.stdout:3/759: chown db/d14/d43/d62/lab 255 1 2024-03-23T08:24:02.010 INFO:tasks.workunit.client.0.smithi070.stdout:1/637: mknod d8/db/d40/d43/d64/d85/ced 0 2024-03-23T08:24:02.011 INFO:tasks.workunit.client.0.smithi070.stdout:3/760: mknod db/df/d5d/c119 0 2024-03-23T08:24:02.012 INFO:tasks.workunit.client.0.smithi070.stdout:3/761: write db/d4e/dd6/dd7/dd9/f104 [544053,68406] 0 2024-03-23T08:24:02.014 INFO:tasks.workunit.client.0.smithi070.stdout:3/762: read - db/d7f/fdb zero size 2024-03-23T08:24:02.017 INFO:tasks.workunit.client.0.smithi070.stdout:1/638: dwrite d8/d1a/f39 [4194304,4194304] 0 2024-03-23T08:24:02.018 INFO:tasks.workunit.client.0.smithi070.stdout:9/691: sync 2024-03-23T08:24:02.019 INFO:tasks.workunit.client.0.smithi070.stdout:6/773: sync 2024-03-23T08:24:02.022 INFO:tasks.workunit.client.0.smithi070.stdout:1/639: creat d8/d42/fee x:0 0 0 2024-03-23T08:24:02.024 INFO:tasks.workunit.client.0.smithi070.stdout:9/692: write db/d12/d14/de5/da2/d58/db5/fc3 [116673,14850] 0 2024-03-23T08:24:02.026 INFO:tasks.workunit.client.0.smithi070.stdout:6/774: creat d0/d23/d74/d84/db3/d38/d9a/ff8 x:0 0 0 2024-03-23T08:24:02.026 INFO:tasks.workunit.client.0.smithi070.stdout:3/763: link db/d14/d6f/c79 db/d4e/dd6/dd7/dd9/de8/c11a 0 2024-03-23T08:24:02.030 INFO:tasks.workunit.client.0.smithi070.stdout:6/775: dread d0/d23/d88/f94 [0,4194304] 0 2024-03-23T08:24:02.031 INFO:tasks.workunit.client.0.smithi070.stdout:9/693: rename db/d12/d14/de5/da2/dc6/c5e to db/d12/d14/de5/ce9 0 2024-03-23T08:24:02.033 INFO:tasks.workunit.client.0.smithi070.stdout:3/764: fdatasync db/d4e/df8/d22/f2c 0 2024-03-23T08:24:02.035 INFO:tasks.workunit.client.0.smithi070.stdout:9/694: creat db/d12/d14/de5/da2/d26/fea x:0 0 0 2024-03-23T08:24:02.037 INFO:tasks.workunit.client.0.smithi070.stdout:3/765: unlink db/df/d5d/c119 0 2024-03-23T08:24:02.038 INFO:tasks.workunit.client.0.smithi070.stdout:6/776: truncate d0/d23/d74/d84/db3/f8f 184890 0 2024-03-23T08:24:02.039 INFO:tasks.workunit.client.0.smithi070.stdout:9/695: fsync db/fbf 0 2024-03-23T08:24:02.042 INFO:tasks.workunit.client.0.smithi070.stdout:6/777: symlink d0/d23/d74/d84/db3/d2f/dab/lf9 0 2024-03-23T08:24:02.044 INFO:tasks.workunit.client.0.smithi070.stdout:3/766: symlink db/d14/d65/l11b 0 2024-03-23T08:24:02.046 INFO:tasks.workunit.client.0.smithi070.stdout:6/778: readlink d0/d23/d97/d98/lb7 0 2024-03-23T08:24:02.047 INFO:tasks.workunit.client.0.smithi070.stdout:3/767: fdatasync db/df/d5d/fa9 0 2024-03-23T08:24:02.051 INFO:tasks.workunit.client.0.smithi070.stdout:9/696: rename db/f65 to db/d12/d14/de5/da2/dc6/dce/d85/feb 0 2024-03-23T08:24:02.054 INFO:tasks.workunit.client.0.smithi070.stdout:3/768: mknod db/d4e/df8/d22/dd8/c11c 0 2024-03-23T08:24:02.054 INFO:tasks.workunit.client.0.smithi070.stdout:6/779: dwrite d0/d23/d88/f94 [4194304,4194304] 0 2024-03-23T08:24:02.071 INFO:tasks.workunit.client.0.smithi070.stdout:9/697: getdents db/d12/d14/de5 0 2024-03-23T08:24:02.074 INFO:tasks.workunit.client.0.smithi070.stdout:9/698: mkdir db/d12/d14/de5/da2/d26/dec 0 2024-03-23T08:24:02.077 INFO:tasks.workunit.client.0.smithi070.stdout:9/699: rename db/d12/d14/de5/da2/dc6/dce/d85/c50 to db/d12/d14/de5/da2/d58/db5/ced 0 2024-03-23T08:24:02.081 INFO:tasks.workunit.client.0.smithi070.stdout:9/700: mknod db/d12/d14/de5/da2/d58/db5/cee 0 2024-03-23T08:24:02.081 INFO:tasks.workunit.client.0.smithi070.stdout:6/780: rmdir d0/d23/d74/d84/db3/d2f/dab 39 2024-03-23T08:24:02.082 INFO:tasks.workunit.client.0.smithi070.stdout:9/701: write db/d12/db7/fcf [404395,5558] 0 2024-03-23T08:24:02.082 INFO:tasks.workunit.client.0.smithi070.stdout:6/781: fsync d0/d23/d28/d35/d85/fe6 0 2024-03-23T08:24:02.084 INFO:tasks.workunit.client.0.smithi070.stdout:6/782: readlink d0/d23/d74/d84/d3c/l56 0 2024-03-23T08:24:02.091 INFO:tasks.workunit.client.0.smithi070.stdout:6/783: rename d0/d3/c67 to d0/d23/d74/d84/db3/d2f/dab/cfa 0 2024-03-23T08:24:02.091 INFO:tasks.workunit.client.0.smithi070.stdout:3/769: write db/df/d1a/f3c [1411370,52528] 0 2024-03-23T08:24:02.100 INFO:tasks.workunit.client.0.smithi070.stdout:6/784: rename d0/d23/d74/d84/d3c/l82 to d0/d23/d28/d35/d6a/lfb 0 2024-03-23T08:24:02.101 INFO:tasks.workunit.client.0.smithi070.stdout:6/785: read d0/d23/d28/f36 [3571120,36242] 0 2024-03-23T08:24:02.107 INFO:tasks.workunit.client.0.smithi070.stdout:9/702: write db/d12/d14/de5/da2/d4f/f72 [429308,111399] 0 2024-03-23T08:24:02.108 INFO:tasks.workunit.client.0.smithi070.stdout:3/770: getdents db/d4e/d92/dbb/de2 0 2024-03-23T08:24:02.108 INFO:tasks.workunit.client.0.smithi070.stdout:9/703: chown db/d12 30622 1 2024-03-23T08:24:02.110 INFO:tasks.workunit.client.0.smithi070.stdout:6/786: mkdir d0/d3/d2c/dd1/dfc 0 2024-03-23T08:24:02.115 INFO:tasks.workunit.client.0.smithi070.stdout:9/704: mknod db/d12/d14/de5/da2/d4f/d5a/cef 0 2024-03-23T08:24:02.120 INFO:tasks.workunit.client.0.smithi070.stdout:9/705: mknod db/d12/dab/cf0 0 2024-03-23T08:24:02.126 INFO:tasks.workunit.client.0.smithi070.stdout:9/706: rename db/d12/d14/de5/da2/d4f/d5a/f9c to db/db8/ff1 0 2024-03-23T08:24:02.127 INFO:tasks.workunit.client.0.smithi070.stdout:9/707: chown db/d12/d14/de5/da2/dc6/dce/d85/fc9 0 1 2024-03-23T08:24:02.128 INFO:tasks.workunit.client.0.smithi070.stdout:8/671: sync 2024-03-23T08:24:02.128 INFO:tasks.workunit.client.0.smithi070.stdout:4/609: sync 2024-03-23T08:24:02.128 INFO:tasks.workunit.client.0.smithi070.stdout:7/577: sync 2024-03-23T08:24:02.130 INFO:tasks.workunit.client.0.smithi070.stdout:4/610: stat d3/d6/d35/d65/d1b/fae 0 2024-03-23T08:24:02.131 INFO:tasks.workunit.client.0.smithi070.stdout:4/611: dread - d3/d6/d35/d65/d6f/fb1 zero size 2024-03-23T08:24:02.132 INFO:tasks.workunit.client.0.smithi070.stdout:4/612: chown d3/d6/d35/d65/d6f/f82 83749975 1 2024-03-23T08:24:02.132 INFO:tasks.workunit.client.0.smithi070.stdout:8/672: fdatasync d5/d8/d16/d3b/f76 0 2024-03-23T08:24:02.134 INFO:tasks.workunit.client.0.smithi070.stdout:9/708: link db/cdf db/d12/d14/de5/da2/d26/cf2 0 2024-03-23T08:24:02.134 INFO:tasks.workunit.client.0.smithi070.stdout:3/771: write db/d4e/df8/d22/db2/fc7 [3516058,94698] 0 2024-03-23T08:24:02.141 INFO:tasks.workunit.client.0.smithi070.stdout:3/772: chown db/d14/d43/d62/c93 26508574 1 2024-03-23T08:24:02.144 INFO:tasks.workunit.client.0.smithi070.stdout:6/787: dwrite d0/d23/d4d/da5/fcf [0,4194304] 0 2024-03-23T08:24:02.145 INFO:tasks.workunit.client.0.smithi070.stdout:9/709: mkdir db/d12/d14/de5/da2/d26/da7/df3 0 2024-03-23T08:24:02.149 INFO:tasks.workunit.client.0.smithi070.stdout:3/773: creat db/d20/d49/de1/f11d x:0 0 0 2024-03-23T08:24:02.150 INFO:tasks.workunit.client.0.smithi070.stdout:9/710: creat db/d12/d14/ff4 x:0 0 0 2024-03-23T08:24:02.155 INFO:tasks.workunit.client.0.smithi070.stdout:7/578: dwrite d25/d27/f6d [0,4194304] 0 2024-03-23T08:24:02.158 INFO:tasks.workunit.client.0.smithi070.stdout:4/613: dwrite d3/d6/d35/f67 [0,4194304] 0 2024-03-23T08:24:02.164 INFO:tasks.workunit.client.0.smithi070.stdout:4/614: fdatasync d3/d6/d35/dcf/db5/f8d 0 2024-03-23T08:24:02.167 INFO:tasks.workunit.client.0.smithi070.stdout:9/711: write db/d12/d14/f95 [4015854,5648] 0 2024-03-23T08:24:02.170 INFO:tasks.workunit.client.0.smithi070.stdout:8/673: dwrite d5/d8/f2c [4194304,4194304] 0 2024-03-23T08:24:02.170 INFO:tasks.workunit.client.0.smithi070.stdout:3/774: getdents db/d20/d49/de1 0 2024-03-23T08:24:02.173 INFO:tasks.workunit.client.0.smithi070.stdout:9/712: creat db/d12/d91/ff5 x:0 0 0 2024-03-23T08:24:02.175 INFO:tasks.workunit.client.0.smithi070.stdout:7/579: write d25/d27/d6a/d44/d85/f98 [802635,124160] 0 2024-03-23T08:24:02.175 INFO:tasks.workunit.client.0.smithi070.stdout:3/775: mkdir db/d4e/dc4/d11e 0 2024-03-23T08:24:02.181 INFO:tasks.workunit.client.0.smithi070.stdout:9/713: rmdir db/d12/d34/d47 39 2024-03-23T08:24:02.182 INFO:tasks.workunit.client.0.smithi070.stdout:7/580: stat d25/d27/d6a/d44/d7c/db4 0 2024-03-23T08:24:02.183 INFO:tasks.workunit.client.0.smithi070.stdout:9/714: creat db/d12/d14/de5/ff6 x:0 0 0 2024-03-23T08:24:02.184 INFO:tasks.workunit.client.0.smithi070.stdout:3/776: dread db/f23 [0,4194304] 0 2024-03-23T08:24:02.184 INFO:tasks.workunit.client.0.smithi070.stdout:9/715: stat db/d12/d67 0 2024-03-23T08:24:02.185 INFO:tasks.workunit.client.0.smithi070.stdout:7/581: dread d25/d27/f5c [0,4194304] 0 2024-03-23T08:24:02.186 INFO:tasks.workunit.client.0.smithi070.stdout:9/716: chown db/d12/d14/de5/da2/dc6/dce/d85/d42/l7f 268 1 2024-03-23T08:24:02.187 INFO:tasks.workunit.client.0.smithi070.stdout:3/777: creat db/d14/d43/d7d/f11f x:0 0 0 2024-03-23T08:24:02.188 INFO:tasks.workunit.client.0.smithi070.stdout:3/778: fsync db/d7f/dcd/f103 0 2024-03-23T08:24:02.189 INFO:tasks.workunit.client.0.smithi070.stdout:8/674: write d5/d8/d16/d32/d84/d98/dde/da3/fb8 [27331,110518] 0 2024-03-23T08:24:02.190 INFO:tasks.workunit.client.0.smithi070.stdout:9/717: mknod db/d12/d14/de5/da2/d4f/dd9/cf7 0 2024-03-23T08:24:02.194 INFO:tasks.workunit.client.0.smithi070.stdout:3/779: rmdir db/d14/d43/d62/d95 39 2024-03-23T08:24:02.194 INFO:tasks.workunit.client.0.smithi070.stdout:7/582: dwrite d25/d27/d6a/fba [0,4194304] 0 2024-03-23T08:24:02.196 INFO:tasks.workunit.client.0.smithi070.stdout:9/718: mkdir db/d12/db7/df8 0 2024-03-23T08:24:02.199 INFO:tasks.workunit.client.0.smithi070.stdout:8/675: dread d5/d8/d16/dd4/d6f/d7c/f91 [0,4194304] 0 2024-03-23T08:24:02.204 INFO:tasks.workunit.client.0.smithi070.stdout:9/719: symlink db/d12/d67/lf9 0 2024-03-23T08:24:02.207 INFO:tasks.workunit.client.0.smithi070.stdout:7/583: fdatasync d25/d27/d6a/d44/f77 0 2024-03-23T08:24:02.208 INFO:tasks.workunit.client.0.smithi070.stdout:7/584: truncate d25/d40/fc9 483647 0 2024-03-23T08:24:02.210 INFO:tasks.workunit.client.0.smithi070.stdout:8/676: mkdir d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/df2 0 2024-03-23T08:24:02.215 INFO:tasks.workunit.client.0.smithi070.stdout:3/780: truncate db/d14/f3f 1104851 0 2024-03-23T08:24:02.216 INFO:tasks.workunit.client.0.smithi070.stdout:3/781: readlink db/d20/d7a/l8b 0 2024-03-23T08:24:02.217 INFO:tasks.workunit.client.0.smithi070.stdout:3/782: stat db/d14/d65 0 2024-03-23T08:24:02.218 INFO:tasks.workunit.client.0.smithi070.stdout:3/783: write db/d4e/df8/d22/db2/fc7 [5085331,29222] 0 2024-03-23T08:24:02.218 INFO:tasks.workunit.client.0.smithi070.stdout:8/677: symlink d5/d8/d16/d30/d38/lf3 0 2024-03-23T08:24:02.219 INFO:tasks.workunit.client.0.smithi070.stdout:7/585: symlink d25/d27/d6a/d44/da4/ld2 0 2024-03-23T08:24:02.220 INFO:tasks.workunit.client.0.smithi070.stdout:8/678: fsync d5/d8/d16/d32/d84/d9f/fe3 0 2024-03-23T08:24:02.222 INFO:tasks.workunit.client.0.smithi070.stdout:9/720: dread db/d12/f3a [0,4194304] 0 2024-03-23T08:24:02.223 INFO:tasks.workunit.client.0.smithi070.stdout:3/784: rename db/d14/d65/daa/dcb/ld3 to db/d14/d9c/l120 0 2024-03-23T08:24:02.223 INFO:tasks.workunit.client.0.smithi070.stdout:9/721: readlink db/d12/d14/de5/da2/l1c 0 2024-03-23T08:24:02.226 INFO:tasks.workunit.client.0.smithi070.stdout:9/722: rmdir db/d12/d14/de5/da2 39 2024-03-23T08:24:02.235 INFO:tasks.workunit.client.0.smithi070.stdout:9/723: mkdir db/d12/d14/de5/da2/d26/dec/dfa 0 2024-03-23T08:24:02.237 INFO:tasks.workunit.client.0.smithi070.stdout:8/679: write d5/d8/d16/d3b/faa [1840659,10391] 0 2024-03-23T08:24:02.240 INFO:tasks.workunit.client.0.smithi070.stdout:9/724: getdents db/d12/d67 0 2024-03-23T08:24:02.242 INFO:tasks.workunit.client.0.smithi070.stdout:7/586: truncate d25/d40/fc9 120283 0 2024-03-23T08:24:02.245 INFO:tasks.workunit.client.0.smithi070.stdout:8/680: dwrite d5/d8/d16/d30/d38/fc2 [0,4194304] 0 2024-03-23T08:24:02.248 INFO:tasks.workunit.client.0.smithi070.stdout:8/681: truncate d5/d8/d16/d32/d84/d98/dde/da3/fb8 554462 0 2024-03-23T08:24:02.250 INFO:tasks.workunit.client.0.smithi070.stdout:9/725: write db/d12/d14/de5/da2/d4f/fa9 [1864718,1338] 0 2024-03-23T08:24:02.250 INFO:tasks.workunit.client.0.smithi070.stdout:8/682: fsync d5/d8/d16/d3b/d8c/f99 0 2024-03-23T08:24:02.251 INFO:tasks.workunit.client.0.smithi070.stdout:8/683: chown d5/d8/d16/d32/d84/fef 15860 1 2024-03-23T08:24:02.258 INFO:tasks.workunit.client.0.smithi070.stdout:8/684: dwrite d5/d8/d9a/da2/fe9 [0,4194304] 0 2024-03-23T08:24:02.262 INFO:tasks.workunit.client.0.smithi070.stdout:8/685: write d5/d8/f17 [7944449,59268] 0 2024-03-23T08:24:02.287 INFO:tasks.workunit.client.0.smithi070.stdout:5/575: sync 2024-03-23T08:24:02.337 INFO:tasks.workunit.client.0.smithi070.stdout:9/726: write db/d12/d14/de5/da2/dc6/d7e/f43 [2934712,95186] 0 2024-03-23T08:24:02.338 INFO:tasks.workunit.client.0.smithi070.stdout:9/727: chown db/d12/d67 633746 1 2024-03-23T08:24:02.339 INFO:tasks.workunit.client.0.smithi070.stdout:9/728: stat db/d12/d14/de5/da2/d26/f30 0 2024-03-23T08:24:02.340 INFO:tasks.workunit.client.0.smithi070.stdout:5/576: symlink dc/d1b/d1d/d25/d38/d39/lb6 0 2024-03-23T08:24:02.341 INFO:tasks.workunit.client.0.smithi070.stdout:9/729: write db/fde [221263,101646] 0 2024-03-23T08:24:02.344 INFO:tasks.workunit.client.0.smithi070.stdout:5/577: mkdir dc/d1b/d1d/d25/d36/d32/db7 0 2024-03-23T08:24:02.346 INFO:tasks.workunit.client.0.smithi070.stdout:8/686: dread d5/d8/d16/d32/f74 [0,4194304] 0 2024-03-23T08:24:02.349 INFO:tasks.workunit.client.0.smithi070.stdout:5/578: unlink dc/d1b/d1d/d25/d36/d32/d5c/f9b 0 2024-03-23T08:24:02.351 INFO:tasks.workunit.client.0.smithi070.stdout:9/730: link db/d12/d14/de5/da2/d4f/db6/cbd db/d12/d14/cfb 0 2024-03-23T08:24:02.362 INFO:tasks.workunit.client.0.smithi070.stdout:8/687: getdents d5/d8 0 2024-03-23T08:24:02.368 INFO:tasks.workunit.client.0.smithi070.stdout:5/579: dread dc/d1b/d1d/f3a [0,4194304] 0 2024-03-23T08:24:02.368 INFO:tasks.workunit.client.0.smithi070.stdout:5/580: chown dc/d1b/d1d/d25/fb4 58 1 2024-03-23T08:24:02.370 INFO:tasks.workunit.client.0.smithi070.stdout:8/688: getdents d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/df2 0 2024-03-23T08:24:02.373 INFO:tasks.workunit.client.0.smithi070.stdout:9/731: write db/d12/d14/f7a [3079370,32866] 0 2024-03-23T08:24:02.374 INFO:tasks.workunit.client.0.smithi070.stdout:8/689: symlink d5/lf4 0 2024-03-23T08:24:02.379 INFO:tasks.workunit.client.0.smithi070.stdout:9/732: rename db/d12/d14/de5/da2/d58/db5/de0 to db/d12/d14/de5/da2/d58/db5/dfc 0 2024-03-23T08:24:02.382 INFO:tasks.workunit.client.0.smithi070.stdout:8/690: dwrite d5/d8/d16/d30/f2f [0,4194304] 0 2024-03-23T08:24:02.385 INFO:tasks.workunit.client.0.smithi070.stdout:5/581: dwrite dc/d1b/d1d/f37 [0,4194304] 0 2024-03-23T08:24:02.389 INFO:tasks.workunit.client.0.smithi070.stdout:5/582: write dc/d1b/d1d/d25/d68/d8a/d99/fb2 [273546,58763] 0 2024-03-23T08:24:02.393 INFO:tasks.workunit.client.0.smithi070.stdout:5/583: rename dc/l24 to dc/d1b/d1d/d25/d38/d53/lb8 0 2024-03-23T08:24:02.399 INFO:tasks.workunit.client.0.smithi070.stdout:8/691: write d5/f7 [3781738,88194] 0 2024-03-23T08:24:02.401 INFO:tasks.workunit.client.0.smithi070.stdout:8/692: write d5/d8/d16/d30/d38/fc2 [5217333,20008] 0 2024-03-23T08:24:02.402 INFO:tasks.workunit.client.0.smithi070.stdout:9/733: dread db/d12/d14/f49 [0,4194304] 0 2024-03-23T08:24:02.406 INFO:tasks.workunit.client.0.smithi070.stdout:5/584: link dc/d1b/c93 dc/d1b/d1d/d25/d68/d8a/dad/cb9 0 2024-03-23T08:24:02.409 INFO:tasks.workunit.client.0.smithi070.stdout:8/693: fsync d5/f50 0 2024-03-23T08:24:02.414 INFO:tasks.workunit.client.0.smithi070.stdout:9/734: symlink db/d12/d14/de5/da2/dc6/dce/lfd 0 2024-03-23T08:24:02.415 INFO:tasks.workunit.client.0.smithi070.stdout:9/735: fdatasync db/d12/d14/de5/da2/d26/f64 0 2024-03-23T08:24:02.418 INFO:tasks.workunit.client.0.smithi070.stdout:8/694: symlink d5/d8/d9a/dbe/lf5 0 2024-03-23T08:24:02.426 INFO:tasks.workunit.client.0.smithi070.stdout:8/695: write d5/d8/d16/f5b [1094153,88010] 0 2024-03-23T08:24:02.429 INFO:tasks.workunit.client.0.smithi070.stdout:9/736: dwrite db/d12/d14/de5/da2/dc6/dce/d85/feb [0,4194304] 0 2024-03-23T08:24:02.430 INFO:tasks.workunit.client.0.smithi070.stdout:8/696: write d5/d8/f11 [1154069,9926] 0 2024-03-23T08:24:02.430 INFO:tasks.workunit.client.0.smithi070.stdout:5/585: dread dc/d1b/d1d/d25/d68/d8a/f8e [0,4194304] 0 2024-03-23T08:24:02.433 INFO:tasks.workunit.client.0.smithi070.stdout:9/737: write db/d12/d14/de5/da2/d26/f30 [4394157,62755] 0 2024-03-23T08:24:02.436 INFO:tasks.workunit.client.0.smithi070.stdout:8/697: rmdir d5/d8/d9a/da2/dae 39 2024-03-23T08:24:02.442 INFO:tasks.workunit.client.0.smithi070.stdout:9/738: write db/d12/d14/de5/da2/dc6/dce/d85/fad [857273,30008] 0 2024-03-23T08:24:02.443 INFO:tasks.workunit.client.0.smithi070.stdout:5/586: getdents dc/d1b/d1d/d25/d36/d32 0 2024-03-23T08:24:02.446 INFO:tasks.workunit.client.0.smithi070.stdout:5/587: creat dc/d1b/d1d/d25/d38/d39/fba x:0 0 0 2024-03-23T08:24:02.446 INFO:tasks.workunit.client.0.smithi070.stdout:1/640: sync 2024-03-23T08:24:02.446 INFO:tasks.workunit.client.0.smithi070.stdout:2/874: sync 2024-03-23T08:24:02.446 INFO:tasks.workunit.client.0.smithi070.stdout:0/827: sync 2024-03-23T08:24:02.446 INFO:tasks.workunit.client.0.smithi070.stdout:6/788: sync 2024-03-23T08:24:02.451 INFO:tasks.workunit.client.0.smithi070.stdout:6/789: write d0/d23/d28/d35/d85/f89 [2975433,34742] 0 2024-03-23T08:24:02.452 INFO:tasks.workunit.client.0.smithi070.stdout:5/588: symlink dc/d1b/d1d/d25/d68/lbb 0 2024-03-23T08:24:02.453 INFO:tasks.workunit.client.0.smithi070.stdout:9/739: dwrite db/d12/d34/d47/fd7 [0,4194304] 0 2024-03-23T08:24:02.454 INFO:tasks.workunit.client.0.smithi070.stdout:2/875: symlink d1/d7/d48/d69/db8/dc7/l126 0 2024-03-23T08:24:02.455 INFO:tasks.workunit.client.0.smithi070.stdout:2/876: stat d1/d7/d39/d47/d7b/cc5 0 2024-03-23T08:24:02.459 INFO:tasks.workunit.client.0.smithi070.stdout:1/641: truncate d8/db/d2c/d3f/f5f 498327 0 2024-03-23T08:24:02.459 INFO:tasks.workunit.client.0.smithi070.stdout:9/740: dread db/d12/d14/de5/da2/d4f/d5a/fa5 [0,4194304] 0 2024-03-23T08:24:02.465 INFO:tasks.workunit.client.0.smithi070.stdout:2/877: creat d1/d98/f127 x:0 0 0 2024-03-23T08:24:02.465 INFO:tasks.workunit.client.0.smithi070.stdout:5/589: dwrite dc/d1b/d1d/d25/d38/f40 [0,4194304] 0 2024-03-23T08:24:02.468 INFO:tasks.workunit.client.0.smithi070.stdout:0/828: getdents d16/d26/d27/d3d/d4d 0 2024-03-23T08:24:02.469 INFO:tasks.workunit.client.0.smithi070.stdout:2/878: symlink d1/d7/df/d99/l128 0 2024-03-23T08:24:02.472 INFO:tasks.workunit.client.0.smithi070.stdout:2/879: readlink d1/d7/df/d99/l128 0 2024-03-23T08:24:02.478 INFO:tasks.workunit.client.0.smithi070.stdout:5/590: dwrite dc/d1b/d1d/d25/d38/d53/f60 [0,4194304] 0 2024-03-23T08:24:02.479 INFO:tasks.workunit.client.0.smithi070.stdout:1/642: rename d8/d11/cd0 to d8/cef 0 2024-03-23T08:24:02.480 INFO:tasks.workunit.client.0.smithi070.stdout:1/643: write d8/d11/d61/f75 [108223,9297] 0 2024-03-23T08:24:02.480 INFO:tasks.workunit.client.0.smithi070.stdout:0/829: dread - d16/d26/d27/d4c/d60/f105 zero size 2024-03-23T08:24:02.486 INFO:tasks.workunit.client.0.smithi070.stdout:1/644: creat d8/d11/d61/ff0 x:0 0 0 2024-03-23T08:24:02.488 INFO:tasks.workunit.client.0.smithi070.stdout:6/790: dread d0/d23/d28/f36 [0,4194304] 0 2024-03-23T08:24:02.489 INFO:tasks.workunit.client.0.smithi070.stdout:0/830: chown d16/d22/c44 0 1 2024-03-23T08:24:02.489 INFO:tasks.workunit.client.0.smithi070.stdout:9/741: dwrite db/d12/d14/de5/da2/dc6/dce/f93 [0,4194304] 0 2024-03-23T08:24:02.491 INFO:tasks.workunit.client.0.smithi070.stdout:1/645: rmdir d8/d1a/d8f 39 2024-03-23T08:24:02.492 INFO:tasks.workunit.client.0.smithi070.stdout:6/791: write d0/d23/d28/d35/f46 [6808479,75003] 0 2024-03-23T08:24:02.492 INFO:tasks.workunit.client.0.smithi070.stdout:1/646: chown d8/d11 3 1 2024-03-23T08:24:02.496 INFO:tasks.workunit.client.0.smithi070.stdout:2/880: dread d1/d7/df/d94/d38/d4c/f6a [0,4194304] 0 2024-03-23T08:24:02.500 INFO:tasks.workunit.client.0.smithi070.stdout:0/831: dread d16/d26/d27/d4c/d60/d7c/f83 [0,4194304] 0 2024-03-23T08:24:02.514 INFO:tasks.workunit.client.0.smithi070.stdout:5/591: dread dc/d1b/d1d/d25/d36/d32/d5c/f65 [0,4194304] 0 2024-03-23T08:24:02.522 INFO:tasks.workunit.client.0.smithi070.stdout:6/792: dread d0/f50 [0,4194304] 0 2024-03-23T08:24:02.524 INFO:tasks.workunit.client.0.smithi070.stdout:5/592: dwrite dc/f9f [0,4194304] 0 2024-03-23T08:24:02.564 INFO:tasks.workunit.client.0.smithi070.stdout:1/647: symlink d8/db/d40/d43/d64/d85/d97/d9a/lf1 0 2024-03-23T08:24:02.574 INFO:tasks.workunit.client.0.smithi070.stdout:5/593: mkdir dc/d1b/d1d/d25/d38/d39/dbc 0 2024-03-23T08:24:02.581 INFO:tasks.workunit.client.0.smithi070.stdout:5/594: dwrite dc/d1b/d1d/d25/d36/d32/d5c/f65 [4194304,4194304] 0 2024-03-23T08:24:02.582 INFO:tasks.workunit.client.0.smithi070.stdout:5/595: chown dc/d1b 16 1 2024-03-23T08:24:02.606 INFO:tasks.workunit.client.0.smithi070.stdout:5/596: dread dc/d1b/d1d/d25/d38/d48/f85 [0,4194304] 0 2024-03-23T08:24:02.638 INFO:tasks.workunit.client.0.smithi070.stdout:0/832: creat d16/d26/d27/d86/d9e/f11f x:0 0 0 2024-03-23T08:24:02.638 INFO:tasks.workunit.client.0.smithi070.stdout:5/597: dread dc/d1b/f54 [0,4194304] 0 2024-03-23T08:24:02.638 INFO:tasks.workunit.client.0.smithi070.stdout:5/598: readlink dc/d1b/d1d/d25/l35 0 2024-03-23T08:24:02.638 INFO:tasks.workunit.client.0.smithi070.stdout:5/599: truncate dc/d1b/d1d/d25/d38/d53/f76 4914257 0 2024-03-23T08:24:02.638 INFO:tasks.workunit.client.0.smithi070.stdout:0/833: mknod d16/d26/d27/d3d/d4d/d9a/df8/d11a/c120 0 2024-03-23T08:24:02.638 INFO:tasks.workunit.client.0.smithi070.stdout:0/834: mknod d16/d26/d27/d4c/d60/c121 0 2024-03-23T08:24:02.638 INFO:tasks.workunit.client.0.smithi070.stdout:5/600: dwrite dc/d1b/d1d/d25/d38/d39/f6b [0,4194304] 0 2024-03-23T08:24:02.638 INFO:tasks.workunit.client.0.smithi070.stdout:1/648: mknod d8/d26/d49/cf2 0 2024-03-23T08:24:02.639 INFO:tasks.workunit.client.0.smithi070.stdout:0/835: symlink d16/d22/df0/df5/l122 0 2024-03-23T08:24:02.639 INFO:tasks.workunit.client.0.smithi070.stdout:5/601: chown dc/d1b/d1d/d61/f94 868360770 1 2024-03-23T08:24:02.639 INFO:tasks.workunit.client.0.smithi070.stdout:0/836: mknod d16/d26/d27/d86/def/c123 0 2024-03-23T08:24:02.639 INFO:tasks.workunit.client.0.smithi070.stdout:0/837: fdatasync d16/d22/d43/f5a 0 2024-03-23T08:24:02.646 INFO:tasks.workunit.client.0.smithi070.stdout:2/881: write d1/d29/d52/f92 [284951,27664] 0 2024-03-23T08:24:02.649 INFO:tasks.workunit.client.0.smithi070.stdout:2/882: creat d1/d7/d48/d5b/f129 x:0 0 0 2024-03-23T08:24:02.659 INFO:tasks.workunit.client.0.smithi070.stdout:4/615: sync 2024-03-23T08:24:02.670 INFO:tasks.workunit.client.0.smithi070.stdout:4/616: dread d3/d6/f6c [0,4194304] 0 2024-03-23T08:24:02.677 INFO:tasks.workunit.client.0.smithi070.stdout:2/883: rename d1/lae to d1/d118/d34/l12a 0 2024-03-23T08:24:02.679 INFO:tasks.workunit.client.0.smithi070.stdout:2/884: symlink d1/d7/df/l12b 0 2024-03-23T08:24:02.681 INFO:tasks.workunit.client.0.smithi070.stdout:2/885: unlink d1/d7/df/d94/f25 0 2024-03-23T08:24:02.685 INFO:tasks.workunit.client.0.smithi070.stdout:6/793: creat d0/d23/d74/d84/db3/ffd x:0 0 0 2024-03-23T08:24:02.686 INFO:tasks.workunit.client.0.smithi070.stdout:6/794: symlink d0/d3/lfe 0 2024-03-23T08:24:02.687 INFO:tasks.workunit.client.0.smithi070.stdout:6/795: chown d0/d3/d2c/ce2 2 1 2024-03-23T08:24:02.687 INFO:tasks.workunit.client.0.smithi070.stdout:4/617: creat d3/d6/d35/d65/d27/d4b/fe0 x:0 0 0 2024-03-23T08:24:02.690 INFO:tasks.workunit.client.0.smithi070.stdout:5/602: truncate dc/d1b/d1d/d25/d38/f73 2434233 0 2024-03-23T08:24:02.690 INFO:tasks.workunit.client.0.smithi070.stdout:6/796: truncate d0/d23/d4d/f63 645461 0 2024-03-23T08:24:02.715 INFO:tasks.workunit.client.0.smithi070.stdout:0/838: rmdir d16/d22/d43/d47 39 2024-03-23T08:24:02.715 INFO:tasks.workunit.client.0.smithi070.stdout:5/603: chown dc/d1b/d1d/d25/d38/d39/lb3 352 1 2024-03-23T08:24:02.715 INFO:tasks.workunit.client.0.smithi070.stdout:2/886: write d1/f9e [590850,5769] 0 2024-03-23T08:24:02.722 INFO:tasks.workunit.client.0.smithi070.stdout:0/839: fdatasync d16/d26/d27/f91 0 2024-03-23T08:24:02.729 INFO:tasks.workunit.client.0.smithi070.stdout:4/618: getdents d3/d6/d35/dcf/db5/dda/db8/d88 0 2024-03-23T08:24:02.732 INFO:tasks.workunit.client.0.smithi070.stdout:6/797: dwrite d0/d23/d74/d84/d3c/f51 [0,4194304] 0 2024-03-23T08:24:02.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:02 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:24:02.734 INFO:tasks.workunit.client.0.smithi070.stdout:2/887: write d1/d29/d52/f9f [3208226,24899] 0 2024-03-23T08:24:02.735 INFO:tasks.workunit.client.0.smithi070.stdout:5/604: rename dc/f9f to dc/d1b/d1d/d61/fbd 0 2024-03-23T08:24:02.737 INFO:tasks.workunit.client.0.smithi070.stdout:0/840: mknod d16/d26/d27/d4c/d60/d7c/daf/c124 0 2024-03-23T08:24:02.742 INFO:tasks.workunit.client.0.smithi070.stdout:2/888: mkdir d1/d7/d39/d47/dda/d12c 0 2024-03-23T08:24:02.752 INFO:tasks.workunit.client.0.smithi070.stdout:4/619: mkdir d3/d6/d35/d65/d27/de1 0 2024-03-23T08:24:02.757 INFO:tasks.workunit.client.0.smithi070.stdout:5/605: mknod dc/d1b/d1d/cbe 0 2024-03-23T08:24:02.761 INFO:tasks.workunit.client.0.smithi070.stdout:6/798: rename d0/d3/d2c/f49 to d0/d23/d74/d84/db3/fff 0 2024-03-23T08:24:02.763 INFO:tasks.workunit.client.0.smithi070.stdout:0/841: link d16/d26/d27/d3d/d4d/d9a/df8/c10b d16/d26/d27/d3d/d4d/dc9/d102/c125 0 2024-03-23T08:24:02.769 INFO:tasks.workunit.client.0.smithi070.stdout:4/620: dread d3/d6/f7 [0,4194304] 0 2024-03-23T08:24:02.773 INFO:tasks.workunit.client.0.smithi070.stdout:4/621: mkdir d3/d6/d35/d65/d1b/de2 0 2024-03-23T08:24:02.774 INFO:tasks.workunit.client.0.smithi070.stdout:4/622: read - d3/d6/d35/d65/d27/d4b/fba zero size 2024-03-23T08:24:02.774 INFO:tasks.workunit.client.0.smithi070.stdout:4/623: stat d3/fc 0 2024-03-23T08:24:02.775 INFO:tasks.workunit.client.0.smithi070.stdout:4/624: dread - d3/d6/d35/dcf/db5/dda/db8/d88/fa6 zero size 2024-03-23T08:24:02.778 INFO:tasks.workunit.client.0.smithi070.stdout:0/842: dread d16/d22/fa6 [0,4194304] 0 2024-03-23T08:24:02.781 INFO:tasks.workunit.client.0.smithi070.stdout:0/843: truncate d16/d26/d27/d86/d67/d7d/f99 1008130 0 2024-03-23T08:24:02.785 INFO:tasks.workunit.client.0.smithi070.stdout:0/844: creat d16/d107/f126 x:0 0 0 2024-03-23T08:24:02.786 INFO:tasks.workunit.client.0.smithi070.stdout:4/625: dread d3/f6d [0,4194304] 0 2024-03-23T08:24:02.795 INFO:tasks.workunit.client.0.smithi070.stdout:4/626: dwrite d3/d6/d35/d65/d27/d4b/fe0 [0,4194304] 0 2024-03-23T08:24:02.800 INFO:tasks.workunit.client.0.smithi070.stdout:4/627: creat d3/d6/d35/d3f/d51/d6e/d97/fe3 x:0 0 0 2024-03-23T08:24:02.803 INFO:tasks.workunit.client.0.smithi070.stdout:4/628: mkdir d3/d6/d35/d65/d27/de4 0 2024-03-23T08:24:02.816 INFO:tasks.workunit.client.0.smithi070.stdout:4/629: dread d3/d6/d35/d65/d41/fb9 [0,4194304] 0 2024-03-23T08:24:02.817 INFO:tasks.workunit.client.0.smithi070.stdout:4/630: fsync d3/d6/d11/fcc 0 2024-03-23T08:24:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:02 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:24:02.826 INFO:tasks.workunit.client.0.smithi070.stdout:0/845: dread d16/d26/d27/f35 [0,4194304] 0 2024-03-23T08:24:02.855 INFO:tasks.workunit.client.0.smithi070.stdout:5/606: dwrite dc/d1b/d1d/f44 [0,4194304] 0 2024-03-23T08:24:02.856 INFO:tasks.workunit.client.0.smithi070.stdout:6/799: dwrite d0/d23/d74/d84/db3/d2f/dab/fac [0,4194304] 0 2024-03-23T08:24:02.864 INFO:tasks.workunit.client.0.smithi070.stdout:6/800: creat d0/d23/d88/db6/f100 x:0 0 0 2024-03-23T08:24:02.865 INFO:tasks.workunit.client.0.smithi070.stdout:5/607: mknod dc/d1b/d1d/d25/d38/d48/cbf 0 2024-03-23T08:24:02.870 INFO:tasks.workunit.client.0.smithi070.stdout:5/608: fdatasync dc/d1b/d1d/d25/d38/d39/f4f 0 2024-03-23T08:24:02.872 INFO:tasks.workunit.client.0.smithi070.stdout:6/801: getdents d0/d23/d74/d84/d3c/d86 0 2024-03-23T08:24:02.873 INFO:tasks.workunit.client.0.smithi070.stdout:5/609: mkdir dc/d1b/d1d/d25/d36/d32/d5c/dc0 0 2024-03-23T08:24:02.874 INFO:tasks.workunit.client.0.smithi070.stdout:5/610: stat dc/d1b/d1d/d25/d36/l31 0 2024-03-23T08:24:02.874 INFO:tasks.workunit.client.0.smithi070.stdout:5/611: readlink dc/l6c 0 2024-03-23T08:24:02.884 INFO:tasks.workunit.client.0.smithi070.stdout:4/631: truncate d3/d6/d35/d65/d27/d4b/fe0 2763493 0 2024-03-23T08:24:02.890 INFO:tasks.workunit.client.0.smithi070.stdout:6/802: write d0/d23/d74/d84/db3/fff [1755902,38611] 0 2024-03-23T08:24:02.897 INFO:tasks.workunit.client.0.smithi070.stdout:0/846: dwrite d16/d22/d43/d47/da9/fe6 [0,4194304] 0 2024-03-23T08:24:02.898 INFO:tasks.workunit.client.0.smithi070.stdout:5/612: dwrite f8 [0,4194304] 0 2024-03-23T08:24:02.922 INFO:tasks.workunit.client.0.smithi070.stdout:6/803: unlink d0/d23/d74/d84/laf 0 2024-03-23T08:24:02.926 INFO:tasks.workunit.client.0.smithi070.stdout:5/613: symlink dc/d1b/d1d/d25/d38/lc1 0 2024-03-23T08:24:02.929 INFO:tasks.workunit.client.0.smithi070.stdout:4/632: mknod d3/d6/d35/d65/d27/d4b/dc0/ce5 0 2024-03-23T08:24:02.929 INFO:tasks.workunit.client.0.smithi070.stdout:6/804: dread d0/d23/d28/f7d [0,4194304] 0 2024-03-23T08:24:02.937 INFO:tasks.workunit.client.0.smithi070.stdout:4/633: dwrite d3/d6/d11/fcc [0,4194304] 0 2024-03-23T08:24:02.939 INFO:tasks.workunit.client.0.smithi070.stdout:4/634: write d3/d6/d35/d65/d27/d4b/d73/fa4 [3068883,59381] 0 2024-03-23T08:24:02.942 INFO:tasks.workunit.client.0.smithi070.stdout:5/614: dread dc/d1b/d1d/d25/d38/d48/d92/f98 [0,4194304] 0 2024-03-23T08:24:02.942 INFO:tasks.workunit.client.0.smithi070.stdout:4/635: mkdir d3/d6/d9/de6 0 2024-03-23T08:24:02.944 INFO:tasks.workunit.client.0.smithi070.stdout:5/615: chown dc/d1b/d1d/d25/d36/d32/c81 0 1 2024-03-23T08:24:02.948 INFO:tasks.workunit.client.0.smithi070.stdout:4/636: rename d3/d6/d35/d3f/d51/d6e/d97/ddf/fc6 to d3/d6/d35/dcf/fe7 0 2024-03-23T08:24:02.956 INFO:tasks.workunit.client.0.smithi070.stdout:5/616: rename dc/d1b/d1d/d25/d38/d48/d92/f98 to dc/d1b/d1d/d25/fc2 0 2024-03-23T08:24:02.959 INFO:tasks.workunit.client.0.smithi070.stdout:7/587: sync 2024-03-23T08:24:02.960 INFO:tasks.workunit.client.0.smithi070.stdout:3/785: sync 2024-03-23T08:24:02.960 INFO:tasks.workunit.client.0.smithi070.stdout:8/698: sync 2024-03-23T08:24:02.960 INFO:tasks.workunit.client.0.smithi070.stdout:4/637: getdents d3/d6/d11 0 2024-03-23T08:24:02.964 INFO:tasks.workunit.client.0.smithi070.stdout:0/847: dwrite d16/d26/d27/d86/d67/d7d/f99 [0,4194304] 0 2024-03-23T08:24:02.964 INFO:tasks.workunit.client.0.smithi070.stdout:4/638: dread - d3/d6/d35/d3f/da9/fc1 zero size 2024-03-23T08:24:02.965 INFO:tasks.workunit.client.0.smithi070.stdout:5/617: symlink dc/d1b/d1d/d61/lc3 0 2024-03-23T08:24:02.966 INFO:tasks.workunit.client.0.smithi070.stdout:6/805: dwrite d0/d3/fc8 [0,4194304] 0 2024-03-23T08:24:02.974 INFO:tasks.workunit.client.0.smithi070.stdout:5/618: truncate dc/d1b/d1d/d25/d36/d32/f87 613014 0 2024-03-23T08:24:02.975 INFO:tasks.workunit.client.0.smithi070.stdout:7/588: fsync d25/d27/d6a/d44/f79 0 2024-03-23T08:24:02.977 INFO:tasks.workunit.client.0.smithi070.stdout:7/589: chown d25/d27/d6a/l3d 12532 1 2024-03-23T08:24:02.977 INFO:tasks.workunit.client.0.smithi070.stdout:3/786: creat db/d4e/dd6/f121 x:0 0 0 2024-03-23T08:24:02.978 INFO:tasks.workunit.client.0.smithi070.stdout:8/699: creat d5/d8/d9a/da2/dae/ff6 x:0 0 0 2024-03-23T08:24:02.979 INFO:tasks.workunit.client.0.smithi070.stdout:0/848: truncate d16/d26/d27/d4c/d60/f69 217139 0 2024-03-23T08:24:02.982 INFO:tasks.workunit.client.0.smithi070.stdout:4/639: dwrite d3/d6/d35/dcf/db5/dda/db8/d88/fdd [0,4194304] 0 2024-03-23T08:24:02.986 INFO:tasks.workunit.client.0.smithi070.stdout:6/806: symlink d0/d23/d74/d84/d3c/l101 0 2024-03-23T08:24:02.988 INFO:tasks.workunit.client.0.smithi070.stdout:6/807: read d0/d23/d4d/da5/fcf [3155283,32540] 0 2024-03-23T08:24:02.989 INFO:tasks.workunit.client.0.smithi070.stdout:6/808: readlink d0/d23/d74/d84/db3/d38/l4e 0 2024-03-23T08:24:02.992 INFO:tasks.workunit.client.0.smithi070.stdout:3/787: creat db/d4e/dd6/dd7/f122 x:0 0 0 2024-03-23T08:24:02.992 INFO:tasks.workunit.client.0.smithi070.stdout:8/700: creat d5/d8/ff7 x:0 0 0 2024-03-23T08:24:02.995 INFO:tasks.workunit.client.0.smithi070.stdout:5/619: fdatasync dc/d1b/d1d/d25/d38/d39/f79 0 2024-03-23T08:24:02.998 INFO:tasks.workunit.client.0.smithi070.stdout:4/640: symlink d3/d6/d35/d65/d27/d4b/le8 0 2024-03-23T08:24:03.003 INFO:tasks.workunit.client.0.smithi070.stdout:7/590: write d25/f38 [9402846,62328] 0 2024-03-23T08:24:03.003 INFO:tasks.workunit.client.0.smithi070.stdout:6/809: mkdir d0/d23/d74/d84/db3/d2f/dab/d102 0 2024-03-23T08:24:03.004 INFO:tasks.workunit.client.0.smithi070.stdout:6/810: chown d0/d23/d74 5650 1 2024-03-23T08:24:03.010 INFO:tasks.workunit.client.0.smithi070.stdout:9/742: sync 2024-03-23T08:24:03.010 INFO:tasks.workunit.client.0.smithi070.stdout:1/649: sync 2024-03-23T08:24:03.010 INFO:tasks.workunit.client.0.smithi070.stdout:2/889: sync 2024-03-23T08:24:03.013 INFO:tasks.workunit.client.0.smithi070.stdout:0/849: mknod d16/d26/d27/c127 0 2024-03-23T08:24:03.013 INFO:tasks.workunit.client.0.smithi070.stdout:6/811: dwrite d0/d23/d74/d84/db3/d38/d9a/ff8 [0,4194304] 0 2024-03-23T08:24:03.013 INFO:tasks.workunit.client.0.smithi070.stdout:0/850: stat f13 0 2024-03-23T08:24:03.015 INFO:tasks.workunit.client.0.smithi070.stdout:6/812: chown d0/d23/d88/cb2 875 1 2024-03-23T08:24:03.015 INFO:tasks.workunit.client.0.smithi070.stdout:0/851: read d16/d26/d27/d86/d67/d7d/f99 [3426517,46763] 0 2024-03-23T08:24:03.017 INFO:tasks.workunit.client.0.smithi070.stdout:6/813: readlink d0/d23/d28/d35/lb9 0 2024-03-23T08:24:03.020 INFO:tasks.workunit.client.0.smithi070.stdout:6/814: write d0/f20 [1325477,51783] 0 2024-03-23T08:24:03.024 INFO:tasks.workunit.client.0.smithi070.stdout:3/788: mkdir db/d14/d43/d62/d95/d123 0 2024-03-23T08:24:03.027 INFO:tasks.workunit.client.0.smithi070.stdout:3/789: write db/d14/d43/d7d/f11f [581028,104755] 0 2024-03-23T08:24:03.029 INFO:tasks.workunit.client.0.smithi070.stdout:4/641: dwrite d3/d6/d35/f76 [0,4194304] 0 2024-03-23T08:24:03.031 INFO:tasks.workunit.client.0.smithi070.stdout:2/890: unlink d1/d118/d34/f3d 0 2024-03-23T08:24:03.032 INFO:tasks.workunit.client.0.smithi070.stdout:1/650: mkdir d8/db/d2c/d3f/d48/dab/df3 0 2024-03-23T08:24:03.035 INFO:tasks.workunit.client.0.smithi070.stdout:2/891: write d1/d7/df/d99/dea/dc3/fd9 [2957718,35934] 0 2024-03-23T08:24:03.038 INFO:tasks.workunit.client.0.smithi070.stdout:7/591: dwrite d25/d40/f76 [0,4194304] 0 2024-03-23T08:24:03.040 INFO:tasks.workunit.client.0.smithi070.stdout:5/620: creat dc/d1b/d1d/d25/d36/d32/db7/fc4 x:0 0 0 2024-03-23T08:24:03.045 INFO:tasks.workunit.client.0.smithi070.stdout:6/815: truncate d0/fa7 706486 0 2024-03-23T08:24:03.047 INFO:tasks.workunit.client.0.smithi070.stdout:3/790: symlink db/d14/d9c/l124 0 2024-03-23T08:24:03.050 INFO:tasks.workunit.client.0.smithi070.stdout:9/743: dwrite db/d12/f89 [0,4194304] 0 2024-03-23T08:24:03.051 INFO:tasks.workunit.client.0.smithi070.stdout:2/892: dread d1/d7/df/d99/dea/fc4 [0,4194304] 0 2024-03-23T08:24:03.054 INFO:tasks.workunit.client.0.smithi070.stdout:7/592: rmdir d25 39 2024-03-23T08:24:03.054 INFO:tasks.workunit.client.0.smithi070.stdout:9/744: chown db/d12/d67/fc1 336 1 2024-03-23T08:24:03.056 INFO:tasks.workunit.client.0.smithi070.stdout:5/621: dwrite f8 [0,4194304] 0 2024-03-23T08:24:03.057 INFO:tasks.workunit.client.0.smithi070.stdout:0/852: dread d16/d26/d27/d86/db6/fce [0,4194304] 0 2024-03-23T08:24:03.062 INFO:tasks.workunit.client.0.smithi070.stdout:8/701: write d5/d8/f73 [674281,106386] 0 2024-03-23T08:24:03.066 INFO:tasks.workunit.client.0.smithi070.stdout:1/651: write d8/db/f45 [3890531,83641] 0 2024-03-23T08:24:03.072 INFO:tasks.workunit.client.0.smithi070.stdout:2/893: write d1/d7/df/d94/d38/f117 [677582,122077] 0 2024-03-23T08:24:03.073 INFO:tasks.workunit.client.0.smithi070.stdout:2/894: chown d1/d21 79 1 2024-03-23T08:24:03.076 INFO:tasks.workunit.client.0.smithi070.stdout:9/745: creat db/d12/d14/de5/da2/d4f/d5a/ffe x:0 0 0 2024-03-23T08:24:03.076 INFO:tasks.workunit.client.0.smithi070.stdout:5/622: symlink dc/d1b/d1d/d25/d68/d8a/d99/lc5 0 2024-03-23T08:24:03.077 INFO:tasks.workunit.client.0.smithi070.stdout:1/652: dwrite d8/d1a/fe3 [0,4194304] 0 2024-03-23T08:24:03.079 INFO:tasks.workunit.client.0.smithi070.stdout:2/895: write d1/d7/d39/d47/d7d/fb4 [4036665,101704] 0 2024-03-23T08:24:03.079 INFO:tasks.workunit.client.0.smithi070.stdout:0/853: mknod d16/d26/d27/d86/def/c128 0 2024-03-23T08:24:03.083 INFO:tasks.workunit.client.0.smithi070.stdout:9/746: write db/d12/d14/f63 [4940808,77394] 0 2024-03-23T08:24:03.084 INFO:tasks.workunit.client.0.smithi070.stdout:8/702: symlink d5/d8/d16/d32/d84/d98/lf8 0 2024-03-23T08:24:03.084 INFO:tasks.workunit.client.0.smithi070.stdout:5/623: stat dc/d1b/d1d/d25/d36/d32/d5c 0 2024-03-23T08:24:03.085 INFO:tasks.workunit.client.0.smithi070.stdout:7/593: symlink d25/d27/d6a/d44/da2/db5/ld3 0 2024-03-23T08:24:03.087 INFO:tasks.workunit.client.0.smithi070.stdout:4/642: dwrite d3/d6/d35/d65/d1b/f90 [0,4194304] 0 2024-03-23T08:24:03.091 INFO:tasks.workunit.client.0.smithi070.stdout:7/594: creat d25/d27/d6a/d44/d85/d95/da0/fd4 x:0 0 0 2024-03-23T08:24:03.094 INFO:tasks.workunit.client.0.smithi070.stdout:5/624: rename dc/d1b/d1d/d25/d36/d32 to dc/d1b/d1d/d25/d36/d32/dc6 22 2024-03-23T08:24:03.094 INFO:tasks.workunit.client.0.smithi070.stdout:7/595: chown d25/d27/d6a/d44/d85/d95/da0/fd4 1651408 1 2024-03-23T08:24:03.095 INFO:tasks.workunit.client.0.smithi070.stdout:6/816: link d0/d23/d74/d84/cd7 d0/d23/d74/d84/d3c/c103 0 2024-03-23T08:24:03.099 INFO:tasks.workunit.client.0.smithi070.stdout:0/854: creat d16/d26/d27/d3d/d4d/d9a/df8/f129 x:0 0 0 2024-03-23T08:24:03.101 INFO:tasks.workunit.client.0.smithi070.stdout:4/643: chown d3/d6/d35/d65/c29 61245966 1 2024-03-23T08:24:03.102 INFO:tasks.workunit.client.0.smithi070.stdout:8/703: creat d5/d8/d16/d32/d84/d98/dde/da3/dab/ff9 x:0 0 0 2024-03-23T08:24:03.103 INFO:tasks.workunit.client.0.smithi070.stdout:9/747: rmdir db/d12/db7/df8 0 2024-03-23T08:24:03.106 INFO:tasks.workunit.client.0.smithi070.stdout:3/791: dwrite db/d14/f3f [0,4194304] 0 2024-03-23T08:24:03.106 INFO:tasks.workunit.client.0.smithi070.stdout:4/644: write d3/d6/d35/dcf/db5/dda/db8/d88/fa6 [265058,99671] 0 2024-03-23T08:24:03.108 INFO:tasks.workunit.client.0.smithi070.stdout:7/596: symlink d25/d27/d6a/d44/d85/d95/ld5 0 2024-03-23T08:24:03.116 INFO:tasks.workunit.client.0.smithi070.stdout:7/597: dread d25/d40/f8f [0,4194304] 0 2024-03-23T08:24:03.117 INFO:tasks.workunit.client.0.smithi070.stdout:8/704: fsync d5/d8/d9a/da2/dae/ff6 0 2024-03-23T08:24:03.119 INFO:tasks.workunit.client.0.smithi070.stdout:5/625: link dc/d1b/d1d/f44 dc/d1b/d1d/d25/d68/d8a/dad/fc7 0 2024-03-23T08:24:03.123 INFO:tasks.workunit.client.0.smithi070.stdout:1/653: dwrite d8/d26/f38 [0,4194304] 0 2024-03-23T08:24:03.124 INFO:tasks.workunit.client.0.smithi070.stdout:2/896: dread d1/d7/d39/d47/dda/faa [4194304,4194304] 0 2024-03-23T08:24:03.129 INFO:tasks.workunit.client.0.smithi070.stdout:6/817: creat d0/d23/d74/d84/db3/d2f/dab/d102/f104 x:0 0 0 2024-03-23T08:24:03.130 INFO:tasks.workunit.client.0.smithi070.stdout:5/626: truncate dc/d1b/d1d/d25/d38/d53/f9a 972057 0 2024-03-23T08:24:03.130 INFO:tasks.workunit.client.0.smithi070.stdout:9/748: dwrite db/db8/ff1 [0,4194304] 0 2024-03-23T08:24:03.132 INFO:tasks.workunit.client.0.smithi070.stdout:3/792: dread db/d4e/df8/f3e [0,4194304] 0 2024-03-23T08:24:03.133 INFO:tasks.workunit.client.0.smithi070.stdout:6/818: stat d0/d23/d74/d84/d3c/c59 0 2024-03-23T08:24:03.134 INFO:tasks.workunit.client.0.smithi070.stdout:4/645: rename d3/d6/d35/dcf/db5/f8d to d3/d6/d35/d3f/da9/fe9 0 2024-03-23T08:24:03.135 INFO:tasks.workunit.client.0.smithi070.stdout:6/819: stat d0/d23/d74/d84/d3c/l101 0 2024-03-23T08:24:03.139 INFO:tasks.workunit.client.0.smithi070.stdout:4/646: chown d3/d6/d35/d3f/f87 11 1 2024-03-23T08:24:03.140 INFO:tasks.workunit.client.0.smithi070.stdout:0/855: link l0 d16/d26/d27/d86/d9e/ddd/l12a 0 2024-03-23T08:24:03.143 INFO:tasks.workunit.client.0.smithi070.stdout:5/627: rename dc/d1b/d1d/d25/d38/d48/d92 to dc/d1b/d1d/d25/d38/d48/dc8 0 2024-03-23T08:24:03.150 INFO:tasks.workunit.client.0.smithi070.stdout:7/598: rename d25/d27/c3f to d25/d27/d6a/d44/d6f/da3/cd6 0 2024-03-23T08:24:03.152 INFO:tasks.workunit.client.0.smithi070.stdout:9/749: creat db/d12/d14/de5/da2/d58/fff x:0 0 0 2024-03-23T08:24:03.153 INFO:tasks.workunit.client.0.smithi070.stdout:2/897: mknod d1/d118/c12d 0 2024-03-23T08:24:03.153 INFO:tasks.workunit.client.0.smithi070.stdout:6/820: dwrite d0/d23/d28/d35/d85/f89 [0,4194304] 0 2024-03-23T08:24:03.157 INFO:tasks.workunit.client.0.smithi070.stdout:8/705: dread d5/d8/d16/d30/f61 [0,4194304] 0 2024-03-23T08:24:03.158 INFO:tasks.workunit.client.0.smithi070.stdout:1/654: write d8/db/f3e [760595,103747] 0 2024-03-23T08:24:03.161 INFO:tasks.workunit.client.0.smithi070.stdout:6/821: write d0/d23/d88/db6/f100 [890261,109170] 0 2024-03-23T08:24:03.161 INFO:tasks.workunit.client.0.smithi070.stdout:2/898: read - d1/d29/d52/fdb zero size 2024-03-23T08:24:03.163 INFO:tasks.workunit.client.0.smithi070.stdout:3/793: rename db/d4e/d92/f74 to db/d20/d49/f125 0 2024-03-23T08:24:03.163 INFO:tasks.workunit.client.0.smithi070.stdout:6/822: dread - d0/d23/d28/d35/d85/fe6 zero size 2024-03-23T08:24:03.166 INFO:tasks.workunit.client.0.smithi070.stdout:7/599: symlink d25/d27/d6a/d44/d7c/db4/ld7 0 2024-03-23T08:24:03.170 INFO:tasks.workunit.client.0.smithi070.stdout:9/750: truncate db/d12/d14/de5/da2/d4f/d5a/f71 711422 0 2024-03-23T08:24:03.170 INFO:tasks.workunit.client.0.smithi070.stdout:1/655: chown d8/d11/d61/f78 965481 1 2024-03-23T08:24:03.172 INFO:tasks.workunit.client.0.smithi070.stdout:9/751: fsync db/d12/d14/de5/da2/d26/f64 0 2024-03-23T08:24:03.173 INFO:tasks.workunit.client.0.smithi070.stdout:9/752: chown db/d12/d14/de5/da2/l81 7598047 1 2024-03-23T08:24:03.174 INFO:tasks.workunit.client.0.smithi070.stdout:5/628: dread dc/d1b/d1d/d25/d38/d39/f66 [0,4194304] 0 2024-03-23T08:24:03.178 INFO:tasks.workunit.client.0.smithi070.stdout:4/647: rename d3/d6/d35/d65/d27/d4b/d73/fa4 to d3/d6/d6a/d7c/fea 0 2024-03-23T08:24:03.179 INFO:tasks.workunit.client.0.smithi070.stdout:6/823: creat d0/d23/d74/d84/d3c/d86/f105 x:0 0 0 2024-03-23T08:24:03.180 INFO:tasks.workunit.client.0.smithi070.stdout:4/648: stat d3/d6/d35/d65/l8e 0 2024-03-23T08:24:03.182 INFO:tasks.workunit.client.0.smithi070.stdout:4/649: readlink d3/d6/d35/d65/d6f/l9f 0 2024-03-23T08:24:03.190 INFO:tasks.workunit.client.0.smithi070.stdout:9/753: truncate db/d12/f2b 3004670 0 2024-03-23T08:24:03.191 INFO:tasks.workunit.client.0.smithi070.stdout:3/794: dread db/d4e/f73 [4194304,4194304] 0 2024-03-23T08:24:03.192 INFO:tasks.workunit.client.0.smithi070.stdout:9/754: write db/d12/d14/de5/da2/d58/fff [488781,43144] 0 2024-03-23T08:24:03.195 INFO:tasks.workunit.client.0.smithi070.stdout:1/656: rename d8/db/d40/d43/f7a to d8/db/d40/d43/d64/dc2/ff4 0 2024-03-23T08:24:03.196 INFO:tasks.workunit.client.0.smithi070.stdout:1/657: dread - d8/d11/f73 zero size 2024-03-23T08:24:03.197 INFO:tasks.workunit.client.0.smithi070.stdout:6/824: mknod d0/d23/d74/d84/db3/d38/d52/c106 0 2024-03-23T08:24:03.197 INFO:tasks.workunit.client.0.smithi070.stdout:7/600: rmdir d25/d27/d6a/d44 39 2024-03-23T08:24:03.198 INFO:tasks.workunit.client.0.smithi070.stdout:9/755: dread db/d12/d14/f49 [0,4194304] 0 2024-03-23T08:24:03.201 INFO:tasks.workunit.client.0.smithi070.stdout:6/825: dread d0/d23/d74/fe8 [0,4194304] 0 2024-03-23T08:24:03.205 INFO:tasks.workunit.client.0.smithi070.stdout:3/795: mkdir db/df/d5d/d126 0 2024-03-23T08:24:03.207 INFO:tasks.workunit.client.0.smithi070.stdout:0/856: dwrite d16/f1c [0,4194304] 0 2024-03-23T08:24:03.211 INFO:tasks.workunit.client.0.smithi070.stdout:8/706: dwrite d5/d8/d16/d32/f56 [0,4194304] 0 2024-03-23T08:24:03.214 INFO:tasks.workunit.client.0.smithi070.stdout:5/629: dwrite dc/f58 [0,4194304] 0 2024-03-23T08:24:03.220 INFO:tasks.workunit.client.0.smithi070.stdout:2/899: getdents d1/d7/d39/d47/d7b 0 2024-03-23T08:24:03.224 INFO:tasks.workunit.client.0.smithi070.stdout:6/826: mknod d0/d23/d88/c107 0 2024-03-23T08:24:03.229 INFO:tasks.workunit.client.0.smithi070.stdout:8/707: write d5/d8/f11 [43881,125349] 0 2024-03-23T08:24:03.236 INFO:tasks.workunit.client.0.smithi070.stdout:8/708: dwrite d5/d8/d9a/da2/dae/f94 [4194304,4194304] 0 2024-03-23T08:24:03.237 INFO:tasks.workunit.client.0.smithi070.stdout:3/796: symlink db/d14/d65/l127 0 2024-03-23T08:24:03.241 INFO:tasks.workunit.client.0.smithi070.stdout:4/650: rename d3/d6/d35/d65/d6f/f82 to d3/d6/d35/d65/d27/feb 0 2024-03-23T08:24:03.243 INFO:tasks.workunit.client.0.smithi070.stdout:4/651: stat d3/d6/d35/d3f/d51/cb6 0 2024-03-23T08:24:03.243 INFO:tasks.workunit.client.0.smithi070.stdout:1/658: creat d8/d26/ff5 x:0 0 0 2024-03-23T08:24:03.246 INFO:tasks.workunit.client.0.smithi070.stdout:9/756: dwrite db/d12/d14/f35 [0,4194304] 0 2024-03-23T08:24:03.246 INFO:tasks.workunit.client.0.smithi070.stdout:0/857: write d16/d26/d27/d3d/fc4 [958743,90207] 0 2024-03-23T08:24:03.250 INFO:tasks.workunit.client.0.smithi070.stdout:5/630: getdents dc/d1b/d1d/d25/d36/d32/d5c/dc0 0 2024-03-23T08:24:03.250 INFO:tasks.workunit.client.0.smithi070.stdout:1/659: dread d8/d11/d61/f75 [0,4194304] 0 2024-03-23T08:24:03.260 INFO:tasks.workunit.client.0.smithi070.stdout:4/652: mknod d3/d6/d35/d3f/d51/d6e/d97/cec 0 2024-03-23T08:24:03.262 INFO:tasks.workunit.client.0.smithi070.stdout:8/709: dwrite d5/d8/f2c [4194304,4194304] 0 2024-03-23T08:24:03.263 INFO:tasks.workunit.client.0.smithi070.stdout:0/858: rename d16/d22/l38 to d16/d26/d27/d4c/d60/d7c/daf/l12b 0 2024-03-23T08:24:03.268 INFO:tasks.workunit.client.0.smithi070.stdout:9/757: dread db/d12/d14/de5/da2/d4f/d5a/f71 [0,4194304] 0 2024-03-23T08:24:03.272 INFO:tasks.workunit.client.0.smithi070.stdout:6/827: creat d0/d3/d2c/f108 x:0 0 0 2024-03-23T08:24:03.275 INFO:tasks.workunit.client.0.smithi070.stdout:3/797: link db/d4e/dd6/dd7/dd9/f104 db/d4e/dc4/d10c/f128 0 2024-03-23T08:24:03.277 INFO:tasks.workunit.client.0.smithi070.stdout:4/653: mkdir d3/d6/d11/ded 0 2024-03-23T08:24:03.278 INFO:tasks.workunit.client.0.smithi070.stdout:0/859: fsync d16/f30 0 2024-03-23T08:24:03.280 INFO:tasks.workunit.client.0.smithi070.stdout:2/900: truncate d1/d7/d48/d69/d7a/fa5 4414408 0 2024-03-23T08:24:03.282 INFO:tasks.workunit.client.0.smithi070.stdout:2/901: dread - d1/d7/d48/d69/d7a/f10c zero size 2024-03-23T08:24:03.283 INFO:tasks.workunit.client.0.smithi070.stdout:9/758: symlink db/d12/d14/de5/da2/dc6/dce/d85/d42/l100 0 2024-03-23T08:24:03.283 INFO:tasks.workunit.client.0.smithi070.stdout:2/902: readlink d1/d7/df/l12b 0 2024-03-23T08:24:03.288 INFO:tasks.workunit.client.0.smithi070.stdout:8/710: mkdir d5/d8/d16/d32/d84/da7/deb/dfa 0 2024-03-23T08:24:03.289 INFO:tasks.workunit.client.0.smithi070.stdout:7/601: dwrite d25/d27/d6a/d44/f90 [0,4194304] 0 2024-03-23T08:24:03.295 INFO:tasks.workunit.client.0.smithi070.stdout:0/860: read d16/d26/d27/d4c/d60/d7c/daf/fe9 [675238,68926] 0 2024-03-23T08:24:03.297 INFO:tasks.workunit.client.0.smithi070.stdout:4/654: rename d3/d6/d35/d65/d41 to d3/d6/d35/d65/d6f/dee 0 2024-03-23T08:24:03.299 INFO:tasks.workunit.client.0.smithi070.stdout:6/828: unlink d0/d3/l73 0 2024-03-23T08:24:03.302 INFO:tasks.workunit.client.0.smithi070.stdout:4/655: readlink d3/d6/d35/dcf/db5/dda/db8/la1 0 2024-03-23T08:24:03.303 INFO:tasks.workunit.client.0.smithi070.stdout:9/759: mkdir db/d12/d14/de5/da2/dc6/d7e/d101 0 2024-03-23T08:24:03.303 INFO:tasks.workunit.client.0.smithi070.stdout:7/602: dwrite d25/d27/d6a/d44/da2/db5/fcf [0,4194304] 0 2024-03-23T08:24:03.304 INFO:tasks.workunit.client.0.smithi070.stdout:1/660: dwrite d8/d26/d49/d65/f77 [0,4194304] 0 2024-03-23T08:24:03.309 INFO:tasks.workunit.client.0.smithi070.stdout:3/798: truncate db/df/d6a/fa2 42531 0 2024-03-23T08:24:03.311 INFO:tasks.workunit.client.0.smithi070.stdout:8/711: creat d5/d8/d16/d30/d38/d6b/da9/ffb x:0 0 0 2024-03-23T08:24:03.312 INFO:tasks.workunit.client.0.smithi070.stdout:5/631: getdents dc/d1b/d1d/d25/d68/d8a/d99 0 2024-03-23T08:24:03.318 INFO:tasks.workunit.client.0.smithi070.stdout:9/760: dwrite db/d12/d14/de5/da2/d26/f64 [4194304,4194304] 0 2024-03-23T08:24:03.323 INFO:tasks.workunit.client.0.smithi070.stdout:0/861: dread d16/d22/f2c [0,4194304] 0 2024-03-23T08:24:03.324 INFO:tasks.workunit.client.0.smithi070.stdout:0/862: stat d16/d26/d27 0 2024-03-23T08:24:03.326 INFO:tasks.workunit.client.0.smithi070.stdout:7/603: dread d25/d40/fc9 [0,4194304] 0 2024-03-23T08:24:03.326 INFO:tasks.workunit.client.0.smithi070.stdout:3/799: symlink db/d7f/l129 0 2024-03-23T08:24:03.329 INFO:tasks.workunit.client.0.smithi070.stdout:5/632: creat dc/d1b/d1d/d25/d38/d53/fc9 x:0 0 0 2024-03-23T08:24:03.330 INFO:tasks.workunit.client.0.smithi070.stdout:9/761: symlink db/d12/d91/l102 0 2024-03-23T08:24:03.331 INFO:tasks.workunit.client.0.smithi070.stdout:3/800: dread db/d14/d43/d7d/f11f [0,4194304] 0 2024-03-23T08:24:03.331 INFO:tasks.workunit.client.0.smithi070.stdout:5/633: readlink dc/d1b/d1d/d25/d38/d39/l6d 0 2024-03-23T08:24:03.334 INFO:tasks.workunit.client.0.smithi070.stdout:2/903: link d1/d118/d34/l88 d1/d118/d34/l12e 0 2024-03-23T08:24:03.342 INFO:tasks.workunit.client.0.smithi070.stdout:7/604: fdatasync d25/d40/fb0 0 2024-03-23T08:24:03.345 INFO:tasks.workunit.client.0.smithi070.stdout:8/712: symlink d5/d8/d16/d3b/db9/lfc 0 2024-03-23T08:24:03.347 INFO:tasks.workunit.client.0.smithi070.stdout:8/713: truncate d5/d8/d9a/da2/dae/ff6 380716 0 2024-03-23T08:24:03.351 INFO:tasks.workunit.client.0.smithi070.stdout:5/634: creat dc/fca x:0 0 0 2024-03-23T08:24:03.353 INFO:tasks.workunit.client.0.smithi070.stdout:1/661: write d8/db/d40/d43/d64/d85/dc7/fcc [566405,5250] 0 2024-03-23T08:24:03.354 INFO:tasks.workunit.client.0.smithi070.stdout:8/714: dwrite d5/d8/d16/d3b/f5d [0,4194304] 0 2024-03-23T08:24:03.357 INFO:tasks.workunit.client.0.smithi070.stdout:2/904: mkdir d1/d7/d39/d12f 0 2024-03-23T08:24:03.358 INFO:tasks.workunit.client.0.smithi070.stdout:2/905: dread - d1/d7/df/d99/f11b zero size 2024-03-23T08:24:03.362 INFO:tasks.workunit.client.0.smithi070.stdout:4/656: creat d3/d6/d35/d65/fef x:0 0 0 2024-03-23T08:24:03.363 INFO:tasks.workunit.client.0.smithi070.stdout:0/863: read d16/d26/d27/d4c/d60/d75/f9c [508140,23380] 0 2024-03-23T08:24:03.366 INFO:tasks.workunit.client.0.smithi070.stdout:7/605: rename d25/d27/d6a/f5b to d25/d40/fd8 0 2024-03-23T08:24:03.367 INFO:tasks.workunit.client.0.smithi070.stdout:7/606: readlink d25/d27/d6a/d44/d6f/l97 0 2024-03-23T08:24:03.373 INFO:tasks.workunit.client.0.smithi070.stdout:5/635: rmdir dc/d1b/d1d/d25/d68 39 2024-03-23T08:24:03.375 INFO:tasks.workunit.client.0.smithi070.stdout:3/801: dwrite db/d4e/df8/d3d/f56 [0,4194304] 0 2024-03-23T08:24:03.376 INFO:tasks.workunit.client.0.smithi070.stdout:9/762: getdents db/d12/d14/de5/da2/d26/dec/dfa 0 2024-03-23T08:24:03.377 INFO:tasks.workunit.client.0.smithi070.stdout:9/763: readlink db/d12/d14/de5/da2/l6b 0 2024-03-23T08:24:03.379 INFO:tasks.workunit.client.0.smithi070.stdout:6/829: link d0/d23/d74/d84/db3/c21 d0/d3/d2c/c109 0 2024-03-23T08:24:03.380 INFO:tasks.workunit.client.0.smithi070.stdout:4/657: fsync d3/d6/d9/f22 0 2024-03-23T08:24:03.382 INFO:tasks.workunit.client.0.smithi070.stdout:0/864: unlink d16/d26/d27/d86/d9e/ddd/ff3 0 2024-03-23T08:24:03.387 INFO:tasks.workunit.client.0.smithi070.stdout:2/906: rename d1/d118/fdf to d1/d7/d39/d47/d7b/f130 0 2024-03-23T08:24:03.392 INFO:tasks.workunit.client.0.smithi070.stdout:1/662: creat d8/d42/d4e/d99/ff6 x:0 0 0 2024-03-23T08:24:03.396 INFO:tasks.workunit.client.0.smithi070.stdout:5/636: mkdir dc/d1b/d1d/d25/d36/d32/d5c/dcb 0 2024-03-23T08:24:03.398 INFO:tasks.workunit.client.0.smithi070.stdout:3/802: unlink db/df/f90 0 2024-03-23T08:24:03.400 INFO:tasks.workunit.client.0.smithi070.stdout:9/764: mknod db/d12/d14/de5/da2/d58/db5/c103 0 2024-03-23T08:24:03.455 INFO:tasks.workunit.client.0.smithi070.stdout:8/715: write d5/d8/d16/d32/f46 [512996,121561] 0 2024-03-23T08:24:03.459 INFO:tasks.workunit.client.0.smithi070.stdout:2/907: creat d1/d118/d34/da2/f131 x:0 0 0 2024-03-23T08:24:03.459 INFO:tasks.workunit.client.0.smithi070.stdout:1/663: creat d8/d42/d4e/ff7 x:0 0 0 2024-03-23T08:24:03.463 INFO:tasks.workunit.client.0.smithi070.stdout:7/607: dread d25/d27/d6a/d44/f8c [0,4194304] 0 2024-03-23T08:24:03.463 INFO:tasks.workunit.client.0.smithi070.stdout:2/908: readlink d1/d21/d5f/l11a 0 2024-03-23T08:24:03.463 INFO:tasks.workunit.client.0.smithi070.stdout:8/716: chown d5/d8/d16/f24 1585446 1 2024-03-23T08:24:03.490 INFO:tasks.workunit.client.0.smithi070.stdout:5/637: truncate dc/d1b/d1d/d25/d36/f27 4893710 0 2024-03-23T08:24:03.491 INFO:tasks.workunit.client.0.smithi070.stdout:7/608: readlink d25/d27/d6a/l32 0 2024-03-23T08:24:03.491 INFO:tasks.workunit.client.0.smithi070.stdout:3/803: dread db/d20/d49/f125 [0,4194304] 0 2024-03-23T08:24:03.493 INFO:tasks.workunit.client.0.smithi070.stdout:0/865: creat d16/d26/d27/d4c/d60/f12c x:0 0 0 2024-03-23T08:24:03.495 INFO:tasks.workunit.client.0.smithi070.stdout:4/658: dwrite d3/d6/f2f [0,4194304] 0 2024-03-23T08:24:03.495 INFO:tasks.workunit.client.0.smithi070.stdout:2/909: creat d1/d7/d39/d58/f132 x:0 0 0 2024-03-23T08:24:03.495 INFO:tasks.workunit.client.0.smithi070.stdout:8/717: fdatasync d5/d8/d16/dd4/d6f/d7c/f91 0 2024-03-23T08:24:03.497 INFO:tasks.workunit.client.0.smithi070.stdout:9/765: rename db/d12/d14/de5/da2/dc6/dce/l52 to db/d12/d14/de5/da2/d26/d80/l104 0 2024-03-23T08:24:03.499 INFO:tasks.workunit.client.0.smithi070.stdout:4/659: chown d3/d6/d35/cd4 518578538 1 2024-03-23T08:24:03.501 INFO:tasks.workunit.client.0.smithi070.stdout:0/866: dread - d16/d26/d27/ffe zero size 2024-03-23T08:24:03.502 INFO:tasks.workunit.client.0.smithi070.stdout:9/766: unlink db/f1b 0 2024-03-23T08:24:03.505 INFO:tasks.workunit.client.0.smithi070.stdout:2/910: mkdir d1/d29/d52/de4/d133 0 2024-03-23T08:24:03.506 INFO:tasks.workunit.client.0.smithi070.stdout:2/911: stat d1/d21/fa9 0 2024-03-23T08:24:03.507 INFO:tasks.workunit.client.0.smithi070.stdout:4/660: dwrite d3/d6/d35/d65/d1b/f1c [0,4194304] 0 2024-03-23T08:24:03.508 INFO:tasks.workunit.client.0.smithi070.stdout:4/661: stat d3/f8f 0 2024-03-23T08:24:03.508 INFO:tasks.workunit.client.0.smithi070.stdout:4/662: readlink d3/d6/d35/d65/d6f/lc5 0 2024-03-23T08:24:03.513 INFO:tasks.workunit.client.0.smithi070.stdout:1/664: getdents d8/d26/d41/dd9 0 2024-03-23T08:24:03.515 INFO:tasks.workunit.client.0.smithi070.stdout:1/665: fdatasync d8/db/d40/d43/d64/d85/d97/fa7 0 2024-03-23T08:24:03.522 INFO:tasks.workunit.client.0.smithi070.stdout:9/767: truncate db/d12/d91/f97 628470 0 2024-03-23T08:24:03.523 INFO:tasks.workunit.client.0.smithi070.stdout:2/912: rmdir d1/d7/d48/d69/db8/dc7 39 2024-03-23T08:24:03.526 INFO:tasks.workunit.client.0.smithi070.stdout:8/718: symlink d5/d8/d16/d30/d38/lfd 0 2024-03-23T08:24:03.526 INFO:tasks.workunit.client.0.smithi070.stdout:4/663: read d3/d6/d9/fb4 [393686,44346] 0 2024-03-23T08:24:03.529 INFO:tasks.workunit.client.0.smithi070.stdout:1/666: rmdir d8/db 39 2024-03-23T08:24:03.532 INFO:tasks.workunit.client.0.smithi070.stdout:0/867: fdatasync d16/f1d 0 2024-03-23T08:24:03.533 INFO:tasks.workunit.client.0.smithi070.stdout:3/804: link db/d4e/d92/l68 db/d20/d49/l12a 0 2024-03-23T08:24:03.534 INFO:tasks.workunit.client.0.smithi070.stdout:0/868: chown d16/d26/d27/d86/d67/d7d/faa 222 1 2024-03-23T08:24:03.534 INFO:tasks.workunit.client.0.smithi070.stdout:7/609: link d25/d27/l33 d25/d27/d6a/d44/d85/d95/da0/ld9 0 2024-03-23T08:24:03.536 INFO:tasks.workunit.client.0.smithi070.stdout:9/768: dread - db/d12/d14/de5/da2/dc6/dce/d85/fc9 zero size 2024-03-23T08:24:03.538 INFO:tasks.workunit.client.0.smithi070.stdout:9/769: chown db/d12/d14/de5/da2/d4f/lb3 80 1 2024-03-23T08:24:03.538 INFO:tasks.workunit.client.0.smithi070.stdout:8/719: chown d5/d8/f57 3 1 2024-03-23T08:24:03.540 INFO:tasks.workunit.client.0.smithi070.stdout:4/664: symlink d3/d6/d35/dcf/db5/dda/db8/lf0 0 2024-03-23T08:24:03.544 INFO:tasks.workunit.client.0.smithi070.stdout:5/638: truncate dc/d1b/d1d/d61/fbd 2238923 0 2024-03-23T08:24:03.546 INFO:tasks.workunit.client.0.smithi070.stdout:2/913: dread d1/d7/d39/d58/f89 [0,4194304] 0 2024-03-23T08:24:03.552 INFO:tasks.workunit.client.0.smithi070.stdout:3/805: creat db/d14/d43/d62/d95/d123/f12b x:0 0 0 2024-03-23T08:24:03.553 INFO:tasks.workunit.client.0.smithi070.stdout:3/806: write db/f17 [2555689,60249] 0 2024-03-23T08:24:03.558 INFO:tasks.workunit.client.0.smithi070.stdout:0/869: dread d16/d22/f29 [0,4194304] 0 2024-03-23T08:24:03.561 INFO:tasks.workunit.client.0.smithi070.stdout:2/914: dread d1/d7/d39/d47/d7b/f105 [0,4194304] 0 2024-03-23T08:24:03.562 INFO:tasks.workunit.client.0.smithi070.stdout:4/665: fsync d3/d6/d35/d65/d1b/f59 0 2024-03-23T08:24:03.565 INFO:tasks.workunit.client.0.smithi070.stdout:1/667: truncate d8/db/d40/d43/d64/d85/d97/fa7 7628938 0 2024-03-23T08:24:03.565 INFO:tasks.workunit.client.0.smithi070.stdout:4/666: chown d3/d6/d35/dcf/d7e/d9a 10 1 2024-03-23T08:24:03.566 INFO:tasks.workunit.client.0.smithi070.stdout:9/770: write db/d12/f2b [1953301,3416] 0 2024-03-23T08:24:03.567 INFO:tasks.workunit.client.0.smithi070.stdout:3/807: dwrite db/df/d1a/f3c [0,4194304] 0 2024-03-23T08:24:03.569 INFO:tasks.workunit.client.0.smithi070.stdout:7/610: creat d25/d27/d6a/fda x:0 0 0 2024-03-23T08:24:03.572 INFO:tasks.workunit.client.0.smithi070.stdout:5/639: dwrite dc/d1b/d1d/d25/d38/f69 [0,4194304] 0 2024-03-23T08:24:03.572 INFO:tasks.workunit.client.0.smithi070.stdout:0/870: stat d16/l2d 0 2024-03-23T08:24:03.574 INFO:tasks.workunit.client.0.smithi070.stdout:8/720: creat d5/d8/ffe x:0 0 0 2024-03-23T08:24:03.580 INFO:tasks.workunit.client.0.smithi070.stdout:8/721: read d5/d8/d16/d32/f3c [18799,23170] 0 2024-03-23T08:24:03.583 INFO:tasks.workunit.client.0.smithi070.stdout:2/915: rename d1/d7/df/d99/dea/dc3/c1c to d1/d7/d48/d69/d7a/d108/c134 0 2024-03-23T08:24:03.588 INFO:tasks.workunit.client.0.smithi070.stdout:7/611: stat d25/d27/d6a/d44/fa8 0 2024-03-23T08:24:03.589 INFO:tasks.workunit.client.0.smithi070.stdout:4/667: write d3/d6/d9/f4a [1782191,54146] 0 2024-03-23T08:24:03.591 INFO:tasks.workunit.client.0.smithi070.stdout:9/771: creat db/d12/d14/de5/da2/dc6/d7e/f105 x:0 0 0 2024-03-23T08:24:03.596 INFO:tasks.workunit.client.0.smithi070.stdout:0/871: mknod d16/d26/d27/d86/d67/d7d/c12d 0 2024-03-23T08:24:03.598 INFO:tasks.workunit.client.0.smithi070.stdout:3/808: dread db/df/d1a/de9/fff [0,4194304] 0 2024-03-23T08:24:03.598 INFO:tasks.workunit.client.0.smithi070.stdout:2/916: rename d1/d7/df/d99/dea/fc4 to d1/d7/df/f135 0 2024-03-23T08:24:03.602 INFO:tasks.workunit.client.0.smithi070.stdout:5/640: creat dc/d1b/d1d/d25/da9/fcc x:0 0 0 2024-03-23T08:24:03.605 INFO:tasks.workunit.client.0.smithi070.stdout:1/668: getdents d8/d11/d61/d74 0 2024-03-23T08:24:03.606 INFO:tasks.workunit.client.0.smithi070.stdout:2/917: dwrite d1/d7/df/d94/d38/d4c/fde [0,4194304] 0 2024-03-23T08:24:03.608 INFO:tasks.workunit.client.0.smithi070.stdout:0/872: mknod d16/d26/d27/d86/db6/dde/c12e 0 2024-03-23T08:24:03.609 INFO:tasks.workunit.client.0.smithi070.stdout:7/612: rename d25/d27/d6a/d44/d7c/c9b to d25/d27/d6a/d44/d7c/db4/cdb 0 2024-03-23T08:24:03.611 INFO:tasks.workunit.client.0.smithi070.stdout:4/668: dread - d3/d6/d11/fd5 zero size 2024-03-23T08:24:03.614 INFO:tasks.workunit.client.0.smithi070.stdout:2/918: mknod d1/d7/d48/d5b/c136 0 2024-03-23T08:24:03.617 INFO:tasks.workunit.client.0.smithi070.stdout:3/809: symlink db/d7f/dcd/d100/l12c 0 2024-03-23T08:24:03.619 INFO:tasks.workunit.client.0.smithi070.stdout:5/641: mkdir dc/d1b/d1d/d25/dcd 0 2024-03-23T08:24:03.622 INFO:tasks.workunit.client.0.smithi070.stdout:8/722: getdents d5/d8 0 2024-03-23T08:24:03.625 INFO:tasks.workunit.client.0.smithi070.stdout:7/613: mkdir d25/d27/d6a/d44/d6f/dab/dcd/ddc 0 2024-03-23T08:24:03.627 INFO:tasks.workunit.client.0.smithi070.stdout:9/772: write db/d12/d91/f97 [754568,3176] 0 2024-03-23T08:24:03.628 INFO:tasks.workunit.client.0.smithi070.stdout:3/810: unlink db/d4e/d92/c6b 0 2024-03-23T08:24:03.630 INFO:tasks.workunit.client.0.smithi070.stdout:9/773: write db/d12/d14/de5/da2/f78 [1794609,34252] 0 2024-03-23T08:24:03.630 INFO:tasks.workunit.client.0.smithi070.stdout:5/642: mknod dc/d1b/d1d/d25/da9/cce 0 2024-03-23T08:24:03.631 INFO:tasks.workunit.client.0.smithi070.stdout:0/873: write d16/d26/d27/d3d/d4d/d9a/fe3 [442223,127989] 0 2024-03-23T08:24:03.631 INFO:tasks.workunit.client.0.smithi070.stdout:3/811: read db/d4e/df8/d3d/f56 [3305729,102115] 0 2024-03-23T08:24:03.632 INFO:tasks.workunit.client.0.smithi070.stdout:2/919: creat d1/d29/d52/de4/d133/f137 x:0 0 0 2024-03-23T08:24:03.634 INFO:tasks.workunit.client.0.smithi070.stdout:7/614: dwrite d25/d27/fac [4194304,4194304] 0 2024-03-23T08:24:03.637 INFO:tasks.workunit.client.0.smithi070.stdout:8/723: mknod d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/df2/cff 0 2024-03-23T08:24:03.637 INFO:tasks.workunit.client.0.smithi070.stdout:7/615: dread d25/d40/fc9 [0,4194304] 0 2024-03-23T08:24:03.639 INFO:tasks.workunit.client.0.smithi070.stdout:5/643: rename dc/d1b/d1d to dc/d1b/d1d/dcf 22 2024-03-23T08:24:03.640 INFO:tasks.workunit.client.0.smithi070.stdout:2/920: mkdir d1/d7/df/d99/dea/d138 0 2024-03-23T08:24:03.649 INFO:tasks.workunit.client.0.smithi070.stdout:0/874: symlink d16/d26/d27/d4c/l12f 0 2024-03-23T08:24:03.651 INFO:tasks.workunit.client.0.smithi070.stdout:7/616: creat d25/d27/d6a/d44/d85/fdd x:0 0 0 2024-03-23T08:24:03.651 INFO:tasks.workunit.client.0.smithi070.stdout:5/644: creat dc/d1b/d1d/d25/da9/fd0 x:0 0 0 2024-03-23T08:24:03.654 INFO:tasks.workunit.client.0.smithi070.stdout:8/724: fdatasync d5/d8/d9a/dbe/fed 0 2024-03-23T08:24:03.655 INFO:tasks.workunit.client.0.smithi070.stdout:8/725: dread - d5/d8/d9a/da2/fd2 zero size 2024-03-23T08:24:03.657 INFO:tasks.workunit.client.0.smithi070.stdout:7/617: mknod d25/d27/d6a/d44/d7c/cde 0 2024-03-23T08:24:03.657 INFO:tasks.workunit.client.0.smithi070.stdout:8/726: fdatasync d5/d8/d16/d32/d84/d98/dde/da3/dab/ff9 0 2024-03-23T08:24:03.669 INFO:tasks.workunit.client.0.smithi070.stdout:0/875: rename d16/d26/d27/d86/db6/dde/lb7 to d16/d26/d27/dee/l130 0 2024-03-23T08:24:03.673 INFO:tasks.workunit.client.0.smithi070.stdout:9/774: link db/d12/d67/fc1 db/d12/d14/de5/da2/dc6/f106 0 2024-03-23T08:24:03.677 INFO:tasks.workunit.client.0.smithi070.stdout:0/876: rename d16/d22/f8d to d16/d22/d43/d47/da9/de7/f131 0 2024-03-23T08:24:03.677 INFO:tasks.workunit.client.0.smithi070.stdout:2/921: dwrite d1/d7/d48/d69/d7a/fa5 [0,4194304] 0 2024-03-23T08:24:03.682 INFO:tasks.workunit.client.0.smithi070.stdout:8/727: creat d5/d8/d9a/da2/f100 x:0 0 0 2024-03-23T08:24:03.683 INFO:tasks.workunit.client.0.smithi070.stdout:9/775: dwrite db/d12/d14/de5/da2/d26/f64 [0,4194304] 0 2024-03-23T08:24:03.685 INFO:tasks.workunit.client.0.smithi070.stdout:0/877: chown d16/d22/f39 69 1 2024-03-23T08:24:03.686 INFO:tasks.workunit.client.0.smithi070.stdout:5/645: link dc/d1b/c45 dc/d1b/d1d/d25/d68/d8a/cd1 0 2024-03-23T08:24:03.687 INFO:tasks.workunit.client.0.smithi070.stdout:9/776: fsync db/d12/d14/de5/da2/d58/fff 0 2024-03-23T08:24:03.689 INFO:tasks.workunit.client.0.smithi070.stdout:7/618: dwrite d25/d27/d6a/d44/d85/faf [0,4194304] 0 2024-03-23T08:24:03.698 INFO:tasks.workunit.client.0.smithi070.stdout:9/777: fdatasync f8 0 2024-03-23T08:24:03.701 INFO:tasks.workunit.client.0.smithi070.stdout:5/646: unlink dc/d1b/d1d/d25/d38/l50 0 2024-03-23T08:24:03.702 INFO:tasks.workunit.client.0.smithi070.stdout:7/619: dwrite d25/d27/d6a/d44/f90 [0,4194304] 0 2024-03-23T08:24:03.705 INFO:tasks.workunit.client.0.smithi070.stdout:5/647: write dc/d1b/d1d/d25/d38/d53/f60 [3456172,353] 0 2024-03-23T08:24:03.707 INFO:tasks.workunit.client.0.smithi070.stdout:8/728: link d5/d8/f17 d5/d8/d9a/da2/dae/f101 0 2024-03-23T08:24:03.713 INFO:tasks.workunit.client.0.smithi070.stdout:5/648: dwrite dc/fca [0,4194304] 0 2024-03-23T08:24:03.717 INFO:tasks.workunit.client.0.smithi070.stdout:0/878: write f3 [5204008,124784] 0 2024-03-23T08:24:03.725 INFO:tasks.workunit.client.0.smithi070.stdout:7/620: mknod d25/d27/d6a/d44/d7c/db4/cdf 0 2024-03-23T08:24:03.726 INFO:tasks.workunit.client.0.smithi070.stdout:5/649: dwrite dc/d1b/d1d/d25/da9/fcc [0,4194304] 0 2024-03-23T08:24:03.728 INFO:tasks.workunit.client.0.smithi070.stdout:8/729: mknod d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/c102 0 2024-03-23T08:24:03.729 INFO:tasks.workunit.client.0.smithi070.stdout:9/778: symlink db/d12/d14/de5/da2/d26/dd6/l107 0 2024-03-23T08:24:03.730 INFO:tasks.workunit.client.0.smithi070.stdout:0/879: rmdir d16/d26/d27/d4c 39 2024-03-23T08:24:03.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:03 smithi070 ceph-mon[39699]: pgmap v41: 65 pgs: 65 active+clean; 1.6 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 52 MiB/s rd, 154 MiB/s wr, 427 op/s 2024-03-23T08:24:03.734 INFO:tasks.workunit.client.0.smithi070.stdout:7/621: creat d25/d27/d6a/d44/d7c/db4/fe0 x:0 0 0 2024-03-23T08:24:03.736 INFO:tasks.workunit.client.0.smithi070.stdout:7/622: dread d25/d40/f8f [0,4194304] 0 2024-03-23T08:24:03.737 INFO:tasks.workunit.client.0.smithi070.stdout:5/650: creat dc/d1b/d1d/d25/d36/d32/d5c/fd2 x:0 0 0 2024-03-23T08:24:03.738 INFO:tasks.workunit.client.0.smithi070.stdout:8/730: dread d5/d8/d16/d32/d84/d98/dde/da3/fbb [0,4194304] 0 2024-03-23T08:24:03.744 INFO:tasks.workunit.client.0.smithi070.stdout:2/922: link d1/d7/d48/d69/db8/dc7/d111/l49 d1/d118/d34/l139 0 2024-03-23T08:24:03.745 INFO:tasks.workunit.client.0.smithi070.stdout:2/923: fsync d1/d98/f127 0 2024-03-23T08:24:03.746 INFO:tasks.workunit.client.0.smithi070.stdout:7/623: mkdir d25/d27/d6a/d44/d6f/da3/de1 0 2024-03-23T08:24:03.746 INFO:tasks.workunit.client.0.smithi070.stdout:2/924: write d1/d29/d52/f113 [290727,43952] 0 2024-03-23T08:24:03.756 INFO:tasks.workunit.client.0.smithi070.stdout:9/779: unlink db/d12/d14/de5/da2/dc6/dce/d85/c59 0 2024-03-23T08:24:03.761 INFO:tasks.workunit.client.0.smithi070.stdout:5/651: dwrite dc/d1b/d1d/d25/d68/d8a/f8e [0,4194304] 0 2024-03-23T08:24:03.764 INFO:tasks.workunit.client.0.smithi070.stdout:7/624: mkdir d25/d27/d6a/d44/d85/d95/da0/de2 0 2024-03-23T08:24:03.766 INFO:tasks.workunit.client.0.smithi070.stdout:0/880: dread d16/d26/d27/d4c/d60/d75/fd2 [0,4194304] 0 2024-03-23T08:24:03.769 INFO:tasks.workunit.client.0.smithi070.stdout:2/925: dwrite d1/d7/df/f27 [0,4194304] 0 2024-03-23T08:24:03.770 INFO:tasks.workunit.client.0.smithi070.stdout:9/780: rename db/d12/d14/de5/da2/dc6/dce/d85/d42/fa4 to db/d12/d14/de5/da2/d4f/d5a/f108 0 2024-03-23T08:24:03.772 INFO:tasks.workunit.client.0.smithi070.stdout:2/926: fdatasync d1/d7/df/d94/d38/f117 0 2024-03-23T08:24:03.773 INFO:tasks.workunit.client.0.smithi070.stdout:5/652: creat dc/d1b/d1d/d25/d36/fd3 x:0 0 0 2024-03-23T08:24:03.776 INFO:tasks.workunit.client.0.smithi070.stdout:5/653: truncate dc/d1b/d1d/d25/d36/d32/d5c/fd2 362062 0 2024-03-23T08:24:03.799 INFO:tasks.workunit.client.0.smithi070.stdout:7/625: fsync d25/f43 0 2024-03-23T08:24:03.799 INFO:tasks.workunit.client.0.smithi070.stdout:7/626: readlink d25/d27/l69 0 2024-03-23T08:24:03.800 INFO:tasks.workunit.client.0.smithi070.stdout:7/627: chown d25/d27/d6a/d44 217014 1 2024-03-23T08:24:03.802 INFO:tasks.workunit.client.0.smithi070.stdout:0/881: fsync d16/d22/f23 0 2024-03-23T08:24:03.807 INFO:tasks.workunit.client.0.smithi070.stdout:2/927: mkdir d1/d7/df/d99/dea/dc3/d13a 0 2024-03-23T08:24:03.807 INFO:tasks.workunit.client.0.smithi070.stdout:9/781: dread db/d12/d34/f92 [0,4194304] 0 2024-03-23T08:24:03.810 INFO:tasks.workunit.client.0.smithi070.stdout:9/782: dread db/d12/d14/de5/da2/d4f/d5a/fa5 [0,4194304] 0 2024-03-23T08:24:03.813 INFO:tasks.workunit.client.0.smithi070.stdout:7/628: creat d25/d27/dc2/fe3 x:0 0 0 2024-03-23T08:24:03.821 INFO:tasks.workunit.client.0.smithi070.stdout:2/928: rename d1/d7/df/ccf to d1/d7/d48/d69/d7a/c13b 0 2024-03-23T08:24:03.821 INFO:tasks.workunit.client.0.smithi070.stdout:8/731: truncate d5/d8/d16/dd4/d6f/fb5 2882602 0 2024-03-23T08:24:03.823 INFO:tasks.workunit.client.0.smithi070.stdout:8/732: chown d5/d8/d16/d30/d38/ca8 1017137 1 2024-03-23T08:24:03.823 INFO:tasks.workunit.client.0.smithi070.stdout:9/783: mknod db/d12/d14/de5/da2/d4f/db6/d79/c109 0 2024-03-23T08:24:03.824 INFO:tasks.workunit.client.0.smithi070.stdout:8/733: chown d5/d8/d16/d3b/faa 542 1 2024-03-23T08:24:03.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:03 smithi193 ceph-mon[48370]: pgmap v41: 65 pgs: 65 active+clean; 1.6 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 52 MiB/s rd, 154 MiB/s wr, 427 op/s 2024-03-23T08:24:03.826 INFO:tasks.workunit.client.0.smithi070.stdout:7/629: chown d25/cc0 0 1 2024-03-23T08:24:03.826 INFO:tasks.workunit.client.0.smithi070.stdout:7/630: read - d25/fb8 zero size 2024-03-23T08:24:03.830 INFO:tasks.workunit.client.0.smithi070.stdout:8/734: dwrite d5/d8/d16/d3b/faa [0,4194304] 0 2024-03-23T08:24:03.835 INFO:tasks.workunit.client.0.smithi070.stdout:8/735: truncate d5/d8/d16/dd4/d6f/d7c/d7d/fe4 267776 0 2024-03-23T08:24:03.835 INFO:tasks.workunit.client.0.smithi070.stdout:7/631: dwrite d25/d27/d6a/d44/d7c/db4/fb6 [0,4194304] 0 2024-03-23T08:24:03.836 INFO:tasks.workunit.client.0.smithi070.stdout:8/736: chown d5/d8/d16/d30/f2f 49652 1 2024-03-23T08:24:03.837 INFO:tasks.workunit.client.0.smithi070.stdout:8/737: dread - d5/d8/d16/dd4/d6f/f7e zero size 2024-03-23T08:24:03.930 INFO:tasks.workunit.client.0.smithi070.stdout:9/784: unlink db/d12/d34/f92 0 2024-03-23T08:24:03.954 INFO:tasks.workunit.client.0.smithi070.stdout:5/654: dwrite dc/d19/f2c [0,4194304] 0 2024-03-23T08:24:03.964 INFO:tasks.workunit.client.0.smithi070.stdout:0/882: dwrite d16/d22/d43/d47/fa5 [4194304,4194304] 0 2024-03-23T08:24:03.978 INFO:tasks.workunit.client.0.smithi070.stdout:7/632: write d25/d27/f4c [2029924,107187] 0 2024-03-23T08:24:03.989 INFO:tasks.workunit.client.0.smithi070.stdout:0/883: mkdir d16/d107/d132 0 2024-03-23T08:24:03.992 INFO:tasks.workunit.client.0.smithi070.stdout:9/785: mkdir db/d12/d14/de5/da2/d4f/dd9/dc2/de7/d10a 0 2024-03-23T08:24:03.996 INFO:tasks.workunit.client.0.smithi070.stdout:7/633: mkdir d25/d27/d6a/d44/d6f/dab/dcd/ddc/de4 0 2024-03-23T08:24:03.999 INFO:tasks.workunit.client.0.smithi070.stdout:8/738: link d5/l15 d5/d8/d16/d32/d84/d98/l103 0 2024-03-23T08:24:04.002 INFO:tasks.workunit.client.0.smithi070.stdout:5/655: creat dc/d1b/fd4 x:0 0 0 2024-03-23T08:24:04.003 INFO:tasks.workunit.client.0.smithi070.stdout:0/884: creat d16/d26/d27/d86/d67/d109/f133 x:0 0 0 2024-03-23T08:24:04.006 INFO:tasks.workunit.client.0.smithi070.stdout:7/634: read d25/d27/d6a/d44/d7c/f9a [164878,57910] 0 2024-03-23T08:24:04.008 INFO:tasks.workunit.client.0.smithi070.stdout:5/656: dwrite dc/d1b/d1d/d25/f43 [4194304,4194304] 0 2024-03-23T08:24:04.009 INFO:tasks.workunit.client.0.smithi070.stdout:8/739: rmdir d5/d8/d16/d32/d84/da7 39 2024-03-23T08:24:04.017 INFO:tasks.workunit.client.0.smithi070.stdout:0/885: rename d16/d26/d27/d3d/d4d/dad/fe8 to d16/d107/d132/f134 0 2024-03-23T08:24:04.021 INFO:tasks.workunit.client.0.smithi070.stdout:5/657: unlink dc/d1b/d1d/d61/lc3 0 2024-03-23T08:24:04.023 INFO:tasks.workunit.client.0.smithi070.stdout:5/658: write dc/d1b/d1d/d25/f43 [6944486,56268] 0 2024-03-23T08:24:04.029 INFO:tasks.workunit.client.0.smithi070.stdout:9/786: getdents db/d12/d14/de5/da2/d26/dec 0 2024-03-23T08:24:04.030 INFO:tasks.workunit.client.0.smithi070.stdout:7/635: dread d25/d27/f70 [0,4194304] 0 2024-03-23T08:24:04.045 INFO:tasks.workunit.client.0.smithi070.stdout:7/636: mknod d25/ce5 0 2024-03-23T08:24:04.048 INFO:tasks.workunit.client.0.smithi070.stdout:9/787: creat db/d12/d14/de5/f10b x:0 0 0 2024-03-23T08:24:04.053 INFO:tasks.workunit.client.0.smithi070.stdout:7/637: unlink d25/d27/d6a/d44/d7c/c9f 0 2024-03-23T08:24:04.055 INFO:tasks.workunit.client.0.smithi070.stdout:9/788: mkdir db/d12/d14/de5/da2/d26/d80/d10c 0 2024-03-23T08:24:04.057 INFO:tasks.workunit.client.0.smithi070.stdout:7/638: write d25/d27/d6a/d44/d7c/db4/fc1 [958578,4811] 0 2024-03-23T08:24:04.089 INFO:tasks.workunit.client.0.smithi070.stdout:8/740: write d5/d8/d16/d32/d84/d98/fc5 [22932,22628] 0 2024-03-23T08:24:04.093 INFO:tasks.workunit.client.0.smithi070.stdout:8/741: unlink d5/d8/d16/d3b/db9/fd1 0 2024-03-23T08:24:04.094 INFO:tasks.workunit.client.0.smithi070.stdout:0/886: dwrite f13 [4194304,4194304] 0 2024-03-23T08:24:04.095 INFO:tasks.workunit.client.0.smithi070.stdout:8/742: fsync d5/d8/d16/d30/d38/d6b/d72/d88/f8a 0 2024-03-23T08:24:04.103 INFO:tasks.workunit.client.0.smithi070.stdout:8/743: truncate d5/d8/d16/d30/f3e 367339 0 2024-03-23T08:24:04.103 INFO:tasks.workunit.client.0.smithi070.stdout:9/789: write db/d12/d14/de5/da2/d4f/db6/f99 [2192197,106754] 0 2024-03-23T08:24:04.105 INFO:tasks.workunit.client.0.smithi070.stdout:7/639: write d25/d27/f57 [306334,90523] 0 2024-03-23T08:24:04.106 INFO:tasks.workunit.client.0.smithi070.stdout:9/790: fdatasync db/d12/d14/de5/da2/d58/fff 0 2024-03-23T08:24:04.108 INFO:tasks.workunit.client.0.smithi070.stdout:7/640: dread - d25/d27/d6a/d44/da2/fbf zero size 2024-03-23T08:24:04.108 INFO:tasks.workunit.client.0.smithi070.stdout:9/791: write db/d12/d14/de5/da2/fca [822759,77573] 0 2024-03-23T08:24:04.112 INFO:tasks.workunit.client.0.smithi070.stdout:7/641: mknod d25/d27/d6a/d44/da2/ce6 0 2024-03-23T08:24:04.115 INFO:tasks.workunit.client.0.smithi070.stdout:9/792: rename db/d12/d14/de5/ff6 to db/db8/dd8/f10d 0 2024-03-23T08:24:04.118 INFO:tasks.workunit.client.0.smithi070.stdout:8/744: dread d5/d8/f18 [0,4194304] 0 2024-03-23T08:24:04.118 INFO:tasks.workunit.client.0.smithi070.stdout:0/887: dwrite d16/d26/d27/d4c/d60/d75/f9c [0,4194304] 0 2024-03-23T08:24:04.121 INFO:tasks.workunit.client.0.smithi070.stdout:9/793: rmdir db/d12/d67 39 2024-03-23T08:24:04.123 INFO:tasks.workunit.client.0.smithi070.stdout:8/745: write d5/d8/d9a/da2/fd2 [78679,55414] 0 2024-03-23T08:24:04.130 INFO:tasks.workunit.client.0.smithi070.stdout:9/794: mkdir db/d12/d14/de5/da2/d26/d80/d10c/d10e 0 2024-03-23T08:24:04.174 INFO:tasks.workunit.client.0.smithi070.stdout:8/746: getdents d5/d8/d16 0 2024-03-23T08:24:04.174 INFO:tasks.workunit.client.0.smithi070.stdout:7/642: dread d25/f34 [0,4194304] 0 2024-03-23T08:24:04.178 INFO:tasks.workunit.client.0.smithi070.stdout:8/747: read d5/d8/d16/d32/f58 [2707310,78886] 0 2024-03-23T08:24:04.179 INFO:tasks.workunit.client.0.smithi070.stdout:0/888: dread d16/d26/d27/d86/db6/fdc [0,4194304] 0 2024-03-23T08:24:04.180 INFO:tasks.workunit.client.0.smithi070.stdout:7/643: fsync d25/d27/d6a/d44/d85/d95/fb9 0 2024-03-23T08:24:04.186 INFO:tasks.workunit.client.0.smithi070.stdout:9/795: write db/d12/d14/de5/da2/d4f/d5a/f108 [4976520,37302] 0 2024-03-23T08:24:04.186 INFO:tasks.workunit.client.0.smithi070.stdout:7/644: fsync d25/f87 0 2024-03-23T08:24:04.194 INFO:tasks.workunit.client.0.smithi070.stdout:9/796: dwrite db/d12/d14/de5/da2/d26/da7/fac [0,4194304] 0 2024-03-23T08:24:04.197 INFO:tasks.workunit.client.0.smithi070.stdout:7/645: symlink d25/d27/d6a/d44/d85/d95/da0/le7 0 2024-03-23T08:24:04.201 INFO:tasks.workunit.client.0.smithi070.stdout:0/889: creat d16/d26/d27/d4c/f135 x:0 0 0 2024-03-23T08:24:04.202 INFO:tasks.workunit.client.0.smithi070.stdout:9/797: chown db/d12/d14/de5/da2/d26/d80/lcb 34 1 2024-03-23T08:24:04.205 INFO:tasks.workunit.client.0.smithi070.stdout:8/748: getdents d5/d8/d16/dd4/d6f/d7c/d7d 0 2024-03-23T08:24:04.209 INFO:tasks.workunit.client.0.smithi070.stdout:9/798: dwrite db/d12/d14/de5/da2/dc6/dce/f93 [0,4194304] 0 2024-03-23T08:24:04.210 INFO:tasks.workunit.client.0.smithi070.stdout:0/890: dread - d16/d22/d43/d47/da9/fc5 zero size 2024-03-23T08:24:04.212 INFO:tasks.workunit.client.0.smithi070.stdout:0/891: chown d16/d26/d27/d3d/d4d/d9a/df8/d11a/c120 7637 1 2024-03-23T08:24:04.218 INFO:tasks.workunit.client.0.smithi070.stdout:7/646: dwrite d25/d27/d6a/fb1 [0,4194304] 0 2024-03-23T08:24:04.219 INFO:tasks.workunit.client.0.smithi070.stdout:0/892: rename d16/f1d to d16/d26/d27/d86/d9e/f136 0 2024-03-23T08:24:04.222 INFO:tasks.workunit.client.0.smithi070.stdout:9/799: mkdir db/de8/d10f 0 2024-03-23T08:24:04.225 INFO:tasks.workunit.client.0.smithi070.stdout:0/893: symlink d16/d26/d27/d86/d9e/l137 0 2024-03-23T08:24:04.233 INFO:tasks.workunit.client.0.smithi070.stdout:7/647: dread d25/d27/f6d [0,4194304] 0 2024-03-23T08:24:04.233 INFO:tasks.workunit.client.0.smithi070.stdout:9/800: read db/d12/d14/de5/da2/d58/db5/fc3 [108923,79621] 0 2024-03-23T08:24:04.234 INFO:tasks.workunit.client.0.smithi070.stdout:0/894: dread d16/d26/d27/d4c/d60/d75/fc3 [0,4194304] 0 2024-03-23T08:24:04.235 INFO:tasks.workunit.client.0.smithi070.stdout:7/648: fdatasync d25/d27/d6a/d44/d85/d95/da0/fd4 0 2024-03-23T08:24:04.237 INFO:tasks.workunit.client.0.smithi070.stdout:0/895: write d16/d26/d27/d3d/fc4 [219201,58508] 0 2024-03-23T08:24:04.238 INFO:tasks.workunit.client.0.smithi070.stdout:0/896: write d16/f1c [3889799,59693] 0 2024-03-23T08:24:04.238 INFO:tasks.workunit.client.0.smithi070.stdout:9/801: creat db/d12/d14/de5/da2/d4f/d5a/f110 x:0 0 0 2024-03-23T08:24:04.240 INFO:tasks.workunit.client.0.smithi070.stdout:7/649: readlink d25/d27/d6a/d44/dc8/lce 0 2024-03-23T08:24:04.240 INFO:tasks.workunit.client.0.smithi070.stdout:0/897: stat d16/d26/d27/d4c/d60/d7c/daf/c124 0 2024-03-23T08:24:04.244 INFO:tasks.workunit.client.0.smithi070.stdout:9/802: symlink db/d12/d14/de5/da2/d26/d80/d10c/d10e/l111 0 2024-03-23T08:24:04.245 INFO:tasks.workunit.client.0.smithi070.stdout:9/803: stat db/d12/d34/d47/fd7 0 2024-03-23T08:24:04.247 INFO:tasks.workunit.client.0.smithi070.stdout:0/898: link d16/d26/d27/d3d/d4d/d9a/fe3 d16/d26/d27/d86/d9e/ddd/f138 0 2024-03-23T08:24:04.248 INFO:tasks.workunit.client.0.smithi070.stdout:7/650: dwrite d25/d27/d6a/fda [0,4194304] 0 2024-03-23T08:24:04.248 INFO:tasks.workunit.client.0.smithi070.stdout:0/899: readlink d16/d26/d27/l77 0 2024-03-23T08:24:04.250 INFO:tasks.workunit.client.0.smithi070.stdout:9/804: write db/db8/dd8/f10d [841761,25873] 0 2024-03-23T08:24:04.252 INFO:tasks.workunit.client.0.smithi070.stdout:0/900: fdatasync d16/d22/df0/f11b 0 2024-03-23T08:24:04.254 INFO:tasks.workunit.client.0.smithi070.stdout:7/651: creat d25/d27/d6a/d44/d85/fe8 x:0 0 0 2024-03-23T08:24:04.255 INFO:tasks.workunit.client.0.smithi070.stdout:7/652: readlink d25/d27/d6a/d44/da4/ld2 0 2024-03-23T08:24:04.256 INFO:tasks.workunit.client.0.smithi070.stdout:9/805: rmdir db/d12 39 2024-03-23T08:24:04.256 INFO:tasks.workunit.client.0.smithi070.stdout:7/653: fsync d25/d27/d6a/d44/d85/faf 0 2024-03-23T08:24:04.257 INFO:tasks.workunit.client.0.smithi070.stdout:7/654: read - d25/fb8 zero size 2024-03-23T08:24:04.258 INFO:tasks.workunit.client.0.smithi070.stdout:0/901: truncate d16/d26/d27/d86/db6/fdc 3487599 0 2024-03-23T08:24:04.259 INFO:tasks.workunit.client.0.smithi070.stdout:9/806: chown db/d12/d14/de5/da2/d58/la0 289854720 1 2024-03-23T08:24:04.262 INFO:tasks.workunit.client.0.smithi070.stdout:0/902: mknod d16/d107/d132/c139 0 2024-03-23T08:24:04.263 INFO:tasks.workunit.client.0.smithi070.stdout:7/655: creat d25/d27/d6a/d44/d6f/da3/fe9 x:0 0 0 2024-03-23T08:24:04.266 INFO:tasks.workunit.client.0.smithi070.stdout:0/903: symlink d16/d26/d27/d4c/d60/d7c/l13a 0 2024-03-23T08:24:04.273 INFO:tasks.workunit.client.0.smithi070.stdout:9/807: getdents db/d12/d14/de5/da2/dc6/dce/d85 0 2024-03-23T08:24:04.277 INFO:tasks.workunit.client.0.smithi070.stdout:7/656: write d25/d27/d6a/d44/f5a [1817205,42762] 0 2024-03-23T08:24:04.280 INFO:tasks.workunit.client.0.smithi070.stdout:0/904: creat d16/d26/d27/d86/d67/f13b x:0 0 0 2024-03-23T08:24:04.286 INFO:tasks.workunit.client.0.smithi070.stdout:9/808: getdents db/d12/d14/de5/da2/d4f 0 2024-03-23T08:24:04.289 INFO:tasks.workunit.client.0.smithi070.stdout:9/809: mknod db/d12/d14/de5/da2/d4f/db6/d79/c112 0 2024-03-23T08:24:04.290 INFO:tasks.workunit.client.0.smithi070.stdout:9/810: read - db/d12/d14/de5/da2/dc6/d7e/f105 zero size 2024-03-23T08:24:04.290 INFO:tasks.workunit.client.0.smithi070.stdout:0/905: link d16/d26/c63 d16/d26/d27/d3d/d4d/d9a/df8/d11a/c13c 0 2024-03-23T08:24:04.291 INFO:tasks.workunit.client.0.smithi070.stdout:9/811: chown db/d12/d14/f35 6 1 2024-03-23T08:24:04.291 INFO:tasks.workunit.client.0.smithi070.stdout:0/906: read - d16/d26/d27/ffe zero size 2024-03-23T08:24:04.292 INFO:tasks.workunit.client.0.smithi070.stdout:0/907: stat d16/d26/d27/d3d/d4d/c88 0 2024-03-23T08:24:04.295 INFO:tasks.workunit.client.0.smithi070.stdout:0/908: chown d16/d26/d27/d86/d9e/f11f 6772 1 2024-03-23T08:24:04.296 INFO:tasks.workunit.client.0.smithi070.stdout:9/812: getdents db/d12/d14/de5/da2/dc6 0 2024-03-23T08:24:04.297 INFO:tasks.workunit.client.0.smithi070.stdout:7/657: dwrite d25/f87 [4194304,4194304] 0 2024-03-23T08:24:04.298 INFO:tasks.workunit.client.0.smithi070.stdout:7/658: readlink d25/d27/d6a/d44/da2/db5/lc7 0 2024-03-23T08:24:04.299 INFO:tasks.workunit.client.0.smithi070.stdout:0/909: read d16/d22/d43/f96 [473870,60122] 0 2024-03-23T08:24:04.301 INFO:tasks.workunit.client.0.smithi070.stdout:9/813: rmdir db/d12/d14/de5/da2/dc6 39 2024-03-23T08:24:04.303 INFO:tasks.workunit.client.0.smithi070.stdout:7/659: unlink f20 0 2024-03-23T08:24:04.306 INFO:tasks.workunit.client.0.smithi070.stdout:0/910: creat d16/d26/d27/d86/db6/dde/f13d x:0 0 0 2024-03-23T08:24:04.307 INFO:tasks.workunit.client.0.smithi070.stdout:9/814: symlink db/d12/d14/de5/da2/d58/db5/l113 0 2024-03-23T08:24:04.309 INFO:tasks.workunit.client.0.smithi070.stdout:7/660: mknod d25/d27/d6a/d44/d85/d95/da0/cea 0 2024-03-23T08:24:04.311 INFO:tasks.workunit.client.0.smithi070.stdout:0/911: creat d16/d26/d27/d3d/d4d/d9a/df8/d11a/f13e x:0 0 0 2024-03-23T08:24:04.311 INFO:tasks.workunit.client.0.smithi070.stdout:9/815: creat db/d12/d14/de5/da2/d4f/db6/d79/f114 x:0 0 0 2024-03-23T08:24:04.312 INFO:tasks.workunit.client.0.smithi070.stdout:9/816: stat db/d12/d14 0 2024-03-23T08:24:04.313 INFO:tasks.workunit.client.0.smithi070.stdout:0/912: symlink d16/d22/d43/d47/l13f 0 2024-03-23T08:24:04.314 INFO:tasks.workunit.client.0.smithi070.stdout:0/913: dread - d16/d26/d27/d86/d9e/f11f zero size 2024-03-23T08:24:04.316 INFO:tasks.workunit.client.0.smithi070.stdout:0/914: write d16/d22/d43/d47/fd6 [3844336,18042] 0 2024-03-23T08:24:04.320 INFO:tasks.workunit.client.0.smithi070.stdout:7/661: getdents d25/d27/dc2 0 2024-03-23T08:24:04.321 INFO:tasks.workunit.client.0.smithi070.stdout:7/662: write d25/d27/f4c [2702766,59152] 0 2024-03-23T08:24:04.327 INFO:tasks.workunit.client.0.smithi070.stdout:7/663: mknod d25/d27/d6a/d44/da2/db5/ceb 0 2024-03-23T08:24:04.335 INFO:tasks.workunit.client.0.smithi070.stdout:9/817: dwrite db/d12/d14/de5/da2/d4f/db6/d79/f9d [0,4194304] 0 2024-03-23T08:24:04.339 INFO:tasks.workunit.client.0.smithi070.stdout:7/664: creat d25/d27/d6a/fec x:0 0 0 2024-03-23T08:24:04.340 INFO:tasks.workunit.client.0.smithi070.stdout:7/665: chown d25/d27/d6a/d44/d6f/da3/fa7 1 1 2024-03-23T08:24:04.342 INFO:tasks.workunit.client.0.smithi070.stdout:9/818: mkdir db/d12/d115 0 2024-03-23T08:24:04.343 INFO:tasks.workunit.client.0.smithi070.stdout:0/915: getdents d16/d26/d27/d3d/d4d 0 2024-03-23T08:24:04.347 INFO:tasks.workunit.client.0.smithi070.stdout:7/666: mknod d25/d27/d6a/d44/dc8/ced 0 2024-03-23T08:24:04.347 INFO:tasks.workunit.client.0.smithi070.stdout:9/819: dread - db/d12/d34/d47/fa3 zero size 2024-03-23T08:24:04.349 INFO:tasks.workunit.client.0.smithi070.stdout:7/667: write d25/d27/d6a/d44/d85/f98 [225356,111960] 0 2024-03-23T08:24:04.350 INFO:tasks.workunit.client.0.smithi070.stdout:9/820: mkdir db/db8/dd8/d116 0 2024-03-23T08:24:04.352 INFO:tasks.workunit.client.0.smithi070.stdout:7/668: creat d25/d27/dc2/fee x:0 0 0 2024-03-23T08:24:04.356 INFO:tasks.workunit.client.0.smithi070.stdout:0/916: rename d16/d26/d27/d3d/d4d/d9a to d16/d26/d27/d86/d140 0 2024-03-23T08:24:04.360 INFO:tasks.workunit.client.0.smithi070.stdout:0/917: symlink d16/d26/d27/d86/def/l141 0 2024-03-23T08:24:04.361 INFO:tasks.workunit.client.0.smithi070.stdout:7/669: dread d25/d40/f61 [0,4194304] 0 2024-03-23T08:24:04.364 INFO:tasks.workunit.client.0.smithi070.stdout:9/821: getdents db/d12/d14 0 2024-03-23T08:24:04.366 INFO:tasks.workunit.client.0.smithi070.stdout:0/918: mkdir d16/d22/d43/d47/da9/de7/d142 0 2024-03-23T08:24:04.367 INFO:tasks.workunit.client.0.smithi070.stdout:0/919: chown d16/d22/d43/d47/d87/fcb 87124894 1 2024-03-23T08:24:04.367 INFO:tasks.workunit.client.0.smithi070.stdout:7/670: creat d25/d27/d6a/d44/d6f/dab/dcd/ddc/fef x:0 0 0 2024-03-23T08:24:04.371 INFO:tasks.workunit.client.0.smithi070.stdout:9/822: chown db/d12/d14/de5/da2/dc6/d7e/f74 845997 1 2024-03-23T08:24:04.376 INFO:tasks.workunit.client.0.smithi070.stdout:7/671: symlink d25/d27/d6a/d44/d6f/dab/dcd/ddc/lf0 0 2024-03-23T08:24:04.380 INFO:tasks.workunit.client.0.smithi070.stdout:7/672: mkdir d25/d27/d6a/d44/d6f/dab/dcd/df1 0 2024-03-23T08:24:04.383 INFO:tasks.workunit.client.0.smithi070.stdout:0/920: rename d16/d26/d27/d86/d9e/ddd/l12a to d16/d26/d27/l143 0 2024-03-23T08:24:04.384 INFO:tasks.workunit.client.0.smithi070.stdout:7/673: mkdir d25/d27/d6a/d44/d6f/dcc/df2 0 2024-03-23T08:24:04.388 INFO:tasks.workunit.client.0.smithi070.stdout:7/674: link d25/d27/d6a/c6b d25/d27/d6a/d44/d7c/db4/cf3 0 2024-03-23T08:24:04.390 INFO:tasks.workunit.client.0.smithi070.stdout:7/675: mkdir d25/d27/d6a/d44/da2/df4 0 2024-03-23T08:24:04.392 INFO:tasks.workunit.client.0.smithi070.stdout:7/676: rename d25/d27/d6a/d44/d85/fad to d25/d27/d6a/d44/da2/df4/ff5 0 2024-03-23T08:24:04.393 INFO:tasks.workunit.client.0.smithi070.stdout:7/677: dread - d25/d27/d6a/d44/d85/d95/da0/fb2 zero size 2024-03-23T08:24:04.394 INFO:tasks.workunit.client.0.smithi070.stdout:7/678: chown d25/d40/f4d 5 1 2024-03-23T08:24:04.400 INFO:tasks.workunit.client.0.smithi070.stdout:7/679: dwrite d25/d27/d6a/d44/d85/faf [0,4194304] 0 2024-03-23T08:24:04.401 INFO:tasks.workunit.client.0.smithi070.stdout:7/680: dread - d25/d27/d6a/d44/fa8 zero size 2024-03-23T08:24:04.417 INFO:tasks.workunit.client.0.smithi070.stdout:7/681: dwrite d25/d27/f3c [0,4194304] 0 2024-03-23T08:24:04.429 INFO:tasks.workunit.client.0.smithi070.stdout:7/682: dwrite d25/d27/d6a/d44/fa8 [0,4194304] 0 2024-03-23T08:24:04.436 INFO:tasks.workunit.client.0.smithi070.stdout:7/683: dwrite d25/d27/d6a/d44/d85/d95/da0/fb2 [0,4194304] 0 2024-03-23T08:24:04.438 INFO:tasks.workunit.client.0.smithi070.stdout:7/684: chown d25/d27 128 1 2024-03-23T08:24:04.438 INFO:tasks.workunit.client.0.smithi070.stdout:7/685: stat d25/d27/d6a/l3d 0 2024-03-23T08:24:04.445 INFO:tasks.workunit.client.0.smithi070.stdout:7/686: read d25/d27/d6a/d44/d85/f93 [221844,125825] 0 2024-03-23T08:24:04.449 INFO:tasks.workunit.client.0.smithi070.stdout:7/687: creat d25/d27/d6a/d44/d6f/ff6 x:0 0 0 2024-03-23T08:24:04.452 INFO:tasks.workunit.client.0.smithi070.stdout:7/688: dread d25/d27/d6a/d44/d6f/f71 [0,4194304] 0 2024-03-23T08:24:04.454 INFO:tasks.workunit.client.0.smithi070.stdout:7/689: truncate d25/d27/d6a/d44/d7c/db4/fe0 225612 0 2024-03-23T08:24:04.456 INFO:tasks.workunit.client.0.smithi070.stdout:7/690: fsync d25/d40/f61 0 2024-03-23T08:24:04.459 INFO:tasks.workunit.client.0.smithi070.stdout:7/691: fsync d25/d27/d6a/d44/d7c/db4/fb7 0 2024-03-23T08:24:04.465 INFO:tasks.workunit.client.0.smithi070.stdout:7/692: write d25/f34 [3162899,100772] 0 2024-03-23T08:24:04.469 INFO:tasks.workunit.client.0.smithi070.stdout:7/693: symlink d25/d27/d6a/d44/da4/lf7 0 2024-03-23T08:24:04.473 INFO:tasks.workunit.client.0.smithi070.stdout:7/694: getdents d25/d27/d6a/d44/d6f/dab/dcd 0 2024-03-23T08:24:04.474 INFO:tasks.workunit.client.0.smithi070.stdout:7/695: write d25/d27/d6a/d44/d7c/fd0 [461772,118234] 0 2024-03-23T08:24:04.475 INFO:tasks.workunit.client.0.smithi070.stdout:7/696: write d25/d27/d6a/d44/f90 [577168,38223] 0 2024-03-23T08:24:04.478 INFO:tasks.workunit.client.0.smithi070.stdout:7/697: mkdir d25/d27/dc2/df8 0 2024-03-23T08:24:04.480 INFO:tasks.workunit.client.0.smithi070.stdout:7/698: dread - d25/d40/fd8 zero size 2024-03-23T08:24:04.482 INFO:tasks.workunit.client.0.smithi070.stdout:7/699: mkdir d25/d27/d6a/d44/df9 0 2024-03-23T08:24:04.483 INFO:tasks.workunit.client.0.smithi070.stdout:7/700: write d25/fb8 [68758,75918] 0 2024-03-23T08:24:04.484 INFO:tasks.workunit.client.0.smithi070.stdout:7/701: dread d25/d27/d6a/d44/d6f/f71 [0,4194304] 0 2024-03-23T08:24:04.486 INFO:tasks.workunit.client.0.smithi070.stdout:7/702: truncate d25/f43 796849 0 2024-03-23T08:24:04.490 INFO:tasks.workunit.client.0.smithi070.stdout:7/703: dread d25/d27/d6a/d44/f90 [0,4194304] 0 2024-03-23T08:24:04.492 INFO:tasks.workunit.client.0.smithi070.stdout:7/704: fdatasync d25/d27/d6a/d44/f79 0 2024-03-23T08:24:04.495 INFO:tasks.workunit.client.0.smithi070.stdout:7/705: creat d25/d27/d6a/d44/d6f/dab/ffa x:0 0 0 2024-03-23T08:24:04.499 INFO:tasks.workunit.client.0.smithi070.stdout:7/706: dread d25/d27/d6a/d44/d7c/fd0 [0,4194304] 0 2024-03-23T08:24:04.500 INFO:tasks.workunit.client.0.smithi070.stdout:7/707: write d25/f38 [7352359,106821] 0 2024-03-23T08:24:04.511 INFO:tasks.workunit.client.0.smithi070.stdout:7/708: dwrite d25/d27/d6a/d44/f54 [0,4194304] 0 2024-03-23T08:24:04.512 INFO:tasks.workunit.client.0.smithi070.stdout:6/830: sync 2024-03-23T08:24:04.512 INFO:tasks.workunit.client.0.smithi070.stdout:4/669: sync 2024-03-23T08:24:04.513 INFO:tasks.workunit.client.0.smithi070.stdout:1/669: sync 2024-03-23T08:24:04.514 INFO:tasks.workunit.client.0.smithi070.stdout:1/670: write d8/db/d40/d43/d64/d85/dc7/fcc [774089,118509] 0 2024-03-23T08:24:04.515 INFO:tasks.workunit.client.0.smithi070.stdout:7/709: unlink d25/d27/d6a/d44/f90 0 2024-03-23T08:24:04.520 INFO:tasks.workunit.client.0.smithi070.stdout:1/671: read d8/db/d2c/f3c [732732,71033] 0 2024-03-23T08:24:04.525 INFO:tasks.workunit.client.0.smithi070.stdout:1/672: rename d8/d26/d41/dd9/fea to d8/db/d8e/ff8 0 2024-03-23T08:24:04.530 INFO:tasks.workunit.client.0.smithi070.stdout:1/673: getdents d8/d11/d61/d74/d5d/d70 0 2024-03-23T08:24:04.533 INFO:tasks.workunit.client.0.smithi070.stdout:1/674: symlink d8/db/d40/d43/lf9 0 2024-03-23T08:24:04.534 INFO:tasks.workunit.client.0.smithi070.stdout:1/675: rmdir d8/d11 39 2024-03-23T08:24:04.535 INFO:tasks.workunit.client.0.smithi070.stdout:1/676: mknod d8/d42/d4e/cfa 0 2024-03-23T08:24:04.536 INFO:tasks.workunit.client.0.smithi070.stdout:1/677: chown d8/d1a/f47 556 1 2024-03-23T08:24:04.542 INFO:tasks.workunit.client.0.smithi070.stdout:6/831: dwrite d0/d23/d28/f37 [0,4194304] 0 2024-03-23T08:24:04.542 INFO:tasks.workunit.client.0.smithi070.stdout:7/710: dwrite d25/d27/d6a/d44/d7c/f8b [0,4194304] 0 2024-03-23T08:24:04.544 INFO:tasks.workunit.client.0.smithi070.stdout:1/678: write d8/f90 [384042,85374] 0 2024-03-23T08:24:04.548 INFO:tasks.workunit.client.0.smithi070.stdout:1/679: chown d8/d1a/f39 62436 1 2024-03-23T08:24:04.553 INFO:tasks.workunit.client.0.smithi070.stdout:1/680: read d8/db/d2c/d3f/d48/f58 [1431480,66005] 0 2024-03-23T08:24:04.553 INFO:tasks.workunit.client.0.smithi070.stdout:7/711: dread d25/d27/d6a/f5f [0,4194304] 0 2024-03-23T08:24:04.558 INFO:tasks.workunit.client.0.smithi070.stdout:6/832: write d0/fa7 [819069,13287] 0 2024-03-23T08:24:04.559 INFO:tasks.workunit.client.0.smithi070.stdout:1/681: mkdir d8/db/d40/d43/d64/dc2/dfb 0 2024-03-23T08:24:04.560 INFO:tasks.workunit.client.0.smithi070.stdout:7/712: mkdir d25/d27/d6a/d44/d7c/dfb 0 2024-03-23T08:24:04.562 INFO:tasks.workunit.client.0.smithi070.stdout:1/682: chown d8/d42/d51/lc0 51419 1 2024-03-23T08:24:04.565 INFO:tasks.workunit.client.0.smithi070.stdout:1/683: unlink d8/d1b/f86 0 2024-03-23T08:24:04.569 INFO:tasks.workunit.client.0.smithi070.stdout:7/713: write d25/d27/d6a/f7f [862227,32968] 0 2024-03-23T08:24:04.572 INFO:tasks.workunit.client.0.smithi070.stdout:1/684: dwrite d8/d11/f3b [0,4194304] 0 2024-03-23T08:24:04.577 INFO:tasks.workunit.client.0.smithi070.stdout:7/714: unlink d25/d40/l58 0 2024-03-23T08:24:04.577 INFO:tasks.workunit.client.0.smithi070.stdout:7/715: chown c10 16 1 2024-03-23T08:24:04.577 INFO:tasks.workunit.client.0.smithi070.stdout:1/685: truncate d8/db/d2c/d3f/f56 1228968 0 2024-03-23T08:24:04.579 INFO:tasks.workunit.client.0.smithi070.stdout:7/716: write d25/d27/d6a/d44/d7c/db4/fb6 [1740954,18845] 0 2024-03-23T08:24:04.586 INFO:tasks.workunit.client.0.smithi070.stdout:7/717: dwrite d25/d27/d6a/d44/f8c [0,4194304] 0 2024-03-23T08:24:04.590 INFO:tasks.workunit.client.0.smithi070.stdout:1/686: write d8/d1a/f1e [3313191,122008] 0 2024-03-23T08:24:04.591 INFO:tasks.workunit.client.0.smithi070.stdout:7/718: creat d25/d27/d6a/d44/d85/d95/da0/de2/ffc x:0 0 0 2024-03-23T08:24:04.597 INFO:tasks.workunit.client.0.smithi070.stdout:7/719: unlink d25/f43 0 2024-03-23T08:24:04.604 INFO:tasks.workunit.client.0.smithi070.stdout:7/720: mkdir d25/d27/d6a/d44/d7c/dfb/dfd 0 2024-03-23T08:24:04.604 INFO:tasks.workunit.client.0.smithi070.stdout:7/721: chown d25/f87 44 1 2024-03-23T08:24:04.605 INFO:tasks.workunit.client.0.smithi070.stdout:7/722: write d25/d27/d6a/fda [2694227,24015] 0 2024-03-23T08:24:04.607 INFO:tasks.workunit.client.0.smithi070.stdout:1/687: stat d8/db/d2c/d3f/f56 0 2024-03-23T08:24:04.608 INFO:tasks.workunit.client.0.smithi070.stdout:1/688: stat d8/d42/d51/cb9 0 2024-03-23T08:24:04.612 INFO:tasks.workunit.client.0.smithi070.stdout:1/689: mknod d8/d11/d61/d74/cfc 0 2024-03-23T08:24:04.616 INFO:tasks.workunit.client.0.smithi070.stdout:7/723: write d25/d27/d6a/d44/d85/f93 [955038,11659] 0 2024-03-23T08:24:04.620 INFO:tasks.workunit.client.0.smithi070.stdout:7/724: mkdir d25/d27/d6a/d44/dfe 0 2024-03-23T08:24:04.622 INFO:tasks.workunit.client.0.smithi070.stdout:1/690: dread d8/db/d2c/d3f/d48/f58 [0,4194304] 0 2024-03-23T08:24:04.625 INFO:tasks.workunit.client.0.smithi070.stdout:7/725: creat d25/d27/d6a/d44/d6f/dab/dcd/df1/fff x:0 0 0 2024-03-23T08:24:04.626 INFO:tasks.workunit.client.0.smithi070.stdout:1/691: fdatasync d8/d11/d61/d74/d5d/d70/fa8 0 2024-03-23T08:24:04.628 INFO:tasks.workunit.client.0.smithi070.stdout:1/692: fsync d8/db/d40/d43/f54 0 2024-03-23T08:24:04.633 INFO:tasks.workunit.client.0.smithi070.stdout:7/726: getdents d25/d27/d6a/d44/d7c 0 2024-03-23T08:24:04.636 INFO:tasks.workunit.client.0.smithi070.stdout:7/727: symlink d25/d27/d6a/d44/d6f/da3/de1/l100 0 2024-03-23T08:24:04.641 INFO:tasks.workunit.client.0.smithi070.stdout:1/693: dread d8/d1a/f39 [4194304,4194304] 0 2024-03-23T08:24:04.641 INFO:tasks.workunit.client.0.smithi070.stdout:1/694: chown d8/db/l17 1282257114 1 2024-03-23T08:24:04.642 INFO:tasks.workunit.client.0.smithi070.stdout:1/695: chown d8/db/d2c/d3f/f6e 3688146 1 2024-03-23T08:24:04.645 INFO:tasks.workunit.client.0.smithi070.stdout:1/696: mkdir d8/db/d40/d43/d64/d85/d97/dfd 0 2024-03-23T08:24:04.647 INFO:tasks.workunit.client.0.smithi070.stdout:1/697: rename d8/d1b/l35 to d8/d26/d41/lfe 0 2024-03-23T08:24:04.649 INFO:tasks.workunit.client.0.smithi070.stdout:1/698: creat d8/db/d40/d43/d64/d85/d97/fff x:0 0 0 2024-03-23T08:24:04.650 INFO:tasks.workunit.client.0.smithi070.stdout:1/699: stat d8 0 2024-03-23T08:24:04.654 INFO:tasks.workunit.client.0.smithi070.stdout:1/700: rename d8/d11/d61 to d8/db/d100 0 2024-03-23T08:24:04.656 INFO:tasks.workunit.client.0.smithi070.stdout:1/701: truncate d8/db/d40/d43/feb 1389597 0 2024-03-23T08:24:04.657 INFO:tasks.workunit.client.0.smithi070.stdout:1/702: chown d8/f90 13965642 1 2024-03-23T08:24:04.675 INFO:tasks.workunit.client.0.smithi070.stdout:1/703: dwrite d8/db/f27 [0,4194304] 0 2024-03-23T08:24:04.704 INFO:tasks.workunit.client.0.smithi070.stdout:1/704: dread d8/d42/d4e/d99/fda [0,4194304] 0 2024-03-23T08:24:04.710 INFO:tasks.workunit.client.0.smithi070.stdout:1/705: dwrite f3 [0,4194304] 0 2024-03-23T08:24:04.715 INFO:tasks.workunit.client.0.smithi070.stdout:1/706: dread d8/db/d40/d43/f6c [0,4194304] 0 2024-03-23T08:24:04.715 INFO:tasks.workunit.client.0.smithi070.stdout:1/707: chown d8/d26/d41/dd9 119476 1 2024-03-23T08:24:04.722 INFO:tasks.workunit.client.0.smithi070.stdout:1/708: dwrite d8/d26/d7b/fe0 [0,4194304] 0 2024-03-23T08:24:04.731 INFO:tasks.workunit.client.0.smithi070.stdout:3/812: sync 2024-03-23T08:24:04.731 INFO:tasks.workunit.client.0.smithi070.stdout:3/813: chown db/d20/d49/c7b 813 1 2024-03-23T08:24:04.734 INFO:tasks.workunit.client.0.smithi070.stdout:3/814: chown db/df/d1a/ff4 2503360 1 2024-03-23T08:24:04.737 INFO:tasks.workunit.client.0.smithi070.stdout:3/815: mknod db/d20/d7a/d80/ded/c12d 0 2024-03-23T08:24:04.743 INFO:tasks.workunit.client.0.smithi070.stdout:3/816: dwrite db/d14/d65/f85 [0,4194304] 0 2024-03-23T08:24:04.744 INFO:tasks.workunit.client.0.smithi070.stdout:3/817: chown db/d20/d49 0 1 2024-03-23T08:24:04.749 INFO:tasks.workunit.client.0.smithi070.stdout:1/709: dread d8/d26/d41/fae [0,4194304] 0 2024-03-23T08:24:04.752 INFO:tasks.workunit.client.0.smithi070.stdout:1/710: fdatasync d8/d42/d4e/f66 0 2024-03-23T08:24:04.763 INFO:tasks.workunit.client.0.smithi070.stdout:1/711: symlink d8/db/d100/d74/l101 0 2024-03-23T08:24:04.766 INFO:tasks.workunit.client.0.smithi070.stdout:1/712: mknod d8/d42/d4e/d99/c102 0 2024-03-23T08:24:04.767 INFO:tasks.workunit.client.0.smithi070.stdout:3/818: dwrite db/d4e/d92/f101 [0,4194304] 0 2024-03-23T08:24:04.771 INFO:tasks.workunit.client.0.smithi070.stdout:1/713: symlink d8/db/d40/d43/d64/dc2/l103 0 2024-03-23T08:24:04.774 INFO:tasks.workunit.client.0.smithi070.stdout:1/714: fsync d8/db/d40/d43/d64/d85/fb6 0 2024-03-23T08:24:04.775 INFO:tasks.workunit.client.0.smithi070.stdout:1/715: dread - d8/d42/d4e/d99/ff6 zero size 2024-03-23T08:24:04.780 INFO:tasks.workunit.client.0.smithi070.stdout:1/716: symlink d8/l104 0 2024-03-23T08:24:04.787 INFO:tasks.workunit.client.0.smithi070.stdout:1/717: link d8/db/d2c/c30 d8/db/d40/d43/d64/d85/c105 0 2024-03-23T08:24:04.787 INFO:tasks.workunit.client.0.smithi070.stdout:1/718: chown d8/db/d100/d74/d5d/f87 0 1 2024-03-23T08:24:04.793 INFO:tasks.workunit.client.0.smithi070.stdout:3/819: dread db/df/fec [0,4194304] 0 2024-03-23T08:24:04.800 INFO:tasks.workunit.client.0.smithi070.stdout:3/820: link db/d14/d43/d7d/l10b db/d20/d63/d83/l12e 0 2024-03-23T08:24:04.802 INFO:tasks.workunit.client.0.smithi070.stdout:3/821: mkdir db/d4e/dd6/dd7/dd9/d12f 0 2024-03-23T08:24:04.805 INFO:tasks.workunit.client.0.smithi070.stdout:3/822: fsync db/d20/fee 0 2024-03-23T08:24:04.806 INFO:tasks.workunit.client.0.smithi070.stdout:3/823: chown db/d14/d43/d62/d95/l96 5886145 1 2024-03-23T08:24:04.809 INFO:tasks.workunit.client.0.smithi070.stdout:3/824: mkdir db/d4e/dd6/d130 0 2024-03-23T08:24:04.812 INFO:tasks.workunit.client.0.smithi070.stdout:3/825: symlink db/d4e/d92/l131 0 2024-03-23T08:24:04.814 INFO:tasks.workunit.client.0.smithi070.stdout:3/826: creat db/d7f/f132 x:0 0 0 2024-03-23T08:24:04.817 INFO:tasks.workunit.client.0.smithi070.stdout:3/827: mknod db/d4e/df8/dca/c133 0 2024-03-23T08:24:04.822 INFO:tasks.workunit.client.0.smithi070.stdout:1/719: truncate d8/d1a/fe3 2253367 0 2024-03-23T08:24:04.824 INFO:tasks.workunit.client.0.smithi070.stdout:1/720: mknod d8/db/c106 0 2024-03-23T08:24:04.852 INFO:tasks.workunit.client.0.smithi070.stdout:3/828: dwrite db/d4e/dd6/dd7/dd9/f104 [0,4194304] 0 2024-03-23T08:24:04.854 INFO:tasks.workunit.client.0.smithi070.stdout:3/829: chown db/d4e/dc4/d10c 29 1 2024-03-23T08:24:04.969 INFO:tasks.workunit.client.0.smithi070.stdout:2/929: sync 2024-03-23T08:24:04.977 INFO:tasks.workunit.client.0.smithi070.stdout:2/930: truncate d1/d29/f35 2552553 0 2024-03-23T08:24:04.992 INFO:tasks.workunit.client.0.smithi070.stdout:2/931: dread d1/d7/df/f91 [0,4194304] 0 2024-03-23T08:24:04.995 INFO:tasks.workunit.client.0.smithi070.stdout:2/932: fsync d1/d7/f36 0 2024-03-23T08:24:05.011 INFO:tasks.workunit.client.0.smithi070.stdout:2/933: dread d1/d7/f7e [0,4194304] 0 2024-03-23T08:24:05.013 INFO:tasks.workunit.client.0.smithi070.stdout:2/934: mknod d1/d118/d103/c13c 0 2024-03-23T08:24:05.015 INFO:tasks.workunit.client.0.smithi070.stdout:2/935: unlink d1/d7/d48/d5b/cd7 0 2024-03-23T08:24:05.018 INFO:tasks.workunit.client.0.smithi070.stdout:2/936: mkdir d1/d7/d39/d47/d90/d13d 0 2024-03-23T08:24:05.020 INFO:tasks.workunit.client.0.smithi070.stdout:2/937: rmdir d1/d118/dc0 39 2024-03-23T08:24:05.026 INFO:tasks.workunit.client.0.smithi070.stdout:2/938: rename d1/d7/d39/d58/ff4 to d1/f13e 0 2024-03-23T08:24:05.029 INFO:tasks.workunit.client.0.smithi070.stdout:2/939: fdatasync d1/d7/df/d99/dea/dc3/f3c 0 2024-03-23T08:24:05.035 INFO:tasks.workunit.client.0.smithi070.stdout:2/940: dread d1/d7/df/d94/f28 [4194304,4194304] 0 2024-03-23T08:24:05.045 INFO:tasks.workunit.client.0.smithi070.stdout:2/941: dwrite d1/d21/d5f/fa8 [0,4194304] 0 2024-03-23T08:24:05.049 INFO:tasks.workunit.client.0.smithi070.stdout:2/942: truncate d1/d7/d48/d69/db8/f110 111395 0 2024-03-23T08:24:05.056 INFO:tasks.workunit.client.0.smithi070.stdout:2/943: dwrite d1/d21/fa9 [0,4194304] 0 2024-03-23T08:24:05.080 INFO:tasks.workunit.client.0.smithi070.stdout:2/944: dread d1/d7/df/d94/fb0 [0,4194304] 0 2024-03-23T08:24:05.094 INFO:tasks.workunit.client.0.smithi070.stdout:2/945: dwrite d1/d7/d39/fe3 [0,4194304] 0 2024-03-23T08:24:05.097 INFO:tasks.workunit.client.0.smithi070.stdout:2/946: write d1/d118/d34/da2/f131 [658039,67378] 0 2024-03-23T08:24:05.118 INFO:tasks.workunit.client.0.smithi070.stdout:2/947: dwrite d1/d7/df/d99/dea/dc3/f3c [0,4194304] 0 2024-03-23T08:24:05.132 INFO:tasks.workunit.client.0.smithi070.stdout:2/948: write d1/d7/d48/d69/d7a/fec [1064080,93201] 0 2024-03-23T08:24:05.133 INFO:tasks.workunit.client.0.smithi070.stdout:2/949: chown d1/d118/d34/da2 2963 1 2024-03-23T08:24:05.134 INFO:tasks.workunit.client.0.smithi070.stdout:2/950: symlink d1/d98/l13f 0 2024-03-23T08:24:05.137 INFO:tasks.workunit.client.0.smithi070.stdout:2/951: mkdir d1/d7/df/d99/dea/d140 0 2024-03-23T08:24:05.138 INFO:tasks.workunit.client.0.smithi070.stdout:2/952: chown d1/d7/f40 37 1 2024-03-23T08:24:05.138 INFO:tasks.workunit.client.0.smithi070.stdout:2/953: readlink d1/d7/d48/d69/d7a/l8a 0 2024-03-23T08:24:05.141 INFO:tasks.workunit.client.0.smithi070.stdout:2/954: mknod d1/d118/dc0/c141 0 2024-03-23T08:24:05.155 INFO:tasks.workunit.client.0.smithi070.stdout:9/823: sync 2024-03-23T08:24:05.155 INFO:tasks.workunit.client.0.smithi070.stdout:8/749: sync 2024-03-23T08:24:05.155 INFO:tasks.workunit.client.0.smithi070.stdout:0/921: sync 2024-03-23T08:24:05.155 INFO:tasks.workunit.client.0.smithi070.stdout:4/670: sync 2024-03-23T08:24:05.155 INFO:tasks.workunit.client.0.smithi070.stdout:5/659: sync 2024-03-23T08:24:05.155 INFO:tasks.workunit.client.0.smithi070.stdout:6/833: sync 2024-03-23T08:24:05.157 INFO:tasks.workunit.client.0.smithi070.stdout:4/671: chown d3/d6/d35/c83 251305913 1 2024-03-23T08:24:05.158 INFO:tasks.workunit.client.0.smithi070.stdout:9/824: chown db/d12/d34/d47/fa3 1888 1 2024-03-23T08:24:05.159 INFO:tasks.workunit.client.0.smithi070.stdout:4/672: chown d3/d6/d35/d3f/d51/d6e/d97/ddf/cbc 0 1 2024-03-23T08:24:05.164 INFO:tasks.workunit.client.0.smithi070.stdout:6/834: read d0/d23/d74/d84/d3c/f60 [572759,114373] 0 2024-03-23T08:24:05.165 INFO:tasks.workunit.client.0.smithi070.stdout:8/750: dread - d5/d8/d16/d32/d84/d98/dde/da3/dab/fad zero size 2024-03-23T08:24:05.168 INFO:tasks.workunit.client.0.smithi070.stdout:2/955: dwrite d1/d7/d48/d69/db8/dc7/d111/f3a [0,4194304] 0 2024-03-23T08:24:05.174 INFO:tasks.workunit.client.0.smithi070.stdout:0/922: write d16/d22/d43/f56 [5620617,93336] 0 2024-03-23T08:24:05.176 INFO:tasks.workunit.client.0.smithi070.stdout:6/835: mknod d0/d23/d88/c10a 0 2024-03-23T08:24:05.178 INFO:tasks.workunit.client.0.smithi070.stdout:8/751: mknod d5/d8/d16/dd4/d6f/d7c/d7d/c104 0 2024-03-23T08:24:05.181 INFO:tasks.workunit.client.0.smithi070.stdout:5/660: dwrite dc/d1b/d1d/d25/d38/d39/f79 [0,4194304] 0 2024-03-23T08:24:05.184 INFO:tasks.workunit.client.0.smithi070.stdout:4/673: mkdir d3/d6/d35/d65/df1 0 2024-03-23T08:24:05.186 INFO:tasks.workunit.client.0.smithi070.stdout:9/825: creat db/d12/d14/de5/da2/f117 x:0 0 0 2024-03-23T08:24:05.186 INFO:tasks.workunit.client.0.smithi070.stdout:0/923: chown d16/d22/d43/d47/d87/l89 245391639 1 2024-03-23T08:24:05.189 INFO:tasks.workunit.client.0.smithi070.stdout:5/661: creat dc/d1b/d1d/d25/d38/d39/fd5 x:0 0 0 2024-03-23T08:24:05.191 INFO:tasks.workunit.client.0.smithi070.stdout:6/836: symlink d0/d23/d74/l10b 0 2024-03-23T08:24:05.193 INFO:tasks.workunit.client.0.smithi070.stdout:0/924: rmdir d16/d26/d27/d3d/d4d/dca 39 2024-03-23T08:24:05.197 INFO:tasks.workunit.client.0.smithi070.stdout:5/662: creat dc/d1b/d1d/d25/d38/fd6 x:0 0 0 2024-03-23T08:24:05.198 INFO:tasks.workunit.client.0.smithi070.stdout:9/826: truncate db/d12/f9a 3260619 0 2024-03-23T08:24:05.200 INFO:tasks.workunit.client.0.smithi070.stdout:8/752: dwrite d5/d8/d9a/da2/ff1 [0,4194304] 0 2024-03-23T08:24:05.201 INFO:tasks.workunit.client.0.smithi070.stdout:6/837: mknod d0/d23/d74/d84/db3/d38/d9a/c10c 0 2024-03-23T08:24:05.203 INFO:tasks.workunit.client.0.smithi070.stdout:2/956: dwrite d1/d29/d52/f113 [0,4194304] 0 2024-03-23T08:24:05.206 INFO:tasks.workunit.client.0.smithi070.stdout:4/674: dread d3/f17 [0,4194304] 0 2024-03-23T08:24:05.208 INFO:tasks.workunit.client.0.smithi070.stdout:9/827: write db/d12/d14/de5/da2/d4f/db6/d79/f114 [644311,7825] 0 2024-03-23T08:24:05.208 INFO:tasks.workunit.client.0.smithi070.stdout:5/663: unlink dc/d1b/d1d/d25/c4a 0 2024-03-23T08:24:05.217 INFO:tasks.workunit.client.0.smithi070.stdout:5/664: fsync dc/d1b/d1d/d25/d36/d32/d5c/fab 0 2024-03-23T08:24:05.219 INFO:tasks.workunit.client.0.smithi070.stdout:0/925: dwrite d16/d26/d27/d86/d67/d109/f133 [0,4194304] 0 2024-03-23T08:24:05.220 INFO:tasks.workunit.client.0.smithi070.stdout:4/675: unlink d3/d6/d35/c68 0 2024-03-23T08:24:05.222 INFO:tasks.workunit.client.0.smithi070.stdout:8/753: dwrite d5/d8/d9a/da2/dae/f94 [8388608,4194304] 0 2024-03-23T08:24:05.223 INFO:tasks.workunit.client.0.smithi070.stdout:0/926: read - d16/d26/d27/d86/d67/f13b zero size 2024-03-23T08:24:05.230 INFO:tasks.workunit.client.0.smithi070.stdout:6/838: truncate d0/d23/d74/d84/faa 3976143 0 2024-03-23T08:24:05.234 INFO:tasks.workunit.client.0.smithi070.stdout:9/828: dread db/d12/d14/de5/da2/dc6/dce/d85/d42/f5f [0,4194304] 0 2024-03-23T08:24:05.236 INFO:tasks.workunit.client.0.smithi070.stdout:5/665: mkdir dc/d1b/d1d/d25/d68/d8a/d99/dd7 0 2024-03-23T08:24:05.241 INFO:tasks.workunit.client.0.smithi070.stdout:6/839: dwrite d0/d23/d74/d84/db3/d2f/fe5 [0,4194304] 0 2024-03-23T08:24:05.244 INFO:tasks.workunit.client.0.smithi070.stdout:6/840: chown d0/d23/d97/d98/lb7 3 1 2024-03-23T08:24:05.248 INFO:tasks.workunit.client.0.smithi070.stdout:8/754: unlink d5/d8/d16/d30/f3e 0 2024-03-23T08:24:05.249 INFO:tasks.workunit.client.0.smithi070.stdout:4/676: write d3/d6/d9/f2e [234953,9788] 0 2024-03-23T08:24:05.251 INFO:tasks.workunit.client.0.smithi070.stdout:0/927: link d16/d26/d27/d4c/d60/d75/f9c d16/d26/d27/d3d/d4d/dc9/f144 0 2024-03-23T08:24:05.254 INFO:tasks.workunit.client.0.smithi070.stdout:6/841: mkdir d0/d23/d74/d84/d3c/d86/d10d 0 2024-03-23T08:24:05.259 INFO:tasks.workunit.client.0.smithi070.stdout:5/666: creat dc/d1b/fd8 x:0 0 0 2024-03-23T08:24:05.260 INFO:tasks.workunit.client.0.smithi070.stdout:9/829: dwrite db/d12/f3a [0,4194304] 0 2024-03-23T08:24:05.263 INFO:tasks.workunit.client.0.smithi070.stdout:6/842: creat d0/d23/d4d/da5/f10e x:0 0 0 2024-03-23T08:24:05.267 INFO:tasks.workunit.client.0.smithi070.stdout:2/957: dread d1/fd [0,4194304] 0 2024-03-23T08:24:05.269 INFO:tasks.workunit.client.0.smithi070.stdout:6/843: dread - d0/d23/d28/d35/d6a/fce zero size 2024-03-23T08:24:05.272 INFO:tasks.workunit.client.0.smithi070.stdout:4/677: dread d3/d6/d35/d3f/d51/d6e/f99 [0,4194304] 0 2024-03-23T08:24:05.283 INFO:tasks.workunit.client.0.smithi070.stdout:2/958: write d1/d7/f7e [5169431,48667] 0 2024-03-23T08:24:05.290 INFO:tasks.workunit.client.0.smithi070.stdout:2/959: dread - d1/d7/df/d99/f11b zero size 2024-03-23T08:24:05.290 INFO:tasks.workunit.client.0.smithi070.stdout:8/755: dwrite d5/d8/d16/dd4/fcb [0,4194304] 0 2024-03-23T08:24:05.292 INFO:tasks.workunit.client.0.smithi070.stdout:9/830: dwrite f4 [0,4194304] 0 2024-03-23T08:24:05.297 INFO:tasks.workunit.client.0.smithi070.stdout:2/960: symlink d1/d21/d5f/l142 0 2024-03-23T08:24:05.299 INFO:tasks.workunit.client.0.smithi070.stdout:5/667: dwrite dc/d1b/d1d/f3a [0,4194304] 0 2024-03-23T08:24:05.300 INFO:tasks.workunit.client.0.smithi070.stdout:6/844: dwrite d0/d23/d28/d35/d85/f8b [0,4194304] 0 2024-03-23T08:24:05.311 INFO:tasks.workunit.client.0.smithi070.stdout:8/756: creat d5/d8/d16/d30/d38/d6b/d72/d88/dbf/f105 x:0 0 0 2024-03-23T08:24:05.312 INFO:tasks.workunit.client.0.smithi070.stdout:9/831: truncate db/d12/d14/de5/da2/f7c 889575 0 2024-03-23T08:24:05.312 INFO:tasks.workunit.client.0.smithi070.stdout:2/961: getdents d1/d29/d52/de4/d10a 0 2024-03-23T08:24:05.314 INFO:tasks.workunit.client.0.smithi070.stdout:4/678: dread d3/d6/d35/d65/d27/d4b/fe0 [0,4194304] 0 2024-03-23T08:24:05.315 INFO:tasks.workunit.client.0.smithi070.stdout:5/668: rename dc/d1b/d1d/d25/d36/l75 to dc/d1b/d1d/d25/d36/ld9 0 2024-03-23T08:24:05.316 INFO:tasks.workunit.client.0.smithi070.stdout:8/757: creat d5/d8/d9a/f106 x:0 0 0 2024-03-23T08:24:05.316 INFO:tasks.workunit.client.0.smithi070.stdout:4/679: readlink d3/d6/d35/d65/d27/d4b/le8 0 2024-03-23T08:24:05.318 INFO:tasks.workunit.client.0.smithi070.stdout:2/962: readlink d1/d29/d52/de4/lfc 0 2024-03-23T08:24:05.318 INFO:tasks.workunit.client.0.smithi070.stdout:9/832: dread - db/d12/d14/de5/da2/d4f/db6/fc4 zero size 2024-03-23T08:24:05.319 INFO:tasks.workunit.client.0.smithi070.stdout:2/963: truncate d1/d118/d34/da2/f131 1130829 0 2024-03-23T08:24:05.325 INFO:tasks.workunit.client.0.smithi070.stdout:9/833: rename db/de8 to db/d12/d115/d118 0 2024-03-23T08:24:05.326 INFO:tasks.workunit.client.0.smithi070.stdout:6/845: dwrite d0/d3/fd0 [0,4194304] 0 2024-03-23T08:24:05.326 INFO:tasks.workunit.client.0.smithi070.stdout:5/669: mknod dc/d1b/d1d/d25/d68/cda 0 2024-03-23T08:24:05.327 INFO:tasks.workunit.client.0.smithi070.stdout:9/834: chown db/d12/d14/de5/da2/d26/da7/fbc 420 1 2024-03-23T08:24:05.335 INFO:tasks.workunit.client.0.smithi070.stdout:6/846: dwrite d0/d23/d74/d84/db3/d2f/fe4 [0,4194304] 0 2024-03-23T08:24:05.336 INFO:tasks.workunit.client.0.smithi070.stdout:6/847: truncate d0/fa7 1066355 0 2024-03-23T08:24:05.337 INFO:tasks.workunit.client.0.smithi070.stdout:5/670: mknod dc/d1b/d1d/d25/da9/cdb 0 2024-03-23T08:24:05.340 INFO:tasks.workunit.client.0.smithi070.stdout:9/835: rename db/d12/d14/de5/da2/c20 to db/d12/d14/de5/da2/d26/da7/df3/c119 0 2024-03-23T08:24:05.340 INFO:tasks.workunit.client.0.smithi070.stdout:6/848: dread d0/d23/d28/d35/f77 [0,4194304] 0 2024-03-23T08:24:05.342 INFO:tasks.workunit.client.0.smithi070.stdout:6/849: stat d0/d23/d88/db6/fe9 0 2024-03-23T08:24:05.345 INFO:tasks.workunit.client.0.smithi070.stdout:4/680: dwrite d3/d6/d6a/f58 [0,4194304] 0 2024-03-23T08:24:05.352 INFO:tasks.workunit.client.0.smithi070.stdout:9/836: unlink db/d12/d14/de5/da2/d26/c2e 0 2024-03-23T08:24:05.353 INFO:tasks.workunit.client.0.smithi070.stdout:8/758: link d5/d8/d16/dd4/c9b d5/d8/d16/d32/d84/d9f/c107 0 2024-03-23T08:24:05.357 INFO:tasks.workunit.client.0.smithi070.stdout:2/964: getdents d1/d7/d48/d69 0 2024-03-23T08:24:05.362 INFO:tasks.workunit.client.0.smithi070.stdout:9/837: unlink db/d12/d14/de5/da2/d26/f2c 0 2024-03-23T08:24:05.364 INFO:tasks.workunit.client.0.smithi070.stdout:8/759: rmdir d5/d8/d16/d32/d84 39 2024-03-23T08:24:05.366 INFO:tasks.workunit.client.0.smithi070.stdout:9/838: dread db/d12/d14/de5/da2/dc6/dce/d85/f66 [0,4194304] 0 2024-03-23T08:24:05.367 INFO:tasks.workunit.client.0.smithi070.stdout:4/681: fdatasync d3/d6/d35/d65/d27/feb 0 2024-03-23T08:24:05.374 INFO:tasks.workunit.client.0.smithi070.stdout:8/760: stat d5/d8/d16/d32/d84/d98/dde/da3/fb2 0 2024-03-23T08:24:05.375 INFO:tasks.workunit.client.0.smithi070.stdout:9/839: fsync db/d12/f82 0 2024-03-23T08:24:05.376 INFO:tasks.workunit.client.0.smithi070.stdout:9/840: chown db/d12/d14/de5/da2/d4f/db6/f99 900574 1 2024-03-23T08:24:05.384 INFO:tasks.workunit.client.0.smithi070.stdout:2/965: fdatasync d1/d29/f35 0 2024-03-23T08:24:05.386 INFO:tasks.workunit.client.0.smithi070.stdout:5/671: link dc/d1b/d1d/c30 dc/d1b/d1d/d25/d68/d8a/d99/dd7/cdc 0 2024-03-23T08:24:05.387 INFO:tasks.workunit.client.0.smithi070.stdout:5/672: write dc/d1b/fd8 [845822,62282] 0 2024-03-23T08:24:05.388 INFO:tasks.workunit.client.0.smithi070.stdout:6/850: getdents d0/d23/d74/d84/d3c/d86 0 2024-03-23T08:24:05.388 INFO:tasks.workunit.client.0.smithi070.stdout:8/761: creat d5/d8/d9a/f108 x:0 0 0 2024-03-23T08:24:05.391 INFO:tasks.workunit.client.0.smithi070.stdout:9/841: mkdir db/d12/d14/de5/da2/dc6/dce/d85/d11a 0 2024-03-23T08:24:05.393 INFO:tasks.workunit.client.0.smithi070.stdout:4/682: creat d3/d6/dca/ff2 x:0 0 0 2024-03-23T08:24:05.394 INFO:tasks.workunit.client.0.smithi070.stdout:8/762: symlink d5/d8/d9a/l109 0 2024-03-23T08:24:05.401 INFO:tasks.workunit.client.0.smithi070.stdout:9/842: creat db/d12/d14/de5/da2/d26/d80/f11b x:0 0 0 2024-03-23T08:24:05.403 INFO:tasks.workunit.client.0.smithi070.stdout:2/966: dwrite d1/d7/d39/d47/d7b/f130 [0,4194304] 0 2024-03-23T08:24:05.404 INFO:tasks.workunit.client.0.smithi070.stdout:9/843: chown db/d12/d14/de5/da2/d4f/dd9/dc2 226542 1 2024-03-23T08:24:05.408 INFO:tasks.workunit.client.0.smithi070.stdout:8/763: dwrite d5/d8/f11 [0,4194304] 0 2024-03-23T08:24:05.410 INFO:tasks.workunit.client.0.smithi070.stdout:4/683: read d3/d6/d35/d3f/d51/f63 [3074905,47971] 0 2024-03-23T08:24:05.410 INFO:tasks.workunit.client.0.smithi070.stdout:8/764: chown d5/d8/d16/d32/d84/d98/dde/ce1 26522 1 2024-03-23T08:24:05.413 INFO:tasks.workunit.client.0.smithi070.stdout:5/673: truncate dc/d19/f2c 497240 0 2024-03-23T08:24:05.415 INFO:tasks.workunit.client.0.smithi070.stdout:6/851: rename d0/d23/d74/d84/db3/d38/la9 to d0/d23/d74/d84/l10f 0 2024-03-23T08:24:05.419 INFO:tasks.workunit.client.0.smithi070.stdout:8/765: rmdir d5/d8/d16/d30/d38/d6b/d72/d88/dbf 39 2024-03-23T08:24:05.420 INFO:tasks.workunit.client.0.smithi070.stdout:8/766: readlink d5/d8/d16/d32/d84/d98/dde/da3/dab/ld0 0 2024-03-23T08:24:05.423 INFO:tasks.workunit.client.0.smithi070.stdout:9/844: link db/d12/d14/de5/da2/d26/da7/fac db/d12/f11c 0 2024-03-23T08:24:05.425 INFO:tasks.workunit.client.0.smithi070.stdout:8/767: truncate d5/d8/d16/f22 2657088 0 2024-03-23T08:24:05.434 INFO:tasks.workunit.client.0.smithi070.stdout:7/728: sync 2024-03-23T08:24:05.439 INFO:tasks.workunit.client.0.smithi070.stdout:5/674: link dc/d1b/d1d/d25/d68/d8a/dad/fc7 dc/d1b/d1d/d25/dcd/fdd 0 2024-03-23T08:24:05.441 INFO:tasks.workunit.client.0.smithi070.stdout:6/852: write d0/d23/d28/f36 [4317227,59409] 0 2024-03-23T08:24:05.446 INFO:tasks.workunit.client.0.smithi070.stdout:7/729: link d25/d27/d6a/f9c d25/d27/d6a/d44/d6f/da3/f101 0 2024-03-23T08:24:05.448 INFO:tasks.workunit.client.0.smithi070.stdout:7/730: read - d25/d27/d6a/d44/d6f/dab/dcd/ddc/fef zero size 2024-03-23T08:24:05.451 INFO:tasks.workunit.client.0.smithi070.stdout:9/845: unlink db/d12/d14/de5/da2/dc6/dce/d85/d42/l53 0 2024-03-23T08:24:05.562 INFO:tasks.workunit.client.0.smithi070.stdout:5/675: rename dc/d1b/d1d/d25/d38/d48/f57 to dc/d1b/d1d/fde 0 2024-03-23T08:24:05.562 INFO:tasks.workunit.client.0.smithi070.stdout:8/768: write d5/d8/d16/f44 [108190,37644] 0 2024-03-23T08:24:05.565 INFO:tasks.workunit.client.0.smithi070.stdout:5/676: read dc/d1b/d1d/d25/d38/d53/f9a [618412,36908] 0 2024-03-23T08:24:05.569 INFO:tasks.workunit.client.0.smithi070.stdout:5/677: chown dc/d1b/d1d/d25/d68/d8a/dad 4 1 2024-03-23T08:24:05.571 INFO:tasks.workunit.client.0.smithi070.stdout:7/731: dread d25/d27/d6a/d44/f67 [0,4194304] 0 2024-03-23T08:24:05.572 INFO:tasks.workunit.client.0.smithi070.stdout:8/769: dwrite d5/d8/d9a/f106 [0,4194304] 0 2024-03-23T08:24:05.577 INFO:tasks.workunit.client.0.smithi070.stdout:5/678: mknod dc/d1b/d1d/d25/d38/d48/dc8/cdf 0 2024-03-23T08:24:05.580 INFO:tasks.workunit.client.0.smithi070.stdout:9/846: dread db/d12/d14/f23 [0,4194304] 0 2024-03-23T08:24:05.581 INFO:tasks.workunit.client.0.smithi070.stdout:7/732: rename d25/d40/fb0 to d25/d27/d6a/d44/d6f/da3/f102 0 2024-03-23T08:24:05.582 INFO:tasks.workunit.client.0.smithi070.stdout:7/733: chown d25/d27/d6a/cc3 30199098 1 2024-03-23T08:24:05.585 INFO:tasks.workunit.client.0.smithi070.stdout:9/847: unlink db/d12/d14/de5/da2/d26/d80/l83 0 2024-03-23T08:24:05.588 INFO:tasks.workunit.client.0.smithi070.stdout:7/734: creat d25/d27/d6a/d44/d6f/da3/f103 x:0 0 0 2024-03-23T08:24:05.592 INFO:tasks.workunit.client.0.smithi070.stdout:9/848: rename db/d12/d14/de5/da2/d26/da7/fac to db/d12/d14/de5/da2/d4f/dd9/dc2/de7/d10a/f11d 0 2024-03-23T08:24:05.594 INFO:tasks.workunit.client.0.smithi070.stdout:8/770: dwrite d5/d8/f4a [0,4194304] 0 2024-03-23T08:24:05.600 INFO:tasks.workunit.client.0.smithi070.stdout:9/849: unlink db/d12/d14/f4e 0 2024-03-23T08:24:05.601 INFO:tasks.workunit.client.0.smithi070.stdout:7/735: dwrite d25/d27/d6a/d44/d7c/f8b [0,4194304] 0 2024-03-23T08:24:05.605 INFO:tasks.workunit.client.0.smithi070.stdout:5/679: dwrite dc/d1b/d1d/d25/d36/d32/d5c/fa2 [0,4194304] 0 2024-03-23T08:24:05.610 INFO:tasks.workunit.client.0.smithi070.stdout:8/771: dwrite d5/d8/f73 [0,4194304] 0 2024-03-23T08:24:05.619 INFO:tasks.workunit.client.0.smithi070.stdout:8/772: read - d5/d8/d16/d30/fe7 zero size 2024-03-23T08:24:05.624 INFO:tasks.workunit.client.0.smithi070.stdout:8/773: truncate d5/d8/d16/dd4/d6f/fa0 539557 0 2024-03-23T08:24:05.631 INFO:tasks.workunit.client.0.smithi070.stdout:9/850: symlink db/d12/d14/de5/da2/dc6/dce/d85/d11a/l11e 0 2024-03-23T08:24:05.632 INFO:tasks.workunit.client.0.smithi070.stdout:5/680: dread dc/d1b/d1d/d25/d68/d8a/dad/fc7 [0,4194304] 0 2024-03-23T08:24:05.635 INFO:tasks.workunit.client.0.smithi070.stdout:7/736: getdents d25/d27/d6a/d44/d6f/dab/dcd/ddc 0 2024-03-23T08:24:05.639 INFO:tasks.workunit.client.0.smithi070.stdout:8/774: creat d5/d8/d16/d30/f10a x:0 0 0 2024-03-23T08:24:05.644 INFO:tasks.workunit.client.0.smithi070.stdout:5/681: dread dc/d1b/d1d/d25/d38/d39/f91 [0,4194304] 0 2024-03-23T08:24:05.646 INFO:tasks.workunit.client.0.smithi070.stdout:5/682: chown dc/d1b/d1d/d25/d38/d39/f79 4054 1 2024-03-23T08:24:05.646 INFO:tasks.workunit.client.0.smithi070.stdout:9/851: getdents db/d12/d67 0 2024-03-23T08:24:05.647 INFO:tasks.workunit.client.0.smithi070.stdout:8/775: write d5/d8/d16/d3b/f76 [1755125,103313] 0 2024-03-23T08:24:05.650 INFO:tasks.workunit.client.0.smithi070.stdout:5/683: write dc/d1b/d1d/d25/d38/d39/f79 [5103049,89425] 0 2024-03-23T08:24:05.652 INFO:tasks.workunit.client.0.smithi070.stdout:9/852: fdatasync db/d12/d14/de5/da2/d26/da7/fbc 0 2024-03-23T08:24:05.654 INFO:tasks.workunit.client.0.smithi070.stdout:7/737: dwrite d25/d27/d6a/d44/d7c/f9a [0,4194304] 0 2024-03-23T08:24:05.656 INFO:tasks.workunit.client.0.smithi070.stdout:8/776: creat d5/d8/d16/dd4/dc9/f10b x:0 0 0 2024-03-23T08:24:05.665 INFO:tasks.workunit.client.0.smithi070.stdout:5/684: dwrite dc/d1b/d1d/d61/f94 [0,4194304] 0 2024-03-23T08:24:05.666 INFO:tasks.workunit.client.0.smithi070.stdout:7/738: dwrite d25/d27/d6a/d44/d7c/f8b [0,4194304] 0 2024-03-23T08:24:05.668 INFO:tasks.workunit.client.0.smithi070.stdout:7/739: write d25/d27/d6a/f73 [4691762,21290] 0 2024-03-23T08:24:05.668 INFO:tasks.workunit.client.0.smithi070.stdout:5/685: write dc/d1b/d1d/d25/da9/fcc [1149375,45430] 0 2024-03-23T08:24:05.670 INFO:tasks.workunit.client.0.smithi070.stdout:8/777: rename d5/d8/c34 to d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/c10c 0 2024-03-23T08:24:05.675 INFO:tasks.workunit.client.0.smithi070.stdout:8/778: chown d5/d8/f4a 190 1 2024-03-23T08:24:05.676 INFO:tasks.workunit.client.0.smithi070.stdout:7/740: mkdir d25/d27/d6a/d44/d6f/d104 0 2024-03-23T08:24:05.677 INFO:tasks.workunit.client.0.smithi070.stdout:7/741: stat d25/d27/d6a/d44/da2/df4 0 2024-03-23T08:24:05.679 INFO:tasks.workunit.client.0.smithi070.stdout:7/742: dread - d25/d27/d6a/d44/da2/fbf zero size 2024-03-23T08:24:05.681 INFO:tasks.workunit.client.0.smithi070.stdout:5/686: rename dc/d1b/d1d/d25/d38/d39/l6d to dc/d1b/d1d/d25/d36/d32/db7/le0 0 2024-03-23T08:24:05.681 INFO:tasks.workunit.client.0.smithi070.stdout:9/853: write db/d12/d14/de5/da2/dc6/dce/d85/fd4 [31105,92944] 0 2024-03-23T08:24:05.703 INFO:tasks.workunit.client.0.smithi070.stdout:7/743: dread d25/d27/d6a/f80 [0,4194304] 0 2024-03-23T08:24:05.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:05 smithi070 ceph-mon[39699]: pgmap v42: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 45 MiB/s rd, 127 MiB/s wr, 354 op/s 2024-03-23T08:24:05.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:05 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:05.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:05 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:05.783 INFO:tasks.workunit.client.0.smithi070.stdout:1/721: sync 2024-03-23T08:24:05.783 INFO:tasks.workunit.client.0.smithi070.stdout:3/830: sync 2024-03-23T08:24:05.797 INFO:tasks.workunit.client.0.smithi070.stdout:9/854: mkdir db/d12/d115/d118/d11f 0 2024-03-23T08:24:05.806 INFO:tasks.workunit.client.0.smithi070.stdout:5/687: rename dc/d1b/d1d/d25/da9/cdb to dc/d1b/d1d/d25/d38/d53/ce1 0 2024-03-23T08:24:05.807 INFO:tasks.workunit.client.0.smithi070.stdout:8/779: truncate d5/d8/f2c 3260742 0 2024-03-23T08:24:05.808 INFO:tasks.workunit.client.0.smithi070.stdout:8/780: write d5/fe [1092175,54532] 0 2024-03-23T08:24:05.810 INFO:tasks.workunit.client.0.smithi070.stdout:3/831: read db/d20/d63/f75 [1149753,107752] 0 2024-03-23T08:24:05.818 INFO:tasks.workunit.client.0.smithi070.stdout:3/832: mkdir db/df/d1a/d134 0 2024-03-23T08:24:05.820 INFO:tasks.workunit.client.0.smithi070.stdout:7/744: creat d25/d27/d6a/d44/f105 x:0 0 0 2024-03-23T08:24:05.820 INFO:tasks.workunit.client.0.smithi070.stdout:9/855: dread db/d12/d14/de5/da2/d4f/d5a/f71 [0,4194304] 0 2024-03-23T08:24:05.821 INFO:tasks.workunit.client.0.smithi070.stdout:1/722: link d8/db/d40/d43/d64/f8b d8/db/d2c/d3f/d48/f107 0 2024-03-23T08:24:05.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:05 smithi193 ceph-mon[48370]: pgmap v42: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 45 MiB/s rd, 127 MiB/s wr, 354 op/s 2024-03-23T08:24:05.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:05 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:05.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:05 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:05.829 INFO:tasks.workunit.client.0.smithi070.stdout:7/745: dwrite d25/d27/d6a/d44/d6f/dab/ffa [0,4194304] 0 2024-03-23T08:24:05.830 INFO:tasks.workunit.client.0.smithi070.stdout:3/833: symlink db/l135 0 2024-03-23T08:24:05.831 INFO:tasks.workunit.client.0.smithi070.stdout:5/688: getdents dc/d1b/d1d/d25/d68/d8a/d99 0 2024-03-23T08:24:05.833 INFO:tasks.workunit.client.0.smithi070.stdout:5/689: write dc/d1b/fd8 [1273402,48294] 0 2024-03-23T08:24:05.833 INFO:tasks.workunit.client.0.smithi070.stdout:9/856: mkdir db/d12/d14/de5/da2/d4f/de6/d120 0 2024-03-23T08:24:05.835 INFO:tasks.workunit.client.0.smithi070.stdout:9/857: fdatasync db/d12/d14/de5/da2/d4f/f72 0 2024-03-23T08:24:05.837 INFO:tasks.workunit.client.0.smithi070.stdout:9/858: truncate db/d12/d14/de5/da2/d26/fea 993545 0 2024-03-23T08:24:05.837 INFO:tasks.workunit.client.0.smithi070.stdout:7/746: dread d25/fb8 [0,4194304] 0 2024-03-23T08:24:05.838 INFO:tasks.workunit.client.0.smithi070.stdout:3/834: symlink db/d4e/d105/l136 0 2024-03-23T08:24:05.840 INFO:tasks.workunit.client.0.smithi070.stdout:7/747: dread - d25/d27/d6a/d44/d85/d95/da0/de2/ffc zero size 2024-03-23T08:24:05.843 INFO:tasks.workunit.client.0.smithi070.stdout:5/690: symlink dc/d1b/d1d/d25/d68/d8a/d99/le2 0 2024-03-23T08:24:05.843 INFO:tasks.workunit.client.0.smithi070.stdout:5/691: stat dc 0 2024-03-23T08:24:05.845 INFO:tasks.workunit.client.0.smithi070.stdout:3/835: mkdir db/d4e/dc4/d137 0 2024-03-23T08:24:05.845 INFO:tasks.workunit.client.0.smithi070.stdout:3/836: stat db/d4e/df8/l114 0 2024-03-23T08:24:05.849 INFO:tasks.workunit.client.0.smithi070.stdout:5/692: mknod dc/d1b/d1d/d25/d38/d39/ce3 0 2024-03-23T08:24:05.850 INFO:tasks.workunit.client.0.smithi070.stdout:5/693: readlink dc/d1b/lb1 0 2024-03-23T08:24:05.851 INFO:tasks.workunit.client.0.smithi070.stdout:9/859: truncate db/d12/f11c 3158772 0 2024-03-23T08:24:05.854 INFO:tasks.workunit.client.0.smithi070.stdout:7/748: mknod d25/d27/d6a/d44/dfe/c106 0 2024-03-23T08:24:05.854 INFO:tasks.workunit.client.0.smithi070.stdout:8/781: truncate d5/d8/d16/d3b/f76 3834769 0 2024-03-23T08:24:05.855 INFO:tasks.workunit.client.0.smithi070.stdout:3/837: mknod db/c138 0 2024-03-23T08:24:05.857 INFO:tasks.workunit.client.0.smithi070.stdout:9/860: fdatasync db/fbf 0 2024-03-23T08:24:05.858 INFO:tasks.workunit.client.0.smithi070.stdout:9/861: stat db/d12/d14/de5/da2/d4f/d5a/f71 0 2024-03-23T08:24:05.860 INFO:tasks.workunit.client.0.smithi070.stdout:7/749: mkdir d25/d27/d6a/d44/dfe/d107 0 2024-03-23T08:24:05.865 INFO:tasks.workunit.client.0.smithi070.stdout:1/723: write d8/db/d2c/d3f/faa [5068055,3085] 0 2024-03-23T08:24:05.865 INFO:tasks.workunit.client.0.smithi070.stdout:8/782: dread d5/d8/f18 [0,4194304] 0 2024-03-23T08:24:05.866 INFO:tasks.workunit.client.0.smithi070.stdout:3/838: rename db/d14/d43/d62/d95 to db/d20/d49/de1/d139 0 2024-03-23T08:24:05.866 INFO:tasks.workunit.client.0.smithi070.stdout:3/839: stat db/df/d5d 0 2024-03-23T08:24:05.866 INFO:tasks.workunit.client.0.smithi070.stdout:9/862: symlink db/d12/d115/d118/l121 0 2024-03-23T08:24:05.866 INFO:tasks.workunit.client.0.smithi070.stdout:3/840: chown db/df/d5d/d126 15 1 2024-03-23T08:24:05.867 INFO:tasks.workunit.client.0.smithi070.stdout:9/863: rename db/d12 to db/d12/d14/de5/da2/d58/db5/d122 22 2024-03-23T08:24:05.868 INFO:tasks.workunit.client.0.smithi070.stdout:7/750: mkdir d25/d27/d6a/d44/d6f/dab/dcd/d108 0 2024-03-23T08:24:05.868 INFO:tasks.workunit.client.0.smithi070.stdout:1/724: dread - d8/d1b/f89 zero size 2024-03-23T08:24:05.871 INFO:tasks.workunit.client.0.smithi070.stdout:3/841: unlink db/d4e/df8/c2b 0 2024-03-23T08:24:05.872 INFO:tasks.workunit.client.0.smithi070.stdout:9/864: read db/d12/f17 [5225154,46607] 0 2024-03-23T08:24:05.874 INFO:tasks.workunit.client.0.smithi070.stdout:8/783: mknod d5/d8/d16/d3b/c10d 0 2024-03-23T08:24:05.875 INFO:tasks.workunit.client.0.smithi070.stdout:3/842: fsync db/d20/d49/f125 0 2024-03-23T08:24:05.878 INFO:tasks.workunit.client.0.smithi070.stdout:1/725: mkdir d8/d26/d108 0 2024-03-23T08:24:05.879 INFO:tasks.workunit.client.0.smithi070.stdout:9/865: fdatasync db/d12/d14/de5/da2/d58/db5/fc3 0 2024-03-23T08:24:05.880 INFO:tasks.workunit.client.0.smithi070.stdout:8/784: creat d5/d8/d16/d32/d84/d9f/f10e x:0 0 0 2024-03-23T08:24:05.882 INFO:tasks.workunit.client.0.smithi070.stdout:1/726: fsync d8/d1b/f4b 0 2024-03-23T08:24:05.883 INFO:tasks.workunit.client.0.smithi070.stdout:7/751: dread d25/d27/d6a/fb1 [0,4194304] 0 2024-03-23T08:24:05.887 INFO:tasks.workunit.client.0.smithi070.stdout:3/843: creat db/d20/d7a/dcc/f13a x:0 0 0 2024-03-23T08:24:05.889 INFO:tasks.workunit.client.0.smithi070.stdout:7/752: symlink d25/d27/d6a/d44/d7c/db4/l109 0 2024-03-23T08:24:05.893 INFO:tasks.workunit.client.0.smithi070.stdout:7/753: truncate d25/d40/f8f 644158 0 2024-03-23T08:24:05.896 INFO:tasks.workunit.client.0.smithi070.stdout:7/754: rename d25/d27/d6a/d44/l6c to d25/d27/d6a/d44/da4/l10a 0 2024-03-23T08:24:05.898 INFO:tasks.workunit.client.0.smithi070.stdout:7/755: rmdir d25/d27/d6a/d44/d7c 39 2024-03-23T08:24:05.899 INFO:tasks.workunit.client.0.smithi070.stdout:9/866: rmdir db/d12/d14 39 2024-03-23T08:24:05.910 INFO:tasks.workunit.client.0.smithi070.stdout:9/867: readlink db/d12/d14/de5/da2/dc6/dce/d85/d42/l8e 0 2024-03-23T08:24:05.912 INFO:tasks.workunit.client.0.smithi070.stdout:9/868: write db/db8/ff1 [240169,118441] 0 2024-03-23T08:24:05.960 INFO:tasks.workunit.client.0.smithi070.stdout:8/785: dwrite d5/d8/d16/d30/d38/fa4 [0,4194304] 0 2024-03-23T08:24:05.962 INFO:tasks.workunit.client.0.smithi070.stdout:3/844: dwrite db/d14/d65/f78 [0,4194304] 0 2024-03-23T08:24:05.967 INFO:tasks.workunit.client.0.smithi070.stdout:3/845: mknod db/d7f/dcd/d100/c13b 0 2024-03-23T08:24:05.971 INFO:tasks.workunit.client.0.smithi070.stdout:3/846: creat db/d4e/dfc/f13c x:0 0 0 2024-03-23T08:24:05.974 INFO:tasks.workunit.client.0.smithi070.stdout:7/756: truncate d25/d27/d6a/d44/d85/d95/da0/fb2 4005569 0 2024-03-23T08:24:05.980 INFO:tasks.workunit.client.0.smithi070.stdout:7/757: getdents d25/d27/dc2 0 2024-03-23T08:24:05.987 INFO:tasks.workunit.client.0.smithi070.stdout:7/758: dwrite d25/d27/d6a/d44/f54 [0,4194304] 0 2024-03-23T08:24:05.991 INFO:tasks.workunit.client.0.smithi070.stdout:7/759: rename d25/d27/dc2/fc4 to d25/d27/d6a/d44/d6f/dab/f10b 0 2024-03-23T08:24:05.992 INFO:tasks.workunit.client.0.smithi070.stdout:7/760: readlink d25/d27/d6a/d44/l4e 0 2024-03-23T08:24:05.993 INFO:tasks.workunit.client.0.smithi070.stdout:7/761: fsync d25/d27/d6a/d44/d6f/dab/ffa 0 2024-03-23T08:24:05.999 INFO:tasks.workunit.client.0.smithi070.stdout:9/869: write db/d12/d14/de5/da2/d4f/db6/faf [4466338,70702] 0 2024-03-23T08:24:06.005 INFO:tasks.workunit.client.0.smithi070.stdout:7/762: dread d25/d40/f4b [0,4194304] 0 2024-03-23T08:24:06.006 INFO:tasks.workunit.client.0.smithi070.stdout:3/847: dread db/d14/d43/f4a [0,4194304] 0 2024-03-23T08:24:06.012 INFO:tasks.workunit.client.0.smithi070.stdout:9/870: getdents db/d12/d34/d47 0 2024-03-23T08:24:06.013 INFO:tasks.workunit.client.0.smithi070.stdout:8/786: dwrite d5/d8/d16/d32/d84/d98/dde/da3/dab/fad [0,4194304] 0 2024-03-23T08:24:06.013 INFO:tasks.workunit.client.0.smithi070.stdout:7/763: symlink d25/d27/d6a/d44/d6f/dcc/l10c 0 2024-03-23T08:24:06.017 INFO:tasks.workunit.client.0.smithi070.stdout:7/764: creat d25/d40/f10d x:0 0 0 2024-03-23T08:24:06.019 INFO:tasks.workunit.client.0.smithi070.stdout:9/871: dwrite db/d12/d14/f95 [0,4194304] 0 2024-03-23T08:24:06.026 INFO:tasks.workunit.client.0.smithi070.stdout:8/787: rename d5/d8/f21 to d5/f10f 0 2024-03-23T08:24:06.028 INFO:tasks.workunit.client.0.smithi070.stdout:9/872: creat db/db8/dd8/d116/f123 x:0 0 0 2024-03-23T08:24:06.031 INFO:tasks.workunit.client.0.smithi070.stdout:7/765: rename d25/d27/d6a/d44/l4e to d25/d27/d6a/d44/d85/d95/da0/l10e 0 2024-03-23T08:24:06.032 INFO:tasks.workunit.client.0.smithi070.stdout:3/848: write db/d7f/fdb [382573,102222] 0 2024-03-23T08:24:06.033 INFO:tasks.workunit.client.0.smithi070.stdout:8/788: mknod d5/d8/d16/d30/d38/d6b/da9/c110 0 2024-03-23T08:24:06.034 INFO:tasks.workunit.client.0.smithi070.stdout:9/873: dread - db/d12/dab/fba zero size 2024-03-23T08:24:06.041 INFO:tasks.workunit.client.0.smithi070.stdout:9/874: dwrite db/d12/d14/de5/da2/f78 [0,4194304] 0 2024-03-23T08:24:06.042 INFO:tasks.workunit.client.0.smithi070.stdout:3/849: creat db/d4e/dc4/d10c/f13d x:0 0 0 2024-03-23T08:24:06.049 INFO:tasks.workunit.client.0.smithi070.stdout:7/766: mkdir d25/d27/d6a/d44/d7c/dfb/dfd/d10f 0 2024-03-23T08:24:06.049 INFO:tasks.workunit.client.0.smithi070.stdout:9/875: dwrite db/d12/d14/de5/da2/dc6/dce/d85/fd4 [0,4194304] 0 2024-03-23T08:24:06.052 INFO:tasks.workunit.client.0.smithi070.stdout:3/850: dread - db/d7f/dcd/ff0 zero size 2024-03-23T08:24:06.052 INFO:tasks.workunit.client.0.smithi070.stdout:9/876: dread db/fbf [0,4194304] 0 2024-03-23T08:24:06.057 INFO:tasks.workunit.client.0.smithi070.stdout:7/767: dread d25/d27/f57 [0,4194304] 0 2024-03-23T08:24:06.061 INFO:tasks.workunit.client.0.smithi070.stdout:9/877: mknod db/db8/dd8/c124 0 2024-03-23T08:24:06.065 INFO:tasks.workunit.client.0.smithi070.stdout:8/789: dwrite d5/d8/d16/dd4/fa6 [0,4194304] 0 2024-03-23T08:24:06.068 INFO:tasks.workunit.client.0.smithi070.stdout:7/768: dread d25/d27/d6a/d44/d85/f93 [0,4194304] 0 2024-03-23T08:24:06.070 INFO:tasks.workunit.client.0.smithi070.stdout:9/878: truncate db/d12/d14/de5/da2/d4f/d5a/f110 359824 0 2024-03-23T08:24:06.074 INFO:tasks.workunit.client.0.smithi070.stdout:3/851: write db/d20/d49/f125 [1319326,126826] 0 2024-03-23T08:24:06.075 INFO:tasks.workunit.client.0.smithi070.stdout:9/879: symlink db/d12/d14/de5/da2/d4f/de6/l125 0 2024-03-23T08:24:06.087 INFO:tasks.workunit.client.0.smithi070.stdout:7/769: rmdir d25/d27/d6a/d44/d6f/dab/dcd/d108 0 2024-03-23T08:24:06.089 INFO:tasks.workunit.client.0.smithi070.stdout:8/790: dwrite d5/d8/d16/dd4/d6f/fa0 [0,4194304] 0 2024-03-23T08:24:06.090 INFO:tasks.workunit.client.0.smithi070.stdout:7/770: read d25/d27/d6a/d44/d85/faf [1624588,107071] 0 2024-03-23T08:24:06.091 INFO:tasks.workunit.client.0.smithi070.stdout:9/880: write db/fd5 [293461,58618] 0 2024-03-23T08:24:06.098 INFO:tasks.workunit.client.0.smithi070.stdout:3/852: dwrite db/d14/d43/f4a [0,4194304] 0 2024-03-23T08:24:06.101 INFO:tasks.workunit.client.0.smithi070.stdout:9/881: creat db/d12/d14/de5/da2/d4f/db6/f126 x:0 0 0 2024-03-23T08:24:06.103 INFO:tasks.workunit.client.0.smithi070.stdout:8/791: mknod d5/d8/d16/d32/d84/d98/dde/da3/c111 0 2024-03-23T08:24:06.105 INFO:tasks.workunit.client.0.smithi070.stdout:3/853: symlink db/d14/l13e 0 2024-03-23T08:24:06.107 INFO:tasks.workunit.client.0.smithi070.stdout:3/854: chown db/df/f55 157 1 2024-03-23T08:24:06.111 INFO:tasks.workunit.client.0.smithi070.stdout:3/855: symlink db/d7f/l13f 0 2024-03-23T08:24:06.112 INFO:tasks.workunit.client.0.smithi070.stdout:8/792: creat d5/d8/d16/d30/f112 x:0 0 0 2024-03-23T08:24:06.114 INFO:tasks.workunit.client.0.smithi070.stdout:3/856: mkdir db/d4e/df8/d3d/d140 0 2024-03-23T08:24:06.116 INFO:tasks.workunit.client.0.smithi070.stdout:8/793: creat d5/d8/f113 x:0 0 0 2024-03-23T08:24:06.117 INFO:tasks.workunit.client.0.smithi070.stdout:9/882: truncate db/d12/da1/fbe 2060610 0 2024-03-23T08:24:06.119 INFO:tasks.workunit.client.0.smithi070.stdout:7/771: dwrite d25/d27/d6a/d44/d85/d95/da0/fb2 [0,4194304] 0 2024-03-23T08:24:06.122 INFO:tasks.workunit.client.0.smithi070.stdout:7/772: chown d25/d27/d6a/d44/d6f/dab/dcd 42 1 2024-03-23T08:24:06.126 INFO:tasks.workunit.client.0.smithi070.stdout:3/857: getdents db/d4e/df8 0 2024-03-23T08:24:06.127 INFO:tasks.workunit.client.0.smithi070.stdout:3/858: chown db/cb3 2975 1 2024-03-23T08:24:06.131 INFO:tasks.workunit.client.0.smithi070.stdout:3/859: mkdir db/d20/d7a/dcc/d141 0 2024-03-23T08:24:06.136 INFO:tasks.workunit.client.0.smithi070.stdout:3/860: creat db/d20/d63/f142 x:0 0 0 2024-03-23T08:24:06.136 INFO:tasks.workunit.client.0.smithi070.stdout:8/794: write d5/d8/d9a/dbe/fed [669559,127423] 0 2024-03-23T08:24:06.137 INFO:tasks.workunit.client.0.smithi070.stdout:3/861: fdatasync db/d14/d65/f78 0 2024-03-23T08:24:06.142 INFO:tasks.workunit.client.0.smithi070.stdout:8/795: mknod d5/d8/d16/d32/d84/d98/dde/da3/dab/c114 0 2024-03-23T08:24:06.144 INFO:tasks.workunit.client.0.smithi070.stdout:9/883: dwrite db/d12/d34/d47/fa3 [0,4194304] 0 2024-03-23T08:24:06.151 INFO:tasks.workunit.client.0.smithi070.stdout:7/773: dread d25/d27/d6a/d44/d6f/da3/f102 [0,4194304] 0 2024-03-23T08:24:06.152 INFO:tasks.workunit.client.0.smithi070.stdout:3/862: link db/cb3 db/d20/d63/d83/dfe/c143 0 2024-03-23T08:24:06.153 INFO:tasks.workunit.client.0.smithi070.stdout:9/884: dwrite db/d12/d14/de5/da2/f117 [0,4194304] 0 2024-03-23T08:24:06.154 INFO:tasks.workunit.client.0.smithi070.stdout:8/796: mkdir d5/d8/d16/dd4/d115 0 2024-03-23T08:24:06.154 INFO:tasks.workunit.client.0.smithi070.stdout:7/774: read d25/d40/fc9 [117434,51564] 0 2024-03-23T08:24:06.156 INFO:tasks.workunit.client.0.smithi070.stdout:8/797: write d5/d8/d9a/f108 [237379,103402] 0 2024-03-23T08:24:06.157 INFO:tasks.workunit.client.0.smithi070.stdout:9/885: dread - db/d12/d14/ff4 zero size 2024-03-23T08:24:06.158 INFO:tasks.workunit.client.0.smithi070.stdout:9/886: stat db/d12/d67 0 2024-03-23T08:24:06.160 INFO:tasks.workunit.client.0.smithi070.stdout:9/887: chown db/d12/d34/d47/c4c 3561431 1 2024-03-23T08:24:06.161 INFO:tasks.workunit.client.0.smithi070.stdout:8/798: mkdir d5/d8/d16/d3b/db9/d116 0 2024-03-23T08:24:06.161 INFO:tasks.workunit.client.0.smithi070.stdout:9/888: write f4 [1986344,8811] 0 2024-03-23T08:24:06.163 INFO:tasks.workunit.client.0.smithi070.stdout:8/799: chown d5/d8/d16/d30/d38/d6b/db0 144153960 1 2024-03-23T08:24:06.165 INFO:tasks.workunit.client.0.smithi070.stdout:8/800: truncate d5/d8/d16/dd4/d6f/fa0 4546624 0 2024-03-23T08:24:06.167 INFO:tasks.workunit.client.0.smithi070.stdout:3/863: write db/d20/d7a/d80/f116 [513652,125524] 0 2024-03-23T08:24:06.168 INFO:tasks.workunit.client.0.smithi070.stdout:9/889: mkdir db/d12/d14/de5/da2/d4f/de6/d127 0 2024-03-23T08:24:06.168 INFO:tasks.workunit.client.0.smithi070.stdout:7/775: getdents d25/d27/d6a/d44/d6f/dab/dcd/df1 0 2024-03-23T08:24:06.169 INFO:tasks.workunit.client.0.smithi070.stdout:3/864: fdatasync db/d20/d49/f125 0 2024-03-23T08:24:06.170 INFO:tasks.workunit.client.0.smithi070.stdout:9/890: chown db/d12/c6e 88 1 2024-03-23T08:24:06.171 INFO:tasks.workunit.client.0.smithi070.stdout:8/801: rename d5/d8/f18 to d5/d8/d9a/dbe/f117 0 2024-03-23T08:24:06.175 INFO:tasks.workunit.client.0.smithi070.stdout:9/891: mkdir db/d12/d14/de5/da2/dc6/dce/d85/d42/d128 0 2024-03-23T08:24:06.180 INFO:tasks.workunit.client.0.smithi070.stdout:7/776: rename d25/d27/d6a/d44/d85/d95/da0/l10e to d25/d27/d6a/d44/da4/l110 0 2024-03-23T08:24:06.181 INFO:tasks.workunit.client.0.smithi070.stdout:8/802: mkdir d5/d8/d16/d30/d38/d6b/d72/d118 0 2024-03-23T08:24:06.182 INFO:tasks.workunit.client.0.smithi070.stdout:9/892: mknod db/d12/d14/de5/da2/dc6/dce/d85/d42/c129 0 2024-03-23T08:24:06.184 INFO:tasks.workunit.client.0.smithi070.stdout:7/777: creat d25/d27/d6a/d44/d85/d95/da0/de2/f111 x:0 0 0 2024-03-23T08:24:06.186 INFO:tasks.workunit.client.0.smithi070.stdout:3/865: dread db/d4e/d92/f101 [0,4194304] 0 2024-03-23T08:24:06.187 INFO:tasks.workunit.client.0.smithi070.stdout:9/893: rename db/d12/d14/de5/da2/d58/la0 to db/d12/d115/d118/l12a 0 2024-03-23T08:24:06.188 INFO:tasks.workunit.client.0.smithi070.stdout:7/778: mkdir d25/d27/d112 0 2024-03-23T08:24:06.190 INFO:tasks.workunit.client.0.smithi070.stdout:7/779: write d25/d40/fa5 [1284830,66609] 0 2024-03-23T08:24:06.191 INFO:tasks.workunit.client.0.smithi070.stdout:8/803: mknod d5/d8/d16/d30/d38/d6b/d72/c119 0 2024-03-23T08:24:06.194 INFO:tasks.workunit.client.0.smithi070.stdout:3/866: dwrite db/d20/d63/f142 [0,4194304] 0 2024-03-23T08:24:06.194 INFO:tasks.workunit.client.0.smithi070.stdout:9/894: rename db/d12/d34/d47 to db/d12/d14/de5/da2/d26/dec/d12b 0 2024-03-23T08:24:06.198 INFO:tasks.workunit.client.0.smithi070.stdout:7/780: read f24 [1683,118974] 0 2024-03-23T08:24:06.199 INFO:tasks.workunit.client.0.smithi070.stdout:8/804: unlink d5/d8/f57 0 2024-03-23T08:24:06.199 INFO:tasks.workunit.client.0.smithi070.stdout:7/781: dread - d25/d27/d6a/d44/d6f/da3/f103 zero size 2024-03-23T08:24:06.202 INFO:tasks.workunit.client.0.smithi070.stdout:3/867: mknod db/df/d1a/de9/c144 0 2024-03-23T08:24:06.204 INFO:tasks.workunit.client.0.smithi070.stdout:8/805: mkdir d5/d8/d16/d3b/d8c/d11a 0 2024-03-23T08:24:06.208 INFO:tasks.workunit.client.0.smithi070.stdout:3/868: write db/d4e/d92/fc1 [2078699,41976] 0 2024-03-23T08:24:06.208 INFO:tasks.workunit.client.0.smithi070.stdout:8/806: dread d5/d8/f4a [0,4194304] 0 2024-03-23T08:24:06.209 INFO:tasks.workunit.client.0.smithi070.stdout:7/782: creat d25/d27/d6a/d44/d6f/dab/dcd/ddc/de4/f113 x:0 0 0 2024-03-23T08:24:06.210 INFO:tasks.workunit.client.0.smithi070.stdout:8/807: write d5/d8/d16/d32/d84/d98/fc5 [963837,27037] 0 2024-03-23T08:24:06.216 INFO:tasks.workunit.client.0.smithi070.stdout:8/808: dread d5/d8/d16/dd4/fcb [0,4194304] 0 2024-03-23T08:24:06.216 INFO:tasks.workunit.client.0.smithi070.stdout:0/928: sync 2024-03-23T08:24:06.219 INFO:tasks.workunit.client.0.smithi070.stdout:7/783: dwrite d25/d27/d6a/d44/d6f/dab/dcd/df1/fff [0,4194304] 0 2024-03-23T08:24:06.225 INFO:tasks.workunit.client.0.smithi070.stdout:3/869: creat db/d14/d65/f145 x:0 0 0 2024-03-23T08:24:06.225 INFO:tasks.workunit.client.0.smithi070.stdout:9/895: dwrite f8 [0,4194304] 0 2024-03-23T08:24:06.226 INFO:tasks.workunit.client.0.smithi070.stdout:9/896: read - db/d12/d14/de5/da2/d4f/db6/fc4 zero size 2024-03-23T08:24:06.227 INFO:tasks.workunit.client.0.smithi070.stdout:9/897: write db/d12/d91/f97 [708964,97239] 0 2024-03-23T08:24:06.230 INFO:tasks.workunit.client.0.smithi070.stdout:9/898: dread - db/d12/d14/de5/da2/d26/d80/f11b zero size 2024-03-23T08:24:06.232 INFO:tasks.workunit.client.0.smithi070.stdout:0/929: mknod d16/d26/d27/d86/d67/d7d/c145 0 2024-03-23T08:24:06.233 INFO:tasks.workunit.client.0.smithi070.stdout:7/784: fdatasync d25/d40/f4d 0 2024-03-23T08:24:06.233 INFO:tasks.workunit.client.0.smithi070.stdout:0/930: chown d16/d26/d27/d86/d9e 16985188 1 2024-03-23T08:24:06.234 INFO:tasks.workunit.client.0.smithi070.stdout:8/809: mknod d5/d8/d16/dd4/d115/c11b 0 2024-03-23T08:24:06.235 INFO:tasks.workunit.client.0.smithi070.stdout:3/870: chown db/d14/d43/f54 493907091 1 2024-03-23T08:24:06.238 INFO:tasks.workunit.client.0.smithi070.stdout:7/785: truncate d25/d40/fd8 30740 0 2024-03-23T08:24:06.239 INFO:tasks.workunit.client.0.smithi070.stdout:3/871: mknod db/d4e/d92/dbb/de2/c146 0 2024-03-23T08:24:06.244 INFO:tasks.workunit.client.0.smithi070.stdout:8/810: mkdir d5/d8/d16/d30/d11c 0 2024-03-23T08:24:06.246 INFO:tasks.workunit.client.0.smithi070.stdout:8/811: write d5/d8/d16/d30/d38/fa4 [3680147,66292] 0 2024-03-23T08:24:06.246 INFO:tasks.workunit.client.0.smithi070.stdout:7/786: mkdir d25/d27/d6a/d44/d7c/db4/d114 0 2024-03-23T08:24:06.247 INFO:tasks.workunit.client.0.smithi070.stdout:3/872: chown db/d14/d43/d7d/l10b 223 1 2024-03-23T08:24:06.252 INFO:tasks.workunit.client.0.smithi070.stdout:7/787: dwrite d25/f87 [4194304,4194304] 0 2024-03-23T08:24:06.258 INFO:tasks.workunit.client.0.smithi070.stdout:9/899: dwrite db/d12/d14/f63 [0,4194304] 0 2024-03-23T08:24:06.263 INFO:tasks.workunit.client.0.smithi070.stdout:0/931: getdents d16/d22/d43/d47 0 2024-03-23T08:24:06.263 INFO:tasks.workunit.client.0.smithi070.stdout:7/788: read d25/d27/f6d [75095,98410] 0 2024-03-23T08:24:06.271 INFO:tasks.workunit.client.0.smithi070.stdout:8/812: creat d5/d8/d16/d30/d38/f11d x:0 0 0 2024-03-23T08:24:06.274 INFO:tasks.workunit.client.0.smithi070.stdout:0/932: mkdir d16/d26/d27/d3d/d4d/dca/d146 0 2024-03-23T08:24:06.276 INFO:tasks.workunit.client.0.smithi070.stdout:3/873: getdents db/df/d4d 0 2024-03-23T08:24:06.278 INFO:tasks.workunit.client.0.smithi070.stdout:7/789: link d25/d27/d6a/d44/dfe/c106 d25/d40/c115 0 2024-03-23T08:24:06.281 INFO:tasks.workunit.client.0.smithi070.stdout:0/933: mknod d16/d22/d43/c147 0 2024-03-23T08:24:06.281 INFO:tasks.workunit.client.0.smithi070.stdout:3/874: truncate db/d14/d43/d7d/f11f 251177 0 2024-03-23T08:24:06.282 INFO:tasks.workunit.client.0.smithi070.stdout:3/875: dread - db/d4e/dd6/f121 zero size 2024-03-23T08:24:06.284 INFO:tasks.workunit.client.0.smithi070.stdout:7/790: fdatasync d25/d27/d6a/d44/f79 0 2024-03-23T08:24:06.286 INFO:tasks.workunit.client.0.smithi070.stdout:3/876: symlink db/d4e/df8/d22/l147 0 2024-03-23T08:24:06.288 INFO:tasks.workunit.client.0.smithi070.stdout:8/813: getdents d5/d8/d16/d30/d38/d6b/d72/d88/dbf 0 2024-03-23T08:24:06.290 INFO:tasks.workunit.client.0.smithi070.stdout:0/934: rename d16/d26/d27/d4c/d60/d7c/daf/cb1 to d16/d26/d27/d86/d140/c148 0 2024-03-23T08:24:06.292 INFO:tasks.workunit.client.0.smithi070.stdout:9/900: dread db/d12/d14/de5/da2/dc6/d7e/f56 [0,4194304] 0 2024-03-23T08:24:06.292 INFO:tasks.workunit.client.0.smithi070.stdout:9/901: chown db/d12 2061034 1 2024-03-23T08:24:06.295 INFO:tasks.workunit.client.0.smithi070.stdout:9/902: read db/d12/d14/de5/da2/d26/dec/d12b/fd7 [3514713,80106] 0 2024-03-23T08:24:06.296 INFO:tasks.workunit.client.0.smithi070.stdout:9/903: write db/d12/d14/f35 [2855419,71913] 0 2024-03-23T08:24:06.298 INFO:tasks.workunit.client.0.smithi070.stdout:9/904: write db/d12/d14/de5/da2/d4f/d5a/f110 [183193,22369] 0 2024-03-23T08:24:06.304 INFO:tasks.workunit.client.0.smithi070.stdout:7/791: rename d25/d40/la9 to d25/d27/dc2/df8/l116 0 2024-03-23T08:24:06.304 INFO:tasks.workunit.client.0.smithi070.stdout:7/792: stat d25/d27/d6a/l9d 0 2024-03-23T08:24:06.309 INFO:tasks.workunit.client.0.smithi070.stdout:2/967: sync 2024-03-23T08:24:06.310 INFO:tasks.workunit.client.0.smithi070.stdout:3/877: write db/d4e/fb7 [686571,106413] 0 2024-03-23T08:24:06.312 INFO:tasks.workunit.client.0.smithi070.stdout:9/905: rmdir db/d12/d14/de5/da2/d26/da7 39 2024-03-23T08:24:06.313 INFO:tasks.workunit.client.0.smithi070.stdout:9/906: chown db/d12/d91/ff5 321 1 2024-03-23T08:24:06.314 INFO:tasks.workunit.client.0.smithi070.stdout:9/907: chown db/d12/f89 57 1 2024-03-23T08:24:06.317 INFO:tasks.workunit.client.0.smithi070.stdout:7/793: symlink d25/d27/d6a/d44/d7c/dfb/l117 0 2024-03-23T08:24:06.317 INFO:tasks.workunit.client.0.smithi070.stdout:0/935: dwrite d16/d26/d27/f91 [4194304,4194304] 0 2024-03-23T08:24:06.321 INFO:tasks.workunit.client.0.smithi070.stdout:9/908: chown db/d12/d67/fc1 946832 1 2024-03-23T08:24:06.324 INFO:tasks.workunit.client.0.smithi070.stdout:8/814: link d5/c35 d5/d8/d9a/da2/c11e 0 2024-03-23T08:24:06.325 INFO:tasks.workunit.client.0.smithi070.stdout:9/909: dread db/fd5 [0,4194304] 0 2024-03-23T08:24:06.327 INFO:tasks.workunit.client.0.smithi070.stdout:2/968: creat d1/d7/df/d99/dea/d138/f143 x:0 0 0 2024-03-23T08:24:06.327 INFO:tasks.workunit.client.0.smithi070.stdout:9/910: write db/d12/f89 [1366897,43677] 0 2024-03-23T08:24:06.332 INFO:tasks.workunit.client.0.smithi070.stdout:3/878: getdents db/de6 0 2024-03-23T08:24:06.334 INFO:tasks.workunit.client.0.smithi070.stdout:3/879: chown db/d4e/d92 317 1 2024-03-23T08:24:06.334 INFO:tasks.workunit.client.0.smithi070.stdout:8/815: dwrite d5/d8/d16/d30/f112 [0,4194304] 0 2024-03-23T08:24:06.341 INFO:tasks.workunit.client.0.smithi070.stdout:9/911: chown db/d12/d14/de5/da2/d26/da7/fbc 1 1 2024-03-23T08:24:06.342 INFO:tasks.workunit.client.0.smithi070.stdout:0/936: dwrite d16/d22/fa3 [0,4194304] 0 2024-03-23T08:24:06.345 INFO:tasks.workunit.client.0.smithi070.stdout:9/912: rename db/d12/d14/de5 to db/d12/d14/de5/da2/dc6/d7e/d12c 22 2024-03-23T08:24:06.345 INFO:tasks.workunit.client.0.smithi070.stdout:0/937: read d16/d26/f37 [265437,95769] 0 2024-03-23T08:24:06.347 INFO:tasks.workunit.client.0.smithi070.stdout:0/938: chown d16/d26/d27/d4c/d60/d75/fa7 138883804 1 2024-03-23T08:24:06.349 INFO:tasks.workunit.client.0.smithi070.stdout:7/794: creat d25/f118 x:0 0 0 2024-03-23T08:24:06.349 INFO:tasks.workunit.client.0.smithi070.stdout:7/795: chown d25/f83 336549 1 2024-03-23T08:24:06.352 INFO:tasks.workunit.client.0.smithi070.stdout:2/969: mkdir d1/d7/df/d99/dea/dc3/d13a/d144 0 2024-03-23T08:24:06.353 INFO:tasks.workunit.client.0.smithi070.stdout:2/970: chown d1/d7/df/d94/l6f 0 1 2024-03-23T08:24:06.353 INFO:tasks.workunit.client.0.smithi070.stdout:9/913: creat db/db8/dd8/d116/f12d x:0 0 0 2024-03-23T08:24:06.354 INFO:tasks.workunit.client.0.smithi070.stdout:0/939: mknod d16/d26/d27/d86/d140/df8/d11a/c149 0 2024-03-23T08:24:06.356 INFO:tasks.workunit.client.0.smithi070.stdout:3/880: mkdir db/d14/d65/d148 0 2024-03-23T08:24:06.356 INFO:tasks.workunit.client.0.smithi070.stdout:0/940: dread - d16/d26/d27/d86/db6/dde/f13d zero size 2024-03-23T08:24:06.359 INFO:tasks.workunit.client.0.smithi070.stdout:0/941: read d16/d22/d43/d47/da9/fe6 [2096502,86331] 0 2024-03-23T08:24:06.360 INFO:tasks.workunit.client.0.smithi070.stdout:9/914: creat db/d12/d91/f12e x:0 0 0 2024-03-23T08:24:06.365 INFO:tasks.workunit.client.0.smithi070.stdout:3/881: rename db/df/d1a/f3c to db/d20/d7a/d80/ded/f149 0 2024-03-23T08:24:06.366 INFO:tasks.workunit.client.0.smithi070.stdout:7/796: write d25/f84 [1474965,33347] 0 2024-03-23T08:24:06.368 INFO:tasks.workunit.client.0.smithi070.stdout:8/816: rmdir d5/d8/d16/d3b/d8c/d11a 0 2024-03-23T08:24:06.369 INFO:tasks.workunit.client.0.smithi070.stdout:0/942: rmdir d16/d22/d43/d47/da9 39 2024-03-23T08:24:06.373 INFO:tasks.workunit.client.0.smithi070.stdout:4/684: sync 2024-03-23T08:24:06.373 INFO:tasks.workunit.client.0.smithi070.stdout:1/727: sync 2024-03-23T08:24:06.373 INFO:tasks.workunit.client.0.smithi070.stdout:6/853: sync 2024-03-23T08:24:06.374 INFO:tasks.workunit.client.0.smithi070.stdout:5/694: sync 2024-03-23T08:24:06.375 INFO:tasks.workunit.client.0.smithi070.stdout:9/915: unlink db/d12/d14/de5/da2/f117 0 2024-03-23T08:24:06.379 INFO:tasks.workunit.client.0.smithi070.stdout:5/695: truncate dc/d1b/d1d/d25/d68/d8a/d99/fb2 963505 0 2024-03-23T08:24:06.380 INFO:tasks.workunit.client.0.smithi070.stdout:2/971: truncate d1/d7/d39/d47/d7d/fb4 3907941 0 2024-03-23T08:24:06.381 INFO:tasks.workunit.client.0.smithi070.stdout:6/854: dwrite d0/d23/d88/f94 [4194304,4194304] 0 2024-03-23T08:24:06.382 INFO:tasks.workunit.client.0.smithi070.stdout:9/916: dread f8 [0,4194304] 0 2024-03-23T08:24:06.383 INFO:tasks.workunit.client.0.smithi070.stdout:2/972: fdatasync d1/d7/df/d94/d38/d4c/fde 0 2024-03-23T08:24:06.383 INFO:tasks.workunit.client.0.smithi070.stdout:3/882: fdatasync db/df/d4d/fbf 0 2024-03-23T08:24:06.386 INFO:tasks.workunit.client.0.smithi070.stdout:7/797: rmdir d25/d27/d6a/d44/d85/d95/da0/de2 39 2024-03-23T08:24:06.387 INFO:tasks.workunit.client.0.smithi070.stdout:3/883: dread - db/d7f/f132 zero size 2024-03-23T08:24:06.388 INFO:tasks.workunit.client.0.smithi070.stdout:8/817: creat d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/f11f x:0 0 0 2024-03-23T08:24:06.390 INFO:tasks.workunit.client.0.smithi070.stdout:4/685: mknod d3/d6/d45/cf3 0 2024-03-23T08:24:06.391 INFO:tasks.workunit.client.0.smithi070.stdout:3/884: write db/d20/d49/de1/f11d [227043,37369] 0 2024-03-23T08:24:06.394 INFO:tasks.workunit.client.0.smithi070.stdout:5/696: symlink dc/d1b/d1d/d61/le4 0 2024-03-23T08:24:06.395 INFO:tasks.workunit.client.0.smithi070.stdout:7/798: dwrite d25/f118 [0,4194304] 0 2024-03-23T08:24:06.404 INFO:tasks.workunit.client.0.smithi070.stdout:6/855: rename d0/d3/d2c/dd1 to d0/d23/d74/d84/db3/d2f/dab/db8/d110 0 2024-03-23T08:24:06.407 INFO:tasks.workunit.client.0.smithi070.stdout:8/818: unlink d5/fe 0 2024-03-23T08:24:06.409 INFO:tasks.workunit.client.0.smithi070.stdout:9/917: creat db/d12/d14/de5/da2/d26/d80/d10c/d10e/f12f x:0 0 0 2024-03-23T08:24:06.422 INFO:tasks.workunit.client.0.smithi070.stdout:7/799: unlink d25/d27/d6a/d44/d85/l8e 0 2024-03-23T08:24:06.424 INFO:tasks.workunit.client.0.smithi070.stdout:4/686: write d3/f62 [318245,108450] 0 2024-03-23T08:24:06.426 INFO:tasks.workunit.client.0.smithi070.stdout:0/943: creat d16/f14a x:0 0 0 2024-03-23T08:24:06.427 INFO:tasks.workunit.client.0.smithi070.stdout:3/885: write db/d14/d43/d62/f69 [19216,89247] 0 2024-03-23T08:24:06.430 INFO:tasks.workunit.client.0.smithi070.stdout:7/800: dwrite d25/d40/f76 [0,4194304] 0 2024-03-23T08:24:06.431 INFO:tasks.workunit.client.0.smithi070.stdout:6/856: mkdir d0/d23/d88/dd9/d111 0 2024-03-23T08:24:06.435 INFO:tasks.workunit.client.0.smithi070.stdout:0/944: dread d16/d26/d27/d4c/d60/d75/fc3 [0,4194304] 0 2024-03-23T08:24:06.441 INFO:tasks.workunit.client.0.smithi070.stdout:9/918: dread db/d12/d14/de5/da2/f7c [0,4194304] 0 2024-03-23T08:24:06.442 INFO:tasks.workunit.client.0.smithi070.stdout:4/687: creat d3/d6/d35/dcf/db5/dda/dc7/ff4 x:0 0 0 2024-03-23T08:24:06.444 INFO:tasks.workunit.client.0.smithi070.stdout:3/886: creat db/df/d4d/f14a x:0 0 0 2024-03-23T08:24:06.446 INFO:tasks.workunit.client.0.smithi070.stdout:8/819: dread d5/d8/d16/f22 [0,4194304] 0 2024-03-23T08:24:06.446 INFO:tasks.workunit.client.0.smithi070.stdout:7/801: fsync d25/d40/fc9 0 2024-03-23T08:24:06.447 INFO:tasks.workunit.client.0.smithi070.stdout:8/820: chown d5/d8/d16/d30/d38/ca8 228590 1 2024-03-23T08:24:06.450 INFO:tasks.workunit.client.0.smithi070.stdout:1/728: getdents d8/db/d2c/d3f/d48/dab 0 2024-03-23T08:24:06.454 INFO:tasks.workunit.client.0.smithi070.stdout:5/697: creat dc/d1b/fe5 x:0 0 0 2024-03-23T08:24:06.458 INFO:tasks.workunit.client.0.smithi070.stdout:2/973: getdents d1/d118/d34 0 2024-03-23T08:24:06.460 INFO:tasks.workunit.client.0.smithi070.stdout:6/857: write d0/d23/d74/d84/db3/d38/d9a/fad [311605,21028] 0 2024-03-23T08:24:06.461 INFO:tasks.workunit.client.0.smithi070.stdout:9/919: mkdir db/d12/d14/de5/da2/dc6/d130 0 2024-03-23T08:24:06.463 INFO:tasks.workunit.client.0.smithi070.stdout:6/858: chown d0/d23/d74/d84/db3/d2f/dab/db8/cf7 508 1 2024-03-23T08:24:06.464 INFO:tasks.workunit.client.0.smithi070.stdout:4/688: truncate d3/d6/d35/d65/d6f/dee/fc2 536236 0 2024-03-23T08:24:06.465 INFO:tasks.workunit.client.0.smithi070.stdout:4/689: readlink d3/d6/d35/d65/d6f/l9f 0 2024-03-23T08:24:06.468 INFO:tasks.workunit.client.0.smithi070.stdout:5/698: mkdir dc/d1b/d1d/d25/d36/d32/db7/de6 0 2024-03-23T08:24:06.477 INFO:tasks.workunit.client.0.smithi070.stdout:6/859: symlink d0/d23/d74/d84/db3/d38/d52/l112 0 2024-03-23T08:24:06.481 INFO:tasks.workunit.client.0.smithi070.stdout:3/887: write db/df/d1a/de9/ff7 [718414,67924] 0 2024-03-23T08:24:06.485 INFO:tasks.workunit.client.0.smithi070.stdout:5/699: stat dc/d1b/d1d/d25/d38/c3b 0 2024-03-23T08:24:06.485 INFO:tasks.workunit.client.0.smithi070.stdout:0/945: dwrite d16/d26/d27/d86/db6/fdc [0,4194304] 0 2024-03-23T08:24:06.487 INFO:tasks.workunit.client.0.smithi070.stdout:2/974: dread d1/d7/df/d94/f56 [0,4194304] 0 2024-03-23T08:24:06.488 INFO:tasks.workunit.client.0.smithi070.stdout:5/700: write dc/d1b/d1d/d25/d68/d8a/d99/fb2 [531215,129617] 0 2024-03-23T08:24:06.488 INFO:tasks.workunit.client.0.smithi070.stdout:9/920: truncate db/d12/d14/de5/da2/d58/fff 521127 0 2024-03-23T08:24:06.492 INFO:tasks.workunit.client.0.smithi070.stdout:6/860: creat d0/d23/d97/d98/f113 x:0 0 0 2024-03-23T08:24:06.493 INFO:tasks.workunit.client.0.smithi070.stdout:5/701: read dc/d1b/d1d/f3a [105583,128886] 0 2024-03-23T08:24:06.496 INFO:tasks.workunit.client.0.smithi070.stdout:7/802: creat d25/d27/d6a/f119 x:0 0 0 2024-03-23T08:24:06.497 INFO:tasks.workunit.client.0.smithi070.stdout:1/729: creat d8/db/d40/d43/f109 x:0 0 0 2024-03-23T08:24:06.500 INFO:tasks.workunit.client.0.smithi070.stdout:0/946: unlink d16/d22/d43/d47/fa5 0 2024-03-23T08:24:06.503 INFO:tasks.workunit.client.0.smithi070.stdout:2/975: write d1/d7/df/d94/d38/d4c/f6a [5553229,18044] 0 2024-03-23T08:24:06.504 INFO:tasks.workunit.client.0.smithi070.stdout:2/976: truncate d1/d21/fa9 4814680 0 2024-03-23T08:24:06.505 INFO:tasks.workunit.client.0.smithi070.stdout:2/977: chown d1/d7/d39/d58/f75 52 1 2024-03-23T08:24:06.507 INFO:tasks.workunit.client.0.smithi070.stdout:6/861: fdatasync d0/d23/d88/db6/fe9 0 2024-03-23T08:24:06.513 INFO:tasks.workunit.client.0.smithi070.stdout:7/803: mkdir d25/d27/d6a/d44/d6f/dcc/d11a 0 2024-03-23T08:24:06.514 INFO:tasks.workunit.client.0.smithi070.stdout:7/804: chown d25/d27/d6a/f86 15644 1 2024-03-23T08:24:06.514 INFO:tasks.workunit.client.0.smithi070.stdout:2/978: dread d1/fd [0,4194304] 0 2024-03-23T08:24:06.515 INFO:tasks.workunit.client.0.smithi070.stdout:7/805: write f12 [3709821,64258] 0 2024-03-23T08:24:06.516 INFO:tasks.workunit.client.0.smithi070.stdout:3/888: mknod db/d4e/dd6/dd7/dd9/d12f/c14b 0 2024-03-23T08:24:06.517 INFO:tasks.workunit.client.0.smithi070.stdout:1/730: mkdir d8/db/d100/d74/d5d/d10a 0 2024-03-23T08:24:06.518 INFO:tasks.workunit.client.0.smithi070.stdout:3/889: chown db/d4e/dd6/dd7/dd9/f104 6578849 1 2024-03-23T08:24:06.518 INFO:tasks.workunit.client.0.smithi070.stdout:7/806: dread d25/d27/d6a/d44/f5a [0,4194304] 0 2024-03-23T08:24:06.524 INFO:tasks.workunit.client.0.smithi070.stdout:3/890: dwrite db/d20/d49/f125 [0,4194304] 0 2024-03-23T08:24:06.528 INFO:tasks.workunit.client.0.smithi070.stdout:4/690: getdents d3/d6/d45 0 2024-03-23T08:24:06.532 INFO:tasks.workunit.client.0.smithi070.stdout:6/862: symlink d0/d23/d74/d84/db3/d38/d52/l114 0 2024-03-23T08:24:06.536 INFO:tasks.workunit.client.0.smithi070.stdout:9/921: dwrite db/d12/d14/de5/da2/dc6/dce/d85/f66 [0,4194304] 0 2024-03-23T08:24:06.537 INFO:tasks.workunit.client.0.smithi070.stdout:2/979: chown d1/d7/d48/d69/db8/dc7/d111/d104/f123 1 1 2024-03-23T08:24:06.537 INFO:tasks.workunit.client.0.smithi070.stdout:1/731: rmdir d8/db/d2c/d3f/dd3 39 2024-03-23T08:24:06.538 INFO:tasks.workunit.client.0.smithi070.stdout:9/922: read - db/db8/dd8/d116/f12d zero size 2024-03-23T08:24:06.539 INFO:tasks.workunit.client.0.smithi070.stdout:1/732: write d8/d42/d4e/d99/ff6 [1046574,96109] 0 2024-03-23T08:24:06.545 INFO:tasks.workunit.client.0.smithi070.stdout:7/807: creat d25/d27/d6a/d44/d7c/dfb/dfd/f11b x:0 0 0 2024-03-23T08:24:06.546 INFO:tasks.workunit.client.0.smithi070.stdout:3/891: creat db/d4e/d92/f14c x:0 0 0 2024-03-23T08:24:06.547 INFO:tasks.workunit.client.0.smithi070.stdout:5/702: dwrite dc/d19/f2c [0,4194304] 0 2024-03-23T08:24:06.547 INFO:tasks.workunit.client.0.smithi070.stdout:6/863: mknod d0/d23/d88/c115 0 2024-03-23T08:24:06.551 INFO:tasks.workunit.client.0.smithi070.stdout:9/923: creat db/d12/d14/de5/da2/d26/d80/d10c/d10e/f131 x:0 0 0 2024-03-23T08:24:06.551 INFO:tasks.workunit.client.0.smithi070.stdout:1/733: mknod d8/db/d40/d43/d64/d85/c10b 0 2024-03-23T08:24:06.551 INFO:tasks.workunit.client.0.smithi070.stdout:7/808: write d25/d27/d6a/d44/d85/fe8 [960902,76492] 0 2024-03-23T08:24:06.553 INFO:tasks.workunit.client.0.smithi070.stdout:0/947: rename d16/d26/d27/d86/def/c123 to d16/d26/d27/d86/d67/c14b 0 2024-03-23T08:24:06.553 INFO:tasks.workunit.client.0.smithi070.stdout:4/691: getdents d3/d6/dd8 0 2024-03-23T08:24:06.554 INFO:tasks.workunit.client.0.smithi070.stdout:5/703: fsync dc/d1b/d1d/d25/d36/d32/f5a 0 2024-03-23T08:24:06.558 INFO:tasks.workunit.client.0.smithi070.stdout:3/892: rmdir db/d4e/df8/d22/dd8 39 2024-03-23T08:24:06.559 INFO:tasks.workunit.client.0.smithi070.stdout:7/809: unlink d25/d27/d6a/f73 0 2024-03-23T08:24:06.563 INFO:tasks.workunit.client.0.smithi070.stdout:9/924: mknod db/d12/d115/d118/d10f/c132 0 2024-03-23T08:24:06.569 INFO:tasks.workunit.client.0.smithi070.stdout:2/980: write d1/d118/d34/fad [3248380,75392] 0 2024-03-23T08:24:06.571 INFO:tasks.workunit.client.0.smithi070.stdout:4/692: dread d3/d6/d35/d3f/d51/d6e/f99 [0,4194304] 0 2024-03-23T08:24:06.571 INFO:tasks.workunit.client.0.smithi070.stdout:7/810: unlink d25/d27/d6a/d44/da2/db5/ld3 0 2024-03-23T08:24:06.573 INFO:tasks.workunit.client.0.smithi070.stdout:4/693: write d3/d6/d35/d65/fef [240093,54280] 0 2024-03-23T08:24:06.614 INFO:tasks.workunit.client.0.smithi070.stdout:3/893: mkdir db/df/d1a/d134/d14d 0 2024-03-23T08:24:06.615 INFO:tasks.workunit.client.0.smithi070.stdout:1/734: link d8/db/d100/d74/d5d/fe2 d8/db/d40/d43/d64/dc2/dfb/f10c 0 2024-03-23T08:24:06.616 INFO:tasks.workunit.client.0.smithi070.stdout:7/811: mknod d25/d27/c11c 0 2024-03-23T08:24:06.618 INFO:tasks.workunit.client.0.smithi070.stdout:5/704: write dc/d1b/d1d/d25/d38/d53/fa7 [771520,54693] 0 2024-03-23T08:24:06.620 INFO:tasks.workunit.client.0.smithi070.stdout:0/948: dread f14 [4194304,4194304] 0 2024-03-23T08:24:06.623 INFO:tasks.workunit.client.0.smithi070.stdout:2/981: dwrite d1/d98/f127 [0,4194304] 0 2024-03-23T08:24:06.625 INFO:tasks.workunit.client.0.smithi070.stdout:1/735: creat d8/d42/d51/f10d x:0 0 0 2024-03-23T08:24:06.632 INFO:tasks.workunit.client.0.smithi070.stdout:0/949: mkdir d16/d22/d14c 0 2024-03-23T08:24:06.632 INFO:tasks.workunit.client.0.smithi070.stdout:5/705: write dc/d1b/d1d/d25/d68/d8a/f8e [2698875,100409] 0 2024-03-23T08:24:06.634 INFO:tasks.workunit.client.0.smithi070.stdout:6/864: dwrite d0/d3/d2c/f76 [0,4194304] 0 2024-03-23T08:24:06.637 INFO:tasks.workunit.client.0.smithi070.stdout:9/925: dwrite db/d12/f39 [0,4194304] 0 2024-03-23T08:24:06.640 INFO:tasks.workunit.client.0.smithi070.stdout:1/736: rmdir d8/d26/d7b 39 2024-03-23T08:24:06.641 INFO:tasks.workunit.client.0.smithi070.stdout:2/982: truncate d1/d118/dc0/fc6 3596360 0 2024-03-23T08:24:06.642 INFO:tasks.workunit.client.0.smithi070.stdout:7/812: dread d25/f34 [0,4194304] 0 2024-03-23T08:24:06.643 INFO:tasks.workunit.client.0.smithi070.stdout:4/694: dwrite d3/d6/d35/dcf/db5/dda/db8/d88/fa6 [0,4194304] 0 2024-03-23T08:24:06.646 INFO:tasks.workunit.client.0.smithi070.stdout:5/706: mknod dc/d1b/d1d/d25/d36/d32/d5c/ce7 0 2024-03-23T08:24:06.646 INFO:tasks.workunit.client.0.smithi070.stdout:2/983: stat d1/d7/df/d99/l128 0 2024-03-23T08:24:06.649 INFO:tasks.workunit.client.0.smithi070.stdout:4/695: chown d3/d6/d35/dcf/db5/dda/db8/c42 409 1 2024-03-23T08:24:06.649 INFO:tasks.workunit.client.0.smithi070.stdout:5/707: dread - dc/d1b/d1d/d25/d38/d53/fc9 zero size 2024-03-23T08:24:06.651 INFO:tasks.workunit.client.0.smithi070.stdout:5/708: write dc/d1b/d1d/d25/d38/d39/f79 [3575796,452] 0 2024-03-23T08:24:06.652 INFO:tasks.workunit.client.0.smithi070.stdout:9/926: creat db/d12/dab/f133 x:0 0 0 2024-03-23T08:24:06.653 INFO:tasks.workunit.client.0.smithi070.stdout:5/709: dread - dc/d1b/d1d/d25/d36/fd3 zero size 2024-03-23T08:24:06.658 INFO:tasks.workunit.client.0.smithi070.stdout:6/865: write d0/d23/d74/d84/db3/fea [602354,97097] 0 2024-03-23T08:24:06.660 INFO:tasks.workunit.client.0.smithi070.stdout:2/984: dread - d1/d7/d48/d69/d7a/fcd zero size 2024-03-23T08:24:06.661 INFO:tasks.workunit.client.0.smithi070.stdout:2/985: readlink d1/l80 0 2024-03-23T08:24:06.661 INFO:tasks.workunit.client.0.smithi070.stdout:4/696: unlink d3/d6/d35/d65/d27/feb 0 2024-03-23T08:24:06.662 INFO:tasks.workunit.client.0.smithi070.stdout:5/710: dwrite dc/d1b/d1d/d25/d38/d39/fba [0,4194304] 0 2024-03-23T08:24:06.663 INFO:tasks.workunit.client.0.smithi070.stdout:4/697: dread - d3/d6/d35/dcf/fe7 zero size 2024-03-23T08:24:06.664 INFO:tasks.workunit.client.0.smithi070.stdout:1/737: mknod d8/d26/d7b/c10e 0 2024-03-23T08:24:06.666 INFO:tasks.workunit.client.0.smithi070.stdout:9/927: mknod db/d12/d14/de5/da2/dc6/d7e/c134 0 2024-03-23T08:24:06.668 INFO:tasks.workunit.client.0.smithi070.stdout:5/711: truncate dc/d1b/d1d/d25/d38/d53/fa7 1179350 0 2024-03-23T08:24:06.669 INFO:tasks.workunit.client.0.smithi070.stdout:0/950: getdents d16/d22/d43/d47/da9/de7/d142 0 2024-03-23T08:24:06.670 INFO:tasks.workunit.client.0.smithi070.stdout:9/928: dread db/d12/d14/de5/da2/d4f/d5a/f71 [0,4194304] 0 2024-03-23T08:24:06.674 INFO:tasks.workunit.client.0.smithi070.stdout:4/698: creat d3/d6/d6a/ff5 x:0 0 0 2024-03-23T08:24:06.676 INFO:tasks.workunit.client.0.smithi070.stdout:0/951: mknod d16/d22/d43/d47/da9/c14d 0 2024-03-23T08:24:06.677 INFO:tasks.workunit.client.0.smithi070.stdout:9/929: rmdir db/d12/d14/de5/da2/d4f/d5a 39 2024-03-23T08:24:06.677 INFO:tasks.workunit.client.0.smithi070.stdout:0/952: stat d16/d26/d27/lc7 0 2024-03-23T08:24:06.679 INFO:tasks.workunit.client.0.smithi070.stdout:1/738: mkdir d8/db/d40/d43/d64/d85/d97/de7/d10f 0 2024-03-23T08:24:06.680 INFO:tasks.workunit.client.0.smithi070.stdout:1/739: readlink d8/l104 0 2024-03-23T08:24:06.684 INFO:tasks.workunit.client.0.smithi070.stdout:4/699: dread d3/d6/d35/d65/d1b/f26 [0,4194304] 0 2024-03-23T08:24:06.686 INFO:tasks.workunit.client.0.smithi070.stdout:4/700: chown d3/d6/d35/d3f/da9/fe9 1 1 2024-03-23T08:24:06.686 INFO:tasks.workunit.client.0.smithi070.stdout:2/986: rename d1/d7/d39/d47/dda/lf8 to d1/d7/d48/d69/db8/dc7/d111/l145 0 2024-03-23T08:24:06.687 INFO:tasks.workunit.client.0.smithi070.stdout:2/987: chown d1/d21/ld6 138446614 1 2024-03-23T08:24:06.689 INFO:tasks.workunit.client.0.smithi070.stdout:7/813: getdents d25/d27/d6a/d44/d85/d95/da0/de2 0 2024-03-23T08:24:06.690 INFO:tasks.workunit.client.0.smithi070.stdout:6/866: write d0/f9c [809484,71618] 0 2024-03-23T08:24:06.692 INFO:tasks.workunit.client.0.smithi070.stdout:1/740: symlink d8/db/d40/d43/d64/dc2/dfb/l110 0 2024-03-23T08:24:06.693 INFO:tasks.workunit.client.0.smithi070.stdout:4/701: mknod d3/d6/d35/dcf/d7e/cf6 0 2024-03-23T08:24:06.694 INFO:tasks.workunit.client.0.smithi070.stdout:5/712: write dc/d1b/d1d/d25/d38/d53/f9a [760554,105799] 0 2024-03-23T08:24:06.694 INFO:tasks.workunit.client.0.smithi070.stdout:2/988: symlink d1/d7/df/l146 0 2024-03-23T08:24:06.697 INFO:tasks.workunit.client.0.smithi070.stdout:1/741: dread - d8/db/d40/d43/d64/d85/d97/fff zero size 2024-03-23T08:24:06.698 INFO:tasks.workunit.client.0.smithi070.stdout:6/867: fsync d0/d23/d28/f4a 0 2024-03-23T08:24:06.698 INFO:tasks.workunit.client.0.smithi070.stdout:4/702: read d3/d6/d35/d65/d1b/f1c [694625,30120] 0 2024-03-23T08:24:06.701 INFO:tasks.workunit.client.0.smithi070.stdout:0/953: dread d16/d26/d27/d86/d67/d109/f133 [0,4194304] 0 2024-03-23T08:24:06.701 INFO:tasks.workunit.client.0.smithi070.stdout:7/814: dwrite d25/d27/d6a/f119 [0,4194304] 0 2024-03-23T08:24:06.704 INFO:tasks.workunit.client.0.smithi070.stdout:1/742: read d8/db/d100/d74/d5d/f87 [2277276,32615] 0 2024-03-23T08:24:06.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:06 smithi070 ceph-mon[39699]: Reconfiguring grafana.smithi070 (dependencies changed)... 2024-03-23T08:24:06.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:06 smithi070 ceph-mon[39699]: Reconfiguring daemon grafana.smithi070 on smithi070 2024-03-23T08:24:06.759 INFO:tasks.workunit.client.0.smithi070.stdout:5/713: creat dc/d1b/d1d/d25/d36/d32/db7/fe8 x:0 0 0 2024-03-23T08:24:06.761 INFO:tasks.workunit.client.0.smithi070.stdout:2/989: dwrite d1/d29/d52/f113 [0,4194304] 0 2024-03-23T08:24:06.761 INFO:tasks.workunit.client.0.smithi070.stdout:5/714: fdatasync dc/f3d 0 2024-03-23T08:24:06.809 INFO:tasks.workunit.client.0.smithi070.stdout:5/715: dwrite dc/d1b/d1d/d25/d36/d32/d5c/f65 [4194304,4194304] 0 2024-03-23T08:24:06.810 INFO:tasks.workunit.client.0.smithi070.stdout:5/716: chown dc/d1b/d1d/d25/d68/d8a/dad/fc7 3499737 1 2024-03-23T08:24:06.817 INFO:tasks.workunit.client.0.smithi070.stdout:1/743: mkdir d8/db/d2c/d3f/dd3/d111 0 2024-03-23T08:24:06.817 INFO:tasks.workunit.client.0.smithi070.stdout:7/815: dread f23 [0,4194304] 0 2024-03-23T08:24:06.817 INFO:tasks.workunit.client.0.smithi070.stdout:0/954: link d16/d22/d43/d47/f70 d16/d22/d43/d47/da9/de7/f14e 0 2024-03-23T08:24:06.818 INFO:tasks.workunit.client.0.smithi070.stdout:4/703: getdents d3/d6/d35/dcf/d7e/d9a 0 2024-03-23T08:24:06.819 INFO:tasks.workunit.client.0.smithi070.stdout:4/704: chown d3/d6/d35/d3f/fd0 116 1 2024-03-23T08:24:06.821 INFO:tasks.workunit.client.0.smithi070.stdout:4/705: rename d3/d6/d35 to d3/d6/d35/dcf/db5/dda/db8/d91/df7 22 2024-03-23T08:24:06.823 INFO:tasks.workunit.client.0.smithi070.stdout:2/990: truncate d1/d7/d39/d47/fe0 133202 0 2024-03-23T08:24:06.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:06 smithi193 ceph-mon[48370]: Reconfiguring grafana.smithi070 (dependencies changed)... 2024-03-23T08:24:06.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:06 smithi193 ceph-mon[48370]: Reconfiguring daemon grafana.smithi070 on smithi070 2024-03-23T08:24:06.825 INFO:tasks.workunit.client.0.smithi070.stdout:1/744: dread d8/d1b/f22 [0,4194304] 0 2024-03-23T08:24:06.826 INFO:tasks.workunit.client.0.smithi070.stdout:7/816: symlink d25/d27/d6a/d44/d6f/dab/dcd/l11d 0 2024-03-23T08:24:06.828 INFO:tasks.workunit.client.0.smithi070.stdout:5/717: link dc/d1b/fd4 dc/d1b/d1d/d61/fe9 0 2024-03-23T08:24:06.832 INFO:tasks.workunit.client.0.smithi070.stdout:4/706: dread d3/d6/d9/fb4 [0,4194304] 0 2024-03-23T08:24:06.833 INFO:tasks.workunit.client.0.smithi070.stdout:6/868: link d0/d23/d4d/lc6 d0/d23/d74/d84/db3/l116 0 2024-03-23T08:24:06.834 INFO:tasks.workunit.client.0.smithi070.stdout:5/718: fdatasync dc/d1b/d1d/d25/d38/d53/f7a 0 2024-03-23T08:24:06.845 INFO:tasks.workunit.client.0.smithi070.stdout:6/869: mknod d0/d23/d74/d84/db3/d38/c117 0 2024-03-23T08:24:06.846 INFO:tasks.workunit.client.0.smithi070.stdout:6/870: chown d0/d23/d74/d84/db3/d38/d9a/lc2 33 1 2024-03-23T08:24:06.850 INFO:tasks.workunit.client.0.smithi070.stdout:4/707: link d3/d6/d35/d65/fef d3/d6/d35/d65/d27/d4b/ff8 0 2024-03-23T08:24:06.852 INFO:tasks.workunit.client.0.smithi070.stdout:4/708: write d3/d6/d9/f4a [2559730,33965] 0 2024-03-23T08:24:06.853 INFO:tasks.workunit.client.0.smithi070.stdout:6/871: dwrite d0/fa7 [0,4194304] 0 2024-03-23T08:24:06.856 INFO:tasks.workunit.client.0.smithi070.stdout:7/817: getdents d25/d27 0 2024-03-23T08:24:06.858 INFO:tasks.workunit.client.0.smithi070.stdout:5/719: dread dc/d1b/d1d/d25/d36/d32/d5c/f5d [0,4194304] 0 2024-03-23T08:24:06.862 INFO:tasks.workunit.client.0.smithi070.stdout:4/709: rmdir d3/d6/d35/d65/d27/d4b/d73/d89 39 2024-03-23T08:24:06.865 INFO:tasks.workunit.client.0.smithi070.stdout:6/872: symlink d0/d23/d4d/da5/l118 0 2024-03-23T08:24:06.868 INFO:tasks.workunit.client.0.smithi070.stdout:4/710: creat d3/d6/d35/dcf/db5/dda/ff9 x:0 0 0 2024-03-23T08:24:06.874 INFO:tasks.workunit.client.0.smithi070.stdout:2/991: truncate d1/d7/df/f135 3476410 0 2024-03-23T08:24:06.875 INFO:tasks.workunit.client.0.smithi070.stdout:2/992: fsync d1/f9e 0 2024-03-23T08:24:06.875 INFO:tasks.workunit.client.0.smithi070.stdout:4/711: write d3/d6/d35/d3f/da9/fe9 [1022276,9455] 0 2024-03-23T08:24:06.876 INFO:tasks.workunit.client.0.smithi070.stdout:4/712: chown d3/d6/d6a/f4c 30218 1 2024-03-23T08:24:06.877 INFO:tasks.workunit.client.0.smithi070.stdout:4/713: chown d3/d6/d35/d65/fef 38 1 2024-03-23T08:24:06.877 INFO:tasks.workunit.client.0.smithi070.stdout:6/873: link d0/d23/d28/f37 d0/d23/d74/d84/db3/d38/f119 0 2024-03-23T08:24:06.878 INFO:tasks.workunit.client.0.smithi070.stdout:1/745: write d8/db/d40/d43/d64/f8b [369644,22006] 0 2024-03-23T08:24:06.879 INFO:tasks.workunit.client.0.smithi070.stdout:2/993: creat d1/d29/d52/de4/d10a/f147 x:0 0 0 2024-03-23T08:24:06.880 INFO:tasks.workunit.client.0.smithi070.stdout:1/746: dread - d8/d26/d41/fe4 zero size 2024-03-23T08:24:06.881 INFO:tasks.workunit.client.0.smithi070.stdout:6/874: rmdir d0/d23/d74/d84/db3/d2f 39 2024-03-23T08:24:06.883 INFO:tasks.workunit.client.0.smithi070.stdout:1/747: symlink d8/db/d100/d74/l112 0 2024-03-23T08:24:06.885 INFO:tasks.workunit.client.0.smithi070.stdout:6/875: symlink d0/d23/d74/d84/d3c/d5b/dde/l11a 0 2024-03-23T08:24:06.886 INFO:tasks.workunit.client.0.smithi070.stdout:4/714: link d3/d6/d9/f7f d3/d6/d35/dcf/db5/dbb/ffa 0 2024-03-23T08:24:06.891 INFO:tasks.workunit.client.0.smithi070.stdout:6/876: rename d0/d23/d74/d84/d3c/f51 to d0/d23/d74/d84/db3/d38/d52/f11b 0 2024-03-23T08:24:06.893 INFO:tasks.workunit.client.0.smithi070.stdout:2/994: dread d1/d7/d39/d47/fb1 [0,4194304] 0 2024-03-23T08:24:06.895 INFO:tasks.workunit.client.0.smithi070.stdout:2/995: dread - d1/d29/d52/de4/d133/f137 zero size 2024-03-23T08:24:06.896 INFO:tasks.workunit.client.0.smithi070.stdout:2/996: read - d1/d118/d34/f100 zero size 2024-03-23T08:24:06.899 INFO:tasks.workunit.client.0.smithi070.stdout:6/877: rename d0/d23/d74/d84/d3c/f53 to d0/d23/d74/d84/db3/d2f/dab/db8/f11c 0 2024-03-23T08:24:06.900 INFO:tasks.workunit.client.0.smithi070.stdout:2/997: symlink d1/d21/d5f/l148 0 2024-03-23T08:24:06.901 INFO:tasks.workunit.client.0.smithi070.stdout:1/748: dread d8/db/d2c/d3f/faa [0,4194304] 0 2024-03-23T08:24:06.901 INFO:tasks.workunit.client.0.smithi070.stdout:2/998: stat d1/d29/d52/de4/d10a/f120 0 2024-03-23T08:24:06.902 INFO:tasks.workunit.client.0.smithi070.stdout:1/749: read d8/d1a/f39 [6605803,528] 0 2024-03-23T08:24:06.906 INFO:tasks.workunit.client.0.smithi070.stdout:2/999: symlink d1/d29/l149 0 2024-03-23T08:24:06.908 INFO:tasks.workunit.client.0.smithi070.stdout:1/750: creat d8/db/d40/d43/d64/d85/d97/dfd/f113 x:0 0 0 2024-03-23T08:24:06.919 INFO:tasks.workunit.client.0.smithi070.stdout:7/818: truncate d25/d27/d6a/d44/d85/d95/da0/fb2 2022360 0 2024-03-23T08:24:06.920 INFO:tasks.workunit.client.0.smithi070.stdout:5/720: truncate dc/d1b/d1d/d25/d38/d53/f60 5253419 0 2024-03-23T08:24:06.947 INFO:tasks.workunit.client.0.smithi070.stdout:4/715: dwrite d3/d6/d35/d65/d27/f4e [0,4194304] 0 2024-03-23T08:24:06.958 INFO:tasks.workunit.client.0.smithi070.stdout:6/878: truncate d0/d23/d4d/da5/fcf 3008524 0 2024-03-23T08:24:06.964 INFO:tasks.workunit.client.0.smithi070.stdout:4/716: rename d3/d6/d35/dcf/db5/l86 to d3/d6/d35/d65/d27/lfb 0 2024-03-23T08:24:06.965 INFO:tasks.workunit.client.0.smithi070.stdout:6/879: mkdir d0/d23/d28/d11d 0 2024-03-23T08:24:06.968 INFO:tasks.workunit.client.0.smithi070.stdout:1/751: write d8/db/f32 [420763,15424] 0 2024-03-23T08:24:06.971 INFO:tasks.workunit.client.0.smithi070.stdout:6/880: rename d0/d23/d28/d35/dd6 to d0/d23/d97/d98/d11e 0 2024-03-23T08:24:06.974 INFO:tasks.workunit.client.0.smithi070.stdout:1/752: creat d8/db/d40/d43/d64/dc2/dfb/f114 x:0 0 0 2024-03-23T08:24:06.975 INFO:tasks.workunit.client.0.smithi070.stdout:6/881: stat d0/d3/d2c/c109 0 2024-03-23T08:24:06.978 INFO:tasks.workunit.client.0.smithi070.stdout:6/882: chown d0/d23/d74/d84/l10f 6616 1 2024-03-23T08:24:06.980 INFO:tasks.workunit.client.0.smithi070.stdout:1/753: getdents d8/db/d2c/d3f/d48/dab 0 2024-03-23T08:24:06.983 INFO:tasks.workunit.client.0.smithi070.stdout:1/754: mknod d8/db/d40/d43/d64/d85/d97/de7/c115 0 2024-03-23T08:24:06.985 INFO:tasks.workunit.client.0.smithi070.stdout:1/755: fsync d8/db/d100/f95 0 2024-03-23T08:24:06.985 INFO:tasks.workunit.client.0.smithi070.stdout:1/756: fdatasync d8/d26/d7b/d9c/fde 0 2024-03-23T08:24:06.987 INFO:tasks.workunit.client.0.smithi070.stdout:6/883: dread d0/d23/d74/d84/d3c/f60 [0,4194304] 0 2024-03-23T08:24:06.987 INFO:tasks.workunit.client.0.smithi070.stdout:1/757: truncate d8/db/d100/d74/f71 5452072 0 2024-03-23T08:24:06.988 INFO:tasks.workunit.client.0.smithi070.stdout:6/884: stat d0/d23/d28/d35/lb9 0 2024-03-23T08:24:06.990 INFO:tasks.workunit.client.0.smithi070.stdout:1/758: truncate d8/d26/d41/fca 710220 0 2024-03-23T08:24:06.995 INFO:tasks.workunit.client.0.smithi070.stdout:1/759: rmdir d8/d1b 39 2024-03-23T08:24:07.089 INFO:tasks.workunit.client.0.smithi070.stdout:5/721: write dc/d1b/d1d/d25/d38/d53/f7a [630043,28064] 0 2024-03-23T08:24:07.089 INFO:tasks.workunit.client.0.smithi070.stdout:4/717: write d3/d6/d35/d65/f3e [1994856,110741] 0 2024-03-23T08:24:07.092 INFO:tasks.workunit.client.0.smithi070.stdout:5/722: write dc/d1b/d1d/d25/d36/d32/f87 [638717,90917] 0 2024-03-23T08:24:07.096 INFO:tasks.workunit.client.0.smithi070.stdout:4/718: fsync d3/d6/d35/d65/d27/d4b/fba 0 2024-03-23T08:24:07.101 INFO:tasks.workunit.client.0.smithi070.stdout:7/819: dwrite d25/d27/d6a/d44/d6f/da3/f101 [0,4194304] 0 2024-03-23T08:24:07.102 INFO:tasks.workunit.client.0.smithi070.stdout:6/885: dread d0/d23/f19 [4194304,4194304] 0 2024-03-23T08:24:07.112 INFO:tasks.workunit.client.0.smithi070.stdout:7/820: stat d25/d27/d6a/d44/d6f/dab/dcd/ddc/lf0 0 2024-03-23T08:24:07.112 INFO:tasks.workunit.client.0.smithi070.stdout:5/723: symlink dc/d1b/d1d/d25/d36/d32/d5c/dcb/lea 0 2024-03-23T08:24:07.121 INFO:tasks.workunit.client.0.smithi070.stdout:7/821: dread - d25/d27/d6a/fbe zero size 2024-03-23T08:24:07.122 INFO:tasks.workunit.client.0.smithi070.stdout:5/724: mkdir dc/d1b/d1d/d25/da9/deb 0 2024-03-23T08:24:07.122 INFO:tasks.workunit.client.0.smithi070.stdout:7/822: chown f24 1574516 1 2024-03-23T08:24:07.127 INFO:tasks.workunit.client.0.smithi070.stdout:6/886: rename d0/d3/ld to d0/d23/d88/l11f 0 2024-03-23T08:24:07.134 INFO:tasks.workunit.client.0.smithi070.stdout:7/823: symlink d25/d27/d6a/d44/d6f/l11e 0 2024-03-23T08:24:07.136 INFO:tasks.workunit.client.0.smithi070.stdout:7/824: read d25/d27/d6a/d44/f5a [1299372,28517] 0 2024-03-23T08:24:07.138 INFO:tasks.workunit.client.0.smithi070.stdout:4/719: dread d3/d6/d35/dcf/db5/dda/db8/d88/fdd [0,4194304] 0 2024-03-23T08:24:07.140 INFO:tasks.workunit.client.0.smithi070.stdout:7/825: write d25/d40/f4b [1050748,72669] 0 2024-03-23T08:24:07.142 INFO:tasks.workunit.client.0.smithi070.stdout:5/725: dread dc/d1b/d1d/d25/d36/f2a [0,4194304] 0 2024-03-23T08:24:07.148 INFO:tasks.workunit.client.0.smithi070.stdout:5/726: rmdir dc/d1b/d1d/d25/d36/d32/d5c/dc0 0 2024-03-23T08:24:07.149 INFO:tasks.workunit.client.0.smithi070.stdout:6/887: write d0/d23/f10 [1768952,88334] 0 2024-03-23T08:24:07.152 INFO:tasks.workunit.client.0.smithi070.stdout:5/727: mkdir dc/d1b/d1d/d25/d36/d32/d5c/dcb/dec 0 2024-03-23T08:24:07.153 INFO:tasks.workunit.client.0.smithi070.stdout:5/728: dread - dc/d1b/d1d/d25/d36/fd3 zero size 2024-03-23T08:24:07.153 INFO:tasks.workunit.client.0.smithi070.stdout:6/888: unlink d0/d23/d74/d84/db3/d38/d9a/ff8 0 2024-03-23T08:24:07.158 INFO:tasks.workunit.client.0.smithi070.stdout:6/889: mknod d0/d23/d97/d98/d11e/c120 0 2024-03-23T08:24:07.158 INFO:tasks.workunit.client.0.smithi070.stdout:6/890: readlink d0/d23/d74/d84/l10f 0 2024-03-23T08:24:07.159 INFO:tasks.workunit.client.0.smithi070.stdout:6/891: chown d0/d23/d88/fdb 0 1 2024-03-23T08:24:07.167 INFO:tasks.workunit.client.0.smithi070.stdout:5/729: write dc/d1b/d1d/f6f [138424,2016] 0 2024-03-23T08:24:07.173 INFO:tasks.workunit.client.0.smithi070.stdout:5/730: getdents dc/d1b/d1d/d25/d38/d39 0 2024-03-23T08:24:07.175 INFO:tasks.workunit.client.0.smithi070.stdout:6/892: dread d0/d23/d74/d84/db3/d2f/dab/fac [0,4194304] 0 2024-03-23T08:24:07.177 INFO:tasks.workunit.client.0.smithi070.stdout:5/731: rename dc/d1b/d1d/d25/dcd/fdd to dc/d1b/d1d/d25/d36/d32/d5c/fed 0 2024-03-23T08:24:07.179 INFO:tasks.workunit.client.0.smithi070.stdout:6/893: unlink d0/d23/d88/c10a 0 2024-03-23T08:24:07.185 INFO:tasks.workunit.client.0.smithi070.stdout:5/732: dwrite dc/d1b/d1d/d25/d68/d8a/d99/fb2 [0,4194304] 0 2024-03-23T08:24:07.198 INFO:tasks.workunit.client.0.smithi070.stdout:6/894: dwrite d0/d23/d28/f4a [0,4194304] 0 2024-03-23T08:24:07.200 INFO:tasks.workunit.client.0.smithi070.stdout:5/733: truncate dc/d19/f2c 176313 0 2024-03-23T08:24:07.204 INFO:tasks.workunit.client.0.smithi070.stdout:5/734: chown dc/d1b/d1d/d25/d38/d39/c4d 14569137 1 2024-03-23T08:24:07.204 INFO:tasks.workunit.client.0.smithi070.stdout:6/895: creat d0/d23/d74/d84/db3/d2f/f121 x:0 0 0 2024-03-23T08:24:07.205 INFO:tasks.workunit.client.0.smithi070.stdout:5/735: dread - dc/d1b/d1d/d25/d36/d32/db7/fe8 zero size 2024-03-23T08:24:07.207 INFO:tasks.workunit.client.0.smithi070.stdout:6/896: read d0/d23/d74/d84/db3/d38/d52/f11b [1965263,34336] 0 2024-03-23T08:24:07.208 INFO:tasks.workunit.client.0.smithi070.stdout:5/736: getdents dc/d1b/d1d/d25/d36/d32/db7/de6 0 2024-03-23T08:24:07.210 INFO:tasks.workunit.client.0.smithi070.stdout:6/897: symlink d0/d23/d28/d35/l122 0 2024-03-23T08:24:07.211 INFO:tasks.workunit.client.0.smithi070.stdout:6/898: chown d0/d23/d28/d35/lb9 208695 1 2024-03-23T08:24:07.212 INFO:tasks.workunit.client.0.smithi070.stdout:6/899: chown d0/d23/d74/d84/d3c/d86/cd5 2532178 1 2024-03-23T08:24:07.214 INFO:tasks.workunit.client.0.smithi070.stdout:5/737: link dc/d1b/d1d/d25/d38/d48/cb0 dc/d19/cee 0 2024-03-23T08:24:07.214 INFO:tasks.workunit.client.0.smithi070.stdout:6/900: creat d0/d23/d97/d98/f123 x:0 0 0 2024-03-23T08:24:07.222 INFO:tasks.workunit.client.0.smithi070.stdout:6/901: getdents d0/d3/d2c 0 2024-03-23T08:24:07.227 INFO:tasks.workunit.client.0.smithi070.stdout:6/902: dwrite d0/d23/d28/d35/d85/f8b [0,4194304] 0 2024-03-23T08:24:07.229 INFO:tasks.workunit.client.0.smithi070.stdout:6/903: write d0/f9c [1161569,21891] 0 2024-03-23T08:24:07.240 INFO:tasks.workunit.client.0.smithi070.stdout:5/738: dwrite dc/d1b/d1d/d25/d38/d53/f60 [4194304,4194304] 0 2024-03-23T08:24:07.259 INFO:tasks.workunit.client.0.smithi070.stdout:5/739: dwrite dc/d1b/d1d/d25/fc2 [0,4194304] 0 2024-03-23T08:24:07.262 INFO:tasks.workunit.client.0.smithi070.stdout:5/740: creat dc/d1b/d1d/d25/d36/d32/db7/fef x:0 0 0 2024-03-23T08:24:07.264 INFO:tasks.workunit.client.0.smithi070.stdout:5/741: dread dc/f74 [0,4194304] 0 2024-03-23T08:24:07.267 INFO:tasks.workunit.client.0.smithi070.stdout:5/742: mkdir dc/d1b/d1d/d25/d36/d32/db7/de6/df0 0 2024-03-23T08:24:07.268 INFO:tasks.workunit.client.0.smithi070.stdout:5/743: truncate dc/d1b/fd4 261472 0 2024-03-23T08:24:07.271 INFO:tasks.workunit.client.0.smithi070.stdout:5/744: dread dc/d1b/fd4 [0,4194304] 0 2024-03-23T08:24:07.289 INFO:tasks.workunit.client.0.smithi070.stdout:5/745: write dc/d1b/d1d/fde [745441,66466] 0 2024-03-23T08:24:07.290 INFO:tasks.workunit.client.0.smithi070.stdout:5/746: fsync dc/d1b/d1d/d25/d38/d53/f7a 0 2024-03-23T08:24:07.293 INFO:tasks.workunit.client.0.smithi070.stdout:5/747: mknod dc/d1b/d1d/cf1 0 2024-03-23T08:24:07.299 INFO:tasks.workunit.client.0.smithi070.stdout:8/821: sync 2024-03-23T08:24:07.303 INFO:tasks.workunit.client.0.smithi070.stdout:8/822: creat d5/d8/d16/d30/d38/d6b/d72/f120 x:0 0 0 2024-03-23T08:24:07.308 INFO:tasks.workunit.client.0.smithi070.stdout:5/748: dread dc/f58 [0,4194304] 0 2024-03-23T08:24:07.309 INFO:tasks.workunit.client.0.smithi070.stdout:8/823: dwrite d5/d8/d16/d30/d38/fa4 [0,4194304] 0 2024-03-23T08:24:07.311 INFO:tasks.workunit.client.0.smithi070.stdout:5/749: mknod dc/d1b/d1d/d25/d38/d39/cf2 0 2024-03-23T08:24:07.312 INFO:tasks.workunit.client.0.smithi070.stdout:8/824: chown d5/d8/d16/d32/c36 371569726 1 2024-03-23T08:24:07.314 INFO:tasks.workunit.client.0.smithi070.stdout:8/825: mkdir d5/d8/d16/d121 0 2024-03-23T08:24:07.317 INFO:tasks.workunit.client.0.smithi070.stdout:8/826: creat d5/d8/d16/d30/d38/d6b/d72/d88/dbf/f122 x:0 0 0 2024-03-23T08:24:07.318 INFO:tasks.workunit.client.0.smithi070.stdout:5/750: dread dc/d1b/d1d/d25/d38/d53/f9a [0,4194304] 0 2024-03-23T08:24:07.320 INFO:tasks.workunit.client.0.smithi070.stdout:8/827: rename d5/l14 to d5/d8/d16/l123 0 2024-03-23T08:24:07.324 INFO:tasks.workunit.client.0.smithi070.stdout:8/828: mknod d5/d8/d16/d30/d38/d6b/c124 0 2024-03-23T08:24:07.335 INFO:tasks.workunit.client.0.smithi070.stdout:8/829: write d5/d8/f4e [500497,61437] 0 2024-03-23T08:24:07.338 INFO:tasks.workunit.client.0.smithi070.stdout:8/830: rename d5/lf4 to d5/d8/d16/d32/l125 0 2024-03-23T08:24:07.341 INFO:tasks.workunit.client.0.smithi070.stdout:8/831: creat d5/d8/d16/dd4/d115/f126 x:0 0 0 2024-03-23T08:24:07.342 INFO:tasks.workunit.client.0.smithi070.stdout:8/832: fsync d5/d8/f113 0 2024-03-23T08:24:07.347 INFO:tasks.workunit.client.0.smithi070.stdout:8/833: dwrite d5/d8/d16/d30/d38/fa4 [0,4194304] 0 2024-03-23T08:24:07.349 INFO:tasks.workunit.client.0.smithi070.stdout:8/834: stat d5/d8/d16/d3b/d8c/f99 0 2024-03-23T08:24:07.352 INFO:tasks.workunit.client.0.smithi070.stdout:8/835: mkdir d5/d8/d16/d30/d127 0 2024-03-23T08:24:07.355 INFO:tasks.workunit.client.0.smithi070.stdout:8/836: truncate d5/d8/d16/dd4/d6f/f89 5139685 0 2024-03-23T08:24:07.357 INFO:tasks.workunit.client.0.smithi070.stdout:8/837: read d5/d8/d16/d3b/d8c/f99 [682193,37598] 0 2024-03-23T08:24:07.363 INFO:tasks.workunit.client.0.smithi070.stdout:8/838: link d5/d8/d16/d32/d84/fdc d5/d8/d16/d30/d38/f128 0 2024-03-23T08:24:07.367 INFO:tasks.workunit.client.0.smithi070.stdout:8/839: creat d5/d8/d16/d30/d38/d6b/f129 x:0 0 0 2024-03-23T08:24:07.368 INFO:tasks.workunit.client.0.smithi070.stdout:8/840: readlink d5/d8/d16/d32/d84/d98/lf8 0 2024-03-23T08:24:07.381 INFO:tasks.workunit.client.0.smithi070.stdout:8/841: dwrite d5/d8/d16/dd4/d6f/f7e [0,4194304] 0 2024-03-23T08:24:07.387 INFO:tasks.workunit.client.0.smithi070.stdout:8/842: mknod d5/d8/d16/d3b/db9/d116/c12a 0 2024-03-23T08:24:07.388 INFO:tasks.workunit.client.0.smithi070.stdout:8/843: stat d5/d8/d16/d30/d38/lf3 0 2024-03-23T08:24:07.388 INFO:tasks.workunit.client.0.smithi070.stdout:8/844: read d5/d8/d16/f44 [120613,12545] 0 2024-03-23T08:24:07.399 INFO:tasks.workunit.client.0.smithi070.stdout:8/845: truncate d5/d8/d16/d30/f2f 3400890 0 2024-03-23T08:24:07.400 INFO:tasks.workunit.client.0.smithi070.stdout:8/846: write d5/d8/f4e [724606,121581] 0 2024-03-23T08:24:07.403 INFO:tasks.workunit.client.0.smithi070.stdout:8/847: mkdir d5/d8/d16/dd4/d6f/d7c/d12b 0 2024-03-23T08:24:07.406 INFO:tasks.workunit.client.0.smithi070.stdout:8/848: truncate d5/d8/d16/d30/fbc 6348 0 2024-03-23T08:24:07.406 INFO:tasks.workunit.client.0.smithi070.stdout:8/849: readlink d5/d8/d9a/dbe/lf5 0 2024-03-23T08:24:07.414 INFO:tasks.workunit.client.0.smithi070.stdout:8/850: creat d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/df2/f12c x:0 0 0 2024-03-23T08:24:07.416 INFO:tasks.workunit.client.0.smithi070.stdout:8/851: readlink d5/d8/d16/d30/d38/d6b/d72/l82 0 2024-03-23T08:24:07.425 INFO:tasks.workunit.client.0.smithi070.stdout:8/852: getdents d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/df2 0 2024-03-23T08:24:07.427 INFO:tasks.workunit.client.0.smithi070.stdout:8/853: creat d5/d8/d16/f12d x:0 0 0 2024-03-23T08:24:07.437 INFO:tasks.workunit.client.0.smithi070.stdout:8/854: write d5/d8/d16/dd4/fcb [4024985,33808] 0 2024-03-23T08:24:07.438 INFO:tasks.workunit.client.0.smithi070.stdout:8/855: stat d5/d8/d16/dd4/d6f/d7c/f80 0 2024-03-23T08:24:07.451 INFO:tasks.workunit.client.0.smithi070.stdout:8/856: write d5/d8/d16/dd4/d6f/d7c/d7d/f7f [3439197,48219] 0 2024-03-23T08:24:07.456 INFO:tasks.workunit.client.0.smithi070.stdout:8/857: rename d5/d8/d16/d30/d38/d6b/da9/c110 to d5/d8/d16/dd4/dc9/c12e 0 2024-03-23T08:24:07.458 INFO:tasks.workunit.client.0.smithi070.stdout:8/858: creat d5/d8/f12f x:0 0 0 2024-03-23T08:24:07.461 INFO:tasks.workunit.client.0.smithi070.stdout:8/859: rmdir d5/d8/dcc 0 2024-03-23T08:24:07.461 INFO:tasks.workunit.client.0.smithi070.stdout:8/860: chown d5/d8/d16/d30/d38/d6b/l78 124762 1 2024-03-23T08:24:07.464 INFO:tasks.workunit.client.0.smithi070.stdout:8/861: mknod d5/d8/d16/d32/d84/c130 0 2024-03-23T08:24:07.469 INFO:tasks.workunit.client.0.smithi070.stdout:8/862: dwrite d5/d8/d16/d30/f61 [0,4194304] 0 2024-03-23T08:24:07.470 INFO:tasks.workunit.client.0.smithi070.stdout:8/863: dread - d5/d8/d16/f12d zero size 2024-03-23T08:24:07.472 INFO:tasks.workunit.client.0.smithi070.stdout:8/864: fsync d5/d8/d16/d30/d38/fa4 0 2024-03-23T08:24:07.475 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:07 smithi070 ceph-mon[39699]: pgmap v43: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 36 MiB/s rd, 105 MiB/s wr, 315 op/s 2024-03-23T08:24:07.478 INFO:tasks.workunit.client.0.smithi070.stdout:8/865: truncate d5/d8/f25 549439 0 2024-03-23T08:24:07.487 INFO:tasks.workunit.client.0.smithi070.stdout:8/866: write d5/d8/d16/d3b/f76 [4732046,101737] 0 2024-03-23T08:24:07.490 INFO:tasks.workunit.client.0.smithi070.stdout:8/867: creat d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/f131 x:0 0 0 2024-03-23T08:24:07.490 INFO:tasks.workunit.client.0.smithi070.stdout:8/868: readlink d5/d8/d16/d32/d84/d98/dde/l9e 0 2024-03-23T08:24:07.492 INFO:tasks.workunit.client.0.smithi070.stdout:8/869: rmdir d5/d8/d16/d3b 39 2024-03-23T08:24:07.495 INFO:tasks.workunit.client.0.smithi070.stdout:8/870: creat d5/d8/d16/dd4/f132 x:0 0 0 2024-03-23T08:24:07.518 INFO:tasks.workunit.client.0.smithi070.stdout:3/894: sync 2024-03-23T08:24:07.519 INFO:tasks.workunit.client.0.smithi070.stdout:1/760: sync 2024-03-23T08:24:07.519 INFO:tasks.workunit.client.0.smithi070.stdout:0/955: sync 2024-03-23T08:24:07.519 INFO:tasks.workunit.client.0.smithi070.stdout:9/930: sync 2024-03-23T08:24:07.551 INFO:tasks.workunit.client.0.smithi070.stdout:8/871: symlink d5/d8/d16/d30/l133 0 2024-03-23T08:24:07.553 INFO:tasks.workunit.client.0.smithi070.stdout:0/956: creat d16/d26/d27/d4c/d60/d75/f14f x:0 0 0 2024-03-23T08:24:07.557 INFO:tasks.workunit.client.0.smithi070.stdout:8/872: dwrite d5/d8/f11 [0,4194304] 0 2024-03-23T08:24:07.561 INFO:tasks.workunit.client.0.smithi070.stdout:0/957: read d16/d22/fd3 [22648,22422] 0 2024-03-23T08:24:07.567 INFO:tasks.workunit.client.0.smithi070.stdout:9/931: creat db/d12/d115/d118/d11f/f135 x:0 0 0 2024-03-23T08:24:07.572 INFO:tasks.workunit.client.0.smithi070.stdout:4/720: sync 2024-03-23T08:24:07.572 INFO:tasks.workunit.client.0.smithi070.stdout:5/751: sync 2024-03-23T08:24:07.572 INFO:tasks.workunit.client.0.smithi070.stdout:7/826: sync 2024-03-23T08:24:07.572 INFO:tasks.workunit.client.0.smithi070.stdout:6/904: sync 2024-03-23T08:24:07.574 INFO:tasks.workunit.client.0.smithi070.stdout:7/827: chown d25/l2a 340101 1 2024-03-23T08:24:07.574 INFO:tasks.workunit.client.0.smithi070.stdout:0/958: chown d16/c1e 241074178 1 2024-03-23T08:24:07.577 INFO:tasks.workunit.client.0.smithi070.stdout:9/932: chown db/d12/d14/de5/da2/dc6/dce/d85/d42/c5b 24734 1 2024-03-23T08:24:07.578 INFO:tasks.workunit.client.0.smithi070.stdout:0/959: chown d16/d26/d27/d3d/d4d/dca/d113 1 1 2024-03-23T08:24:07.581 INFO:tasks.workunit.client.0.smithi070.stdout:5/752: dwrite dc/d1b/fd8 [0,4194304] 0 2024-03-23T08:24:07.588 INFO:tasks.workunit.client.0.smithi070.stdout:9/933: dwrite db/d12/d14/de5/da2/d4f/db6/d79/f114 [0,4194304] 0 2024-03-23T08:24:07.591 INFO:tasks.workunit.client.0.smithi070.stdout:9/934: fsync db/d12/d14/de5/da2/d26/f30 0 2024-03-23T08:24:07.592 INFO:tasks.workunit.client.0.smithi070.stdout:3/895: getdents db/d20/d63/d83/dfe 0 2024-03-23T08:24:07.594 INFO:tasks.workunit.client.0.smithi070.stdout:3/896: stat db/d14/d9c/ca5 0 2024-03-23T08:24:07.597 INFO:tasks.workunit.client.0.smithi070.stdout:1/761: getdents d8/d1b 0 2024-03-23T08:24:07.599 INFO:tasks.workunit.client.0.smithi070.stdout:0/960: chown d16/lf4 1 1 2024-03-23T08:24:07.602 INFO:tasks.workunit.client.0.smithi070.stdout:0/961: readlink d16/d26/d27/d3d/d4d/dca/l117 0 2024-03-23T08:24:07.602 INFO:tasks.workunit.client.0.smithi070.stdout:0/962: chown d16/d26/d27/d3d/lbb 6 1 2024-03-23T08:24:07.604 INFO:tasks.workunit.client.0.smithi070.stdout:4/721: creat d3/d6/d35/d65/df1/ffc x:0 0 0 2024-03-23T08:24:07.608 INFO:tasks.workunit.client.0.smithi070.stdout:9/935: mkdir db/d12/d14/de5/da2/d26/dec/d12b/d136 0 2024-03-23T08:24:07.611 INFO:tasks.workunit.client.0.smithi070.stdout:3/897: rename db/d20/d7a/ld2 to db/d4e/dc4/l14e 0 2024-03-23T08:24:07.616 INFO:tasks.workunit.client.0.smithi070.stdout:5/753: mknod dc/d1b/d1d/d25/da9/deb/cf3 0 2024-03-23T08:24:07.618 INFO:tasks.workunit.client.0.smithi070.stdout:0/963: truncate d16/d26/d27/ffe 852631 0 2024-03-23T08:24:07.619 INFO:tasks.workunit.client.0.smithi070.stdout:4/722: dread - d3/d6/d35/d65/d27/d4b/fd7 zero size 2024-03-23T08:24:07.622 INFO:tasks.workunit.client.0.smithi070.stdout:9/936: creat db/d12/d91/f137 x:0 0 0 2024-03-23T08:24:07.623 INFO:tasks.workunit.client.0.smithi070.stdout:3/898: readlink db/d4e/d92/la3 0 2024-03-23T08:24:07.631 INFO:tasks.workunit.client.0.smithi070.stdout:9/937: dwrite db/d12/d14/de5/f10b [0,4194304] 0 2024-03-23T08:24:07.632 INFO:tasks.workunit.client.0.smithi070.stdout:3/899: dwrite db/d14/d43/d62/f69 [0,4194304] 0 2024-03-23T08:24:07.634 INFO:tasks.workunit.client.0.smithi070.stdout:0/964: write d16/d26/d27/d86/d67/d109/f133 [1928133,86328] 0 2024-03-23T08:24:07.635 INFO:tasks.workunit.client.0.smithi070.stdout:4/723: creat d3/d6/d35/d3f/d51/d6e/ffd x:0 0 0 2024-03-23T08:24:07.638 INFO:tasks.workunit.client.0.smithi070.stdout:0/965: write d16/d26/d27/d86/db6/fdc [1875641,95159] 0 2024-03-23T08:24:07.645 INFO:tasks.workunit.client.0.smithi070.stdout:9/938: chown db/d12/d14/de5/da2/d26/d80/l104 14284 1 2024-03-23T08:24:07.646 INFO:tasks.workunit.client.0.smithi070.stdout:3/900: mknod db/d4e/d92/c14f 0 2024-03-23T08:24:07.650 INFO:tasks.workunit.client.0.smithi070.stdout:4/724: unlink d3/d6/d6a/c4d 0 2024-03-23T08:24:07.650 INFO:tasks.workunit.client.0.smithi070.stdout:4/725: stat d3/f17 0 2024-03-23T08:24:07.655 INFO:tasks.workunit.client.0.smithi070.stdout:0/966: mkdir d16/d26/d27/d86/d67/d7d/d150 0 2024-03-23T08:24:07.657 INFO:tasks.workunit.client.0.smithi070.stdout:4/726: mkdir d3/d6/d35/dcf/db5/dda/db8/dfe 0 2024-03-23T08:24:07.660 INFO:tasks.workunit.client.0.smithi070.stdout:9/939: dread db/d12/d14/de5/da2/f24 [0,4194304] 0 2024-03-23T08:24:07.664 INFO:tasks.workunit.client.0.smithi070.stdout:4/727: dwrite d3/d6/d35/f67 [0,4194304] 0 2024-03-23T08:24:07.667 INFO:tasks.workunit.client.0.smithi070.stdout:9/940: symlink db/d12/d14/de5/da2/d58/db5/l138 0 2024-03-23T08:24:07.672 INFO:tasks.workunit.client.0.smithi070.stdout:9/941: chown db/d12/d14/de5/da2/dc6/dce/d85/l76 2307 1 2024-03-23T08:24:07.673 INFO:tasks.workunit.client.0.smithi070.stdout:9/942: chown db/d12/d14/de5/da2/d26/dec/dfa 39 1 2024-03-23T08:24:07.677 INFO:tasks.workunit.client.0.smithi070.stdout:4/728: link d3/d6/d35/d65/d6f/dee/fb9 d3/d6/d35/dcf/db5/dda/dc7/fff 0 2024-03-23T08:24:07.678 INFO:tasks.workunit.client.0.smithi070.stdout:9/943: symlink db/d12/d14/de5/da2/dc6/dce/d85/d11a/l139 0 2024-03-23T08:24:07.678 INFO:tasks.workunit.client.0.smithi070.stdout:9/944: fsync db/db8/ff1 0 2024-03-23T08:24:07.680 INFO:tasks.workunit.client.0.smithi070.stdout:4/729: mknod d3/d6/d35/dcf/db5/dbb/c100 0 2024-03-23T08:24:07.681 INFO:tasks.workunit.client.0.smithi070.stdout:4/730: chown d3/d6/d35/d65/df1/ffc 45826023 1 2024-03-23T08:24:07.682 INFO:tasks.workunit.client.0.smithi070.stdout:4/731: fdatasync d3/d6/d35/d65/df1/ffc 0 2024-03-23T08:24:07.692 INFO:tasks.workunit.client.0.smithi070.stdout:8/873: truncate d5/d8/d9a/da2/dae/f94 2932172 0 2024-03-23T08:24:07.693 INFO:tasks.workunit.client.0.smithi070.stdout:8/874: write d5/d8/d16/d30/d38/d6b/d72/f120 [699670,76676] 0 2024-03-23T08:24:07.700 INFO:tasks.workunit.client.0.smithi070.stdout:7/828: dwrite d25/d27/d6a/d44/d7c/db4/fb7 [0,4194304] 0 2024-03-23T08:24:07.702 INFO:tasks.workunit.client.0.smithi070.stdout:7/829: read d25/d27/f4c [1351410,48937] 0 2024-03-23T08:24:07.711 INFO:tasks.workunit.client.0.smithi070.stdout:1/762: dwrite d8/db/d40/d43/d64/d85/d97/d9a/fb7 [0,4194304] 0 2024-03-23T08:24:07.712 INFO:tasks.workunit.client.0.smithi070.stdout:5/754: write dc/d1b/fd4 [275847,96021] 0 2024-03-23T08:24:07.713 INFO:tasks.workunit.client.0.smithi070.stdout:1/763: read d8/d42/d4e/d99/fda [3370935,41342] 0 2024-03-23T08:24:07.714 INFO:tasks.workunit.client.0.smithi070.stdout:5/755: write dc/d1b/d1d/d25/d38/d39/f79 [3291096,20031] 0 2024-03-23T08:24:07.716 INFO:tasks.workunit.client.0.smithi070.stdout:1/764: fsync d8/f9 0 2024-03-23T08:24:07.720 INFO:tasks.workunit.client.0.smithi070.stdout:5/756: mknod dc/cf4 0 2024-03-23T08:24:07.723 INFO:tasks.workunit.client.0.smithi070.stdout:1/765: link d8/db/d8e/ff8 d8/db/d2c/d3f/d48/f116 0 2024-03-23T08:24:07.725 INFO:tasks.workunit.client.0.smithi070.stdout:5/757: creat dc/d1b/d1d/d25/d36/d32/db7/de6/ff5 x:0 0 0 2024-03-23T08:24:07.727 INFO:tasks.workunit.client.0.smithi070.stdout:1/766: read d8/d1a/fe3 [1071618,105906] 0 2024-03-23T08:24:07.728 INFO:tasks.workunit.client.0.smithi070.stdout:5/758: getdents dc/d1b/d1d/d25/d38/d39/dbc 0 2024-03-23T08:24:07.730 INFO:tasks.workunit.client.0.smithi070.stdout:1/767: creat d8/d26/d49/f117 x:0 0 0 2024-03-23T08:24:07.730 INFO:tasks.workunit.client.0.smithi070.stdout:1/768: chown d8/d26 1719 1 2024-03-23T08:24:07.731 INFO:tasks.workunit.client.0.smithi070.stdout:1/769: write d8/d42/d4e/f66 [152575,61552] 0 2024-03-23T08:24:07.732 INFO:tasks.workunit.client.0.smithi070.stdout:1/770: dread - d8/db/d2c/d3f/f6e zero size 2024-03-23T08:24:07.734 INFO:tasks.workunit.client.0.smithi070.stdout:8/875: rename d5/d8/d16/d30/d38/ca8 to d5/d8/d16/d32/d84/da7/deb/c134 0 2024-03-23T08:24:07.744 INFO:tasks.workunit.client.0.smithi070.stdout:1/771: dwrite d8/db/d40/d43/d64/d85/d97/fff [0,4194304] 0 2024-03-23T08:24:07.744 INFO:tasks.workunit.client.0.smithi070.stdout:0/967: dwrite d16/d26/d27/f45 [0,4194304] 0 2024-03-23T08:24:07.751 INFO:tasks.workunit.client.0.smithi070.stdout:5/759: dread dc/d1b/d1d/d25/da9/fcc [0,4194304] 0 2024-03-23T08:24:07.751 INFO:tasks.workunit.client.0.smithi070.stdout:1/772: write d8/d26/d7b/d9c/fde [1523442,41686] 0 2024-03-23T08:24:07.751 INFO:tasks.workunit.client.0.smithi070.stdout:0/968: mknod d16/d26/d27/d4c/d60/d75/c151 0 2024-03-23T08:24:07.753 INFO:tasks.workunit.client.0.smithi070.stdout:8/876: dread d5/f23 [0,4194304] 0 2024-03-23T08:24:07.755 INFO:tasks.workunit.client.0.smithi070.stdout:1/773: stat d8/db/d100/d74/d5d/f87 0 2024-03-23T08:24:07.757 INFO:tasks.workunit.client.0.smithi070.stdout:4/732: write d3/d6/d45/f47 [468349,109868] 0 2024-03-23T08:24:07.758 INFO:tasks.workunit.client.0.smithi070.stdout:0/969: chown d16/d22/d43/d47/l13f 351325 1 2024-03-23T08:24:07.762 INFO:tasks.workunit.client.0.smithi070.stdout:7/830: write f24 [117702,20291] 0 2024-03-23T08:24:07.768 INFO:tasks.workunit.client.0.smithi070.stdout:9/945: dwrite f8 [0,4194304] 0 2024-03-23T08:24:07.776 INFO:tasks.workunit.client.0.smithi070.stdout:7/831: creat d25/d27/d6a/d44/da4/f11f x:0 0 0 2024-03-23T08:24:07.778 INFO:tasks.workunit.client.0.smithi070.stdout:4/733: dwrite d3/d6/d35/dcf/fe7 [0,4194304] 0 2024-03-23T08:24:07.779 INFO:tasks.workunit.client.0.smithi070.stdout:7/832: write d25/d27/d6a/d44/d6f/da3/f103 [806745,119765] 0 2024-03-23T08:24:07.780 INFO:tasks.workunit.client.0.smithi070.stdout:5/760: link dc/d1b/l7e dc/d1b/d1d/d25/d36/d32/d5c/dcb/dec/lf6 0 2024-03-23T08:24:07.781 INFO:tasks.workunit.client.0.smithi070.stdout:8/877: write d5/d8/f9d [1105180,113066] 0 2024-03-23T08:24:07.790 INFO:tasks.workunit.client.0.smithi070.stdout:1/774: truncate d8/db/d2c/d3f/faa 207884 0 2024-03-23T08:24:07.792 INFO:tasks.workunit.client.0.smithi070.stdout:7/833: rmdir d25/d27/d6a/d44/da2 39 2024-03-23T08:24:07.793 INFO:tasks.workunit.client.0.smithi070.stdout:4/734: dwrite d3/f62 [0,4194304] 0 2024-03-23T08:24:07.796 INFO:tasks.workunit.client.0.smithi070.stdout:0/970: dwrite d16/f68 [4194304,4194304] 0 2024-03-23T08:24:07.796 INFO:tasks.workunit.client.0.smithi070.stdout:7/834: dread - d25/d27/d6a/fec zero size 2024-03-23T08:24:07.797 INFO:tasks.workunit.client.0.smithi070.stdout:8/878: rename d5/d8/d16/dd4/d6f/cb6 to d5/d8/d16/d30/d38/d6b/d72/d118/c135 0 2024-03-23T08:24:07.798 INFO:tasks.workunit.client.0.smithi070.stdout:5/761: truncate dc/d1b/d1d/d25/d38/d53/fa1 3742455 0 2024-03-23T08:24:07.801 INFO:tasks.workunit.client.0.smithi070.stdout:1/775: mkdir d8/d26/d108/d118 0 2024-03-23T08:24:07.803 INFO:tasks.workunit.client.0.smithi070.stdout:1/776: readlink d8/db/d2c/d3f/la6 0 2024-03-23T08:24:07.803 INFO:tasks.workunit.client.0.smithi070.stdout:7/835: fsync d25/d40/f61 0 2024-03-23T08:24:07.809 INFO:tasks.workunit.client.0.smithi070.stdout:8/879: unlink d5/d8/d16/dd4/d6f/d7c/f80 0 2024-03-23T08:24:07.810 INFO:tasks.workunit.client.0.smithi070.stdout:9/946: write db/d12/d14/de5/da2/dc6/dce/d85/f7d [756209,70747] 0 2024-03-23T08:24:07.810 INFO:tasks.workunit.client.0.smithi070.stdout:4/735: dwrite d3/d6/d35/d65/df1/ffc [0,4194304] 0 2024-03-23T08:24:07.812 INFO:tasks.workunit.client.0.smithi070.stdout:5/762: mknod dc/d1b/d1d/cf7 0 2024-03-23T08:24:07.815 INFO:tasks.workunit.client.0.smithi070.stdout:4/736: chown d3/d6/d9/lab 0 1 2024-03-23T08:24:07.817 INFO:tasks.workunit.client.0.smithi070.stdout:0/971: dwrite d16/d26/d27/d86/d140/df8/d11a/f13e [0,4194304] 0 2024-03-23T08:24:07.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:07 smithi193 ceph-mon[48370]: pgmap v43: 65 pgs: 65 active+clean; 1.7 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 36 MiB/s rd, 105 MiB/s wr, 315 op/s 2024-03-23T08:24:07.826 INFO:tasks.workunit.client.0.smithi070.stdout:7/836: stat d25/d27/d6a/d44/da2/lae 0 2024-03-23T08:24:07.827 INFO:tasks.workunit.client.0.smithi070.stdout:1/777: truncate d8/d26/f7f 4133656 0 2024-03-23T08:24:07.828 INFO:tasks.workunit.client.0.smithi070.stdout:7/837: chown d25/d27/d6a/d44/d85/d95 5746 1 2024-03-23T08:24:07.828 INFO:tasks.workunit.client.0.smithi070.stdout:7/838: chown d25/d27/f4c 208 1 2024-03-23T08:24:07.830 INFO:tasks.workunit.client.0.smithi070.stdout:8/880: symlink d5/d8/d16/d30/d38/d6b/da9/l136 0 2024-03-23T08:24:07.832 INFO:tasks.workunit.client.0.smithi070.stdout:9/947: mkdir db/d12/d115/d118/d11f/d13a 0 2024-03-23T08:24:07.834 INFO:tasks.workunit.client.0.smithi070.stdout:4/737: mkdir d3/d6/dca/d101 0 2024-03-23T08:24:07.835 INFO:tasks.workunit.client.0.smithi070.stdout:4/738: write d3/d6/d35/d65/d1b/f90 [1211802,42090] 0 2024-03-23T08:24:07.842 INFO:tasks.workunit.client.0.smithi070.stdout:8/881: fdatasync d5/d8/d16/d32/d84/fef 0 2024-03-23T08:24:07.848 INFO:tasks.workunit.client.0.smithi070.stdout:5/763: mkdir dc/d1b/d1d/d25/d68/d8a/dad/df8 0 2024-03-23T08:24:07.856 INFO:tasks.workunit.client.0.smithi070.stdout:7/839: dread d25/d27/f92 [0,4194304] 0 2024-03-23T08:24:07.861 INFO:tasks.workunit.client.0.smithi070.stdout:1/778: dwrite d8/d42/d51/f67 [0,4194304] 0 2024-03-23T08:24:07.862 INFO:tasks.workunit.client.0.smithi070.stdout:0/972: rename d16/f3e to d16/d26/d27/d4c/d60/d7c/f152 0 2024-03-23T08:24:07.863 INFO:tasks.workunit.client.0.smithi070.stdout:5/764: dread dc/d1b/d1d/d25/d36/d32/d5c/f65 [0,4194304] 0 2024-03-23T08:24:07.864 INFO:tasks.workunit.client.0.smithi070.stdout:0/973: fdatasync d16/d26/d27/f91 0 2024-03-23T08:24:07.864 INFO:tasks.workunit.client.0.smithi070.stdout:7/840: dread d25/d27/d6a/d44/d85/f93 [0,4194304] 0 2024-03-23T08:24:07.867 INFO:tasks.workunit.client.0.smithi070.stdout:9/948: write db/d12/d14/de5/da2/dc6/d7e/f75 [349921,106895] 0 2024-03-23T08:24:07.867 INFO:tasks.workunit.client.0.smithi070.stdout:7/841: read d25/d27/d6a/f9c [127004,12380] 0 2024-03-23T08:24:07.868 INFO:tasks.workunit.client.0.smithi070.stdout:9/949: chown db/d12/d14/de5/da2/d26/dec/d12b/fd7 7 1 2024-03-23T08:24:07.869 INFO:tasks.workunit.client.0.smithi070.stdout:1/779: rmdir d8/d11 39 2024-03-23T08:24:07.871 INFO:tasks.workunit.client.0.smithi070.stdout:8/882: rename d5/d8/d16/dd4/d6f/d7c/d7d/fe4 to d5/d8/d16/d32/de0/f137 0 2024-03-23T08:24:07.875 INFO:tasks.workunit.client.0.smithi070.stdout:7/842: dwrite d25/d27/d6a/fec [0,4194304] 0 2024-03-23T08:24:07.877 INFO:tasks.workunit.client.0.smithi070.stdout:7/843: stat d25 0 2024-03-23T08:24:07.881 INFO:tasks.workunit.client.0.smithi070.stdout:9/950: creat db/d12/d115/d118/d10f/f13b x:0 0 0 2024-03-23T08:24:07.882 INFO:tasks.workunit.client.0.smithi070.stdout:1/780: dwrite d8/db/d100/ff0 [0,4194304] 0 2024-03-23T08:24:07.885 INFO:tasks.workunit.client.0.smithi070.stdout:8/883: dwrite d5/d8/d16/d30/d38/d6b/f129 [0,4194304] 0 2024-03-23T08:24:07.892 INFO:tasks.workunit.client.0.smithi070.stdout:7/844: fsync d25/d27/d6a/f86 0 2024-03-23T08:24:07.892 INFO:tasks.workunit.client.0.smithi070.stdout:4/739: write d3/d6/d35/d65/f30 [1023577,21054] 0 2024-03-23T08:24:07.894 INFO:tasks.workunit.client.0.smithi070.stdout:7/845: chown d25/c64 8516 1 2024-03-23T08:24:07.895 INFO:tasks.workunit.client.0.smithi070.stdout:5/765: fdatasync dc/d1b/d1d/d25/d38/d53/fa1 0 2024-03-23T08:24:07.898 INFO:tasks.workunit.client.0.smithi070.stdout:5/766: fdatasync dc/d1b/d1d/d25/d36/d32/f87 0 2024-03-23T08:24:07.898 INFO:tasks.workunit.client.0.smithi070.stdout:9/951: mknod db/d12/d115/d118/d10f/c13c 0 2024-03-23T08:24:07.899 INFO:tasks.workunit.client.0.smithi070.stdout:5/767: dread - dc/d1b/d1d/d25/d36/fd3 zero size 2024-03-23T08:24:07.901 INFO:tasks.workunit.client.0.smithi070.stdout:4/740: unlink d3/d6/d9/l15 0 2024-03-23T08:24:07.901 INFO:tasks.workunit.client.0.smithi070.stdout:5/768: truncate dc/d1b/d1d/d25/d36/d32/db7/fef 911965 0 2024-03-23T08:24:07.908 INFO:tasks.workunit.client.0.smithi070.stdout:4/741: rmdir d3/d6/d35/d65/d27 39 2024-03-23T08:24:07.909 INFO:tasks.workunit.client.0.smithi070.stdout:4/742: fdatasync d3/d6/d35/d65/f3e 0 2024-03-23T08:24:07.910 INFO:tasks.workunit.client.0.smithi070.stdout:1/781: rmdir d8/d26/d41/dd9 0 2024-03-23T08:24:07.913 INFO:tasks.workunit.client.0.smithi070.stdout:1/782: stat d8/d26/d41/fae 0 2024-03-23T08:24:07.916 INFO:tasks.workunit.client.0.smithi070.stdout:9/952: dread f6 [4194304,4194304] 0 2024-03-23T08:24:07.916 INFO:tasks.workunit.client.0.smithi070.stdout:8/884: rename d5/c2e to d5/d8/d16/d32/d84/c138 0 2024-03-23T08:24:07.917 INFO:tasks.workunit.client.0.smithi070.stdout:7/846: getdents d25/d27/d6a/d44/d7c 0 2024-03-23T08:24:07.925 INFO:tasks.workunit.client.0.smithi070.stdout:0/974: truncate d16/d26/d27/d4c/d60/d7c/f152 3233548 0 2024-03-23T08:24:07.925 INFO:tasks.workunit.client.0.smithi070.stdout:5/769: rename dc/d1b/d1d/d25/d38/f40 to dc/d1b/d1d/d25/d36/d32/db7/ff9 0 2024-03-23T08:24:07.928 INFO:tasks.workunit.client.0.smithi070.stdout:8/885: mknod d5/d8/d16/d32/d84/d98/c139 0 2024-03-23T08:24:07.928 INFO:tasks.workunit.client.0.smithi070.stdout:9/953: write db/d12/d14/de5/da2/dc6/dce/d85/feb [4698061,66583] 0 2024-03-23T08:24:07.930 INFO:tasks.workunit.client.0.smithi070.stdout:5/770: write dc/d1b/d1d/d61/f94 [670529,78404] 0 2024-03-23T08:24:07.932 INFO:tasks.workunit.client.0.smithi070.stdout:8/886: dread - d5/d8/d16/f12d zero size 2024-03-23T08:24:07.934 INFO:tasks.workunit.client.0.smithi070.stdout:0/975: dread d16/d22/f2c [0,4194304] 0 2024-03-23T08:24:07.936 INFO:tasks.workunit.client.0.smithi070.stdout:9/954: dread db/d12/d14/de5/da2/d58/db5/fc3 [0,4194304] 0 2024-03-23T08:24:07.936 INFO:tasks.workunit.client.0.smithi070.stdout:6/905: sync 2024-03-23T08:24:07.936 INFO:tasks.workunit.client.0.smithi070.stdout:7/847: dread d25/f34 [0,4194304] 0 2024-03-23T08:24:07.943 INFO:tasks.workunit.client.0.smithi070.stdout:6/906: dwrite d0/d23/d97/d98/f123 [0,4194304] 0 2024-03-23T08:24:07.945 INFO:tasks.workunit.client.0.smithi070.stdout:6/907: chown d0/d23/d97/d98/lb7 597836 1 2024-03-23T08:24:07.977 INFO:tasks.workunit.client.0.smithi070.stdout:5/771: rename dc/d1b/fd4 to dc/d1b/d1d/d25/d68/d8a/dad/df8/ffa 0 2024-03-23T08:24:07.983 INFO:tasks.workunit.client.0.smithi070.stdout:0/976: unlink d16/d26/d27/d86/d9e/l7b 0 2024-03-23T08:24:07.983 INFO:tasks.workunit.client.0.smithi070.stdout:0/977: stat d16/d26/d27/d86/d140 0 2024-03-23T08:24:07.984 INFO:tasks.workunit.client.0.smithi070.stdout:7/848: mknod d25/d27/d6a/d44/d6f/dab/dcd/c120 0 2024-03-23T08:24:07.989 INFO:tasks.workunit.client.0.smithi070.stdout:4/743: dwrite d3/d6/d35/d65/d27/d4b/fd7 [0,4194304] 0 2024-03-23T08:24:07.995 INFO:tasks.workunit.client.0.smithi070.stdout:1/783: write d8/db/d40/d43/d64/dc2/ff4 [731496,21618] 0 2024-03-23T08:24:08.000 INFO:tasks.workunit.client.0.smithi070.stdout:5/772: truncate dc/d1b/d1d/d25/d38/d53/f5f 1390079 0 2024-03-23T08:24:08.003 INFO:tasks.workunit.client.0.smithi070.stdout:3/901: sync 2024-03-23T08:24:08.003 INFO:tasks.workunit.client.0.smithi070.stdout:7/849: write d25/d27/d6a/fb1 [2971107,53491] 0 2024-03-23T08:24:08.007 INFO:tasks.workunit.client.0.smithi070.stdout:4/744: rmdir d3/d6/d45 39 2024-03-23T08:24:08.009 INFO:tasks.workunit.client.0.smithi070.stdout:1/784: fdatasync d8/d42/d4e/f59 0 2024-03-23T08:24:08.012 INFO:tasks.workunit.client.0.smithi070.stdout:9/955: rename db/d12/d14/de5/da2/d26/cf2 to db/d12/d115/d118/d11f/d13a/c13d 0 2024-03-23T08:24:08.017 INFO:tasks.workunit.client.0.smithi070.stdout:5/773: symlink dc/d1b/d1d/d25/d38/d48/lfb 0 2024-03-23T08:24:08.018 INFO:tasks.workunit.client.0.smithi070.stdout:3/902: symlink db/df/d6a/l150 0 2024-03-23T08:24:08.021 INFO:tasks.workunit.client.0.smithi070.stdout:4/745: mknod d3/d6/d35/dcf/db5/dda/db8/d88/c102 0 2024-03-23T08:24:08.024 INFO:tasks.workunit.client.0.smithi070.stdout:4/746: truncate d3/d6/d6a/ff5 294930 0 2024-03-23T08:24:08.025 INFO:tasks.workunit.client.0.smithi070.stdout:4/747: chown d3/f8f 115906645 1 2024-03-23T08:24:08.028 INFO:tasks.workunit.client.0.smithi070.stdout:0/978: rename d16/d26/d27/d3d/d4d/fbd to d16/d26/d27/d86/d9e/f153 0 2024-03-23T08:24:08.031 INFO:tasks.workunit.client.0.smithi070.stdout:0/979: write d16/f68 [8833026,24586] 0 2024-03-23T08:24:08.032 INFO:tasks.workunit.client.0.smithi070.stdout:7/850: creat d25/d27/d112/f121 x:0 0 0 2024-03-23T08:24:08.035 INFO:tasks.workunit.client.0.smithi070.stdout:7/851: chown d25/d27/d6a/d44/da2/db5 58 1 2024-03-23T08:24:08.036 INFO:tasks.workunit.client.0.smithi070.stdout:5/774: rmdir dc/d1b/d1d/d25/d38/d48/dc8 39 2024-03-23T08:24:08.040 INFO:tasks.workunit.client.0.smithi070.stdout:4/748: unlink d3/d6/d9/f2e 0 2024-03-23T08:24:08.041 INFO:tasks.workunit.client.0.smithi070.stdout:5/775: chown dc/d1b/d1d/d25/d38/c70 1450 1 2024-03-23T08:24:08.042 INFO:tasks.workunit.client.0.smithi070.stdout:5/776: dread - dc/fa6 zero size 2024-03-23T08:24:08.043 INFO:tasks.workunit.client.0.smithi070.stdout:9/956: fdatasync db/d12/f9a 0 2024-03-23T08:24:08.044 INFO:tasks.workunit.client.0.smithi070.stdout:0/980: truncate d16/f100 4179343 0 2024-03-23T08:24:08.047 INFO:tasks.workunit.client.0.smithi070.stdout:1/785: rmdir d8/db/d2c/d3f/d48/dab/df3 0 2024-03-23T08:24:08.049 INFO:tasks.workunit.client.0.smithi070.stdout:4/749: write d3/d6/d6a/d7c/f9c [3784245,128073] 0 2024-03-23T08:24:08.051 INFO:tasks.workunit.client.0.smithi070.stdout:0/981: readlink d16/d26/d27/d3d/d4d/dca/l117 0 2024-03-23T08:24:08.052 INFO:tasks.workunit.client.0.smithi070.stdout:3/903: creat db/d4e/df8/d22/f151 x:0 0 0 2024-03-23T08:24:08.054 INFO:tasks.workunit.client.0.smithi070.stdout:5/777: rename dc/d1b/d1d/d61 to dc/d1b/d1d/d25/d36/dfc 0 2024-03-23T08:24:08.056 INFO:tasks.workunit.client.0.smithi070.stdout:7/852: dread d25/d27/f4c [0,4194304] 0 2024-03-23T08:24:08.059 INFO:tasks.workunit.client.0.smithi070.stdout:4/750: symlink d3/d6/d35/d65/d6f/l103 0 2024-03-23T08:24:08.062 INFO:tasks.workunit.client.0.smithi070.stdout:1/786: rename d8/d26/d7b/fc1 to d8/db/d40/d43/d64/d85/d97/de7/f119 0 2024-03-23T08:24:08.063 INFO:tasks.workunit.client.0.smithi070.stdout:1/787: chown d8/d1b/f4b 287206384 1 2024-03-23T08:24:08.064 INFO:tasks.workunit.client.0.smithi070.stdout:7/853: creat d25/d27/d6a/d44/dfe/f122 x:0 0 0 2024-03-23T08:24:08.065 INFO:tasks.workunit.client.0.smithi070.stdout:1/788: truncate d8/db/d100/f95 376073 0 2024-03-23T08:24:08.067 INFO:tasks.workunit.client.0.smithi070.stdout:0/982: symlink d16/d26/d27/d86/l154 0 2024-03-23T08:24:08.068 INFO:tasks.workunit.client.0.smithi070.stdout:9/957: dread db/d12/d14/f35 [0,4194304] 0 2024-03-23T08:24:08.071 INFO:tasks.workunit.client.0.smithi070.stdout:7/854: dread - d25/d27/d6a/d44/da2/df4/ff5 zero size 2024-03-23T08:24:08.072 INFO:tasks.workunit.client.0.smithi070.stdout:4/751: getdents d3/d6/d35/d65/d27/d4b/d8b 0 2024-03-23T08:24:08.072 INFO:tasks.workunit.client.0.smithi070.stdout:4/752: chown d3/d6/d35/l44 40 1 2024-03-23T08:24:08.073 INFO:tasks.workunit.client.0.smithi070.stdout:0/983: unlink d16/d26/d27/d4c/d60/d7c/fcf 0 2024-03-23T08:24:08.075 INFO:tasks.workunit.client.0.smithi070.stdout:0/984: truncate d16/d26/d27/d4c/f135 802148 0 2024-03-23T08:24:08.076 INFO:tasks.workunit.client.0.smithi070.stdout:1/789: dread d8/d1a/d8f/fa3 [0,4194304] 0 2024-03-23T08:24:08.076 INFO:tasks.workunit.client.0.smithi070.stdout:9/958: chown db/d12/d14/de5/da2/dc6/dce/d85/d42/ccc 691946086 1 2024-03-23T08:24:08.078 INFO:tasks.workunit.client.0.smithi070.stdout:9/959: chown db/d12/d14/de5/da2/d4f/l73 1154724 1 2024-03-23T08:24:08.078 INFO:tasks.workunit.client.0.smithi070.stdout:4/753: mknod d3/d6/d35/d65/c104 0 2024-03-23T08:24:08.081 INFO:tasks.workunit.client.0.smithi070.stdout:1/790: rename d8/db/d40/d43/d64/d85/dc7/fe9 to d8/db/d2c/d3f/dd3/d111/f11a 0 2024-03-23T08:24:08.082 INFO:tasks.workunit.client.0.smithi070.stdout:7/855: getdents d25/d27/d6a/d44/da2/df4 0 2024-03-23T08:24:08.086 INFO:tasks.workunit.client.0.smithi070.stdout:9/960: mknod db/d12/d14/de5/da2/d4f/c13e 0 2024-03-23T08:24:08.086 INFO:tasks.workunit.client.0.smithi070.stdout:4/754: creat d3/d6/dd8/f105 x:0 0 0 2024-03-23T08:24:08.089 INFO:tasks.workunit.client.0.smithi070.stdout:1/791: read d8/d11/f5c [178102,81232] 0 2024-03-23T08:24:08.089 INFO:tasks.workunit.client.0.smithi070.stdout:9/961: mkdir db/d12/d14/de5/da2/dc6/dce/d85/d42/d13f 0 2024-03-23T08:24:08.092 INFO:tasks.workunit.client.0.smithi070.stdout:0/985: getdents d16/d26/d27/d86/d140/df8 0 2024-03-23T08:24:08.093 INFO:tasks.workunit.client.0.smithi070.stdout:7/856: link d25/d27/d6a/d44/d85/d95/ld5 d25/d27/d6a/d44/d6f/l123 0 2024-03-23T08:24:08.094 INFO:tasks.workunit.client.0.smithi070.stdout:9/962: write db/d12/d14/de5/da2/d4f/db6/f99 [610358,49802] 0 2024-03-23T08:24:08.095 INFO:tasks.workunit.client.0.smithi070.stdout:0/986: readlink d16/d26/d27/d4c/d60/d7c/l13a 0 2024-03-23T08:24:08.096 INFO:tasks.workunit.client.0.smithi070.stdout:4/755: dwrite d3/d6/d35/d3f/d51/d6e/ffd [0,4194304] 0 2024-03-23T08:24:08.097 INFO:tasks.workunit.client.0.smithi070.stdout:1/792: rename d8/d26/l2d to d8/d1b/l11b 0 2024-03-23T08:24:08.100 INFO:tasks.workunit.client.0.smithi070.stdout:4/756: write d3/d6/d35/d65/d27/f4e [2048865,59025] 0 2024-03-23T08:24:08.100 INFO:tasks.workunit.client.0.smithi070.stdout:0/987: readlink ld 0 2024-03-23T08:24:08.103 INFO:tasks.workunit.client.0.smithi070.stdout:1/793: chown d8/d26/d41/fca 4418009 1 2024-03-23T08:24:08.103 INFO:tasks.workunit.client.0.smithi070.stdout:9/963: creat db/d12/d14/de5/da2/d26/dec/d12b/d136/f140 x:0 0 0 2024-03-23T08:24:08.106 INFO:tasks.workunit.client.0.smithi070.stdout:0/988: mkdir d16/d26/d27/d86/def/d155 0 2024-03-23T08:24:08.111 INFO:tasks.workunit.client.0.smithi070.stdout:7/857: creat d25/d27/d6a/d44/d6f/dcc/df2/f124 x:0 0 0 2024-03-23T08:24:08.111 INFO:tasks.workunit.client.0.smithi070.stdout:9/964: dwrite db/d12/d14/de5/da2/dc6/dce/d85/fd4 [0,4194304] 0 2024-03-23T08:24:08.115 INFO:tasks.workunit.client.0.smithi070.stdout:4/757: link d3/d6/d6a/f58 d3/d6/d35/d65/df1/f106 0 2024-03-23T08:24:08.117 INFO:tasks.workunit.client.0.smithi070.stdout:0/989: read d16/d22/d43/f78 [1219261,73698] 0 2024-03-23T08:24:08.119 INFO:tasks.workunit.client.0.smithi070.stdout:9/965: creat db/d12/d14/de5/da2/d4f/de6/f141 x:0 0 0 2024-03-23T08:24:08.119 INFO:tasks.workunit.client.0.smithi070.stdout:6/908: write d0/d23/f19 [9261233,35120] 0 2024-03-23T08:24:08.120 INFO:tasks.workunit.client.0.smithi070.stdout:7/858: unlink d25/d27/d6a/f7f 0 2024-03-23T08:24:08.121 INFO:tasks.workunit.client.0.smithi070.stdout:4/758: fsync d3/d6/f7 0 2024-03-23T08:24:08.122 INFO:tasks.workunit.client.0.smithi070.stdout:9/966: stat db/d12/d14/de5/da2/d4f/f72 0 2024-03-23T08:24:08.126 INFO:tasks.workunit.client.0.smithi070.stdout:8/887: write d5/d8/d16/d32/d84/d98/dde/da3/fbb [84032,77222] 0 2024-03-23T08:24:08.127 INFO:tasks.workunit.client.0.smithi070.stdout:8/888: read d5/f23 [2537138,30534] 0 2024-03-23T08:24:08.129 INFO:tasks.workunit.client.0.smithi070.stdout:7/859: creat d25/d27/d6a/d44/da2/df4/f125 x:0 0 0 2024-03-23T08:24:08.135 INFO:tasks.workunit.client.0.smithi070.stdout:9/967: dwrite db/d12/d14/f35 [0,4194304] 0 2024-03-23T08:24:08.137 INFO:tasks.workunit.client.0.smithi070.stdout:9/968: write db/d12/d14/de5/f10b [4456627,26621] 0 2024-03-23T08:24:08.244 INFO:tasks.workunit.client.0.smithi070.stdout:7/860: creat d25/d27/d6a/d44/d6f/dab/dcd/ddc/de4/f126 x:0 0 0 2024-03-23T08:24:08.246 INFO:tasks.workunit.client.0.smithi070.stdout:4/759: rmdir d3/d6/d35/dcf/db5/dda/db8/dfe 0 2024-03-23T08:24:08.249 INFO:tasks.workunit.client.0.smithi070.stdout:9/969: creat db/d12/d14/f142 x:0 0 0 2024-03-23T08:24:08.252 INFO:tasks.workunit.client.0.smithi070.stdout:4/760: fsync d3/d6/f7 0 2024-03-23T08:24:08.260 INFO:tasks.workunit.client.0.smithi070.stdout:4/761: creat d3/d6/d35/d65/d27/d4b/dc0/f107 x:0 0 0 2024-03-23T08:24:08.263 INFO:tasks.workunit.client.0.smithi070.stdout:1/794: getdents d8/d26/d7b 0 2024-03-23T08:24:08.263 INFO:tasks.workunit.client.0.smithi070.stdout:1/795: truncate d8/db/f3e 1262733 0 2024-03-23T08:24:08.264 INFO:tasks.workunit.client.0.smithi070.stdout:4/762: unlink d3/d6/d9/f4a 0 2024-03-23T08:24:08.266 INFO:tasks.workunit.client.0.smithi070.stdout:3/904: write db/df/d6a/fa2 [762258,6012] 0 2024-03-23T08:24:08.272 INFO:tasks.workunit.client.0.smithi070.stdout:4/763: mknod d3/d6/d35/dcf/da3/c108 0 2024-03-23T08:24:08.272 INFO:tasks.workunit.client.0.smithi070.stdout:1/796: dwrite d8/d26/d49/d65/f77 [0,4194304] 0 2024-03-23T08:24:08.274 INFO:tasks.workunit.client.0.smithi070.stdout:5/778: dwrite dc/d1b/d1d/d25/d36/d32/d5c/fed [0,4194304] 0 2024-03-23T08:24:08.276 INFO:tasks.workunit.client.0.smithi070.stdout:9/970: rename db/d12/d14/de5/da2/dc6/dce/c6d to db/d12/d14/de5/da2/c143 0 2024-03-23T08:24:08.279 INFO:tasks.workunit.client.0.smithi070.stdout:3/905: symlink db/d4e/dd6/l152 0 2024-03-23T08:24:08.281 INFO:tasks.workunit.client.0.smithi070.stdout:3/906: write db/d4e/dc4/d10c/f13d [447026,65080] 0 2024-03-23T08:24:08.282 INFO:tasks.workunit.client.0.smithi070.stdout:1/797: mkdir d8/db/d100/d74/d5d/d70/d11c 0 2024-03-23T08:24:08.285 INFO:tasks.workunit.client.0.smithi070.stdout:9/971: read db/d12/d14/de5/da2/dc6/dce/fc7 [2309718,90173] 0 2024-03-23T08:24:08.287 INFO:tasks.workunit.client.0.smithi070.stdout:1/798: creat d8/db/d2c/d3f/d48/f11d x:0 0 0 2024-03-23T08:24:08.290 INFO:tasks.workunit.client.0.smithi070.stdout:1/799: creat d8/d26/d108/f11e x:0 0 0 2024-03-23T08:24:08.293 INFO:tasks.workunit.client.0.smithi070.stdout:9/972: getdents db/d12 0 2024-03-23T08:24:08.294 INFO:tasks.workunit.client.0.smithi070.stdout:1/800: mkdir d8/d42/d51/d11f 0 2024-03-23T08:24:08.295 INFO:tasks.workunit.client.0.smithi070.stdout:9/973: readlink db/d12/d14/de5/da2/dc6/dce/laa 0 2024-03-23T08:24:08.296 INFO:tasks.workunit.client.0.smithi070.stdout:1/801: creat d8/d26/d7b/f120 x:0 0 0 2024-03-23T08:24:08.297 INFO:tasks.workunit.client.0.smithi070.stdout:1/802: readlink d8/d11/l13 0 2024-03-23T08:24:08.299 INFO:tasks.workunit.client.0.smithi070.stdout:1/803: symlink d8/db/d40/d43/d64/d85/d97/dfd/l121 0 2024-03-23T08:24:08.301 INFO:tasks.workunit.client.0.smithi070.stdout:9/974: link db/d12/d14/de5/da2/dc6/dce/d85/f66 db/dcd/f144 0 2024-03-23T08:24:08.302 INFO:tasks.workunit.client.0.smithi070.stdout:1/804: truncate d8/d1a/f23 925955 0 2024-03-23T08:24:08.305 INFO:tasks.workunit.client.0.smithi070.stdout:1/805: unlink d8/d42/d51/f10d 0 2024-03-23T08:24:08.307 INFO:tasks.workunit.client.0.smithi070.stdout:8/889: write d5/f7 [2377763,15143] 0 2024-03-23T08:24:08.309 INFO:tasks.workunit.client.0.smithi070.stdout:6/909: dwrite d0/d23/d4d/fd4 [0,4194304] 0 2024-03-23T08:24:08.311 INFO:tasks.workunit.client.0.smithi070.stdout:1/806: mknod d8/d26/d49/d65/c122 0 2024-03-23T08:24:08.313 INFO:tasks.workunit.client.0.smithi070.stdout:1/807: write d8/db/d40/d43/d64/d85/dc7/fcc [1014480,113551] 0 2024-03-23T08:24:08.314 INFO:tasks.workunit.client.0.smithi070.stdout:8/890: read d5/d8/d16/d32/d84/d98/fc5 [903963,9305] 0 2024-03-23T08:24:08.318 INFO:tasks.workunit.client.0.smithi070.stdout:1/808: unlink d8/d26/d49/d65/fc3 0 2024-03-23T08:24:08.319 INFO:tasks.workunit.client.0.smithi070.stdout:8/891: truncate d5/d8/f1d 2312784 0 2024-03-23T08:24:08.320 INFO:tasks.workunit.client.0.smithi070.stdout:1/809: chown d8/d42/d51/l6b 2122498626 1 2024-03-23T08:24:08.321 INFO:tasks.workunit.client.0.smithi070.stdout:7/861: dwrite d25/d27/d6a/f80 [0,4194304] 0 2024-03-23T08:24:08.321 INFO:tasks.workunit.client.0.smithi070.stdout:8/892: write d5/d8/f9d [1822783,42808] 0 2024-03-23T08:24:08.324 INFO:tasks.workunit.client.0.smithi070.stdout:8/893: write d5/d8/d16/dd4/d6f/f7e [1660066,100043] 0 2024-03-23T08:24:08.329 INFO:tasks.workunit.client.0.smithi070.stdout:9/975: dread f4 [0,4194304] 0 2024-03-23T08:24:08.332 INFO:tasks.workunit.client.0.smithi070.stdout:1/810: dwrite d8/db/f27 [0,4194304] 0 2024-03-23T08:24:08.333 INFO:tasks.workunit.client.0.smithi070.stdout:9/976: chown db/d12/d14/de5/da2/f44 1248276 1 2024-03-23T08:24:08.339 INFO:tasks.workunit.client.0.smithi070.stdout:9/977: unlink l2 0 2024-03-23T08:24:08.340 INFO:tasks.workunit.client.0.smithi070.stdout:3/907: write db/df/d5d/fcf [1379349,127529] 0 2024-03-23T08:24:08.343 INFO:tasks.workunit.client.0.smithi070.stdout:4/764: dwrite d3/d6/d35/d65/d1b/f28 [0,4194304] 0 2024-03-23T08:24:08.348 INFO:tasks.workunit.client.0.smithi070.stdout:1/811: getdents d8/d42/d4e 0 2024-03-23T08:24:08.351 INFO:tasks.workunit.client.0.smithi070.stdout:4/765: dwrite d3/d6/d35/d65/df1/ffc [0,4194304] 0 2024-03-23T08:24:08.355 INFO:tasks.workunit.client.0.smithi070.stdout:3/908: creat db/d4e/dc4/d137/f153 x:0 0 0 2024-03-23T08:24:08.360 INFO:tasks.workunit.client.0.smithi070.stdout:4/766: creat d3/d6/d35/dcf/db5/dbb/f109 x:0 0 0 2024-03-23T08:24:08.372 INFO:tasks.workunit.client.0.smithi070.stdout:7/862: dwrite d25/d27/f45 [0,4194304] 0 2024-03-23T08:24:08.374 INFO:tasks.workunit.client.0.smithi070.stdout:3/909: link db/d4e/dd6/dd7/dd9/d12f/c14b db/d4e/df8/d22/db2/c154 0 2024-03-23T08:24:08.375 INFO:tasks.workunit.client.0.smithi070.stdout:9/978: write db/d12/d14/de5/da2/dc6/dce/d85/d42/f5f [761508,20315] 0 2024-03-23T08:24:08.377 INFO:tasks.workunit.client.0.smithi070.stdout:8/894: dwrite d5/d8/d9a/da2/dae/f94 [0,4194304] 0 2024-03-23T08:24:08.378 INFO:tasks.workunit.client.0.smithi070.stdout:3/910: write db/df/d5d/fcf [1825398,69801] 0 2024-03-23T08:24:08.381 INFO:tasks.workunit.client.0.smithi070.stdout:9/979: readlink db/d12/d14/de5/da2/dc6/dce/d85/d11a/l11e 0 2024-03-23T08:24:08.385 INFO:tasks.workunit.client.0.smithi070.stdout:4/767: link d3/d6/d9/f7f d3/d6/d35/d3f/da9/f10a 0 2024-03-23T08:24:08.389 INFO:tasks.workunit.client.0.smithi070.stdout:7/863: dwrite d25/fb8 [0,4194304] 0 2024-03-23T08:24:08.390 INFO:tasks.workunit.client.0.smithi070.stdout:8/895: dread d5/d8/d16/d30/fbc [0,4194304] 0 2024-03-23T08:24:08.393 INFO:tasks.workunit.client.0.smithi070.stdout:3/911: chown db/d4e/df8/d22/db2/dba/cf6 6978293 1 2024-03-23T08:24:08.393 INFO:tasks.workunit.client.0.smithi070.stdout:9/980: mknod db/db8/dd8/c145 0 2024-03-23T08:24:08.394 INFO:tasks.workunit.client.0.smithi070.stdout:7/864: write d25/d27/d6a/d44/f105 [730281,88679] 0 2024-03-23T08:24:08.394 INFO:tasks.workunit.client.0.smithi070.stdout:1/812: dwrite d8/db/d100/d74/d5d/fe2 [0,4194304] 0 2024-03-23T08:24:08.398 INFO:tasks.workunit.client.0.smithi070.stdout:4/768: creat d3/d6/d35/dcf/db5/dda/dc7/f10b x:0 0 0 2024-03-23T08:24:08.402 INFO:tasks.workunit.client.0.smithi070.stdout:1/813: creat d8/db/d100/d74/d5d/d70/f123 x:0 0 0 2024-03-23T08:24:08.402 INFO:tasks.workunit.client.0.smithi070.stdout:7/865: mknod d25/c127 0 2024-03-23T08:24:08.403 INFO:tasks.workunit.client.0.smithi070.stdout:4/769: fsync d3/d6/d35/d65/f75 0 2024-03-23T08:24:08.406 INFO:tasks.workunit.client.0.smithi070.stdout:1/814: truncate d8/d26/d41/fae 3109051 0 2024-03-23T08:24:08.408 INFO:tasks.workunit.client.0.smithi070.stdout:7/866: link d25/d27/d6a/c5d d25/d40/c128 0 2024-03-23T08:24:08.408 INFO:tasks.workunit.client.0.smithi070.stdout:1/815: write d8/d26/ff5 [837784,104913] 0 2024-03-23T08:24:08.409 INFO:tasks.workunit.client.0.smithi070.stdout:4/770: rename d3/d6/d35/dcf/db5/dda/db8/la1 to d3/d6/d35/dcf/db5/dda/db8/l10c 0 2024-03-23T08:24:08.426 INFO:tasks.workunit.client.0.smithi070.stdout:1/816: dread d8/db/d100/f78 [0,4194304] 0 2024-03-23T08:24:08.429 INFO:tasks.workunit.client.0.smithi070.stdout:8/896: dread d5/d8/d16/d30/d38/d6b/d72/d88/fc7 [0,4194304] 0 2024-03-23T08:24:08.429 INFO:tasks.workunit.client.0.smithi070.stdout:9/981: dwrite db/d12/d14/de5/da2/d4f/db6/fc4 [0,4194304] 0 2024-03-23T08:24:08.433 INFO:tasks.workunit.client.0.smithi070.stdout:8/897: write d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/df2/f12c [299576,83159] 0 2024-03-23T08:24:08.436 INFO:tasks.workunit.client.0.smithi070.stdout:9/982: fsync f4 0 2024-03-23T08:24:08.437 INFO:tasks.workunit.client.0.smithi070.stdout:9/983: chown db/d12/d14/de5 54 1 2024-03-23T08:24:08.438 INFO:tasks.workunit.client.0.smithi070.stdout:1/817: creat d8/db/d100/d74/d5d/d10a/f124 x:0 0 0 2024-03-23T08:24:08.441 INFO:tasks.workunit.client.0.smithi070.stdout:8/898: fsync d5/d8/d16/d30/d38/d6b/da9/fdf 0 2024-03-23T08:24:08.442 INFO:tasks.workunit.client.0.smithi070.stdout:7/867: dwrite d25/f4f [0,4194304] 0 2024-03-23T08:24:08.444 INFO:tasks.workunit.client.0.smithi070.stdout:4/771: dwrite d3/d6/d35/d3f/d51/f63 [0,4194304] 0 2024-03-23T08:24:08.452 INFO:tasks.workunit.client.0.smithi070.stdout:8/899: unlink d5/d8/d16/d30/d38/d6b/da9/cda 0 2024-03-23T08:24:08.457 INFO:tasks.workunit.client.0.smithi070.stdout:1/818: rmdir d8/db/d2c/d3f/ddd 0 2024-03-23T08:24:08.458 INFO:tasks.workunit.client.0.smithi070.stdout:1/819: fsync d8/d11/f3b 0 2024-03-23T08:24:08.458 INFO:tasks.workunit.client.0.smithi070.stdout:1/820: stat d8/db/d2c/fd2 0 2024-03-23T08:24:08.461 INFO:tasks.workunit.client.0.smithi070.stdout:8/900: getdents d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/df2 0 2024-03-23T08:24:08.464 INFO:tasks.workunit.client.0.smithi070.stdout:8/901: rmdir d5/d8/d16/dd4/dc9 39 2024-03-23T08:24:08.466 INFO:tasks.workunit.client.0.smithi070.stdout:8/902: creat d5/d8/d9a/dbe/f13a x:0 0 0 2024-03-23T08:24:08.468 INFO:tasks.workunit.client.0.smithi070.stdout:8/903: truncate d5/d8/d16/d30/fbc 852872 0 2024-03-23T08:24:08.469 INFO:tasks.workunit.client.0.smithi070.stdout:8/904: symlink d5/d8/d9a/l13b 0 2024-03-23T08:24:08.476 INFO:tasks.workunit.client.0.smithi070.stdout:7/868: dread d25/d27/d6a/d44/d6f/da3/f103 [0,4194304] 0 2024-03-23T08:24:08.482 INFO:tasks.workunit.client.0.smithi070.stdout:4/772: write d3/d6/d35/d65/f75 [262674,66557] 0 2024-03-23T08:24:08.483 INFO:tasks.workunit.client.0.smithi070.stdout:4/773: chown d3/d6/f38 79559099 1 2024-03-23T08:24:08.486 INFO:tasks.workunit.client.0.smithi070.stdout:9/984: truncate db/d12/d14/de5/da2/dc6/dce/f93 3832247 0 2024-03-23T08:24:08.488 INFO:tasks.workunit.client.0.smithi070.stdout:7/869: dread d25/d40/fa5 [0,4194304] 0 2024-03-23T08:24:08.489 INFO:tasks.workunit.client.0.smithi070.stdout:4/774: mkdir d3/d6/d9/de6/d10d 0 2024-03-23T08:24:08.493 INFO:tasks.workunit.client.0.smithi070.stdout:7/870: write d25/d27/d6a/d44/f67 [2685849,116466] 0 2024-03-23T08:24:08.496 INFO:tasks.workunit.client.0.smithi070.stdout:7/871: dread f24 [0,4194304] 0 2024-03-23T08:24:08.496 INFO:tasks.workunit.client.0.smithi070.stdout:8/905: dread d5/d8/d16/d3b/f5d [0,4194304] 0 2024-03-23T08:24:08.499 INFO:tasks.workunit.client.0.smithi070.stdout:9/985: dread db/d12/f89 [0,4194304] 0 2024-03-23T08:24:08.502 INFO:tasks.workunit.client.0.smithi070.stdout:4/775: link d3/d6/d35/d65/df1/f106 d3/d6/d35/d65/d27/d4b/f10e 0 2024-03-23T08:24:08.504 INFO:tasks.workunit.client.0.smithi070.stdout:8/906: symlink d5/d8/d16/d3b/l13c 0 2024-03-23T08:24:08.508 INFO:tasks.workunit.client.0.smithi070.stdout:7/872: rename d25/d27/d6a/d44/l7a to d25/d27/d6a/d44/d85/l129 0 2024-03-23T08:24:08.511 INFO:tasks.workunit.client.0.smithi070.stdout:8/907: dwrite d5/d8/d9a/f108 [0,4194304] 0 2024-03-23T08:24:08.512 INFO:tasks.workunit.client.0.smithi070.stdout:9/986: chown db/d12/d14/de5/da2/d4f/d5a/cef 317852 1 2024-03-23T08:24:08.513 INFO:tasks.workunit.client.0.smithi070.stdout:9/987: chown db/d12/d14/de5/da2/dc6/dce/d85/d42/l8e 13 1 2024-03-23T08:24:08.517 INFO:tasks.workunit.client.0.smithi070.stdout:9/988: write db/d12/d14/de5/da2/d4f/d5a/f110 [1035311,32528] 0 2024-03-23T08:24:08.519 INFO:tasks.workunit.client.0.smithi070.stdout:7/873: creat d25/d27/d6a/d44/d6f/da3/f12a x:0 0 0 2024-03-23T08:24:08.522 INFO:tasks.workunit.client.0.smithi070.stdout:7/874: chown d25/d27/d6a/d44/da4/f11f 3 1 2024-03-23T08:24:08.525 INFO:tasks.workunit.client.0.smithi070.stdout:4/776: dwrite d3/d6/d35/d65/d6f/fb1 [0,4194304] 0 2024-03-23T08:24:08.530 INFO:tasks.workunit.client.0.smithi070.stdout:8/908: rename d5/d8/d16/f44 to d5/d8/d16/d30/d38/d6b/d72/f13d 0 2024-03-23T08:24:08.533 INFO:tasks.workunit.client.0.smithi070.stdout:4/777: dwrite d3/d6/d35/d65/d27/d4b/dc0/f107 [0,4194304] 0 2024-03-23T08:24:08.535 INFO:tasks.workunit.client.0.smithi070.stdout:7/875: rename d25/d27/f3c to d25/d27/d6a/d44/d7c/dfb/dfd/f12b 0 2024-03-23T08:24:08.571 INFO:tasks.workunit.client.0.smithi070.stdout:4/778: fdatasync d3/d6/d35/dcf/db5/dbb/fbd 0 2024-03-23T08:24:08.571 INFO:tasks.workunit.client.0.smithi070.stdout:9/989: write db/d12/d14/de5/da2/dc6/d7e/f74 [403875,50660] 0 2024-03-23T08:24:08.572 INFO:tasks.workunit.client.0.smithi070.stdout:7/876: fsync d25/d27/d6a/f5f 0 2024-03-23T08:24:08.574 INFO:tasks.workunit.client.0.smithi070.stdout:8/909: dread d5/d8/d16/d32/d84/d98/dde/da3/fb8 [0,4194304] 0 2024-03-23T08:24:08.582 INFO:tasks.workunit.client.0.smithi070.stdout:7/877: unlink d25/d27/d6a/d44/d7c/dfb/dfd/f11b 0 2024-03-23T08:24:08.583 INFO:tasks.workunit.client.0.smithi070.stdout:9/990: symlink db/d12/d14/de5/da2/d26/dec/l146 0 2024-03-23T08:24:08.583 INFO:tasks.workunit.client.0.smithi070.stdout:7/878: write d25/d27/d6a/d44/d6f/dab/dcd/df1/fff [3948508,33048] 0 2024-03-23T08:24:08.593 INFO:tasks.workunit.client.0.smithi070.stdout:4/779: write d3/fc [753061,21802] 0 2024-03-23T08:24:08.593 INFO:tasks.workunit.client.0.smithi070.stdout:7/879: symlink d25/d27/d6a/d44/da2/db5/l12c 0 2024-03-23T08:24:08.601 INFO:tasks.workunit.client.0.smithi070.stdout:9/991: rename db/d12/d14/de5/da2/dc6/dce/lfd to db/d12/d14/de5/da2/d4f/de6/d127/l147 0 2024-03-23T08:24:08.608 INFO:tasks.workunit.client.0.smithi070.stdout:8/910: write d5/d8/d16/dd4/d6f/f89 [568999,112087] 0 2024-03-23T08:24:08.609 INFO:tasks.workunit.client.0.smithi070.stdout:4/780: creat d3/d6/d11/f10f x:0 0 0 2024-03-23T08:24:08.609 INFO:tasks.workunit.client.0.smithi070.stdout:7/880: link d25/d27/d6a/d44/d6f/dab/dcd/l11d d25/d27/d6a/d44/d6f/da3/de1/l12d 0 2024-03-23T08:24:08.610 INFO:tasks.workunit.client.0.smithi070.stdout:4/781: write d3/d6/d11/fd5 [939154,33117] 0 2024-03-23T08:24:08.612 INFO:tasks.workunit.client.0.smithi070.stdout:7/881: mknod d25/d40/c12e 0 2024-03-23T08:24:08.614 INFO:tasks.workunit.client.0.smithi070.stdout:7/882: dread - d25/d27/d6a/d44/d6f/dab/dcd/ddc/fef zero size 2024-03-23T08:24:08.615 INFO:tasks.workunit.client.0.smithi070.stdout:8/911: unlink d5/l15 0 2024-03-23T08:24:08.619 INFO:tasks.workunit.client.0.smithi070.stdout:4/782: dwrite d3/d6/d35/dcf/db5/dda/ff9 [0,4194304] 0 2024-03-23T08:24:08.619 INFO:tasks.workunit.client.0.smithi070.stdout:7/883: getdents d25/d27/d6a/d44/d6f/dcc/d11a 0 2024-03-23T08:24:08.622 INFO:tasks.workunit.client.0.smithi070.stdout:8/912: creat d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/df2/f13e x:0 0 0 2024-03-23T08:24:08.626 INFO:tasks.workunit.client.0.smithi070.stdout:7/884: creat d25/d27/dc2/f12f x:0 0 0 2024-03-23T08:24:08.632 INFO:tasks.workunit.client.0.smithi070.stdout:7/885: rename d25/d27/d6a/d44/fa8 to d25/d27/d6a/d44/d7c/f130 0 2024-03-23T08:24:08.640 INFO:tasks.workunit.client.0.smithi070.stdout:7/886: dwrite d25/d27/d6a/d44/d6f/dab/dcd/ddc/de4/f113 [0,4194304] 0 2024-03-23T08:24:08.643 INFO:tasks.workunit.client.0.smithi070.stdout:8/913: dread d5/d8/d16/d32/d84/d98/dde/da3/dab/fad [0,4194304] 0 2024-03-23T08:24:08.646 INFO:tasks.workunit.client.0.smithi070.stdout:7/887: dread - d25/d27/d6a/d44/d6f/fc5 zero size 2024-03-23T08:24:08.648 INFO:tasks.workunit.client.0.smithi070.stdout:8/914: creat d5/d8/d16/d30/d38/d6b/d72/d118/f13f x:0 0 0 2024-03-23T08:24:08.650 INFO:tasks.workunit.client.0.smithi070.stdout:7/888: write d25/d27/d6a/d44/d6f/da3/f102 [4583003,107413] 0 2024-03-23T08:24:08.663 INFO:tasks.workunit.client.0.smithi070.stdout:8/915: dread d5/d8/d9a/da2/fe9 [0,4194304] 0 2024-03-23T08:24:08.669 INFO:tasks.workunit.client.0.smithi070.stdout:7/889: dread d25/d27/f48 [0,4194304] 0 2024-03-23T08:24:08.670 INFO:tasks.workunit.client.0.smithi070.stdout:8/916: dwrite d5/d8/d9a/da2/f100 [0,4194304] 0 2024-03-23T08:24:08.677 INFO:tasks.workunit.client.0.smithi070.stdout:7/890: creat d25/d27/d6a/d44/d6f/dab/dcd/ddc/f131 x:0 0 0 2024-03-23T08:24:08.678 INFO:tasks.workunit.client.0.smithi070.stdout:8/917: mknod d5/d8/d16/d30/d11c/c140 0 2024-03-23T08:24:08.682 INFO:tasks.workunit.client.0.smithi070.stdout:8/918: dread - d5/d8/d16/dd4/dc9/f10b zero size 2024-03-23T08:24:08.684 INFO:tasks.workunit.client.0.smithi070.stdout:7/891: dwrite d25/d27/d6a/d44/f54 [4194304,4194304] 0 2024-03-23T08:24:08.689 INFO:tasks.workunit.client.0.smithi070.stdout:8/919: dwrite d5/d8/ffe [0,4194304] 0 2024-03-23T08:24:08.691 INFO:tasks.workunit.client.0.smithi070.stdout:7/892: rename d25/d27/d6a/f5f to d25/d27/d6a/d44/d7c/db4/d114/f132 0 2024-03-23T08:24:08.692 INFO:tasks.workunit.client.0.smithi070.stdout:8/920: chown d5/d8/d16/d32/c4c 10 1 2024-03-23T08:24:08.693 INFO:tasks.workunit.client.0.smithi070.stdout:8/921: dread - d5/d8/d16/f12d zero size 2024-03-23T08:24:08.697 INFO:tasks.workunit.client.0.smithi070.stdout:7/893: rename d25/d27/f6d to d25/d27/d6a/d44/d6f/dab/dcd/ddc/de4/f133 0 2024-03-23T08:24:08.699 INFO:tasks.workunit.client.0.smithi070.stdout:7/894: truncate d25/d27/d6a/d44/dfe/f122 180854 0 2024-03-23T08:24:08.712 INFO:tasks.workunit.client.0.smithi070.stdout:8/922: dread d5/d8/d9a/dbe/f117 [0,4194304] 0 2024-03-23T08:24:08.712 INFO:tasks.workunit.client.0.smithi070.stdout:7/895: dwrite d25/d27/d6a/fbe [0,4194304] 0 2024-03-23T08:24:08.714 INFO:tasks.workunit.client.0.smithi070.stdout:7/896: write d25/d27/d6a/d44/d7c/db4/fe0 [1077174,37237] 0 2024-03-23T08:24:08.715 INFO:tasks.workunit.client.0.smithi070.stdout:8/923: symlink d5/d8/d16/dd4/d6f/d7c/l141 0 2024-03-23T08:24:08.723 INFO:tasks.workunit.client.0.smithi070.stdout:7/897: dwrite d25/d27/d6a/d44/f8c [0,4194304] 0 2024-03-23T08:24:08.727 INFO:tasks.workunit.client.0.smithi070.stdout:7/898: unlink d25/d27/d6a/d44/d6f/da3/fe9 0 2024-03-23T08:24:08.728 INFO:tasks.workunit.client.0.smithi070.stdout:8/924: dwrite d5/d8/f79 [0,4194304] 0 2024-03-23T08:24:08.730 INFO:tasks.workunit.client.0.smithi070.stdout:7/899: mkdir d25/d27/d6a/d44/da2/d134 0 2024-03-23T08:24:08.772 INFO:tasks.workunit.client.0.smithi070.stdout:5/779: sync 2024-03-23T08:24:08.772 INFO:tasks.workunit.client.0.smithi070.stdout:0/990: sync 2024-03-23T08:24:08.773 INFO:tasks.workunit.client.0.smithi070.stdout:7/900: dwrite d25/d27/d6a/d44/d6f/dab/dcd/ddc/de4/f126 [0,4194304] 0 2024-03-23T08:24:08.775 INFO:tasks.workunit.client.0.smithi070.stdout:8/925: mknod d5/d8/d16/d3b/db9/c142 0 2024-03-23T08:24:08.784 INFO:tasks.workunit.client.0.smithi070.stdout:0/991: creat d16/d26/d27/d86/db6/dec/f156 x:0 0 0 2024-03-23T08:24:08.791 INFO:tasks.workunit.client.0.smithi070.stdout:0/992: creat d16/d26/d27/d86/db6/f157 x:0 0 0 2024-03-23T08:24:08.795 INFO:tasks.workunit.client.0.smithi070.stdout:0/993: creat d16/d22/d43/d47/da9/f158 x:0 0 0 2024-03-23T08:24:08.795 INFO:tasks.workunit.client.0.smithi070.stdout:7/901: dread d25/d27/d6a/fb1 [0,4194304] 0 2024-03-23T08:24:08.798 INFO:tasks.workunit.client.0.smithi070.stdout:5/780: getdents dc/d1b/d1d/d25/d38/d39 0 2024-03-23T08:24:08.798 INFO:tasks.workunit.client.0.smithi070.stdout:0/994: chown d16/d22/d43/f78 806888340 1 2024-03-23T08:24:08.800 INFO:tasks.workunit.client.0.smithi070.stdout:8/926: dwrite d5/d8/d16/d3b/d8c/fd6 [0,4194304] 0 2024-03-23T08:24:08.802 INFO:tasks.workunit.client.0.smithi070.stdout:5/781: readlink dc/d1b/d1d/l49 0 2024-03-23T08:24:08.803 INFO:tasks.workunit.client.0.smithi070.stdout:7/902: dread d25/d27/d6a/d44/d7c/db4/d114/f132 [0,4194304] 0 2024-03-23T08:24:08.812 INFO:tasks.workunit.client.0.smithi070.stdout:7/903: dwrite d25/d27/d6a/d44/da4/f11f [0,4194304] 0 2024-03-23T08:24:08.813 INFO:tasks.workunit.client.0.smithi070.stdout:5/782: unlink dc/d1b/d1d/d25/d38/d39/f4f 0 2024-03-23T08:24:08.814 INFO:tasks.workunit.client.0.smithi070.stdout:0/995: mkdir d16/d22/d43/d47/da9/de7/d142/d159 0 2024-03-23T08:24:08.818 INFO:tasks.workunit.client.0.smithi070.stdout:8/927: write d5/d8/d9a/dbe/f117 [1121597,115447] 0 2024-03-23T08:24:08.820 INFO:tasks.workunit.client.0.smithi070.stdout:8/928: write d5/d8/d16/dd4/d6f/d7c/f91 [1353483,61876] 0 2024-03-23T08:24:08.829 INFO:tasks.workunit.client.0.smithi070.stdout:0/996: creat d16/d26/f15a x:0 0 0 2024-03-23T08:24:08.831 INFO:tasks.workunit.client.0.smithi070.stdout:5/783: write dc/d1b/d1d/d25/d36/d32/d5c/f5d [3909093,124407] 0 2024-03-23T08:24:08.833 INFO:tasks.workunit.client.0.smithi070.stdout:7/904: creat d25/d27/d6a/d44/d6f/d104/f135 x:0 0 0 2024-03-23T08:24:08.835 INFO:tasks.workunit.client.0.smithi070.stdout:0/997: symlink d16/d26/d27/d86/db6/l15b 0 2024-03-23T08:24:08.839 INFO:tasks.workunit.client.0.smithi070.stdout:5/784: rmdir dc/d1b/d1d/d25/d36/dfc 39 2024-03-23T08:24:08.842 INFO:tasks.workunit.client.0.smithi070.stdout:0/998: mkdir d16/d26/d27/d86/d140/df8/d11a/d15c 0 2024-03-23T08:24:08.843 INFO:tasks.workunit.client.0.smithi070.stdout:8/929: dread d5/d8/f2c [0,4194304] 0 2024-03-23T08:24:08.847 INFO:tasks.workunit.client.0.smithi070.stdout:0/999: symlink d16/d26/d27/d86/db6/dec/l15d 0 2024-03-23T08:24:08.848 INFO:tasks.workunit.client.0.smithi070.stdout:8/930: mknod d5/d8/d16/dd4/d6f/d7c/d7d/c143 0 2024-03-23T08:24:08.850 INFO:tasks.workunit.client.0.smithi070.stdout:5/785: dwrite dc/d1b/d1d/d25/d38/d53/f76 [4194304,4194304] 0 2024-03-23T08:24:08.852 INFO:tasks.workunit.client.0.smithi070.stdout:5/786: write dc/d1b/d1d/d25/d38/d39/fd5 [1028258,98366] 0 2024-03-23T08:24:08.854 INFO:tasks.workunit.client.0.smithi070.stdout:8/931: creat d5/d8/d16/dd4/dc9/f144 x:0 0 0 2024-03-23T08:24:08.856 INFO:tasks.workunit.client.0.smithi070.stdout:8/932: chown d5/d8/d16/d3b/l13c 1985 1 2024-03-23T08:24:08.860 INFO:tasks.workunit.client.0.smithi070.stdout:7/905: link d25/d27/d6a/d44/d7c/db4/cf3 d25/d27/d6a/d44/d85/d95/da0/c136 0 2024-03-23T08:24:08.861 INFO:tasks.workunit.client.0.smithi070.stdout:7/906: chown d25/d27/d6a/d44/d85/d95/da0/le7 10 1 2024-03-23T08:24:08.862 INFO:tasks.workunit.client.0.smithi070.stdout:8/933: creat d5/d8/d16/dd4/dc9/f145 x:0 0 0 2024-03-23T08:24:08.863 INFO:tasks.workunit.client.0.smithi070.stdout:8/934: fsync d5/d8/d9a/da2/f100 0 2024-03-23T08:24:08.866 INFO:tasks.workunit.client.0.smithi070.stdout:5/787: creat dc/d1b/d1d/ffd x:0 0 0 2024-03-23T08:24:08.869 INFO:tasks.workunit.client.0.smithi070.stdout:8/935: mkdir d5/d8/d16/d3b/db9/d146 0 2024-03-23T08:24:08.871 INFO:tasks.workunit.client.0.smithi070.stdout:7/907: creat d25/d27/d6a/d44/d6f/dcc/d11a/f137 x:0 0 0 2024-03-23T08:24:08.872 INFO:tasks.workunit.client.0.smithi070.stdout:7/908: write d25/d40/f76 [1449955,103849] 0 2024-03-23T08:24:08.873 INFO:tasks.workunit.client.0.smithi070.stdout:5/788: rename dc/d1b/d1d/d25/l35 to dc/d1b/d1d/d25/d36/d32/d5c/dcb/lfe 0 2024-03-23T08:24:08.876 INFO:tasks.workunit.client.0.smithi070.stdout:8/936: mknod d5/d8/d16/d3b/db9/c147 0 2024-03-23T08:24:08.881 INFO:tasks.workunit.client.0.smithi070.stdout:7/909: creat d25/d27/d6a/d44/d6f/d104/f138 x:0 0 0 2024-03-23T08:24:08.881 INFO:tasks.workunit.client.0.smithi070.stdout:5/789: dwrite dc/d1b/d1d/d25/d36/d32/db7/fc4 [0,4194304] 0 2024-03-23T08:24:08.902 INFO:tasks.workunit.client.0.smithi070.stdout:7/910: getdents d25/d27/d6a/d44/d6f/d104 0 2024-03-23T08:24:08.902 INFO:tasks.workunit.client.0.smithi070.stdout:8/937: write d5/d8/d16/f24 [1736796,19357] 0 2024-03-23T08:24:08.904 INFO:tasks.workunit.client.0.smithi070.stdout:7/911: creat d25/d27/d6a/d44/da2/df4/f139 x:0 0 0 2024-03-23T08:24:08.909 INFO:tasks.workunit.client.0.smithi070.stdout:5/790: dwrite f1 [0,4194304] 0 2024-03-23T08:24:08.911 INFO:tasks.workunit.client.0.smithi070.stdout:8/938: creat d5/d8/d16/d121/f148 x:0 0 0 2024-03-23T08:24:08.915 INFO:tasks.workunit.client.0.smithi070.stdout:5/791: creat dc/d1b/d1d/d25/d38/d39/dbc/fff x:0 0 0 2024-03-23T08:24:08.916 INFO:tasks.workunit.client.0.smithi070.stdout:5/792: dread - dc/d1b/d1d/d25/d38/d53/fc9 zero size 2024-03-23T08:24:08.918 INFO:tasks.workunit.client.0.smithi070.stdout:8/939: mkdir d5/d8/d16/d3b/d149 0 2024-03-23T08:24:08.919 INFO:tasks.workunit.client.0.smithi070.stdout:5/793: fsync dc/d1b/f54 0 2024-03-23T08:24:08.934 INFO:tasks.workunit.client.0.smithi070.stdout:6/910: sync 2024-03-23T08:24:08.972 INFO:tasks.workunit.client.0.smithi070.stdout:7/912: write d25/d27/f57 [693835,44018] 0 2024-03-23T08:24:08.983 INFO:tasks.workunit.client.0.smithi070.stdout:7/913: dwrite d25/d27/d6a/d44/dfe/f122 [0,4194304] 0 2024-03-23T08:24:08.984 INFO:tasks.workunit.client.0.smithi070.stdout:7/914: truncate d25/d40/f10d 233054 0 2024-03-23T08:24:09.003 INFO:tasks.workunit.client.0.smithi070.stdout:6/911: fsync d0/d23/d74/d84/db3/fff 0 2024-03-23T08:24:09.004 INFO:tasks.workunit.client.0.smithi070.stdout:6/912: write d0/f9c [806021,56656] 0 2024-03-23T08:24:09.007 INFO:tasks.workunit.client.0.smithi070.stdout:7/915: write d25/f34 [4277167,86094] 0 2024-03-23T08:24:09.008 INFO:tasks.workunit.client.0.smithi070.stdout:5/794: creat dc/d1b/d1d/d25/d36/f100 x:0 0 0 2024-03-23T08:24:09.012 INFO:tasks.workunit.client.0.smithi070.stdout:7/916: readlink d25/d27/d6a/d44/d6f/l123 0 2024-03-23T08:24:09.013 INFO:tasks.workunit.client.0.smithi070.stdout:5/795: mknod dc/d1b/d1d/d25/dcd/c101 0 2024-03-23T08:24:09.014 INFO:tasks.workunit.client.0.smithi070.stdout:8/940: dread d5/d8/d16/d32/d84/d98/dde/da3/fb2 [0,4194304] 0 2024-03-23T08:24:09.014 INFO:tasks.workunit.client.0.smithi070.stdout:6/913: symlink d0/d23/d74/l124 0 2024-03-23T08:24:09.016 INFO:tasks.workunit.client.0.smithi070.stdout:7/917: symlink d25/d27/d6a/d44/dfe/l13a 0 2024-03-23T08:24:09.017 INFO:tasks.workunit.client.0.smithi070.stdout:8/941: readlink d5/d8/d16/d32/l125 0 2024-03-23T08:24:09.018 INFO:tasks.workunit.client.0.smithi070.stdout:6/914: symlink d0/d23/d97/d98/l125 0 2024-03-23T08:24:09.020 INFO:tasks.workunit.client.0.smithi070.stdout:8/942: creat d5/d8/d16/dd4/d115/f14a x:0 0 0 2024-03-23T08:24:09.020 INFO:tasks.workunit.client.0.smithi070.stdout:7/918: mknod d25/d27/d6a/d44/d7c/dfb/dfd/d10f/c13b 0 2024-03-23T08:24:09.023 INFO:tasks.workunit.client.0.smithi070.stdout:8/943: symlink d5/d8/d16/dd4/d6f/l14b 0 2024-03-23T08:24:09.023 INFO:tasks.workunit.client.0.smithi070.stdout:6/915: dread d0/d23/d97/d98/d11e/fdd [0,4194304] 0 2024-03-23T08:24:09.025 INFO:tasks.workunit.client.0.smithi070.stdout:6/916: readlink d0/d23/d4d/lc6 0 2024-03-23T08:24:09.026 INFO:tasks.workunit.client.0.smithi070.stdout:8/944: creat d5/d8/d16/d30/d38/d6b/da9/f14c x:0 0 0 2024-03-23T08:24:09.027 INFO:tasks.workunit.client.0.smithi070.stdout:7/919: creat d25/f13c x:0 0 0 2024-03-23T08:24:09.028 INFO:tasks.workunit.client.0.smithi070.stdout:8/945: fsync d5/d8/d16/d32/d84/d98/dde/da3/dab/de6/df2/f12c 0 2024-03-23T08:24:09.030 INFO:tasks.workunit.client.0.smithi070.stdout:8/946: fsync d5/d8/d16/d30/d38/d6b/f129 0 2024-03-23T08:24:09.030 INFO:tasks.workunit.client.0.smithi070.stdout:8/947: chown d5/d8/d9a/fdd 759 1 2024-03-23T08:24:09.031 INFO:tasks.workunit.client.0.smithi070.stdout:6/917: creat d0/d23/d74/f126 x:0 0 0 2024-03-23T08:24:09.032 INFO:tasks.workunit.client.0.smithi070.stdout:7/920: rmdir d25/d27/d6a/d44/d6f/dcc/d11a 39 2024-03-23T08:24:09.036 INFO:tasks.workunit.client.0.smithi070.stdout:6/918: mkdir d0/d23/d97/d98/d127 0 2024-03-23T08:24:09.037 INFO:tasks.workunit.client.0.smithi070.stdout:7/921: fdatasync d25/d27/d6a/d44/d7c/fd0 0 2024-03-23T08:24:09.038 INFO:tasks.workunit.client.0.smithi070.stdout:7/922: fsync d25/d27/d6a/d44/f105 0 2024-03-23T08:24:09.038 INFO:tasks.workunit.client.0.smithi070.stdout:8/948: creat d5/d8/d16/d30/d38/d6b/d72/d88/f14d x:0 0 0 2024-03-23T08:24:09.039 INFO:tasks.workunit.client.0.smithi070.stdout:7/923: stat d25/d27/d6a/d44/d7c/dfb 0 2024-03-23T08:24:09.041 INFO:tasks.workunit.client.0.smithi070.stdout:6/919: creat d0/d23/d74/d84/db3/d2f/dab/db8/f128 x:0 0 0 2024-03-23T08:24:09.045 INFO:tasks.workunit.client.0.smithi070.stdout:8/949: dread d5/d8/d16/d3b/d8c/f99 [0,4194304] 0 2024-03-23T08:24:09.046 INFO:tasks.workunit.client.0.smithi070.stdout:6/920: mknod d0/d23/d88/db6/c129 0 2024-03-23T08:24:09.046 INFO:tasks.workunit.client.0.smithi070.stdout:6/921: chown d0/d23/d74/d84/db3/f2b 14932 1 2024-03-23T08:24:09.052 INFO:tasks.workunit.client.0.smithi070.stdout:6/922: creat d0/d23/d97/d98/d127/f12a x:0 0 0 2024-03-23T08:24:09.054 INFO:tasks.workunit.client.0.smithi070.stdout:8/950: creat d5/d8/d16/d30/d38/f14e x:0 0 0 2024-03-23T08:24:09.062 INFO:tasks.workunit.client.0.smithi070.stdout:6/923: rename d0/d23/d74/d84/d3c to d0/d23/d74/d84/d12b 0 2024-03-23T08:24:09.065 INFO:tasks.workunit.client.0.smithi070.stdout:8/951: rename d5/d8/d16/d30/d38/d6b/d72/c119 to d5/d8/d16/d32/d84/d98/c14f 0 2024-03-23T08:24:09.069 INFO:tasks.workunit.client.0.smithi070.stdout:6/924: dwrite d0/d23/d74/d84/db3/d2f/f121 [0,4194304] 0 2024-03-23T08:24:09.071 INFO:tasks.workunit.client.0.smithi070.stdout:8/952: getdents d5/d8/d16/d30/d127 0 2024-03-23T08:24:09.073 INFO:tasks.workunit.client.0.smithi070.stdout:6/925: mkdir d0/d23/d88/d12c 0 2024-03-23T08:24:09.076 INFO:tasks.workunit.client.0.smithi070.stdout:8/953: creat d5/d8/d16/d32/d84/d98/dde/da3/dab/f150 x:0 0 0 2024-03-23T08:24:09.077 INFO:tasks.workunit.client.0.smithi070.stdout:6/926: fsync d0/d23/d74/d84/db3/d38/d9a/fcc 0 2024-03-23T08:24:09.093 INFO:tasks.workunit.client.0.smithi070.stdout:7/924: dwrite d25/d27/d6a/d44/d7c/db4/d114/f132 [0,4194304] 0 2024-03-23T08:24:09.094 INFO:tasks.workunit.client.0.smithi070.stdout:7/925: chown d25/d27/d6a/d44/d6f/dab/dcd/ddc/f131 1 1 2024-03-23T08:24:09.095 INFO:tasks.workunit.client.0.smithi070.stdout:7/926: chown d25/d40/f61 154251050 1 2024-03-23T08:24:09.099 INFO:tasks.workunit.client.0.smithi070.stdout:7/927: mkdir d25/d27/d6a/d44/da2/db5/d13d 0 2024-03-23T08:24:09.101 INFO:tasks.workunit.client.0.smithi070.stdout:7/928: truncate d25/d27/d6a/f5e 284759 0 2024-03-23T08:24:09.102 INFO:tasks.workunit.client.0.smithi070.stdout:6/927: write d0/d3/d2c/f108 [803455,122661] 0 2024-03-23T08:24:09.104 INFO:tasks.workunit.client.0.smithi070.stdout:7/929: mknod d25/d27/c13e 0 2024-03-23T08:24:09.105 INFO:tasks.workunit.client.0.smithi070.stdout:7/930: write d25/d27/dc2/fee [896064,117860] 0 2024-03-23T08:24:09.106 INFO:tasks.workunit.client.0.smithi070.stdout:6/928: creat d0/d23/d74/d84/db3/d2f/dab/db8/d110/f12d x:0 0 0 2024-03-23T08:24:09.113 INFO:tasks.workunit.client.0.smithi070.stdout:7/931: dwrite d25/d27/d6a/d44/d6f/f71 [0,4194304] 0 2024-03-23T08:24:09.116 INFO:tasks.workunit.client.0.smithi070.stdout:6/929: write d0/d23/d74/d84/db3/d2f/f39 [1506931,48025] 0 2024-03-23T08:24:09.129 INFO:tasks.workunit.client.0.smithi070.stdout:7/932: dwrite d25/d27/d6a/d44/da2/df4/ff5 [0,4194304] 0 2024-03-23T08:24:09.137 INFO:tasks.workunit.client.0.smithi070.stdout:7/933: creat d25/d27/d6a/d44/dc8/f13f x:0 0 0 2024-03-23T08:24:09.137 INFO:tasks.workunit.client.0.smithi070.stdout:6/930: dread d0/d23/d28/f4a [0,4194304] 0 2024-03-23T08:24:09.137 INFO:tasks.workunit.client.0.smithi070.stdout:7/934: symlink d25/d27/d6a/d44/da4/l140 0 2024-03-23T08:24:09.137 INFO:tasks.workunit.client.0.smithi070.stdout:6/931: truncate d0/d23/d97/d98/d127/f12a 1027633 0 2024-03-23T08:24:09.139 INFO:tasks.workunit.client.0.smithi070.stdout:7/935: mknod d25/d27/d6a/d44/d85/c141 0 2024-03-23T08:24:09.140 INFO:tasks.workunit.client.0.smithi070.stdout:7/936: chown d25/d27/d6a/l32 0 1 2024-03-23T08:24:09.141 INFO:tasks.workunit.client.0.smithi070.stdout:6/932: rename d0/d23/d74/d84/db3/d38/d9a/lc2 to d0/d23/d97/d98/d11e/l12e 0 2024-03-23T08:24:09.142 INFO:tasks.workunit.client.0.smithi070.stdout:7/937: symlink d25/d27/d6a/d44/d6f/dab/dcd/ddc/de4/l142 0 2024-03-23T08:24:09.143 INFO:tasks.workunit.client.0.smithi070.stdout:7/938: chown d25/f118 78655 1 2024-03-23T08:24:09.143 INFO:tasks.workunit.client.0.smithi070.stdout:6/933: fdatasync d0/f20 0 2024-03-23T08:24:09.154 INFO:tasks.workunit.client.0.smithi070.stdout:7/939: dread d25/f87 [4194304,4194304] 0 2024-03-23T08:24:09.157 INFO:tasks.workunit.client.0.smithi070.stdout:7/940: rename d25/d27/d6a/c65 to d25/d27/d6a/d44/da2/d134/c143 0 2024-03-23T08:24:09.161 INFO:tasks.workunit.client.0.smithi070.stdout:7/941: rename d25/d27/d6a/d44/d6f/dab/ffa to d25/d27/d6a/d44/d6f/dab/dcd/df1/f144 0 2024-03-23T08:24:09.165 INFO:tasks.workunit.client.0.smithi070.stdout:7/942: creat d25/d27/d6a/d44/d6f/dab/dcd/ddc/de4/f145 x:0 0 0 2024-03-23T08:24:09.166 INFO:tasks.workunit.client.0.smithi070.stdout:3/912: sync 2024-03-23T08:24:09.171 INFO:tasks.workunit.client.0.smithi070.stdout:3/913: rmdir db/d20/d7a/dcc 39 2024-03-23T08:24:09.179 INFO:tasks.workunit.client.0.smithi070.stdout:7/943: dread d25/f83 [0,4194304] 0 2024-03-23T08:24:09.181 INFO:tasks.workunit.client.0.smithi070.stdout:7/944: mknod d25/d27/d6a/d44/dfe/d107/c146 0 2024-03-23T08:24:09.183 INFO:tasks.workunit.client.0.smithi070.stdout:7/945: symlink d25/d27/d6a/d44/d7c/dfb/dfd/d10f/l147 0 2024-03-23T08:24:09.190 INFO:tasks.workunit.client.0.smithi070.stdout:7/946: dwrite d25/d27/dc2/f12f [0,4194304] 0 2024-03-23T08:24:09.195 INFO:tasks.workunit.client.0.smithi070.stdout:7/947: creat d25/d27/d6a/d44/d7c/db4/d114/f148 x:0 0 0 2024-03-23T08:24:09.199 INFO:tasks.workunit.client.0.smithi070.stdout:7/948: unlink d25/d27/d6a/d44/d7c/dfb/dfd/f12b 0 2024-03-23T08:24:09.202 INFO:tasks.workunit.client.0.smithi070.stdout:7/949: rename d25/d27/d6a/d44/d7c/cbb to d25/d27/d6a/d44/da2/c149 0 2024-03-23T08:24:09.219 INFO:tasks.workunit.client.0.smithi070.stdout:7/950: write d25/d27/dc2/fe3 [359239,3379] 0 2024-03-23T08:24:09.222 INFO:tasks.workunit.client.0.smithi070.stdout:7/951: rename d25/d27/dc2/cd1 to d25/d27/d6a/d44/d6f/c14a 0 2024-03-23T08:24:09.224 INFO:tasks.workunit.client.0.smithi070.stdout:7/952: write d25/d27/d6a/d44/da2/fbf [457246,65198] 0 2024-03-23T08:24:09.229 INFO:tasks.workunit.client.0.smithi070.stdout:7/953: dwrite d25/d27/dc2/f12f [0,4194304] 0 2024-03-23T08:24:09.234 INFO:tasks.workunit.client.0.smithi070.stdout:7/954: mkdir d25/d27/d6a/d44/d6f/dcc/d14b 0 2024-03-23T08:24:09.236 INFO:tasks.workunit.client.0.smithi070.stdout:7/955: truncate d25/d40/f61 671484 0 2024-03-23T08:24:09.238 INFO:tasks.workunit.client.0.smithi070.stdout:7/956: creat d25/d27/d6a/d44/d6f/da3/de1/f14c x:0 0 0 2024-03-23T08:24:09.240 INFO:tasks.workunit.client.0.smithi070.stdout:7/957: mknod d25/d27/d6a/d44/da2/d134/c14d 0 2024-03-23T08:24:09.242 INFO:tasks.workunit.client.0.smithi070.stdout:7/958: symlink d25/d27/d6a/d44/d7c/l14e 0 2024-03-23T08:24:09.246 INFO:tasks.workunit.client.0.smithi070.stdout:7/959: rename d25/d27/d6a/d44/d6f/dab/dcd/ddc/f131 to d25/d27/d6a/d44/da2/db5/d13d/f14f 0 2024-03-23T08:24:09.248 INFO:tasks.workunit.client.0.smithi070.stdout:7/960: stat d25/d27/d6a/d44/d7c/db4/cdb 0 2024-03-23T08:24:09.263 INFO:tasks.workunit.client.0.smithi070.stdout:7/961: rmdir d25/d27/d6a/d44/d6f 39 2024-03-23T08:24:09.266 INFO:tasks.workunit.client.0.smithi070.stdout:7/962: mkdir d25/d27/d6a/d44/d7c/db4/d114/d150 0 2024-03-23T08:24:09.269 INFO:tasks.workunit.client.0.smithi070.stdout:7/963: mkdir d25/d27/d6a/d44/d6f/dab/dcd/ddc/d151 0 2024-03-23T08:24:09.276 INFO:tasks.workunit.client.0.smithi070.stdout:7/964: dwrite d25/d27/d6a/d44/d6f/d104/f135 [0,4194304] 0 2024-03-23T08:24:09.277 INFO:tasks.workunit.client.0.smithi070.stdout:7/965: read d25/f34 [1413513,13231] 0 2024-03-23T08:24:09.281 INFO:tasks.workunit.client.0.smithi070.stdout:7/966: mkdir d25/d27/d6a/d44/d152 0 2024-03-23T08:24:09.284 INFO:tasks.workunit.client.0.smithi070.stdout:7/967: mkdir d25/d27/d6a/d44/d85/d153 0 2024-03-23T08:24:09.286 INFO:tasks.workunit.client.0.smithi070.stdout:7/968: write d25/d27/d6a/d44/d6f/dcc/d11a/f137 [98628,41939] 0 2024-03-23T08:24:09.391 INFO:tasks.workunit.client.0.smithi070.stdout:9/992: sync 2024-03-23T08:24:09.391 INFO:tasks.workunit.client.0.smithi070.stdout:1/821: sync 2024-03-23T08:24:09.394 INFO:tasks.workunit.client.0.smithi070.stdout:1/822: dread d8/db/d40/d43/d64/f8b [0,4194304] 0 2024-03-23T08:24:09.397 INFO:tasks.workunit.client.0.smithi070.stdout:9/993: dwrite db/db8/dd8/d116/f12d [0,4194304] 0 2024-03-23T08:24:09.407 INFO:tasks.workunit.client.0.smithi070.stdout:9/994: rmdir db/d12/d14/de5/da2/dc6/d130 0 2024-03-23T08:24:09.409 INFO:tasks.workunit.client.0.smithi070.stdout:1/823: write d8/d1a/f23 [1329544,112248] 0 2024-03-23T08:24:09.410 INFO:tasks.workunit.client.0.smithi070.stdout:9/995: truncate db/d12/d14/de5/da2/f6a 511110 0 2024-03-23T08:24:09.413 INFO:tasks.workunit.client.0.smithi070.stdout:9/996: creat db/d12/d14/de5/da2/d4f/dd9/f148 x:0 0 0 2024-03-23T08:24:09.494 INFO:tasks.workunit.client.0.smithi070.stdout:1/824: rename d8/f1c to d8/db/d2c/f125 0 2024-03-23T08:24:09.495 INFO:tasks.workunit.client.0.smithi070.stdout:1/825: truncate d8/db/d40/d43/f109 869137 0 2024-03-23T08:24:09.500 INFO:tasks.workunit.client.0.smithi070.stdout:1/826: write d8/d1b/f22 [5328703,125245] 0 2024-03-23T08:24:09.505 INFO:tasks.workunit.client.0.smithi070.stdout:9/997: dwrite db/d12/d14/de5/da2/dc6/f106 [0,4194304] 0 2024-03-23T08:24:09.506 INFO:tasks.workunit.client.0.smithi070.stdout:1/827: rename d8/d26/d7b/c10e to d8/db/d40/d43/d64/d85/dc7/c126 0 2024-03-23T08:24:09.508 INFO:tasks.workunit.client.0.smithi070.stdout:9/998: creat db/d12/d14/de5/da2/d4f/de6/f149 x:0 0 0 2024-03-23T08:24:09.511 INFO:tasks.workunit.client.0.smithi070.stdout:1/828: creat d8/db/d40/d43/d64/d85/d97/de7/f127 x:0 0 0 2024-03-23T08:24:09.511 INFO:tasks.workunit.client.0.smithi070.stdout:9/999: chown db/d12/d14/de5/da2/l6b 7704884 1 2024-03-23T08:24:09.511 INFO:tasks.workunit.client.0.smithi070.stdout:1/829: chown d8/d26/c82 13 1 2024-03-23T08:24:09.513 INFO:tasks.workunit.client.0.smithi070.stdout:1/830: creat d8/d26/d49/f128 x:0 0 0 2024-03-23T08:24:09.523 INFO:tasks.workunit.client.0.smithi070.stdout:1/831: unlink d8/db/d40/d43/d64/d85/fbc 0 2024-03-23T08:24:09.525 INFO:tasks.workunit.client.0.smithi070.stdout:1/832: chown d8/db/d40/d43/d64/d85/c105 26626 1 2024-03-23T08:24:09.528 INFO:tasks.workunit.client.0.smithi070.stdout:4/783: sync 2024-03-23T08:24:09.528 INFO:tasks.workunit.client.0.smithi070.stdout:5/796: sync 2024-03-23T08:24:09.528 INFO:tasks.workunit.client.0.smithi070.stdout:1/833: dread d8/d11/f5c [0,4194304] 0 2024-03-23T08:24:09.530 INFO:tasks.workunit.client.0.smithi070.stdout:4/784: mkdir d3/d6/d9/d110 0 2024-03-23T08:24:09.533 INFO:tasks.workunit.client.0.smithi070.stdout:4/785: mknod d3/d6/d35/dcf/d7e/c111 0 2024-03-23T08:24:09.534 INFO:tasks.workunit.client.0.smithi070.stdout:4/786: chown d3/d6/d35/dcf/c80 4243651 1 2024-03-23T08:24:09.535 INFO:tasks.workunit.client.0.smithi070.stdout:1/834: dwrite d8/db/d2c/d3f/d48/f58 [0,4194304] 0 2024-03-23T08:24:09.541 INFO:tasks.workunit.client.0.smithi070.stdout:5/797: truncate dc/d1b/d1d/d25/d38/d39/fba 3907687 0 2024-03-23T08:24:09.544 INFO:tasks.workunit.client.0.smithi070.stdout:4/787: dwrite d3/d6/d35/d65/d27/f46 [4194304,4194304] 0 2024-03-23T08:24:09.545 INFO:tasks.workunit.client.0.smithi070.stdout:5/798: unlink dc/d1b/d1d/d25/d38/d39/f6b 0 2024-03-23T08:24:09.549 INFO:tasks.workunit.client.0.smithi070.stdout:1/835: write d8/d11/f29 [1975892,29327] 0 2024-03-23T08:24:09.550 INFO:tasks.workunit.client.0.smithi070.stdout:5/799: creat dc/d1b/d1d/d25/d36/d32/db7/de6/f102 x:0 0 0 2024-03-23T08:24:09.551 INFO:tasks.workunit.client.0.smithi070.stdout:4/788: creat d3/d6/d35/d65/d27/de1/f112 x:0 0 0 2024-03-23T08:24:09.554 INFO:tasks.workunit.client.0.smithi070.stdout:4/789: mkdir d3/d6/d35/d65/d27/de1/d113 0 2024-03-23T08:24:09.555 INFO:tasks.workunit.client.0.smithi070.stdout:4/790: chown d3/d6/d35/dcf/d7e/cf6 22 1 2024-03-23T08:24:09.555 INFO:tasks.workunit.client.0.smithi070.stdout:4/791: readlink d3/d6/d9/l33 0 2024-03-23T08:24:09.557 INFO:tasks.workunit.client.0.smithi070.stdout:4/792: mkdir d3/d114 0 2024-03-23T08:24:09.562 INFO:tasks.workunit.client.0.smithi070.stdout:5/800: dwrite dc/d1b/d1d/d25/d68/d8a/d99/fb2 [0,4194304] 0 2024-03-23T08:24:09.565 INFO:tasks.workunit.client.0.smithi070.stdout:6/934: sync 2024-03-23T08:24:09.565 INFO:tasks.workunit.client.0.smithi070.stdout:8/954: sync 2024-03-23T08:24:09.568 INFO:tasks.workunit.client.0.smithi070.stdout:5/801: truncate dc/d1b/d1d/d25/d36/d32/db7/de6/ff5 532150 0 2024-03-23T08:24:09.572 INFO:tasks.workunit.client.0.smithi070.stdout:6/935: rename d0/d23/d74/d84/db3/d38/d52/f7f to d0/d23/d74/dcd/f12f 0 2024-03-23T08:24:09.577 INFO:tasks.workunit.client.0.smithi070.stdout:1/836: dwrite d8/d11/f4d [0,4194304] 0 2024-03-23T08:24:09.577 INFO:tasks.workunit.client.0.smithi070.stdout:5/802: creat dc/d1b/d1d/d25/d38/f103 x:0 0 0 2024-03-23T08:24:09.578 INFO:tasks.workunit.client.0.smithi070.stdout:1/837: chown d8/db/d2c/d3f/fbf 164221708 1 2024-03-23T08:24:09.579 INFO:tasks.workunit.client.0.smithi070.stdout:5/803: read - dc/d1b/d1d/d25/d36/fd3 zero size 2024-03-23T08:24:09.580 INFO:tasks.workunit.client.0.smithi070.stdout:5/804: read - dc/d1b/fe5 zero size 2024-03-23T08:24:09.582 INFO:tasks.workunit.client.0.smithi070.stdout:5/805: write dc/d1b/d1d/d25/d36/d32/d5c/fab [506462,53812] 0 2024-03-23T08:24:09.585 INFO:tasks.workunit.client.0.smithi070.stdout:6/936: read d0/d23/d74/d84/d12b/f75 [2204954,26858] 0 2024-03-23T08:24:09.591 INFO:tasks.workunit.client.0.smithi070.stdout:5/806: dwrite dc/d1b/d1d/fde [0,4194304] 0 2024-03-23T08:24:09.595 INFO:tasks.workunit.client.0.smithi070.stdout:6/937: mkdir d0/d23/d88/d130 0 2024-03-23T08:24:09.597 INFO:tasks.workunit.client.0.smithi070.stdout:4/793: dwrite d3/d6/d35/d65/d27/d4b/f10e [0,4194304] 0 2024-03-23T08:24:09.597 INFO:tasks.workunit.client.0.smithi070.stdout:6/938: readlink d0/d23/d28/d35/l3e 0 2024-03-23T08:24:09.598 INFO:tasks.workunit.client.0.smithi070.stdout:6/939: dread - d0/d23/d4d/f6e zero size 2024-03-23T08:24:09.601 INFO:tasks.workunit.client.0.smithi070.stdout:8/955: write d5/d8/d16/d32/f54 [2160884,78969] 0 2024-03-23T08:24:09.602 INFO:tasks.workunit.client.0.smithi070.stdout:8/956: chown d5/d8/d16/dd4/fa6 49304 1 2024-03-23T08:24:09.603 INFO:tasks.workunit.client.0.smithi070.stdout:8/957: fsync d5/d8/ffe 0 2024-03-23T08:24:09.605 INFO:tasks.workunit.client.0.smithi070.stdout:4/794: mknod d3/d6/d35/dcf/db5/dbb/c115 0 2024-03-23T08:24:09.606 INFO:tasks.workunit.client.0.smithi070.stdout:6/940: mknod d0/d3/d2c/c131 0 2024-03-23T08:24:09.608 INFO:tasks.workunit.client.0.smithi070.stdout:5/807: getdents dc/d1b/d1d/d25/d36/d32/db7/de6/df0 0 2024-03-23T08:24:09.609 INFO:tasks.workunit.client.0.smithi070.stdout:1/838: link d8/db/l2a d8/d1a/d8f/l129 0 2024-03-23T08:24:09.613 INFO:tasks.workunit.client.0.smithi070.stdout:8/958: creat d5/d8/d9a/f151 x:0 0 0 2024-03-23T08:24:09.614 INFO:tasks.workunit.client.0.smithi070.stdout:8/959: chown d5/d8/d16/d32/c4c 2180 1 2024-03-23T08:24:09.616 INFO:tasks.workunit.client.0.smithi070.stdout:8/960: write d5/d8/d16/f12d [729749,70637] 0 2024-03-23T08:24:09.624 INFO:tasks.workunit.client.0.smithi070.stdout:6/941: symlink d0/d23/d88/dd9/d111/l132 0 2024-03-23T08:24:09.627 INFO:tasks.workunit.client.0.smithi070.stdout:8/961: dread d5/d8/f4e [0,4194304] 0 2024-03-23T08:24:09.627 INFO:tasks.workunit.client.0.smithi070.stdout:8/962: chown d5/d8/d16/dd4/d115/f14a 4276 1 2024-03-23T08:24:09.629 INFO:tasks.workunit.client.0.smithi070.stdout:5/808: symlink dc/d1b/d1d/d25/d68/d8a/l104 0 2024-03-23T08:24:09.630 INFO:tasks.workunit.client.0.smithi070.stdout:5/809: chown dc/d1b/d1d/ffd 0 1 2024-03-23T08:24:09.633 INFO:tasks.workunit.client.0.smithi070.stdout:4/795: creat d3/d6/d35/d65/d27/f116 x:0 0 0 2024-03-23T08:24:09.635 INFO:tasks.workunit.client.0.smithi070.stdout:6/942: rename d0/d23/d4d to d0/d23/d74/d84/db3/d2f/dab/db8/d133 0 2024-03-23T08:24:09.636 INFO:tasks.workunit.client.0.smithi070.stdout:6/943: write d0/f9c [33815,43003] 0 2024-03-23T08:24:09.638 INFO:tasks.workunit.client.0.smithi070.stdout:7/969: sync 2024-03-23T08:24:09.638 INFO:tasks.workunit.client.0.smithi070.stdout:3/914: sync 2024-03-23T08:24:09.639 INFO:tasks.workunit.client.0.smithi070.stdout:8/963: truncate d5/d8/d16/d32/d84/fef 514595 0 2024-03-23T08:24:09.640 INFO:tasks.workunit.client.0.smithi070.stdout:7/970: chown d25/d27/d6a/d44/d7c/db4/cdf 7902 1 2024-03-23T08:24:09.642 INFO:tasks.workunit.client.0.smithi070.stdout:7/971: dread d25/d27/d6a/d44/f77 [0,4194304] 0 2024-03-23T08:24:09.649 INFO:tasks.workunit.client.0.smithi070.stdout:3/915: dread db/d4e/f73 [0,4194304] 0 2024-03-23T08:24:09.656 INFO:tasks.workunit.client.0.smithi070.stdout:1/839: link d8/d1a/l57 d8/db/d2c/l12a 0 2024-03-23T08:24:09.658 INFO:tasks.workunit.client.0.smithi070.stdout:3/916: dwrite db/d4e/dc4/d10c/f13d [0,4194304] 0 2024-03-23T08:24:09.659 INFO:tasks.workunit.client.0.smithi070.stdout:7/972: rename d25/l2e to d25/d27/d6a/d44/d85/l154 0 2024-03-23T08:24:09.663 INFO:tasks.workunit.client.0.smithi070.stdout:4/796: dread d3/d6/d35/dcf/db5/dda/db8/fa7 [0,4194304] 0 2024-03-23T08:24:09.663 INFO:tasks.workunit.client.0.smithi070.stdout:5/810: mkdir dc/d1b/d1d/d25/d68/d8a/dad/d105 0 2024-03-23T08:24:09.678 INFO:tasks.workunit.client.0.smithi070.stdout:8/964: dwrite d5/d8/d16/d32/f56 [0,4194304] 0 2024-03-23T08:24:09.680 INFO:tasks.workunit.client.0.smithi070.stdout:4/797: rename d3/d6/d35/d65/d27/d4b/dc0/f107 to d3/d6/d35/dcf/db5/dda/db8/d88/f117 0 2024-03-23T08:24:09.684 INFO:tasks.workunit.client.0.smithi070.stdout:8/965: dread d5/d8/d9a/f108 [0,4194304] 0 2024-03-23T08:24:09.699 INFO:tasks.workunit.client.0.smithi070.stdout:8/966: dread d5/d8/d16/d30/f112 [0,4194304] 0 2024-03-23T08:24:09.699 INFO:tasks.workunit.client.0.smithi070.stdout:1/840: dwrite d8/d42/d51/f91 [0,4194304] 0 2024-03-23T08:24:09.701 INFO:tasks.workunit.client.0.smithi070.stdout:3/917: dwrite db/df/d5d/fda [4194304,4194304] 0 2024-03-23T08:24:09.702 INFO:tasks.workunit.client.0.smithi070.stdout:1/841: stat d8/db/d2c/d3f/dd3/d111 0 2024-03-23T08:24:09.703 INFO:tasks.workunit.client.0.smithi070.stdout:1/842: readlink d8/db/d100/l8c 0 2024-03-23T08:24:09.703 INFO:tasks.workunit.client.0.smithi070.stdout:6/944: getdents d0/d23/d74/d84/db3/d2f/dab/db8/d110 0 2024-03-23T08:24:09.704 INFO:tasks.workunit.client.0.smithi070.stdout:8/967: fsync d5/d8/d16/d30/d38/d6b/da9/f14c 0 2024-03-23T08:24:09.706 INFO:tasks.workunit.client.0.smithi070.stdout:6/945: chown d0/d23/d74/d84/d12b/f60 43 1 2024-03-23T08:24:09.710 INFO:tasks.workunit.client.0.smithi070.stdout:5/811: link dc/d1b/c93 dc/d1b/d1d/d25/d36/d32/db7/c106 0 2024-03-23T08:24:09.712 INFO:tasks.workunit.client.0.smithi070.stdout:5/812: write dc/d1b/d1d/d25/d38/fd6 [536859,52148] 0 2024-03-23T08:24:09.716 INFO:tasks.workunit.client.0.smithi070.stdout:5/813: readlink dc/d1b/d1d/d25/d38/d39/lb6 0 2024-03-23T08:24:09.716 INFO:tasks.workunit.client.0.smithi070.stdout:4/798: mkdir d3/d6/d11/d118 0 2024-03-23T08:24:09.718 INFO:tasks.workunit.client.0.smithi070.stdout:3/918: creat db/df/d1a/de9/f155 x:0 0 0 2024-03-23T08:24:09.721 INFO:tasks.workunit.client.0.smithi070.stdout:6/946: symlink d0/d23/d97/d98/d127/l134 0 2024-03-23T08:24:09.723 INFO:tasks.workunit.client.0.smithi070.stdout:7/973: link d25/d27/d6a/d44/dfe/d107/c146 d25/d27/d6a/d44/d85/d95/da0/c155 0 2024-03-23T08:24:09.724 INFO:tasks.workunit.client.0.smithi070.stdout:7/974: chown d25/d27/d6a/d44/d7c/db4/d114/f148 22775815 1 2024-03-23T08:24:09.725 INFO:tasks.workunit.client.0.smithi070.stdout:4/799: mknod d3/d6/d35/d3f/d51/d6e/d97/c119 0 2024-03-23T08:24:09.728 INFO:tasks.workunit.client.0.smithi070.stdout:7/975: dread d25/d27/dc2/fee [0,4194304] 0 2024-03-23T08:24:09.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:09 smithi070 ceph-mon[39699]: pgmap v44: 65 pgs: 65 active+clean; 2.0 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 61 MiB/s rd, 159 MiB/s wr, 504 op/s 2024-03-23T08:24:09.733 INFO:tasks.workunit.client.0.smithi070.stdout:6/947: creat d0/d23/d74/d84/d12b/d86/f135 x:0 0 0 2024-03-23T08:24:09.737 INFO:tasks.workunit.client.0.smithi070.stdout:3/919: mknod db/d4e/dc4/d11e/c156 0 2024-03-23T08:24:09.740 INFO:tasks.workunit.client.0.smithi070.stdout:4/800: mknod d3/d6/d35/d65/d6f/c11a 0 2024-03-23T08:24:09.743 INFO:tasks.workunit.client.0.smithi070.stdout:1/843: write d8/db/d2c/d3f/dd3/d111/f11a [688669,64635] 0 2024-03-23T08:24:09.745 INFO:tasks.workunit.client.0.smithi070.stdout:8/968: rmdir d5/d8/d16/d32/d84/da7/deb/dfa 0 2024-03-23T08:24:09.747 INFO:tasks.workunit.client.0.smithi070.stdout:5/814: write dc/d1b/d1d/d25/d68/d8a/fa3 [100046,21695] 0 2024-03-23T08:24:09.748 INFO:tasks.workunit.client.0.smithi070.stdout:1/844: dread d8/d26/ff5 [0,4194304] 0 2024-03-23T08:24:09.753 INFO:tasks.workunit.client.0.smithi070.stdout:7/976: mknod d25/d27/d6a/d44/d152/c156 0 2024-03-23T08:24:09.758 INFO:tasks.workunit.client.0.smithi070.stdout:5/815: unlink dc/d1b/d1d/d25/d38/d39/c82 0 2024-03-23T08:24:09.760 INFO:tasks.workunit.client.0.smithi070.stdout:1/845: dread d8/d11/f29 [0,4194304] 0 2024-03-23T08:24:09.762 INFO:tasks.workunit.client.0.smithi070.stdout:6/948: dread d0/d23/d28/d35/d85/f9e [0,4194304] 0 2024-03-23T08:24:09.765 INFO:tasks.workunit.client.0.smithi070.stdout:4/801: fsync d3/d6/d35/d3f/da9/f10a 0 2024-03-23T08:24:09.770 INFO:tasks.workunit.client.0.smithi070.stdout:4/802: dread d3/d6/d35/d65/d27/d4b/f10e [0,4194304] 0 2024-03-23T08:24:09.771 INFO:tasks.workunit.client.0.smithi070.stdout:4/803: stat d3/d6/d35/d65/d27/l4f 0 2024-03-23T08:24:09.778 INFO:tasks.workunit.client.0.smithi070.stdout:1/846: rename d8/db/d2c/fd2 to d8/db/d40/d43/d64/d85/d97/de7/f12b 0 2024-03-23T08:24:09.778 INFO:tasks.workunit.client.0.smithi070.stdout:8/969: write d5/d8/d16/d30/f39 [1718250,123842] 0 2024-03-23T08:24:09.779 INFO:tasks.workunit.client.0.smithi070.stdout:1/847: write d8/d11/f3b [4414357,100815] 0 2024-03-23T08:24:09.781 INFO:tasks.workunit.client.0.smithi070.stdout:5/816: rmdir dc/d1b/d1d/d25/d38 39 2024-03-23T08:24:09.783 INFO:tasks.workunit.client.0.smithi070.stdout:3/920: truncate db/df/d5d/fda 6088338 0 2024-03-23T08:24:09.787 INFO:tasks.workunit.client.0.smithi070.stdout:4/804: unlink d3/d6/d35/dcf/db5/dbb/c100 0 2024-03-23T08:24:09.788 INFO:tasks.workunit.client.0.smithi070.stdout:4/805: fsync d3/d6/d35/d3f/d51/f63 0 2024-03-23T08:24:09.789 INFO:tasks.workunit.client.0.smithi070.stdout:4/806: stat d3/d6/d35/dcf/db5/dda/db8/d88/f117 0 2024-03-23T08:24:09.790 INFO:tasks.workunit.client.0.smithi070.stdout:4/807: write d3/d6/d11/f10f [507870,63301] 0 2024-03-23T08:24:09.794 INFO:tasks.workunit.client.0.smithi070.stdout:8/970: unlink d5/d8/d16/d30/f112 0 2024-03-23T08:24:09.803 INFO:tasks.workunit.client.0.smithi070.stdout:3/921: symlink db/d4e/df8/d3d/l157 0 2024-03-23T08:24:09.805 INFO:tasks.workunit.client.0.smithi070.stdout:7/977: truncate d25/d27/d6a/d44/d6f/da3/f103 298870 0 2024-03-23T08:24:09.807 INFO:tasks.workunit.client.0.smithi070.stdout:6/949: creat d0/d23/d74/d84/f136 x:0 0 0 2024-03-23T08:24:09.811 INFO:tasks.workunit.client.0.smithi070.stdout:5/817: fdatasync dc/d1b/d1d/f37 0 2024-03-23T08:24:09.813 INFO:tasks.workunit.client.0.smithi070.stdout:5/818: chown dc/d1b/d1d/d25/d36/c78 259027979 1 2024-03-23T08:24:09.815 INFO:tasks.workunit.client.0.smithi070.stdout:7/978: creat d25/d40/f157 x:0 0 0 2024-03-23T08:24:09.818 INFO:tasks.workunit.client.0.smithi070.stdout:4/808: creat d3/d6/d11/d118/f11b x:0 0 0 2024-03-23T08:24:09.822 INFO:tasks.workunit.client.0.smithi070.stdout:1/848: link d8/db/c20 d8/d11/c12c 0 2024-03-23T08:24:09.823 INFO:tasks.workunit.client.0.smithi070.stdout:6/950: dread d0/d23/d74/d84/db3/d2f/f30 [0,4194304] 0 2024-03-23T08:24:09.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:09 smithi193 ceph-mon[48370]: pgmap v44: 65 pgs: 65 active+clean; 2.0 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 61 MiB/s rd, 159 MiB/s wr, 504 op/s 2024-03-23T08:24:09.842 INFO:tasks.workunit.client.0.smithi070.stdout:6/951: mknod d0/d23/d74/d84/db3/d2f/dab/d102/c137 0 2024-03-23T08:24:09.846 INFO:tasks.workunit.client.0.smithi070.stdout:5/819: rename dc/d1b/d1d/d25/d38/d48 to dc/d1b/d1d/d25/da9/deb/d107 0 2024-03-23T08:24:09.847 INFO:tasks.workunit.client.0.smithi070.stdout:3/922: truncate db/d4e/dc4/d10c/f128 3629307 0 2024-03-23T08:24:09.849 INFO:tasks.workunit.client.0.smithi070.stdout:4/809: dread d3/d6/d35/d65/d1b/fae [0,4194304] 0 2024-03-23T08:24:09.851 INFO:tasks.workunit.client.0.smithi070.stdout:4/810: chown d3/f23 126208 1 2024-03-23T08:24:09.853 INFO:tasks.workunit.client.0.smithi070.stdout:1/849: symlink d8/db/d100/d74/d5d/d70/d11c/l12d 0 2024-03-23T08:24:09.854 INFO:tasks.workunit.client.0.smithi070.stdout:5/820: dread dc/d1b/d1d/d25/da9/deb/d107/f56 [0,4194304] 0 2024-03-23T08:24:09.855 INFO:tasks.workunit.client.0.smithi070.stdout:4/811: write d3/d6/d35/d65/df1/ffc [2270877,8793] 0 2024-03-23T08:24:09.857 INFO:tasks.workunit.client.0.smithi070.stdout:3/923: dwrite db/d4e/dc4/d137/f153 [0,4194304] 0 2024-03-23T08:24:09.858 INFO:tasks.workunit.client.0.smithi070.stdout:5/821: readlink dc/d1b/d1d/d25/d36/d32/laf 0 2024-03-23T08:24:09.859 INFO:tasks.workunit.client.0.smithi070.stdout:5/822: readlink dc/d1b/d1d/d25/d36/l31 0 2024-03-23T08:24:09.869 INFO:tasks.workunit.client.0.smithi070.stdout:3/924: creat db/d20/d63/d83/f158 x:0 0 0 2024-03-23T08:24:09.870 INFO:tasks.workunit.client.0.smithi070.stdout:1/850: dwrite d8/db/d40/d43/f6c [0,4194304] 0 2024-03-23T08:24:09.870 INFO:tasks.workunit.client.0.smithi070.stdout:5/823: readlink dc/d1b/d1d/d25/d36/ld9 0 2024-03-23T08:24:09.872 INFO:tasks.workunit.client.0.smithi070.stdout:5/824: chown dc/d1b/d1d/d25/d38/fd6 2567 1 2024-03-23T08:24:09.875 INFO:tasks.workunit.client.0.smithi070.stdout:4/812: symlink d3/d114/l11c 0 2024-03-23T08:24:09.877 INFO:tasks.workunit.client.0.smithi070.stdout:6/952: rename d0/d23/d74/d84/db3/d2f/dab/db8/cf7 to d0/d23/d74/d84/d12b/c138 0 2024-03-23T08:24:09.881 INFO:tasks.workunit.client.0.smithi070.stdout:7/979: write d25/d27/d6a/d44/d85/d95/da0/fb2 [252272,16461] 0 2024-03-23T08:24:09.882 INFO:tasks.workunit.client.0.smithi070.stdout:3/925: unlink db/d14/d65/daa/fad 0 2024-03-23T08:24:09.883 INFO:tasks.workunit.client.0.smithi070.stdout:5/825: truncate dc/d1b/d1d/d25/d36/f2a 2564605 0 2024-03-23T08:24:09.883 INFO:tasks.workunit.client.0.smithi070.stdout:3/926: stat db/d20/d49/de1/d139/f10f 0 2024-03-23T08:24:09.893 INFO:tasks.workunit.client.0.smithi070.stdout:1/851: truncate d8/d26/d49/d65/fdb 1091991 0 2024-03-23T08:24:09.894 INFO:tasks.workunit.client.0.smithi070.stdout:4/813: rename d3/d6/d35/l44 to d3/d6/d35/d65/df1/l11d 0 2024-03-23T08:24:09.897 INFO:tasks.workunit.client.0.smithi070.stdout:6/953: fsync d0/d23/d28/f7d 0 2024-03-23T08:24:09.900 INFO:tasks.workunit.client.0.smithi070.stdout:3/927: truncate db/df/d1a/de9/fff 4113188 0 2024-03-23T08:24:09.904 INFO:tasks.workunit.client.0.smithi070.stdout:4/814: creat d3/d6/d9/de6/f11e x:0 0 0 2024-03-23T08:24:09.905 INFO:tasks.workunit.client.0.smithi070.stdout:4/815: readlink d3/d6/d9/l33 0 2024-03-23T08:24:09.908 INFO:tasks.workunit.client.0.smithi070.stdout:6/954: rename d0/d23/d28/cbf to d0/d23/d97/c139 0 2024-03-23T08:24:09.912 INFO:tasks.workunit.client.0.smithi070.stdout:1/852: link d8/d1a/f28 d8/db/d100/f12e 0 2024-03-23T08:24:09.918 INFO:tasks.workunit.client.0.smithi070.stdout:1/853: dwrite d8/d11/f4d [0,4194304] 0 2024-03-23T08:24:09.922 INFO:tasks.workunit.client.0.smithi070.stdout:5/826: link dc/d1b/d1d/d25/d36/d32/c81 dc/d1b/d1d/d25/d68/d8a/dad/df8/c108 0 2024-03-23T08:24:09.929 INFO:tasks.workunit.client.0.smithi070.stdout:1/854: mknod d8/db/d2c/d3f/d48/c12f 0 2024-03-23T08:24:09.932 INFO:tasks.workunit.client.0.smithi070.stdout:5/827: mknod dc/d1b/d1d/d25/d36/d32/d5c/c109 0 2024-03-23T08:24:09.933 INFO:tasks.workunit.client.0.smithi070.stdout:5/828: dread - dc/d1b/d1d/d25/d38/d39/dbc/fff zero size 2024-03-23T08:24:09.952 INFO:tasks.workunit.client.0.smithi070.stdout:7/980: write d25/d27/d6a/fba [167704,66221] 0 2024-03-23T08:24:09.953 INFO:tasks.workunit.client.0.smithi070.stdout:7/981: write d25/d27/d6a/fba [1464591,95452] 0 2024-03-23T08:24:09.974 INFO:tasks.workunit.client.0.smithi070.stdout:1/855: link d8/d42/fee d8/d26/d49/f130 0 2024-03-23T08:24:09.975 INFO:tasks.workunit.client.0.smithi070.stdout:3/928: write db/d4e/dc4/d10c/f128 [4082358,100903] 0 2024-03-23T08:24:09.982 INFO:tasks.workunit.client.0.smithi070.stdout:7/982: symlink d25/d27/d6a/d44/d85/d95/da0/de2/l158 0 2024-03-23T08:24:09.984 INFO:tasks.workunit.client.0.smithi070.stdout:4/816: write d3/d6/d35/d3f/d51/d6e/f99 [2278750,28164] 0 2024-03-23T08:24:09.990 INFO:tasks.workunit.client.0.smithi070.stdout:7/983: symlink d25/d27/d6a/d44/d6f/da3/l159 0 2024-03-23T08:24:09.992 INFO:tasks.workunit.client.0.smithi070.stdout:7/984: dread d25/d40/f10d [0,4194304] 0 2024-03-23T08:24:09.994 INFO:tasks.workunit.client.0.smithi070.stdout:7/985: fsync d25/d27/d6a/d44/da4/f11f 0 2024-03-23T08:24:09.995 INFO:tasks.workunit.client.0.smithi070.stdout:3/929: mknod db/d20/d7a/c159 0 2024-03-23T08:24:09.997 INFO:tasks.workunit.client.0.smithi070.stdout:6/955: dwrite d0/d3/f7a [0,4194304] 0 2024-03-23T08:24:09.997 INFO:tasks.workunit.client.0.smithi070.stdout:4/817: rename d3/d6/d35/d65/d27/l4f to d3/d6/d6a/l11f 0 2024-03-23T08:24:10.006 INFO:tasks.workunit.client.0.smithi070.stdout:3/930: symlink db/d4e/dd6/dd7/dd9/l15a 0 2024-03-23T08:24:10.011 INFO:tasks.workunit.client.0.smithi070.stdout:4/818: symlink d3/d6/d35/d3f/d51/d6e/d97/l120 0 2024-03-23T08:24:10.027 INFO:tasks.workunit.client.0.smithi070.stdout:4/819: unlink d3/d6/d35/d65/d1b/ld2 0 2024-03-23T08:24:10.028 INFO:tasks.workunit.client.0.smithi070.stdout:4/820: fsync d3/d6/d9/de6/f11e 0 2024-03-23T08:24:10.050 INFO:tasks.workunit.client.0.smithi070.stdout:7/986: link d25/d40/l94 d25/d27/d6a/d44/d85/d95/da0/l15a 0 2024-03-23T08:24:10.052 INFO:tasks.workunit.client.0.smithi070.stdout:3/931: getdents db/d4e/dd6/dd7/dd9/de8 0 2024-03-23T08:24:10.065 INFO:tasks.workunit.client.0.smithi070.stdout:7/987: dwrite d25/d27/d6a/d44/d6f/d104/f138 [0,4194304] 0 2024-03-23T08:24:10.074 INFO:tasks.workunit.client.0.smithi070.stdout:3/932: getdents db/d4e/df8/d22/db2/dba 0 2024-03-23T08:24:10.081 INFO:tasks.workunit.client.0.smithi070.stdout:3/933: creat db/d14/d65/da8/f15b x:0 0 0 2024-03-23T08:24:10.083 INFO:tasks.workunit.client.0.smithi070.stdout:3/934: truncate db/d4e/df8/d22/f2c 1019271 0 2024-03-23T08:24:10.086 INFO:tasks.workunit.client.0.smithi070.stdout:3/935: creat db/d20/f15c x:0 0 0 2024-03-23T08:24:10.089 INFO:tasks.workunit.client.0.smithi070.stdout:3/936: truncate db/df/d1a/f50 2965157 0 2024-03-23T08:24:10.089 INFO:tasks.workunit.client.0.smithi070.stdout:3/937: readlink db/d7f/dcd/d100/l12c 0 2024-03-23T08:24:10.117 INFO:tasks.workunit.client.0.smithi070.stdout:5/829: dwrite dc/d1b/d1d/d25/d36/f27 [0,4194304] 0 2024-03-23T08:24:10.117 INFO:tasks.workunit.client.0.smithi070.stdout:1/856: truncate d8/db/d40/d43/d64/d85/d97/d9a/fb7 299017 0 2024-03-23T08:24:10.119 INFO:tasks.workunit.client.0.smithi070.stdout:5/830: chown dc/d1b/d1d/d25/d38/d39/lb6 162985775 1 2024-03-23T08:24:10.125 INFO:tasks.workunit.client.0.smithi070.stdout:5/831: fdatasync dc/d1b/d1d/d25/d36/d32/d5c/f65 0 2024-03-23T08:24:10.129 INFO:tasks.workunit.client.0.smithi070.stdout:5/832: truncate dc/d1b/d1d/f37 216160 0 2024-03-23T08:24:10.134 INFO:tasks.workunit.client.0.smithi070.stdout:5/833: rename dc/d1b/d1d/d25/d38/d39/dbc to dc/d1b/d1d/d25/d36/d32/d5c/d10a 0 2024-03-23T08:24:10.146 INFO:tasks.workunit.client.0.smithi070.stdout:5/834: dread dc/d1b/d1d/d25/d36/dfc/fbd [0,4194304] 0 2024-03-23T08:24:10.147 INFO:tasks.workunit.client.0.smithi070.stdout:5/835: stat dc/d1b/d1d/d25/d38/d53/f7a 0 2024-03-23T08:24:10.149 INFO:tasks.workunit.client.0.smithi070.stdout:5/836: write dc/d1b/d1d/d25/d36/d32/d5c/d10a/fff [889584,42731] 0 2024-03-23T08:24:10.155 INFO:tasks.workunit.client.0.smithi070.stdout:5/837: unlink dc/d1b/d1d/d25/d38/d53/l63 0 2024-03-23T08:24:10.155 INFO:tasks.workunit.client.0.smithi070.stdout:6/956: write d0/d3/d2c/fc5 [2891419,24640] 0 2024-03-23T08:24:10.161 INFO:tasks.workunit.client.0.smithi070.stdout:6/957: rename d0/c64 to d0/d23/d74/d84/db3/d2f/dab/db8/d110/c13a 0 2024-03-23T08:24:10.166 INFO:tasks.workunit.client.0.smithi070.stdout:4/821: truncate d3/d6/d6a/d7c/fea 3872990 0 2024-03-23T08:24:10.169 INFO:tasks.workunit.client.0.smithi070.stdout:6/958: dwrite d0/d23/d74/d84/db3/d2f/dab/db8/fca [0,4194304] 0 2024-03-23T08:24:10.170 INFO:tasks.workunit.client.0.smithi070.stdout:6/959: fdatasync d0/d23/d97/d98/f123 0 2024-03-23T08:24:10.171 INFO:tasks.workunit.client.0.smithi070.stdout:5/838: rename dc/d1b/d1d/d25/d38/d53/f60 to dc/d1b/d1d/d25/d68/f10b 0 2024-03-23T08:24:10.175 INFO:tasks.workunit.client.0.smithi070.stdout:7/988: write d25/d40/f61 [627458,113426] 0 2024-03-23T08:24:10.176 INFO:tasks.workunit.client.0.smithi070.stdout:5/839: read dc/d1b/d1d/d25/d38/d53/f9a [285226,113609] 0 2024-03-23T08:24:10.180 INFO:tasks.workunit.client.0.smithi070.stdout:6/960: creat d0/d23/d97/d98/d11e/f13b x:0 0 0 2024-03-23T08:24:10.181 INFO:tasks.workunit.client.0.smithi070.stdout:4/822: symlink d3/d6/dca/d101/l121 0 2024-03-23T08:24:10.195 INFO:tasks.workunit.client.0.smithi070.stdout:6/961: symlink d0/d23/d74/d84/db3/d38/l13c 0 2024-03-23T08:24:10.204 INFO:tasks.workunit.client.0.smithi070.stdout:6/962: dread d0/d23/d28/f36 [0,4194304] 0 2024-03-23T08:24:10.210 INFO:tasks.workunit.client.0.smithi070.stdout:7/989: getdents d25/d27/d6a/d44/dfe 0 2024-03-23T08:24:10.213 INFO:tasks.workunit.client.0.smithi070.stdout:4/823: rename d3/f8f to d3/d6/d11/f122 0 2024-03-23T08:24:10.215 INFO:tasks.workunit.client.0.smithi070.stdout:4/824: truncate d3/d6/d35/dcf/db5/dda/dc7/f10b 292470 0 2024-03-23T08:24:10.216 INFO:tasks.workunit.client.0.smithi070.stdout:6/963: symlink d0/d23/d74/d84/db3/d2f/dab/db8/d133/da5/l13d 0 2024-03-23T08:24:10.217 INFO:tasks.workunit.client.0.smithi070.stdout:1/857: truncate d8/d1a/f23 1241496 0 2024-03-23T08:24:10.218 INFO:tasks.workunit.client.0.smithi070.stdout:6/964: chown d0/d23/d74/d84/db3/d38/f119 701 1 2024-03-23T08:24:10.223 INFO:tasks.workunit.client.0.smithi070.stdout:7/990: write d25/f87 [3165612,7536] 0 2024-03-23T08:24:10.223 INFO:tasks.workunit.client.0.smithi070.stdout:4/825: truncate d3/d6/d6a/f4c 999427 0 2024-03-23T08:24:10.226 INFO:tasks.workunit.client.0.smithi070.stdout:1/858: symlink d8/db/d2c/d3f/dd3/l131 0 2024-03-23T08:24:10.226 INFO:tasks.workunit.client.0.smithi070.stdout:6/965: dwrite d0/d23/d74/d84/db3/d38/d9a/fad [0,4194304] 0 2024-03-23T08:24:10.232 INFO:tasks.workunit.client.0.smithi070.stdout:3/938: creat db/d20/d7a/dcc/d141/f15d x:0 0 0 2024-03-23T08:24:10.236 INFO:tasks.workunit.client.0.smithi070.stdout:8/971: sync 2024-03-23T08:24:10.237 INFO:tasks.workunit.client.0.smithi070.stdout:4/826: fdatasync d3/d6/d35/dcf/db5/dda/dc7/fff 0 2024-03-23T08:24:10.249 INFO:tasks.workunit.client.0.smithi070.stdout:8/972: read d5/d8/f4a [556784,117433] 0 2024-03-23T08:24:10.252 INFO:tasks.workunit.client.0.smithi070.stdout:1/859: mknod d8/c132 0 2024-03-23T08:24:10.254 INFO:tasks.workunit.client.0.smithi070.stdout:4/827: truncate d3/d6/d9/fb4 2471213 0 2024-03-23T08:24:10.258 INFO:tasks.workunit.client.0.smithi070.stdout:3/939: creat db/d20/d49/f15e x:0 0 0 2024-03-23T08:24:10.262 INFO:tasks.workunit.client.0.smithi070.stdout:4/828: mknod d3/d6/d35/dcf/c123 0 2024-03-23T08:24:10.264 INFO:tasks.workunit.client.0.smithi070.stdout:3/940: mkdir db/df/d1a/de9/d15f 0 2024-03-23T08:24:10.270 INFO:tasks.workunit.client.0.smithi070.stdout:4/829: mkdir d3/d6/dca/d124 0 2024-03-23T08:24:10.271 INFO:tasks.workunit.client.0.smithi070.stdout:3/941: dwrite db/d4e/d92/f14c [0,4194304] 0 2024-03-23T08:24:10.273 INFO:tasks.workunit.client.0.smithi070.stdout:5/840: dwrite dc/d1b/d1d/d25/d38/d39/f66 [0,4194304] 0 2024-03-23T08:24:10.275 INFO:tasks.workunit.client.0.smithi070.stdout:1/860: creat d8/d1a/f133 x:0 0 0 2024-03-23T08:24:10.278 INFO:tasks.workunit.client.0.smithi070.stdout:5/841: dread dc/f59 [0,4194304] 0 2024-03-23T08:24:10.278 INFO:tasks.workunit.client.0.smithi070.stdout:1/861: write d8/db/d40/d43/d64/dc2/dfb/f114 [877565,127351] 0 2024-03-23T08:24:10.285 INFO:tasks.workunit.client.0.smithi070.stdout:3/942: unlink db/df/d1a/de9/ff7 0 2024-03-23T08:24:10.294 INFO:tasks.workunit.client.0.smithi070.stdout:5/842: rename dc/f74 to dc/d1b/d1d/d25/d38/d39/f10c 0 2024-03-23T08:24:10.295 INFO:tasks.workunit.client.0.smithi070.stdout:5/843: readlink dc/d1b/d1d/d25/d38/d53/l6e 0 2024-03-23T08:24:10.299 INFO:tasks.workunit.client.0.smithi070.stdout:1/862: dread d8/db/d40/d43/feb [0,4194304] 0 2024-03-23T08:24:10.305 INFO:tasks.workunit.client.0.smithi070.stdout:3/943: read db/d4e/df8/d3d/f56 [611915,96086] 0 2024-03-23T08:24:10.309 INFO:tasks.workunit.client.0.smithi070.stdout:1/863: symlink d8/db/d8e/l134 0 2024-03-23T08:24:10.311 INFO:tasks.workunit.client.0.smithi070.stdout:3/944: dread db/df/d5d/fcf [0,4194304] 0 2024-03-23T08:24:10.312 INFO:tasks.workunit.client.0.smithi070.stdout:1/864: dread d8/d11/fb2 [0,4194304] 0 2024-03-23T08:24:10.314 INFO:tasks.workunit.client.0.smithi070.stdout:5/844: truncate dc/d19/f2c 133757 0 2024-03-23T08:24:10.314 INFO:tasks.workunit.client.0.smithi070.stdout:1/865: write d8/d11/f4d [3341539,84937] 0 2024-03-23T08:24:10.316 INFO:tasks.workunit.client.0.smithi070.stdout:7/991: dwrite d25/d27/f92 [0,4194304] 0 2024-03-23T08:24:10.318 INFO:tasks.workunit.client.0.smithi070.stdout:7/992: write d25/d27/d6a/fec [3785686,37566] 0 2024-03-23T08:24:10.323 INFO:tasks.workunit.client.0.smithi070.stdout:6/966: dwrite d0/d23/d74/d84/db3/d2f/f4c [0,4194304] 0 2024-03-23T08:24:10.328 INFO:tasks.workunit.client.0.smithi070.stdout:1/866: mkdir d8/db/d40/d43/d64/d85/d97/d9a/d135 0 2024-03-23T08:24:10.336 INFO:tasks.workunit.client.0.smithi070.stdout:8/973: write d5/f50 [495942,58259] 0 2024-03-23T08:24:10.336 INFO:tasks.workunit.client.0.smithi070.stdout:8/974: dread - d5/d8/d16/dd4/f132 zero size 2024-03-23T08:24:10.338 INFO:tasks.workunit.client.0.smithi070.stdout:6/967: creat d0/d3/f13e x:0 0 0 2024-03-23T08:24:10.340 INFO:tasks.workunit.client.0.smithi070.stdout:6/968: fdatasync d0/d23/d74/d84/db3/fea 0 2024-03-23T08:24:10.354 INFO:tasks.workunit.client.0.smithi070.stdout:7/993: creat d25/d27/d6a/d44/f15b x:0 0 0 2024-03-23T08:24:10.355 INFO:tasks.workunit.client.0.smithi070.stdout:8/975: mkdir d5/d8/d16/d32/d84/d98/d152 0 2024-03-23T08:24:10.358 INFO:tasks.workunit.client.0.smithi070.stdout:6/969: creat d0/d23/d88/d130/f13f x:0 0 0 2024-03-23T08:24:10.359 INFO:tasks.workunit.client.0.smithi070.stdout:7/994: unlink d25/d27/d112/f121 0 2024-03-23T08:24:10.362 INFO:tasks.workunit.client.0.smithi070.stdout:7/995: write d25/d27/d6a/d44/d7c/fd0 [924449,123184] 0 2024-03-23T08:24:10.368 INFO:tasks.workunit.client.0.smithi070.stdout:4/830: dwrite d3/d6/d35/d65/d6f/dee/fc2 [0,4194304] 0 2024-03-23T08:24:10.370 INFO:tasks.workunit.client.0.smithi070.stdout:4/831: stat d3/d6/d35/d65/ca0 0 2024-03-23T08:24:10.370 INFO:tasks.workunit.client.0.smithi070.stdout:6/970: dread d0/d23/d74/d84/db3/d2f/dab/fac [0,4194304] 0 2024-03-23T08:24:10.375 INFO:tasks.workunit.client.0.smithi070.stdout:7/996: rmdir d25/d27/d6a/d44/d6f/dcc/d11a 39 2024-03-23T08:24:10.377 INFO:tasks.workunit.client.0.smithi070.stdout:8/976: getdents d5/d8/d9a/dbe 0 2024-03-23T08:24:10.380 INFO:tasks.workunit.client.0.smithi070.stdout:3/945: write db/f17 [2639969,100278] 0 2024-03-23T08:24:10.385 INFO:tasks.workunit.client.0.smithi070.stdout:7/997: creat d25/d27/d6a/d44/d7c/dfb/dfd/f15c x:0 0 0 2024-03-23T08:24:10.385 INFO:tasks.workunit.client.0.smithi070.stdout:5/845: write dc/d1b/d1d/d25/d36/dfc/fbd [1983782,99059] 0 2024-03-23T08:24:10.386 INFO:tasks.workunit.client.0.smithi070.stdout:7/998: write d25/d27/d6a/fec [3399847,56820] 0 2024-03-23T08:24:10.392 INFO:tasks.workunit.client.0.smithi070.stdout:6/971: getdents d0/d23/d28/df4 0 2024-03-23T08:24:10.394 INFO:tasks.workunit.client.0.smithi070.stdout:6/972: dread - d0/d23/d74/d84/db3/d2f/dab/db8/d133/f6e zero size 2024-03-23T08:24:10.398 INFO:tasks.workunit.client.0.smithi070.stdout:1/867: write d8/db/d40/d43/feb [353394,85512] 0 2024-03-23T08:24:10.402 INFO:tasks.workunit.client.0.smithi070.stdout:7/999: rename d25/ce5 to d25/d27/d6a/d44/dfe/d107/c15d 0 2024-03-23T08:24:10.405 INFO:tasks.workunit.client.0.smithi070.stdout:6/973: rmdir d0/d23/d74/d84/d12b/d86 39 2024-03-23T08:24:10.414 INFO:tasks.workunit.client.0.smithi070.stdout:6/974: dread - d0/d23/d28/d35/d85/fe6 zero size 2024-03-23T08:24:10.415 INFO:tasks.workunit.client.0.smithi070.stdout:6/975: chown d0/d3/f4 1048508 1 2024-03-23T08:24:10.421 INFO:tasks.workunit.client.0.smithi070.stdout:6/976: creat d0/d23/d74/d84/d12b/f140 x:0 0 0 2024-03-23T08:24:10.422 INFO:tasks.workunit.client.0.smithi070.stdout:6/977: read d0/d23/d74/d84/db3/d38/d52/f11b [3733343,11431] 0 2024-03-23T08:24:10.428 INFO:tasks.workunit.client.0.smithi070.stdout:4/832: truncate d3/d6/d35/d65/d6f/fb1 1464834 0 2024-03-23T08:24:10.430 INFO:tasks.workunit.client.0.smithi070.stdout:6/978: dwrite d0/d23/d97/d98/f113 [0,4194304] 0 2024-03-23T08:24:10.432 INFO:tasks.workunit.client.0.smithi070.stdout:3/946: write db/d4e/df8/d22/db2/fc7 [4277456,98941] 0 2024-03-23T08:24:10.434 INFO:tasks.workunit.client.0.smithi070.stdout:3/947: chown db/df/d1a/de9/d15f 618 1 2024-03-23T08:24:10.435 INFO:tasks.workunit.client.0.smithi070.stdout:8/977: write d5/d8/d16/d32/d84/fdc [3279460,11384] 0 2024-03-23T08:24:10.436 INFO:tasks.workunit.client.0.smithi070.stdout:6/979: rmdir d0/d23/d28/d35 39 2024-03-23T08:24:10.436 INFO:tasks.workunit.client.0.smithi070.stdout:3/948: write db/d20/d49/f125 [1316604,123526] 0 2024-03-23T08:24:10.443 INFO:tasks.workunit.client.0.smithi070.stdout:3/949: fsync db/df/f55 0 2024-03-23T08:24:10.445 INFO:tasks.workunit.client.0.smithi070.stdout:8/978: unlink d5/d8/d16/d3b/f5d 0 2024-03-23T08:24:10.446 INFO:tasks.workunit.client.0.smithi070.stdout:1/868: write d8/d42/d4e/fc8 [1099029,87346] 0 2024-03-23T08:24:10.448 INFO:tasks.workunit.client.0.smithi070.stdout:5/846: truncate dc/d1b/d1d/d25/d36/f27 1832951 0 2024-03-23T08:24:10.453 INFO:tasks.workunit.client.0.smithi070.stdout:8/979: mkdir d5/d8/d16/d30/d38/d6b/d72/d88/dbf/d153 0 2024-03-23T08:24:10.456 INFO:tasks.workunit.client.0.smithi070.stdout:6/980: getdents d0/d23/d28/d11d 0 2024-03-23T08:24:10.459 INFO:tasks.workunit.client.0.smithi070.stdout:5/847: creat dc/d1b/d1d/d25/d38/d53/f10d x:0 0 0 2024-03-23T08:24:10.464 INFO:tasks.workunit.client.0.smithi070.stdout:3/950: truncate db/d14/d65/f78 3506384 0 2024-03-23T08:24:10.465 INFO:tasks.workunit.client.0.smithi070.stdout:8/980: truncate d5/d8/d16/d32/f74 161263 0 2024-03-23T08:24:10.465 INFO:tasks.workunit.client.0.smithi070.stdout:4/833: write d3/d6/f38 [1781920,97160] 0 2024-03-23T08:24:10.470 INFO:tasks.workunit.client.0.smithi070.stdout:6/981: rename d0/d23/d74/d84/d12b/c59 to d0/d23/d74/d84/db3/d38/d9a/c141 0 2024-03-23T08:24:10.474 INFO:tasks.workunit.client.0.smithi070.stdout:8/981: fdatasync d5/d8/d16/d3b/f5f 0 2024-03-23T08:24:10.476 INFO:tasks.workunit.client.0.smithi070.stdout:8/982: truncate d5/d8/d16/d30/d38/d6b/d72/d88/f14d 100514 0 2024-03-23T08:24:10.478 INFO:tasks.workunit.client.0.smithi070.stdout:1/869: link d8/db/f16 d8/db/d40/d43/d64/d85/f136 0 2024-03-23T08:24:10.479 INFO:tasks.workunit.client.0.smithi070.stdout:1/870: chown d8/d1b/c34 649735682 1 2024-03-23T08:24:10.481 INFO:tasks.workunit.client.0.smithi070.stdout:3/951: rename db/d14/d43/d7d/le7 to db/d14/d9c/l160 0 2024-03-23T08:24:10.483 INFO:tasks.workunit.client.0.smithi070.stdout:8/983: dwrite d5/d8/d16/d32/f54 [0,4194304] 0 2024-03-23T08:24:10.484 INFO:tasks.workunit.client.0.smithi070.stdout:8/984: readlink d5/d8/lc1 0 2024-03-23T08:24:10.486 INFO:tasks.workunit.client.0.smithi070.stdout:6/982: creat d0/d23/d74/d84/d12b/d5b/dde/f142 x:0 0 0 2024-03-23T08:24:10.499 INFO:tasks.workunit.client.0.smithi070.stdout:3/952: dread db/d14/d43/f4a [0,4194304] 0 2024-03-23T08:24:10.500 INFO:tasks.workunit.client.0.smithi070.stdout:8/985: mknod d5/d8/d16/d32/d84/d98/dde/da3/dab/c154 0 2024-03-23T08:24:10.502 INFO:tasks.workunit.client.0.smithi070.stdout:8/986: write d5/d8/d16/d30/d38/d6b/d72/d88/dbf/f105 [390162,41644] 0 2024-03-23T08:24:10.503 INFO:tasks.workunit.client.0.smithi070.stdout:8/987: write d5/d8/f11 [221976,56852] 0 2024-03-23T08:24:10.521 INFO:tasks.workunit.client.0.smithi070.stdout:5/848: getdents dc/d1b/d1d/d25/d68/d8a/d99/dd7 0 2024-03-23T08:24:10.521 INFO:tasks.workunit.client.0.smithi070.stdout:3/953: dread db/d14/d65/f85 [0,4194304] 0 2024-03-23T08:24:10.529 INFO:tasks.workunit.client.0.smithi070.stdout:3/954: dwrite db/d20/d49/de1/f11d [0,4194304] 0 2024-03-23T08:24:10.529 INFO:tasks.workunit.client.0.smithi070.stdout:1/871: dwrite d8/d26/f38 [0,4194304] 0 2024-03-23T08:24:10.532 INFO:tasks.workunit.client.0.smithi070.stdout:5/849: read dc/d1b/d1d/d25/f43 [6952534,40723] 0 2024-03-23T08:24:10.538 INFO:tasks.workunit.client.0.smithi070.stdout:3/955: symlink db/d4e/df8/d3d/l161 0 2024-03-23T08:24:10.544 INFO:tasks.workunit.client.0.smithi070.stdout:8/988: write d5/d8/f4a [1660128,117655] 0 2024-03-23T08:24:10.545 INFO:tasks.workunit.client.0.smithi070.stdout:1/872: mkdir d8/d42/d51/d11f/d137 0 2024-03-23T08:24:10.547 INFO:tasks.workunit.client.0.smithi070.stdout:6/983: write d0/d23/d74/dcd/f12f [2944878,23241] 0 2024-03-23T08:24:10.548 INFO:tasks.workunit.client.0.smithi070.stdout:3/956: mknod db/d14/d65/daa/c162 0 2024-03-23T08:24:10.548 INFO:tasks.workunit.client.0.smithi070.stdout:6/984: readlink d0/d23/d74/d84/lc9 0 2024-03-23T08:24:10.550 INFO:tasks.workunit.client.0.smithi070.stdout:8/989: mknod d5/d8/d16/d32/d84/d98/dde/c155 0 2024-03-23T08:24:10.551 INFO:tasks.workunit.client.0.smithi070.stdout:1/873: truncate d8/db/d2c/f3c 825808 0 2024-03-23T08:24:10.555 INFO:tasks.workunit.client.0.smithi070.stdout:8/990: symlink d5/d8/d16/dd4/d115/l156 0 2024-03-23T08:24:10.557 INFO:tasks.workunit.client.0.smithi070.stdout:1/874: creat d8/db/d8e/f138 x:0 0 0 2024-03-23T08:24:10.558 INFO:tasks.workunit.client.0.smithi070.stdout:6/985: symlink d0/d23/d28/d35/d85/l143 0 2024-03-23T08:24:10.559 INFO:tasks.workunit.client.0.smithi070.stdout:6/986: write d0/fa7 [2110129,17750] 0 2024-03-23T08:24:10.561 INFO:tasks.workunit.client.0.smithi070.stdout:6/987: write d0/d23/d74/d84/db3/fea [1340991,23176] 0 2024-03-23T08:24:10.563 INFO:tasks.workunit.client.0.smithi070.stdout:8/991: chown d5/d8/d16/d30/l90 0 1 2024-03-23T08:24:10.563 INFO:tasks.workunit.client.0.smithi070.stdout:8/992: dread - d5/d8/d16/dd4/dc9/f144 zero size 2024-03-23T08:24:10.564 INFO:tasks.workunit.client.0.smithi070.stdout:1/875: rename d8/d42/d4e/f59 to d8/d42/d51/d11f/f139 0 2024-03-23T08:24:10.567 INFO:tasks.workunit.client.0.smithi070.stdout:5/850: write dc/f3d [1119125,54209] 0 2024-03-23T08:24:10.569 INFO:tasks.workunit.client.0.smithi070.stdout:5/851: readlink dc/d1b/d1d/d25/d36/d32/d5c/l64 0 2024-03-23T08:24:10.569 INFO:tasks.workunit.client.0.smithi070.stdout:8/993: mkdir d5/d8/d9a/dbe/d157 0 2024-03-23T08:24:10.574 INFO:tasks.workunit.client.0.smithi070.stdout:6/988: mknod d0/d23/c144 0 2024-03-23T08:24:10.576 INFO:tasks.workunit.client.0.smithi070.stdout:8/994: creat d5/d8/d9a/dbe/f158 x:0 0 0 2024-03-23T08:24:10.580 INFO:tasks.workunit.client.0.smithi070.stdout:3/957: write db/df/d5d/fcf [860189,43445] 0 2024-03-23T08:24:10.581 INFO:tasks.workunit.client.0.smithi070.stdout:5/852: chown dc/d1b/d1d/d25/d38/d39/fba 957531686 1 2024-03-23T08:24:10.582 INFO:tasks.workunit.client.0.smithi070.stdout:1/876: dread d8/db/d40/d43/d64/d85/f136 [0,4194304] 0 2024-03-23T08:24:10.588 INFO:tasks.workunit.client.0.smithi070.stdout:5/853: creat dc/d1b/d1d/d25/d36/d32/db7/f10e x:0 0 0 2024-03-23T08:24:10.591 INFO:tasks.workunit.client.0.smithi070.stdout:5/854: dread dc/d1b/d1d/d25/d36/d32/db7/fef [0,4194304] 0 2024-03-23T08:24:10.599 INFO:tasks.workunit.client.0.smithi070.stdout:8/995: rmdir d5/d8/d9a/dbe 39 2024-03-23T08:24:10.600 INFO:tasks.workunit.client.0.smithi070.stdout:6/989: getdents d0/d3/d2c 0 2024-03-23T08:24:10.605 INFO:tasks.workunit.client.0.smithi070.stdout:5/855: creat dc/d1b/d1d/d25/da9/deb/f10f x:0 0 0 2024-03-23T08:24:10.606 INFO:tasks.workunit.client.0.smithi070.stdout:3/958: rename db/d7f/ce5 to db/d14/d65/c163 0 2024-03-23T08:24:10.609 INFO:tasks.workunit.client.0.smithi070.stdout:1/877: write d8/db/d100/d74/d5d/d70/f93 [781480,118550] 0 2024-03-23T08:24:10.610 INFO:tasks.workunit.client.0.smithi070.stdout:1/878: chown d8/db/d40/d43/d64/lc9 261819722 1 2024-03-23T08:24:10.611 INFO:tasks.workunit.client.0.smithi070.stdout:1/879: readlink d8/db/d2c/d3f/d48/l52 0 2024-03-23T08:24:10.612 INFO:tasks.workunit.client.0.smithi070.stdout:8/996: mknod d5/d8/d16/dd4/d6f/c159 0 2024-03-23T08:24:10.613 INFO:tasks.workunit.client.0.smithi070.stdout:6/990: creat d0/d23/d28/d35/d6a/f145 x:0 0 0 2024-03-23T08:24:10.613 INFO:tasks.workunit.client.0.smithi070.stdout:1/880: dread d8/db/d40/d43/feb [0,4194304] 0 2024-03-23T08:24:10.614 INFO:tasks.workunit.client.0.smithi070.stdout:1/881: chown d8/db/d40/d43/d64/dc2/dfb 105 1 2024-03-23T08:24:10.615 INFO:tasks.workunit.client.0.smithi070.stdout:5/856: creat dc/d1b/d1d/d25/da9/deb/f110 x:0 0 0 2024-03-23T08:24:10.621 INFO:tasks.workunit.client.0.smithi070.stdout:6/991: symlink d0/d23/d97/d98/l146 0 2024-03-23T08:24:10.621 INFO:tasks.workunit.client.0.smithi070.stdout:1/882: dwrite d8/db/d40/d43/d64/d85/d97/dfd/f113 [0,4194304] 0 2024-03-23T08:24:10.622 INFO:tasks.workunit.client.0.smithi070.stdout:6/992: readlink d0/d23/d74/d84/d12b/l56 0 2024-03-23T08:24:10.624 INFO:tasks.workunit.client.0.smithi070.stdout:5/857: mkdir dc/d1b/d1d/d25/d38/d39/d111 0 2024-03-23T08:24:10.706 INFO:tasks.workunit.client.0.smithi070.stdout:8/997: mknod d5/d8/d9a/c15a 0 2024-03-23T08:24:10.707 INFO:tasks.workunit.client.0.smithi070.stdout:8/998: chown d5/d8/d16/d32/fe2 12 1 2024-03-23T08:24:10.729 INFO:tasks.workunit.client.0.smithi070.stdout:1/883: mkdir d8/d1b/d13a 0 2024-03-23T08:24:10.732 INFO:tasks.workunit.client.0.smithi070.stdout:6/993: truncate d0/d23/d88/db6/fe9 3587816 0 2024-03-23T08:24:10.733 INFO:tasks.workunit.client.0.smithi070.stdout:5/858: mknod dc/d1b/d1d/d25/d36/d32/db7/de6/c112 0 2024-03-23T08:24:10.736 INFO:tasks.workunit.client.0.smithi070.stdout:8/999: creat d5/d8/d16/d3b/db9/d116/f15b x:0 0 0 2024-03-23T08:24:10.737 INFO:tasks.workunit.client.0.smithi070.stdout:1/884: fsync d8/d1b/f89 0 2024-03-23T08:24:10.740 INFO:tasks.workunit.client.0.smithi070.stdout:3/959: dwrite db/f21 [4194304,4194304] 0 2024-03-23T08:24:10.743 INFO:tasks.workunit.client.0.smithi070.stdout:5/859: truncate dc/d1b/d1d/d25/d68/d8a/f8e 4898277 0 2024-03-23T08:24:10.759 INFO:tasks.workunit.client.0.smithi070.stdout:6/994: link d0/d23/d74/d84/db3/d38/l13c d0/d23/d74/d84/d12b/d5b/l147 0 2024-03-23T08:24:10.761 INFO:tasks.workunit.client.0.smithi070.stdout:6/995: write d0/d23/d74/d84/db3/fea [53482,4045] 0 2024-03-23T08:24:10.761 INFO:tasks.workunit.client.0.smithi070.stdout:3/960: creat db/d4e/dd6/d130/f164 x:0 0 0 2024-03-23T08:24:10.764 INFO:tasks.workunit.client.0.smithi070.stdout:5/860: rename dc/l12 to dc/d1b/d1d/d25/da9/deb/d107/dc8/l113 0 2024-03-23T08:24:10.769 INFO:tasks.workunit.client.0.smithi070.stdout:6/996: fsync d0/d23/d28/f7d 0 2024-03-23T08:24:10.772 INFO:tasks.workunit.client.0.smithi070.stdout:5/861: fdatasync dc/d1b/d1d/d25/d38/d53/fa1 0 2024-03-23T08:24:10.773 INFO:tasks.workunit.client.0.smithi070.stdout:5/862: chown dc/d19 227599956 1 2024-03-23T08:24:10.773 INFO:tasks.workunit.client.0.smithi070.stdout:1/885: truncate d8/d42/d4e/f66 907957 0 2024-03-23T08:24:10.782 INFO:tasks.workunit.client.0.smithi070.stdout:3/961: write db/d14/f3f [671770,41360] 0 2024-03-23T08:24:10.784 INFO:tasks.workunit.client.0.smithi070.stdout:5/863: creat dc/d1b/d1d/d25/d36/d32/d5c/d10a/f114 x:0 0 0 2024-03-23T08:24:10.796 INFO:tasks.workunit.client.0.smithi070.stdout:6/997: getdents d0/d23/d74/d84/db3/d2f/dab/db8/d133 0 2024-03-23T08:24:10.797 INFO:tasks.workunit.client.0.smithi070.stdout:6/998: chown d0/d23/d74/d84/d12b/f140 87659878 1 2024-03-23T08:24:10.800 INFO:tasks.workunit.client.0.smithi070.stdout:3/962: truncate db/df/d4d/fbf 757457 0 2024-03-23T08:24:10.804 INFO:tasks.workunit.client.0.smithi070.stdout:1/886: dwrite d8/d26/f7f [0,4194304] 0 2024-03-23T08:24:10.806 INFO:tasks.workunit.client.0.smithi070.stdout:1/887: write d8/db/d100/d74/d5d/fe2 [326604,61986] 0 2024-03-23T08:24:10.810 INFO:tasks.workunit.client.0.smithi070.stdout:3/963: creat db/d20/d49/de1/f165 x:0 0 0 2024-03-23T08:24:10.810 INFO:tasks.workunit.client.0.smithi070.stdout:6/999: mknod d0/d23/d74/d84/d12b/d86/c148 0 2024-03-23T08:24:10.813 INFO:tasks.workunit.client.0.smithi070.stdout:3/964: read db/d14/d43/f54 [832518,124349] 0 2024-03-23T08:24:10.817 INFO:tasks.workunit.client.0.smithi070.stdout:1/888: rename d8/d11/lac to d8/db/d40/l13b 0 2024-03-23T08:24:10.817 INFO:tasks.workunit.client.0.smithi070.stdout:5/864: dwrite dc/d19/f2c [0,4194304] 0 2024-03-23T08:24:10.819 INFO:tasks.workunit.client.0.smithi070.stdout:5/865: readlink dc/d1b/d1d/d25/d68/lbb 0 2024-03-23T08:24:10.822 INFO:tasks.workunit.client.0.smithi070.stdout:3/965: creat db/d20/d7a/dcc/f166 x:0 0 0 2024-03-23T08:24:10.826 INFO:tasks.workunit.client.0.smithi070.stdout:5/866: creat dc/d1b/d1d/d25/d68/d8a/d99/dd7/f115 x:0 0 0 2024-03-23T08:24:10.827 INFO:tasks.workunit.client.0.smithi070.stdout:3/966: mkdir db/df/d6a/d167 0 2024-03-23T08:24:10.834 INFO:tasks.workunit.client.0.smithi070.stdout:5/867: dread dc/d1b/d1d/d25/d38/d39/f10c [0,4194304] 0 2024-03-23T08:24:10.836 INFO:tasks.workunit.client.0.smithi070.stdout:3/967: creat db/d20/d7a/d80/f168 x:0 0 0 2024-03-23T08:24:10.846 INFO:tasks.workunit.client.0.smithi070.stdout:5/868: creat dc/d1b/d1d/d25/d68/d8a/d99/dd7/f116 x:0 0 0 2024-03-23T08:24:10.847 INFO:tasks.workunit.client.0.smithi070.stdout:1/889: write d8/db/d40/d43/d64/d85/d97/d9a/fb7 [579544,26888] 0 2024-03-23T08:24:10.848 INFO:tasks.workunit.client.0.smithi070.stdout:5/869: dread - dc/d1b/d1d/d25/da9/deb/f110 zero size 2024-03-23T08:24:10.851 INFO:tasks.workunit.client.0.smithi070.stdout:1/890: unlink d8/db/d40/d43/d64/d85/dc7/fcc 0 2024-03-23T08:24:10.853 INFO:tasks.workunit.client.0.smithi070.stdout:1/891: truncate d8/db/d40/d43/d64/d85/d97/de7/f127 282414 0 2024-03-23T08:24:10.855 INFO:tasks.workunit.client.0.smithi070.stdout:5/870: creat dc/d1b/d1d/d25/d68/d8a/d99/f117 x:0 0 0 2024-03-23T08:24:10.860 INFO:tasks.workunit.client.0.smithi070.stdout:1/892: dwrite d8/db/d2c/d3f/d48/f11d [0,4194304] 0 2024-03-23T08:24:10.863 INFO:tasks.workunit.client.0.smithi070.stdout:5/871: mknod dc/d1b/d1d/d25/da9/deb/d107/c118 0 2024-03-23T08:24:10.869 INFO:tasks.workunit.client.0.smithi070.stdout:5/872: rename dc/d1b/d1d/d25/d36/dfc/le4 to dc/d1b/d1d/d25/d36/d32/l119 0 2024-03-23T08:24:10.871 INFO:tasks.workunit.client.0.smithi070.stdout:5/873: fsync dc/d1b/d1d/d25/d68/d8a/d99/dd7/f116 0 2024-03-23T08:24:10.872 INFO:tasks.workunit.client.0.smithi070.stdout:3/968: dwrite db/d4e/df8/d22/fc5 [4194304,4194304] 0 2024-03-23T08:24:10.874 INFO:tasks.workunit.client.0.smithi070.stdout:5/874: chown dc/d1b/c45 1561205563 1 2024-03-23T08:24:10.897 INFO:tasks.workunit.client.0.smithi070.stdout:1/893: dwrite d8/d42/d4e/ff7 [0,4194304] 0 2024-03-23T08:24:10.900 INFO:tasks.workunit.client.0.smithi070.stdout:3/969: dread db/d4e/df8/d22/db2/fc7 [0,4194304] 0 2024-03-23T08:24:10.905 INFO:tasks.workunit.client.0.smithi070.stdout:1/894: read d8/d11/f29 [503341,36606] 0 2024-03-23T08:24:10.906 INFO:tasks.workunit.client.0.smithi070.stdout:5/875: mkdir dc/d1b/d1d/d25/d36/d32/d11a 0 2024-03-23T08:24:10.909 INFO:tasks.workunit.client.0.smithi070.stdout:1/895: dread d8/db/d2c/d3f/d48/f11d [0,4194304] 0 2024-03-23T08:24:10.909 INFO:tasks.workunit.client.0.smithi070.stdout:3/970: symlink db/df/d5d/l169 0 2024-03-23T08:24:10.910 INFO:tasks.workunit.client.0.smithi070.stdout:3/971: stat db/d14/d65/daa/cb6 0 2024-03-23T08:24:10.913 INFO:tasks.workunit.client.0.smithi070.stdout:5/876: mkdir dc/d1b/d1d/d25/d36/dfc/d11b 0 2024-03-23T08:24:10.916 INFO:tasks.workunit.client.0.smithi070.stdout:3/972: readlink db/d20/d49/l12a 0 2024-03-23T08:24:10.919 INFO:tasks.workunit.client.0.smithi070.stdout:5/877: symlink dc/d1b/d1d/l11c 0 2024-03-23T08:24:10.920 INFO:tasks.workunit.client.0.smithi070.stdout:5/878: read - dc/d1b/d1d/d25/da9/deb/f10f zero size 2024-03-23T08:24:10.980 INFO:tasks.workunit.client.0.smithi070.stdout:4/834: sync 2024-03-23T08:24:10.980 INFO:tasks.workunit.client.0.smithi070.stdout:3/973: creat db/d20/d63/d83/dfe/f16a x:0 0 0 2024-03-23T08:24:11.113 INFO:tasks.workunit.client.0.smithi070.stdout:5/879: mknod dc/d1b/d1d/d25/d36/d32/db7/c11d 0 2024-03-23T08:24:11.117 INFO:tasks.workunit.client.0.smithi070.stdout:5/880: dread dc/d1b/d1d/d25/d36/dfc/fbd [0,4194304] 0 2024-03-23T08:24:11.117 INFO:tasks.workunit.client.0.smithi070.stdout:4/835: mkdir d3/d6/d35/d3f/da9/d125 0 2024-03-23T08:24:11.118 INFO:tasks.workunit.client.0.smithi070.stdout:4/836: stat d3/d6/d35/d65/d27/d4b/f10e 0 2024-03-23T08:24:11.119 INFO:tasks.workunit.client.0.smithi070.stdout:4/837: chown d3/d6/d35/d65/d27/d4b/le8 0 1 2024-03-23T08:24:11.126 INFO:tasks.workunit.client.0.smithi070.stdout:4/838: dread d3/d6/d35/d65/d6f/dee/f79 [0,4194304] 0 2024-03-23T08:24:11.135 INFO:tasks.workunit.client.0.smithi070.stdout:4/839: symlink d3/d6/d9/d110/l126 0 2024-03-23T08:24:11.145 INFO:tasks.workunit.client.0.smithi070.stdout:4/840: rename d3/d6/d35/dcf/db5/dda/db8/f37 to d3/d6/dca/d124/f127 0 2024-03-23T08:24:11.146 INFO:tasks.workunit.client.0.smithi070.stdout:4/841: write d3/d6/d35/d65/f3e [2418943,47617] 0 2024-03-23T08:24:11.154 INFO:tasks.workunit.client.0.smithi070.stdout:4/842: mknod d3/d6/dca/c128 0 2024-03-23T08:24:11.164 INFO:tasks.workunit.client.0.smithi070.stdout:4/843: creat d3/d6/d35/d65/d6f/dee/f129 x:0 0 0 2024-03-23T08:24:11.171 INFO:tasks.workunit.client.0.smithi070.stdout:4/844: symlink d3/d6/d35/d65/d27/d4b/d73/l12a 0 2024-03-23T08:24:11.183 INFO:tasks.workunit.client.0.smithi070.stdout:4/845: dread d3/d6/d35/d65/f3e [0,4194304] 0 2024-03-23T08:24:11.184 INFO:tasks.workunit.client.0.smithi070.stdout:4/846: stat d3/d6/d35/dcf 0 2024-03-23T08:24:11.209 INFO:tasks.workunit.client.0.smithi070.stdout:3/974: write db/d14/d43/d62/f112 [1752163,40515] 0 2024-03-23T08:24:11.214 INFO:tasks.workunit.client.0.smithi070.stdout:3/975: dread db/d20/d49/de1/f11d [0,4194304] 0 2024-03-23T08:24:11.217 INFO:tasks.workunit.client.0.smithi070.stdout:3/976: symlink db/d20/d49/de1/d139/l16b 0 2024-03-23T08:24:11.219 INFO:tasks.workunit.client.0.smithi070.stdout:3/977: symlink db/d14/d43/l16c 0 2024-03-23T08:24:11.223 INFO:tasks.workunit.client.0.smithi070.stdout:3/978: rename db/d4e/dd6/dd7 to db/d4e/d105/d16d 0 2024-03-23T08:24:11.228 INFO:tasks.workunit.client.0.smithi070.stdout:5/881: write dc/d1b/d1d/d25/d36/f2a [603039,34772] 0 2024-03-23T08:24:11.229 INFO:tasks.workunit.client.0.smithi070.stdout:3/979: dwrite db/d4e/df8/d22/fc5 [4194304,4194304] 0 2024-03-23T08:24:11.230 INFO:tasks.workunit.client.0.smithi070.stdout:5/882: chown dc/d1b/d1d/d25/dcd/c101 6 1 2024-03-23T08:24:11.240 INFO:tasks.workunit.client.0.smithi070.stdout:5/883: rename dc/d1b/d1d/d25/d36/c77 to dc/d1b/d1d/d25/d36/d32/d5c/dcb/c11e 0 2024-03-23T08:24:11.244 INFO:tasks.workunit.client.0.smithi070.stdout:5/884: unlink dc/d1b/d1d/d25/d36/dfc/fbd 0 2024-03-23T08:24:11.246 INFO:tasks.workunit.client.0.smithi070.stdout:1/896: sync 2024-03-23T08:24:11.251 INFO:tasks.workunit.client.0.smithi070.stdout:5/885: dwrite dc/d1b/d1d/d25/d38/d53/f76 [8388608,4194304] 0 2024-03-23T08:24:11.254 INFO:tasks.workunit.client.0.smithi070.stdout:1/897: dread d8/db/d100/d74/f71 [0,4194304] 0 2024-03-23T08:24:11.255 INFO:tasks.workunit.client.0.smithi070.stdout:5/886: creat dc/d1b/d1d/d25/d36/d32/d5c/dcb/f11f x:0 0 0 2024-03-23T08:24:11.265 INFO:tasks.workunit.client.0.smithi070.stdout:1/898: creat d8/d1a/f13c x:0 0 0 2024-03-23T08:24:11.274 INFO:tasks.workunit.client.0.smithi070.stdout:4/847: dwrite d3/d6/d35/d65/faa [0,4194304] 0 2024-03-23T08:24:11.277 INFO:tasks.workunit.client.0.smithi070.stdout:5/887: dread dc/d1b/d1d/d25/da9/fcc [0,4194304] 0 2024-03-23T08:24:11.278 INFO:tasks.workunit.client.0.smithi070.stdout:4/848: creat d3/d6/d35/d3f/f12b x:0 0 0 2024-03-23T08:24:11.287 INFO:tasks.workunit.client.0.smithi070.stdout:4/849: creat d3/d6/d11/ded/f12c x:0 0 0 2024-03-23T08:24:11.306 INFO:tasks.workunit.client.0.smithi070.stdout:4/850: dread d3/d6/d35/d65/d1b/f28 [0,4194304] 0 2024-03-23T08:24:11.311 INFO:tasks.workunit.client.0.smithi070.stdout:3/980: dwrite db/d4e/df8/d3d/f56 [0,4194304] 0 2024-03-23T08:24:11.312 INFO:tasks.workunit.client.0.smithi070.stdout:3/981: chown db/d20/d49/l8f 78 1 2024-03-23T08:24:11.319 INFO:tasks.workunit.client.0.smithi070.stdout:4/851: link d3/d6/d35/d65/ca0 d3/d6/d35/d3f/d51/d6e/d97/c12d 0 2024-03-23T08:24:11.320 INFO:tasks.workunit.client.0.smithi070.stdout:4/852: chown d3/d6/d35/d3f/d51 0 1 2024-03-23T08:24:11.336 INFO:tasks.workunit.client.0.smithi070.stdout:3/982: dread db/d7f/fbe [0,4194304] 0 2024-03-23T08:24:11.337 INFO:tasks.workunit.client.0.smithi070.stdout:3/983: write db/df/d6a/fa2 [288840,41217] 0 2024-03-23T08:24:11.340 INFO:tasks.workunit.client.0.smithi070.stdout:3/984: chown db/d14/d9c/l160 43534428 1 2024-03-23T08:24:11.343 INFO:tasks.workunit.client.0.smithi070.stdout:3/985: mkdir db/d14/d16e 0 2024-03-23T08:24:11.347 INFO:tasks.workunit.client.0.smithi070.stdout:5/888: dwrite dc/d1b/d1d/d25/d36/f27 [0,4194304] 0 2024-03-23T08:24:11.350 INFO:tasks.workunit.client.0.smithi070.stdout:5/889: write dc/d1b/d1d/d25/d36/d32/d5c/fed [300144,103865] 0 2024-03-23T08:24:11.411 INFO:tasks.workunit.client.0.smithi070.stdout:5/890: rmdir dc/d1b/d1d/d25/d68/d8a/dad 39 2024-03-23T08:24:11.432 INFO:tasks.workunit.client.0.smithi070.stdout:5/891: link dc/d1b/d1d/d25/d36/d32/l119 dc/d19/l120 0 2024-03-23T08:24:11.433 INFO:tasks.workunit.client.0.smithi070.stdout:5/892: stat dc/d1b/d1d/l11c 0 2024-03-23T08:24:11.435 INFO:tasks.workunit.client.0.smithi070.stdout:4/853: write d3/d6/d35/d3f/da9/f10a [545308,56715] 0 2024-03-23T08:24:11.438 INFO:tasks.workunit.client.0.smithi070.stdout:5/893: symlink dc/d1b/d1d/d25/l121 0 2024-03-23T08:24:11.439 INFO:tasks.workunit.client.0.smithi070.stdout:5/894: write dc/f3d [678747,12933] 0 2024-03-23T08:24:11.447 INFO:tasks.workunit.client.0.smithi070.stdout:5/895: truncate f8 1037772 0 2024-03-23T08:24:11.450 INFO:tasks.workunit.client.0.smithi070.stdout:5/896: dread dc/d1b/d1d/f6f [0,4194304] 0 2024-03-23T08:24:11.451 INFO:tasks.workunit.client.0.smithi070.stdout:5/897: chown dc/d1b/d1d/f3a 329470 1 2024-03-23T08:24:11.454 INFO:tasks.workunit.client.0.smithi070.stdout:3/986: dwrite db/df/f10d [0,4194304] 0 2024-03-23T08:24:11.460 INFO:tasks.workunit.client.0.smithi070.stdout:5/898: write dc/d1b/d1d/d25/d38/d53/f9a [540665,124245] 0 2024-03-23T08:24:11.466 INFO:tasks.workunit.client.0.smithi070.stdout:5/899: symlink dc/d1b/d1d/d25/d36/l122 0 2024-03-23T08:24:11.471 INFO:tasks.workunit.client.0.smithi070.stdout:4/854: truncate d3/fc 2527970 0 2024-03-23T08:24:11.476 INFO:tasks.workunit.client.0.smithi070.stdout:5/900: dread dc/d1b/d1d/d25/d38/f73 [0,4194304] 0 2024-03-23T08:24:11.478 INFO:tasks.workunit.client.0.smithi070.stdout:4/855: mknod d3/d6/d35/d65/d1b/de2/c12e 0 2024-03-23T08:24:11.481 INFO:tasks.workunit.client.0.smithi070.stdout:4/856: mknod d3/d6/d9/de6/c12f 0 2024-03-23T08:24:11.484 INFO:tasks.workunit.client.0.smithi070.stdout:5/901: dwrite dc/d1b/d1d/d25/d38/d53/fc9 [0,4194304] 0 2024-03-23T08:24:11.487 INFO:tasks.workunit.client.0.smithi070.stdout:4/857: creat d3/d6/d6a/f130 x:0 0 0 2024-03-23T08:24:11.499 INFO:tasks.workunit.client.0.smithi070.stdout:5/902: creat dc/d1b/d1d/d25/d68/f123 x:0 0 0 2024-03-23T08:24:11.500 INFO:tasks.workunit.client.0.smithi070.stdout:4/858: rmdir d3/d6/d9/de6/d10d 0 2024-03-23T08:24:11.503 INFO:tasks.workunit.client.0.smithi070.stdout:5/903: fdatasync dc/f58 0 2024-03-23T08:24:11.507 INFO:tasks.workunit.client.0.smithi070.stdout:4/859: creat d3/d6/d35/dcf/d7e/f131 x:0 0 0 2024-03-23T08:24:11.509 INFO:tasks.workunit.client.0.smithi070.stdout:5/904: dwrite dc/d1b/d1d/d25/fc2 [0,4194304] 0 2024-03-23T08:24:11.510 INFO:tasks.workunit.client.0.smithi070.stdout:4/860: fdatasync d3/d6/d35/d65/d1b/f28 0 2024-03-23T08:24:11.511 INFO:tasks.workunit.client.0.smithi070.stdout:5/905: chown dc/d1b/d1d/d25/d36/d32/f5a 112 1 2024-03-23T08:24:11.511 INFO:tasks.workunit.client.0.smithi070.stdout:4/861: write d3/d6/d35/d65/d1b/f26 [675251,95462] 0 2024-03-23T08:24:11.515 INFO:tasks.workunit.client.0.smithi070.stdout:4/862: truncate d3/d6/d35/dcf/db5/dbb/f109 1030782 0 2024-03-23T08:24:11.516 INFO:tasks.workunit.client.0.smithi070.stdout:4/863: readlink d3/d6/d9/l33 0 2024-03-23T08:24:11.520 INFO:tasks.workunit.client.0.smithi070.stdout:4/864: mkdir d3/d6/d35/d65/d6f/dee/d132 0 2024-03-23T08:24:11.524 INFO:tasks.workunit.client.0.smithi070.stdout:5/906: rename dc/d1b/d1d/d25/d36/d32/db7 to dc/d1b/d1d/d25/d68/d124 0 2024-03-23T08:24:11.525 INFO:tasks.workunit.client.0.smithi070.stdout:5/907: stat dc/d1b/d1d/d25/dcd 0 2024-03-23T08:24:11.525 INFO:tasks.workunit.client.0.smithi070.stdout:4/865: mkdir d3/d6/d35/dcf/d133 0 2024-03-23T08:24:11.526 INFO:tasks.workunit.client.0.smithi070.stdout:4/866: write d3/d6/d35/d65/d27/f46 [787924,259] 0 2024-03-23T08:24:11.528 INFO:tasks.workunit.client.0.smithi070.stdout:4/867: write d3/d6/d35/d65/d1b/f28 [3259814,36793] 0 2024-03-23T08:24:11.532 INFO:tasks.workunit.client.0.smithi070.stdout:5/908: creat dc/d1b/d1d/d25/d36/d32/d11a/f125 x:0 0 0 2024-03-23T08:24:11.534 INFO:tasks.workunit.client.0.smithi070.stdout:5/909: write dc/d1b/d1d/d25/d36/d32/d5c/d10a/f114 [38630,119764] 0 2024-03-23T08:24:11.543 INFO:tasks.workunit.client.0.smithi070.stdout:5/910: creat dc/d1b/d1d/f126 x:0 0 0 2024-03-23T08:24:11.551 INFO:tasks.workunit.client.0.smithi070.stdout:5/911: creat dc/d1b/d1d/d25/d36/d32/f127 x:0 0 0 2024-03-23T08:24:11.554 INFO:tasks.workunit.client.0.smithi070.stdout:5/912: symlink dc/d1b/d1d/d25/d68/d124/de6/l128 0 2024-03-23T08:24:11.557 INFO:tasks.workunit.client.0.smithi070.stdout:4/868: write d3/f6d [295212,2685] 0 2024-03-23T08:24:11.562 INFO:tasks.workunit.client.0.smithi070.stdout:4/869: getdents d3/d6/d35/d3f/da9 0 2024-03-23T08:24:11.564 INFO:tasks.workunit.client.0.smithi070.stdout:4/870: readlink d3/d6/d11/l77 0 2024-03-23T08:24:11.573 INFO:tasks.workunit.client.0.smithi070.stdout:4/871: dread d3/d6/f6c [0,4194304] 0 2024-03-23T08:24:11.577 INFO:tasks.workunit.client.0.smithi070.stdout:4/872: mknod d3/d6/d9/c134 0 2024-03-23T08:24:11.594 INFO:tasks.workunit.client.0.smithi070.stdout:4/873: truncate d3/f6d 3726856 0 2024-03-23T08:24:11.616 INFO:tasks.workunit.client.0.smithi070.stdout:4/874: dwrite d3/d6/d35/d3f/fd0 [0,4194304] 0 2024-03-23T08:24:11.619 INFO:tasks.workunit.client.0.smithi070.stdout:4/875: symlink d3/d6/d6a/l135 0 2024-03-23T08:24:11.620 INFO:tasks.workunit.client.0.smithi070.stdout:4/876: chown d3/d6/d35/d3f/d51/d6e/ffd 13556838 1 2024-03-23T08:24:11.632 INFO:tasks.workunit.client.0.smithi070.stdout:4/877: dread d3/f17 [0,4194304] 0 2024-03-23T08:24:11.634 INFO:tasks.workunit.client.0.smithi070.stdout:4/878: mkdir d3/d6/d35/d136 0 2024-03-23T08:24:11.637 INFO:tasks.workunit.client.0.smithi070.stdout:4/879: fdatasync d3/d6/d35/d65/d1b/f66 0 2024-03-23T08:24:11.638 INFO:tasks.workunit.client.0.smithi070.stdout:4/880: readlink d3/l3d 0 2024-03-23T08:24:11.644 INFO:tasks.workunit.client.0.smithi070.stdout:4/881: mknod d3/d6/d35/dcf/d133/c137 0 2024-03-23T08:24:11.644 INFO:tasks.workunit.client.0.smithi070.stdout:4/882: stat d3/d6/d9/l69 0 2024-03-23T08:24:11.647 INFO:tasks.workunit.client.0.smithi070.stdout:4/883: rmdir d3/d6/d35/d65/d27/d4b/d73 39 2024-03-23T08:24:11.649 INFO:tasks.workunit.client.0.smithi070.stdout:4/884: truncate d3/d6/f7 2114094 0 2024-03-23T08:24:11.661 INFO:tasks.workunit.client.0.smithi070.stdout:4/885: dread d3/f6d [0,4194304] 0 2024-03-23T08:24:11.662 INFO:tasks.workunit.client.0.smithi070.stdout:4/886: write d3/d6/d35/d65/d1b/f26 [196861,37299] 0 2024-03-23T08:24:11.665 INFO:tasks.workunit.client.0.smithi070.stdout:4/887: getdents d3/d6/d35/d65/d27/d4b/d8b 0 2024-03-23T08:24:11.667 INFO:tasks.workunit.client.0.smithi070.stdout:4/888: creat d3/d6/d35/d65/d27/d4b/dc0/f138 x:0 0 0 2024-03-23T08:24:11.671 INFO:tasks.workunit.client.0.smithi070.stdout:4/889: mkdir d3/d6/d35/d65/d27/d4b/d73/d139 0 2024-03-23T08:24:11.672 INFO:tasks.workunit.client.0.smithi070.stdout:4/890: readlink d3/d6/d35/dcf/db5/dda/lc3 0 2024-03-23T08:24:11.673 INFO:tasks.workunit.client.0.smithi070.stdout:4/891: chown d3/d6/dca/ff2 838424031 1 2024-03-23T08:24:11.675 INFO:tasks.workunit.client.0.smithi070.stdout:4/892: creat d3/d6/d35/d136/f13a x:0 0 0 2024-03-23T08:24:11.676 INFO:tasks.workunit.client.0.smithi070.stdout:4/893: readlink d3/d6/d35/d65/l8e 0 2024-03-23T08:24:11.677 INFO:tasks.workunit.client.0.smithi070.stdout:4/894: chown d3/d6/d35/d65/d6f/c11a 562 1 2024-03-23T08:24:11.678 INFO:tasks.workunit.client.0.smithi070.stdout:4/895: symlink d3/d6/d35/dcf/d7e/l13b 0 2024-03-23T08:24:11.716 INFO:tasks.workunit.client.0.smithi070.stdout:4/896: write d3/d6/dca/d124/f127 [2130807,42813] 0 2024-03-23T08:24:11.723 INFO:tasks.workunit.client.0.smithi070.stdout:4/897: dwrite d3/d6/d35/dcf/fe7 [4194304,4194304] 0 2024-03-23T08:24:11.732 INFO:tasks.workunit.client.0.smithi070.stdout:4/898: creat d3/d6/d35/d65/d27/d4b/d73/d89/f13c x:0 0 0 2024-03-23T08:24:11.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:11 smithi070 ceph-mon[39699]: pgmap v45: 65 pgs: 65 active+clean; 2.0 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 43 MiB/s rd, 103 MiB/s wr, 338 op/s 2024-03-23T08:24:11.735 INFO:tasks.workunit.client.0.smithi070.stdout:4/899: mkdir d3/d6/d35/dcf/db5/dda/d13d 0 2024-03-23T08:24:11.741 INFO:tasks.workunit.client.0.smithi070.stdout:4/900: truncate d3/d6/d11/fce 72874 0 2024-03-23T08:24:11.742 INFO:tasks.workunit.client.0.smithi070.stdout:4/901: dread - d3/d6/d35/d3f/da9/fc1 zero size 2024-03-23T08:24:11.758 INFO:tasks.workunit.client.0.smithi070.stdout:4/902: write d3/d6/d35/d3f/da9/fc1 [557416,89475] 0 2024-03-23T08:24:11.759 INFO:tasks.workunit.client.0.smithi070.stdout:4/903: write d3/d6/d11/d118/f11b [559397,2968] 0 2024-03-23T08:24:11.770 INFO:tasks.workunit.client.0.smithi070.stdout:4/904: dread d3/d6/d35/f67 [0,4194304] 0 2024-03-23T08:24:11.772 INFO:tasks.workunit.client.0.smithi070.stdout:4/905: write d3/d6/d35/d3f/da9/f10a [334462,120957] 0 2024-03-23T08:24:11.774 INFO:tasks.workunit.client.0.smithi070.stdout:4/906: truncate d3/f17 841440 0 2024-03-23T08:24:11.776 INFO:tasks.workunit.client.0.smithi070.stdout:4/907: truncate d3/d6/d9/f6b 3675646 0 2024-03-23T08:24:11.786 INFO:tasks.workunit.client.0.smithi070.stdout:4/908: dread d3/d6/dca/d124/f127 [0,4194304] 0 2024-03-23T08:24:11.787 INFO:tasks.workunit.client.0.smithi070.stdout:4/909: fdatasync d3/d6/dca/ff2 0 2024-03-23T08:24:11.788 INFO:tasks.workunit.client.0.smithi070.stdout:4/910: readlink d3/d6/d11/l77 0 2024-03-23T08:24:11.789 INFO:tasks.workunit.client.0.smithi070.stdout:4/911: readlink d3/d6/d6a/lc4 0 2024-03-23T08:24:11.792 INFO:tasks.workunit.client.0.smithi070.stdout:4/912: mknod d3/d6/d35/dcf/db5/dda/c13e 0 2024-03-23T08:24:11.797 INFO:tasks.workunit.client.0.smithi070.stdout:4/913: mknod d3/d6/d35/dcf/db5/dda/d13d/c13f 0 2024-03-23T08:24:11.801 INFO:tasks.workunit.client.0.smithi070.stdout:4/914: fdatasync d3/d6/d35/dcf/db5/dda/db8/fa7 0 2024-03-23T08:24:11.809 INFO:tasks.workunit.client.0.smithi070.stdout:4/915: link d3/d6/d35/dcf/d7e/c111 d3/d6/d35/dcf/db5/c140 0 2024-03-23T08:24:11.816 INFO:tasks.workunit.client.0.smithi070.stdout:4/916: dwrite d3/d6/d35/d65/d1b/f26 [0,4194304] 0 2024-03-23T08:24:11.818 INFO:tasks.workunit.client.0.smithi070.stdout:4/917: creat d3/d6/d35/dcf/db5/dda/d13d/f141 x:0 0 0 2024-03-23T08:24:11.821 INFO:tasks.workunit.client.0.smithi070.stdout:4/918: mknod d3/d6/d35/d3f/da9/c142 0 2024-03-23T08:24:11.822 INFO:tasks.workunit.client.0.smithi070.stdout:4/919: read d3/d6/f6c [3872236,15489] 0 2024-03-23T08:24:11.824 INFO:tasks.workunit.client.0.smithi070.stdout:4/920: fdatasync d3/d6/d35/d65/d6f/dee/fb9 0 2024-03-23T08:24:11.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:11 smithi193 ceph-mon[48370]: pgmap v45: 65 pgs: 65 active+clean; 2.0 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 43 MiB/s rd, 103 MiB/s wr, 338 op/s 2024-03-23T08:24:11.844 INFO:tasks.workunit.client.0.smithi070.stdout:4/921: dread d3/fc [0,4194304] 0 2024-03-23T08:24:11.851 INFO:tasks.workunit.client.0.smithi070.stdout:4/922: mkdir d3/d6/d6a/d143 0 2024-03-23T08:24:11.853 INFO:tasks.workunit.client.0.smithi070.stdout:4/923: mkdir d3/d6/d35/d65/d27/d144 0 2024-03-23T08:24:11.855 INFO:tasks.workunit.client.0.smithi070.stdout:3/987: sync 2024-03-23T08:24:11.855 INFO:tasks.workunit.client.0.smithi070.stdout:1/899: sync 2024-03-23T08:24:11.855 INFO:tasks.workunit.client.0.smithi070.stdout:5/913: sync 2024-03-23T08:24:11.855 INFO:tasks.workunit.client.0.smithi070.stdout:4/924: mknod d3/d6/d35/d3f/da9/c145 0 2024-03-23T08:24:11.856 INFO:tasks.workunit.client.0.smithi070.stdout:1/900: stat d8/db/d8e/la4 0 2024-03-23T08:24:11.857 INFO:tasks.workunit.client.0.smithi070.stdout:5/914: read - dc/d1b/d1d/d25/d36/f100 zero size 2024-03-23T08:24:11.859 INFO:tasks.workunit.client.0.smithi070.stdout:5/915: chown dc/d1b/d1d/d25/d38/la5 1102 1 2024-03-23T08:24:11.861 INFO:tasks.workunit.client.0.smithi070.stdout:5/916: stat dc/d1b/c55 0 2024-03-23T08:24:11.864 INFO:tasks.workunit.client.0.smithi070.stdout:3/988: dwrite db/d20/d49/de1/f165 [0,4194304] 0 2024-03-23T08:24:11.864 INFO:tasks.workunit.client.0.smithi070.stdout:4/925: creat d3/d6/d35/d65/d27/d4b/f146 x:0 0 0 2024-03-23T08:24:11.866 INFO:tasks.workunit.client.0.smithi070.stdout:1/901: dread d8/d26/d49/d65/fdb [0,4194304] 0 2024-03-23T08:24:11.868 INFO:tasks.workunit.client.0.smithi070.stdout:4/926: write d3/d6/f38 [3226309,69146] 0 2024-03-23T08:24:11.869 INFO:tasks.workunit.client.0.smithi070.stdout:5/917: symlink dc/d1b/d1d/d25/d68/d8a/d99/dd7/l129 0 2024-03-23T08:24:11.870 INFO:tasks.workunit.client.0.smithi070.stdout:4/927: chown d3/d6/d35/dcf/db5/dda/db8/l57 299528196 1 2024-03-23T08:24:11.870 INFO:tasks.workunit.client.0.smithi070.stdout:5/918: read - dc/d1b/d1d/d25/da9/deb/f110 zero size 2024-03-23T08:24:11.879 INFO:tasks.workunit.client.0.smithi070.stdout:4/928: mknod d3/d6/d35/d3f/da9/d125/c147 0 2024-03-23T08:24:11.880 INFO:tasks.workunit.client.0.smithi070.stdout:5/919: creat dc/d1b/d1d/d25/d36/dfc/d11b/f12a x:0 0 0 2024-03-23T08:24:11.881 INFO:tasks.workunit.client.0.smithi070.stdout:4/929: chown d3/d6/d35/d65/f75 106293 1 2024-03-23T08:24:11.884 INFO:tasks.workunit.client.0.smithi070.stdout:5/920: creat dc/d1b/d1d/d25/d36/dfc/d11b/f12b x:0 0 0 2024-03-23T08:24:11.887 INFO:tasks.workunit.client.0.smithi070.stdout:5/921: mkdir dc/d1b/d1d/d25/d68/d8a/d99/dd7/d12c 0 2024-03-23T08:24:11.891 INFO:tasks.workunit.client.0.smithi070.stdout:5/922: getdents dc/d1b/d1d/d25/d68/d8a/d99/dd7/d12c 0 2024-03-23T08:24:11.901 INFO:tasks.workunit.client.0.smithi070.stdout:5/923: link dc/d1b/d1d/d25/da9/deb/d107/lfb dc/d1b/d1d/d25/d38/d53/l12d 0 2024-03-23T08:24:11.905 INFO:tasks.workunit.client.0.smithi070.stdout:5/924: mknod dc/d1b/d1d/d25/d36/d32/c12e 0 2024-03-23T08:24:11.907 INFO:tasks.workunit.client.0.smithi070.stdout:5/925: mkdir dc/d1b/d1d/d25/dcd/d12f 0 2024-03-23T08:24:11.912 INFO:tasks.workunit.client.0.smithi070.stdout:1/902: write d8/d26/d7b/fe0 [8235,115382] 0 2024-03-23T08:24:11.916 INFO:tasks.workunit.client.0.smithi070.stdout:3/989: dwrite db/d14/d65/dae/fb8 [0,4194304] 0 2024-03-23T08:24:11.917 INFO:tasks.workunit.client.0.smithi070.stdout:3/990: stat db/df/d1a/l7c 0 2024-03-23T08:24:11.918 INFO:tasks.workunit.client.0.smithi070.stdout:3/991: stat db/d20/d49/l12a 0 2024-03-23T08:24:11.922 INFO:tasks.workunit.client.0.smithi070.stdout:1/903: symlink d8/db/l13d 0 2024-03-23T08:24:11.923 INFO:tasks.workunit.client.0.smithi070.stdout:1/904: chown d8/d26/d41/fd7 5887072 1 2024-03-23T08:24:11.924 INFO:tasks.workunit.client.0.smithi070.stdout:3/992: mkdir db/d4e/d105/d16d/dd9/d12f/d16f 0 2024-03-23T08:24:11.924 INFO:tasks.workunit.client.0.smithi070.stdout:5/926: link dc/d1b/d1d/d25/d68/d124/le0 dc/d1b/d1d/d25/d36/d32/d5c/dcb/l130 0 2024-03-23T08:24:11.927 INFO:tasks.workunit.client.0.smithi070.stdout:1/905: unlink d8/f9 0 2024-03-23T08:24:11.933 INFO:tasks.workunit.client.0.smithi070.stdout:3/993: symlink db/d14/d65/dae/l170 0 2024-03-23T08:24:11.934 INFO:tasks.workunit.client.0.smithi070.stdout:1/906: creat d8/d42/d4e/f13e x:0 0 0 2024-03-23T08:24:11.934 INFO:tasks.workunit.client.0.smithi070.stdout:5/927: chown dc/d1b/d1d/d25/d36/d32/l3e 1670 1 2024-03-23T08:24:11.934 INFO:tasks.workunit.client.0.smithi070.stdout:3/994: read db/d20/d7a/d80/f116 [418585,27101] 0 2024-03-23T08:24:11.934 INFO:tasks.workunit.client.0.smithi070.stdout:4/930: dwrite d3/d6/d35/f76 [0,4194304] 0 2024-03-23T08:24:11.936 INFO:tasks.workunit.client.0.smithi070.stdout:4/931: read d3/d6/d6a/d7c/f9c [682458,100147] 0 2024-03-23T08:24:11.937 INFO:tasks.workunit.client.0.smithi070.stdout:1/907: symlink d8/db/l13f 0 2024-03-23T08:24:11.944 INFO:tasks.workunit.client.0.smithi070.stdout:3/995: dwrite db/d4e/dc4/d137/f153 [0,4194304] 0 2024-03-23T08:24:11.944 INFO:tasks.workunit.client.0.smithi070.stdout:4/932: unlink d3/d6/d35/d3f/da9/cdb 0 2024-03-23T08:24:11.944 INFO:tasks.workunit.client.0.smithi070.stdout:3/996: chown db/df 3736572 1 2024-03-23T08:24:11.946 INFO:tasks.workunit.client.0.smithi070.stdout:4/933: chown d3/d6/d35/d65 13 1 2024-03-23T08:24:11.949 INFO:tasks.workunit.client.0.smithi070.stdout:1/908: creat d8/d26/d49/d65/de8/f140 x:0 0 0 2024-03-23T08:24:11.949 INFO:tasks.workunit.client.0.smithi070.stdout:3/997: dread - db/df/d1a/ff4 zero size 2024-03-23T08:24:11.951 INFO:tasks.workunit.client.0.smithi070.stdout:3/998: fsync db/d20/d7a/d80/f168 0 2024-03-23T08:24:11.951 INFO:tasks.workunit.client.0.smithi070.stdout:1/909: stat d8/db/d40/d43/d64/dc2/l103 0 2024-03-23T08:24:11.960 INFO:tasks.workunit.client.0.smithi070.stdout:1/910: dwrite d8/db/f3e [0,4194304] 0 2024-03-23T08:24:11.961 INFO:tasks.workunit.client.0.smithi070.stdout:3/999: dwrite db/df/d6a/fa2 [0,4194304] 0 2024-03-23T08:24:11.971 INFO:tasks.workunit.client.0.smithi070.stdout:1/911: getdents d8/d1a 0 2024-03-23T08:24:12.010 INFO:tasks.workunit.client.0.smithi070.stdout:4/934: dwrite d3/d6/d9/fb4 [0,4194304] 0 2024-03-23T08:24:12.182 INFO:tasks.workunit.client.0.smithi070.stdout:4/935: write d3/d6/d35/d3f/d51/d6e/f99 [2336242,38346] 0 2024-03-23T08:24:12.185 INFO:tasks.workunit.client.0.smithi070.stdout:4/936: chown d3/d6/d35/d3f/d51/cb6 3 1 2024-03-23T08:24:12.189 INFO:tasks.workunit.client.0.smithi070.stdout:4/937: dread d3/d6/d35/d65/d27/d4b/f7d [0,4194304] 0 2024-03-23T08:24:12.191 INFO:tasks.workunit.client.0.smithi070.stdout:4/938: creat d3/d6/d35/d65/d27/de1/f148 x:0 0 0 2024-03-23T08:24:12.196 INFO:tasks.workunit.client.0.smithi070.stdout:4/939: truncate d3/d6/d35/d65/f75 162367 0 2024-03-23T08:24:12.202 INFO:tasks.workunit.client.0.smithi070.stdout:4/940: dwrite d3/d6/d35/d65/df1/ffc [0,4194304] 0 2024-03-23T08:24:12.204 INFO:tasks.workunit.client.0.smithi070.stdout:4/941: creat d3/d6/d35/dcf/db5/dda/dc7/f149 x:0 0 0 2024-03-23T08:24:12.207 INFO:tasks.workunit.client.0.smithi070.stdout:4/942: mkdir d3/d6/d6a/d14a 0 2024-03-23T08:24:12.210 INFO:tasks.workunit.client.0.smithi070.stdout:4/943: creat d3/d6/d35/d3f/d51/f14b x:0 0 0 2024-03-23T08:24:12.211 INFO:tasks.workunit.client.0.smithi070.stdout:4/944: readlink d3/d6/d35/d65/d27/d4b/d73/l81 0 2024-03-23T08:24:12.215 INFO:tasks.workunit.client.0.smithi070.stdout:4/945: mkdir d3/d6/d35/d65/d14c 0 2024-03-23T08:24:12.231 INFO:tasks.workunit.client.0.smithi070.stdout:4/946: dread d3/d6/d35/d3f/d51/d6e/ffd [0,4194304] 0 2024-03-23T08:24:12.232 INFO:tasks.workunit.client.0.smithi070.stdout:4/947: chown d3/d6 90 1 2024-03-23T08:24:12.237 INFO:tasks.workunit.client.0.smithi070.stdout:4/948: dwrite d3/d6/d35/d65/d27/f46 [0,4194304] 0 2024-03-23T08:24:12.244 INFO:tasks.workunit.client.0.smithi070.stdout:4/949: creat d3/d6/d35/d65/d27/de4/f14d x:0 0 0 2024-03-23T08:24:12.246 INFO:tasks.workunit.client.0.smithi070.stdout:4/950: truncate d3/d6/d35/d3f/f87 538004 0 2024-03-23T08:24:12.249 INFO:tasks.workunit.client.0.smithi070.stdout:4/951: truncate d3/d6/d35/dcf/db5/dda/dc7/f149 474495 0 2024-03-23T08:24:12.259 INFO:tasks.workunit.client.0.smithi070.stdout:4/952: dread d3/d6/d6a/d7c/fea [0,4194304] 0 2024-03-23T08:24:12.273 INFO:tasks.workunit.client.0.smithi070.stdout:4/953: dread d3/d6/d35/dcf/db5/dda/db8/d88/fa6 [0,4194304] 0 2024-03-23T08:24:12.276 INFO:tasks.workunit.client.0.smithi070.stdout:4/954: truncate d3/d6/d35/d65/f75 1089048 0 2024-03-23T08:24:12.290 INFO:tasks.workunit.client.0.smithi070.stdout:4/955: dread d3/d6/d35/d65/d6f/dee/fc2 [0,4194304] 0 2024-03-23T08:24:12.294 INFO:tasks.workunit.client.0.smithi070.stdout:4/956: rmdir d3/d6/d35/d65/d27/d4b 39 2024-03-23T08:24:12.300 INFO:tasks.workunit.client.0.smithi070.stdout:4/957: creat d3/d6/d35/d65/d27/d4b/dc0/f14e x:0 0 0 2024-03-23T08:24:12.305 INFO:tasks.workunit.client.0.smithi070.stdout:4/958: mknod d3/d6/d6a/d143/c14f 0 2024-03-23T08:24:12.308 INFO:tasks.workunit.client.0.smithi070.stdout:4/959: chown d3/f17 867 1 2024-03-23T08:24:12.310 INFO:tasks.workunit.client.0.smithi070.stdout:4/960: mkdir d3/d6/d35/d65/d6f/d150 0 2024-03-23T08:24:12.313 INFO:tasks.workunit.client.0.smithi070.stdout:4/961: dread d3/d6/d35/dcf/db5/dbb/f109 [0,4194304] 0 2024-03-23T08:24:12.322 INFO:tasks.workunit.client.0.smithi070.stdout:4/962: creat d3/d6/d35/d65/d27/d4b/d73/f151 x:0 0 0 2024-03-23T08:24:12.326 INFO:tasks.workunit.client.0.smithi070.stdout:4/963: symlink d3/d6/d35/d65/l152 0 2024-03-23T08:24:12.345 INFO:tasks.workunit.client.0.smithi070.stdout:4/964: dwrite d3/d6/dca/ff2 [0,4194304] 0 2024-03-23T08:24:12.346 INFO:tasks.workunit.client.0.smithi070.stdout:4/965: write d3/d6/d35/d3f/d51/d6e/f99 [871184,16814] 0 2024-03-23T08:24:12.349 INFO:tasks.workunit.client.0.smithi070.stdout:4/966: mknod d3/d6/d35/d3f/c153 0 2024-03-23T08:24:12.352 INFO:tasks.workunit.client.0.smithi070.stdout:4/967: chown d3/d6/d9/c134 2090023 1 2024-03-23T08:24:12.353 INFO:tasks.workunit.client.0.smithi070.stdout:4/968: fdatasync d3/d6/d35/d65/d1b/f28 0 2024-03-23T08:24:12.354 INFO:tasks.workunit.client.0.smithi070.stdout:4/969: chown d3/d6/d35/d65/d27/d4b/dc0/f14e 14860396 1 2024-03-23T08:24:12.361 INFO:tasks.workunit.client.0.smithi070.stdout:4/970: write d3/d6/d35/d65/d6f/dee/fb9 [4795633,76580] 0 2024-03-23T08:24:12.364 INFO:tasks.workunit.client.0.smithi070.stdout:4/971: dread d3/d6/d35/d3f/f87 [0,4194304] 0 2024-03-23T08:24:12.366 INFO:tasks.workunit.client.0.smithi070.stdout:4/972: mknod d3/d6/d35/d136/c154 0 2024-03-23T08:24:12.373 INFO:tasks.workunit.client.0.smithi070.stdout:4/973: truncate d3/d6/d35/dcf/fe7 32394 0 2024-03-23T08:24:12.380 INFO:tasks.workunit.client.0.smithi070.stdout:4/974: rmdir d3/d6/d35/d65/d27/de1/d113 0 2024-03-23T08:24:12.423 INFO:tasks.workunit.client.0.smithi070.stdout:5/928: sync 2024-03-23T08:24:12.425 INFO:tasks.workunit.client.0.smithi070.stdout:5/929: creat dc/d19/f131 x:0 0 0 2024-03-23T08:24:12.429 INFO:tasks.workunit.client.0.smithi070.stdout:5/930: dread dc/d19/f2c [0,4194304] 0 2024-03-23T08:24:12.431 INFO:tasks.workunit.client.0.smithi070.stdout:5/931: truncate dc/d1b/d1d/d25/d68/d8a/d99/dd7/f116 760626 0 2024-03-23T08:24:12.433 INFO:tasks.workunit.client.0.smithi070.stdout:5/932: write dc/d1b/d1d/d25/d68/d8a/d99/f117 [875486,72246] 0 2024-03-23T08:24:12.434 INFO:tasks.workunit.client.0.smithi070.stdout:5/933: write dc/d1b/d1d/d25/d68/d124/fc4 [2252569,34206] 0 2024-03-23T08:24:12.507 INFO:tasks.workunit.client.0.smithi070.stdout:1/912: sync 2024-03-23T08:24:12.574 INFO:tasks.workunit.client.0.smithi070.stdout:1/913: fdatasync d8/db/d2c/d3f/f56 0 2024-03-23T08:24:12.579 INFO:tasks.workunit.client.0.smithi070.stdout:1/914: stat d8/db/l2a 0 2024-03-23T08:24:12.580 INFO:tasks.workunit.client.0.smithi070.stdout:1/915: fdatasync d8/d11/f3b 0 2024-03-23T08:24:12.584 INFO:tasks.workunit.client.0.smithi070.stdout:1/916: mknod d8/db/d40/d43/d64/d85/dc7/c141 0 2024-03-23T08:24:12.587 INFO:tasks.workunit.client.0.smithi070.stdout:1/917: creat d8/db/d40/d43/d64/d85/d97/f142 x:0 0 0 2024-03-23T08:24:12.587 INFO:tasks.workunit.client.0.smithi070.stdout:1/918: readlink d8/d1a/l5a 0 2024-03-23T08:24:12.589 INFO:tasks.workunit.client.0.smithi070.stdout:1/919: chown d8/db/d40/d43/d64/dc2/dfb/f10c 341 1 2024-03-23T08:24:12.595 INFO:tasks.workunit.client.0.smithi070.stdout:1/920: truncate d8/db/d2c/d3f/f5f 1533470 0 2024-03-23T08:24:12.598 INFO:tasks.workunit.client.0.smithi070.stdout:1/921: mkdir d8/db/d2c/d3f/dd3/d143 0 2024-03-23T08:24:12.598 INFO:tasks.workunit.client.0.smithi070.stdout:1/922: chown d8/d1a/c1d 431914837 1 2024-03-23T08:24:12.606 INFO:tasks.workunit.client.0.smithi070.stdout:1/923: getdents d8/db/d100/d74/d5d/d70 0 2024-03-23T08:24:12.607 INFO:tasks.workunit.client.0.smithi070.stdout:1/924: fsync d8/d11/f4d 0 2024-03-23T08:24:12.744 INFO:tasks.workunit.client.0.smithi070.stdout:1/925: dwrite d8/f90 [0,4194304] 0 2024-03-23T08:24:12.749 INFO:tasks.workunit.client.0.smithi070.stdout:1/926: rename d8/db/d100/d74/d5d/d10a/f124 to d8/db/d40/d43/d64/d85/d97/f144 0 2024-03-23T08:24:13.073 INFO:tasks.workunit.client.0.smithi070.stdout:4/975: sync 2024-03-23T08:24:13.077 INFO:tasks.workunit.client.0.smithi070.stdout:4/976: link d3/d6/d35/dcf/db5/dda/db8/d88/c8a d3/d6/d35/d65/d6f/d150/c155 0 2024-03-23T08:24:13.082 INFO:tasks.workunit.client.0.smithi070.stdout:4/977: rename d3/d6/d35/d65/d27/de4 to d3/d6/d11/d118/d156 0 2024-03-23T08:24:13.083 INFO:tasks.workunit.client.0.smithi070.stdout:4/978: fsync d3/d6/d35/d65/d27/d4b/f146 0 2024-03-23T08:24:13.085 INFO:tasks.workunit.client.0.smithi070.stdout:4/979: creat d3/d6/d35/d3f/d51/d6e/d97/ddf/f157 x:0 0 0 2024-03-23T08:24:13.088 INFO:tasks.workunit.client.0.smithi070.stdout:4/980: mkdir d3/d6/d35/d65/d27/d158 0 2024-03-23T08:24:13.090 INFO:tasks.workunit.client.0.smithi070.stdout:4/981: unlink d3/d6/d35/d65/d6f/dee/cac 0 2024-03-23T08:24:13.101 INFO:tasks.workunit.client.0.smithi070.stdout:4/982: write d3/d6/d35/dcf/faf [927789,57982] 0 2024-03-23T08:24:13.105 INFO:tasks.workunit.client.0.smithi070.stdout:4/983: unlink d3/d6/d35/d65/d6f/d150/c155 0 2024-03-23T08:24:13.106 INFO:tasks.workunit.client.0.smithi070.stdout:4/984: stat d3/d6/dd8 0 2024-03-23T08:24:13.107 INFO:tasks.workunit.client.0.smithi070.stdout:4/985: dread - d3/d6/d35/d65/d27/de1/f148 zero size 2024-03-23T08:24:13.109 INFO:tasks.workunit.client.0.smithi070.stdout:4/986: mknod d3/c159 0 2024-03-23T08:24:13.111 INFO:tasks.workunit.client.0.smithi070.stdout:4/987: dread d3/d6/d11/fce [0,4194304] 0 2024-03-23T08:24:13.114 INFO:tasks.workunit.client.0.smithi070.stdout:4/988: unlink d3/d6/d35/d65/d6f/dee/fc2 0 2024-03-23T08:24:13.114 INFO:tasks.workunit.client.0.smithi070.stdout:5/934: sync 2024-03-23T08:24:13.114 INFO:tasks.workunit.client.0.smithi070.stdout:1/927: sync 2024-03-23T08:24:13.117 INFO:tasks.workunit.client.0.smithi070.stdout:1/928: creat d8/d26/d7b/f145 x:0 0 0 2024-03-23T08:24:13.118 INFO:tasks.workunit.client.0.smithi070.stdout:1/929: readlink d8/db/d2c/d3f/dd3/l131 0 2024-03-23T08:24:13.118 INFO:tasks.workunit.client.0.smithi070.stdout:4/989: dread d3/d6/d35/d65/f3e [0,4194304] 0 2024-03-23T08:24:13.119 INFO:tasks.workunit.client.0.smithi070.stdout:1/930: read d8/d11/f5c [230049,9797] 0 2024-03-23T08:24:13.122 INFO:tasks.workunit.client.0.smithi070.stdout:1/931: chown d8/d1a/f23 0 1 2024-03-23T08:24:13.122 INFO:tasks.workunit.client.0.smithi070.stdout:4/990: truncate d3/d6/d11/ded/f12c 377685 0 2024-03-23T08:24:13.124 INFO:tasks.workunit.client.0.smithi070.stdout:4/991: write d3/d6/d9/de6/f11e [357490,534] 0 2024-03-23T08:24:13.125 INFO:tasks.workunit.client.0.smithi070.stdout:5/935: dwrite dc/d1b/d1d/d25/d36/d32/d5c/d10a/fff [0,4194304] 0 2024-03-23T08:24:13.127 INFO:tasks.workunit.client.0.smithi070.stdout:4/992: write d3/d6/d35/d3f/d51/f63 [2036090,110871] 0 2024-03-23T08:24:13.127 INFO:tasks.workunit.client.0.smithi070.stdout:1/932: read d8/db/d2c/d3f/dd3/d111/f11a [653108,59480] 0 2024-03-23T08:24:13.129 INFO:tasks.workunit.client.0.smithi070.stdout:1/933: chown d8/db/d100/d74/l112 15409037 1 2024-03-23T08:24:13.135 INFO:tasks.workunit.client.0.smithi070.stdout:5/936: dread dc/d1b/d1d/d25/d36/d32/f87 [0,4194304] 0 2024-03-23T08:24:13.136 INFO:tasks.workunit.client.0.smithi070.stdout:1/934: symlink d8/db/d40/d43/d64/l146 0 2024-03-23T08:24:13.138 INFO:tasks.workunit.client.0.smithi070.stdout:5/937: symlink dc/d1b/d1d/d25/dcd/l132 0 2024-03-23T08:24:13.139 INFO:tasks.workunit.client.0.smithi070.stdout:1/935: dread d8/db/d40/d43/d64/f8b [0,4194304] 0 2024-03-23T08:24:13.144 INFO:tasks.workunit.client.0.smithi070.stdout:1/936: mknod d8/db/d2c/d3f/d48/dab/c147 0 2024-03-23T08:24:13.144 INFO:tasks.workunit.client.0.smithi070.stdout:5/938: dwrite dc/d1b/d1d/ffd [0,4194304] 0 2024-03-23T08:24:13.149 INFO:tasks.workunit.client.0.smithi070.stdout:1/937: symlink d8/db/d100/d74/l148 0 2024-03-23T08:24:13.161 INFO:tasks.workunit.client.0.smithi070.stdout:4/993: dwrite d3/d6/d35/dcf/db5/dda/dc7/ff4 [0,4194304] 0 2024-03-23T08:24:13.161 INFO:tasks.workunit.client.0.smithi070.stdout:1/938: dwrite d8/d42/d4e/d99/fda [0,4194304] 0 2024-03-23T08:24:13.162 INFO:tasks.workunit.client.0.smithi070.stdout:5/939: dwrite dc/d1b/d1d/d25/fb4 [4194304,4194304] 0 2024-03-23T08:24:13.167 INFO:tasks.workunit.client.0.smithi070.stdout:4/994: symlink d3/d6/d11/d118/l15a 0 2024-03-23T08:24:13.169 INFO:tasks.workunit.client.0.smithi070.stdout:1/939: creat d8/db/d2c/d3f/dd3/f149 x:0 0 0 2024-03-23T08:24:13.171 INFO:tasks.workunit.client.0.smithi070.stdout:5/940: mknod dc/d1b/d1d/d25/d38/d53/c133 0 2024-03-23T08:24:13.174 INFO:tasks.workunit.client.0.smithi070.stdout:5/941: creat dc/d1b/d1d/d25/d36/d32/d5c/f134 x:0 0 0 2024-03-23T08:24:13.185 INFO:tasks.workunit.client.0.smithi070.stdout:4/995: dwrite d3/d6/d35/d65/d27/d4b/fcb [0,4194304] 0 2024-03-23T08:24:13.188 INFO:tasks.workunit.client.0.smithi070.stdout:4/996: creat d3/d6/d35/d65/d27/d4b/d73/d139/f15b x:0 0 0 2024-03-23T08:24:13.189 INFO:tasks.workunit.client.0.smithi070.stdout:5/942: dwrite dc/d1b/d1d/d25/d68/d8a/f8e [0,4194304] 0 2024-03-23T08:24:13.192 INFO:tasks.workunit.client.0.smithi070.stdout:4/997: mkdir d3/d6/d11/d118/d15c 0 2024-03-23T08:24:13.196 INFO:tasks.workunit.client.0.smithi070.stdout:4/998: truncate d3/d6/d45/f47 8972506 0 2024-03-23T08:24:13.200 INFO:tasks.workunit.client.0.smithi070.stdout:4/999: truncate d3/d6/d35/d65/d27/d4b/fe0 708423 0 2024-03-23T08:24:13.211 INFO:tasks.workunit.client.0.smithi070.stdout:1/940: dread d8/d1b/f25 [0,4194304] 0 2024-03-23T08:24:13.212 INFO:tasks.workunit.client.0.smithi070.stdout:1/941: write d8/d26/f7f [4071161,21036] 0 2024-03-23T08:24:13.221 INFO:tasks.workunit.client.0.smithi070.stdout:1/942: write d8/d1a/d8f/fa3 [608654,8014] 0 2024-03-23T08:24:13.223 INFO:tasks.workunit.client.0.smithi070.stdout:5/943: dread dc/d1b/d1d/d25/d38/f86 [0,4194304] 0 2024-03-23T08:24:13.227 INFO:tasks.workunit.client.0.smithi070.stdout:1/943: dwrite d8/db/d40/d43/d64/d85/d97/f142 [0,4194304] 0 2024-03-23T08:24:13.230 INFO:tasks.workunit.client.0.smithi070.stdout:1/944: write d8/d26/d7b/f120 [521934,65819] 0 2024-03-23T08:24:13.231 INFO:tasks.workunit.client.0.smithi070.stdout:1/945: chown d8/db/d40/d43/d64/dc2/dfb/f114 14 1 2024-03-23T08:24:13.237 INFO:tasks.workunit.client.0.smithi070.stdout:5/944: link dc/d1b/d1d/d25/da9/deb/d107/lfb dc/d1b/d1d/d25/d36/l135 0 2024-03-23T08:24:13.237 INFO:tasks.workunit.client.0.smithi070.stdout:1/946: read d8/d26/d7b/d9c/fde [504347,86711] 0 2024-03-23T08:24:13.238 INFO:tasks.workunit.client.0.smithi070.stdout:1/947: write d8/d1b/f4b [4659080,56800] 0 2024-03-23T08:24:13.240 INFO:tasks.workunit.client.0.smithi070.stdout:1/948: write d8/db/d100/d74/d5d/fe2 [1887758,46737] 0 2024-03-23T08:24:13.241 INFO:tasks.workunit.client.0.smithi070.stdout:1/949: chown d8/d26/d7b/d9c/cec 2004247 1 2024-03-23T08:24:13.243 INFO:tasks.workunit.client.0.smithi070.stdout:5/945: rmdir dc/d1b/d1d/d25/dcd/d12f 0 2024-03-23T08:24:13.243 INFO:tasks.workunit.client.0.smithi070.stdout:5/946: readlink dc/l6c 0 2024-03-23T08:24:13.245 INFO:tasks.workunit.client.0.smithi070.stdout:1/950: creat d8/db/d2c/d3f/dd3/d143/f14a x:0 0 0 2024-03-23T08:24:13.247 INFO:tasks.workunit.client.0.smithi070.stdout:5/947: unlink dc/d1b/d1d/c30 0 2024-03-23T08:24:13.248 INFO:tasks.workunit.client.0.smithi070.stdout:1/951: mknod d8/d26/d49/c14b 0 2024-03-23T08:24:13.251 INFO:tasks.workunit.client.0.smithi070.stdout:1/952: mkdir d8/db/d40/d43/d64/dc2/d14c 0 2024-03-23T08:24:13.254 INFO:tasks.workunit.client.0.smithi070.stdout:5/948: dwrite dc/d1b/d1d/d25/d36/fd3 [0,4194304] 0 2024-03-23T08:24:13.274 INFO:tasks.workunit.client.0.smithi070.stdout:1/953: dwrite d8/d26/d41/f92 [0,4194304] 0 2024-03-23T08:24:13.278 INFO:tasks.workunit.client.0.smithi070.stdout:1/954: rename d8/d42/d4e/f13e to d8/db/d40/d43/d64/d85/d97/f14d 0 2024-03-23T08:24:13.289 INFO:tasks.workunit.client.0.smithi070.stdout:1/955: getdents d8/d26/d49 0 2024-03-23T08:24:13.291 INFO:tasks.workunit.client.0.smithi070.stdout:1/956: symlink d8/d42/d4e/l14e 0 2024-03-23T08:24:13.294 INFO:tasks.workunit.client.0.smithi070.stdout:1/957: creat d8/db/d40/d43/d64/d85/d97/d9a/d135/f14f x:0 0 0 2024-03-23T08:24:13.299 INFO:tasks.workunit.client.0.smithi070.stdout:1/958: rename d8/db/d40/d43/d64/dc2 to d8/db/d2c/d3f/d48/d150 0 2024-03-23T08:24:13.308 INFO:tasks.workunit.client.0.smithi070.stdout:1/959: link d8/d11/c12c d8/d26/d49/d65/c151 0 2024-03-23T08:24:13.310 INFO:tasks.workunit.client.0.smithi070.stdout:1/960: mkdir d8/db/d2c/d3f/d48/d150/d152 0 2024-03-23T08:24:13.313 INFO:tasks.workunit.client.0.smithi070.stdout:1/961: symlink d8/db/d2c/d3f/l153 0 2024-03-23T08:24:13.322 INFO:tasks.workunit.client.0.smithi070.stdout:1/962: write d8/db/d100/f12e [1539126,71057] 0 2024-03-23T08:24:13.324 INFO:tasks.workunit.client.0.smithi070.stdout:1/963: dread d8/db/d100/f95 [0,4194304] 0 2024-03-23T08:24:13.330 INFO:tasks.workunit.client.0.smithi070.stdout:1/964: dwrite d8/db/f3e [0,4194304] 0 2024-03-23T08:24:13.395 INFO:tasks.workunit.client.0.smithi070.stdout:1/965: mkdir d8/db/d40/d154 0 2024-03-23T08:24:13.417 INFO:tasks.workunit.client.0.smithi070.stdout:1/966: write d8/d42/d4e/d99/ff6 [249574,121264] 0 2024-03-23T08:24:13.418 INFO:tasks.workunit.client.0.smithi070.stdout:1/967: dread - d8/db/d40/d43/d64/d85/d97/f144 zero size 2024-03-23T08:24:13.424 INFO:tasks.workunit.client.0.smithi070.stdout:1/968: getdents d8/db/d40/d43/d64/d85/d97/d9a 0 2024-03-23T08:24:13.427 INFO:tasks.workunit.client.0.smithi070.stdout:1/969: dread d8/d42/d51/d11f/f139 [0,4194304] 0 2024-03-23T08:24:13.430 INFO:tasks.workunit.client.0.smithi070.stdout:1/970: mknod d8/db/d100/d74/d5d/d70/d11c/c155 0 2024-03-23T08:24:13.431 INFO:tasks.workunit.client.0.smithi070.stdout:1/971: dread - d8/d1a/fb0 zero size 2024-03-23T08:24:13.433 INFO:tasks.workunit.client.0.smithi070.stdout:1/972: symlink d8/db/d8e/l156 0 2024-03-23T08:24:13.438 INFO:tasks.workunit.client.0.smithi070.stdout:1/973: dwrite d8/d26/d7b/fe0 [0,4194304] 0 2024-03-23T08:24:13.448 INFO:tasks.workunit.client.0.smithi070.stdout:1/974: dwrite d8/d1b/f25 [0,4194304] 0 2024-03-23T08:24:13.449 INFO:tasks.workunit.client.0.smithi070.stdout:1/975: readlink d8/d42/l60 0 2024-03-23T08:24:13.453 INFO:tasks.workunit.client.0.smithi070.stdout:1/976: mkdir d8/db/d2c/d3f/d48/d150/d152/d157 0 2024-03-23T08:24:13.459 INFO:tasks.workunit.client.0.smithi070.stdout:1/977: dwrite d8/d26/f7f [0,4194304] 0 2024-03-23T08:24:13.464 INFO:tasks.workunit.client.0.smithi070.stdout:1/978: rename d8/ld to d8/db/d8e/l158 0 2024-03-23T08:24:13.469 INFO:tasks.workunit.client.0.smithi070.stdout:1/979: mkdir d8/db/d2c/d159 0 2024-03-23T08:24:13.478 INFO:tasks.workunit.client.0.smithi070.stdout:1/980: truncate d8/d42/d4e/ff7 2069054 0 2024-03-23T08:24:13.495 INFO:tasks.workunit.client.0.smithi070.stdout:1/981: dread d8/d11/f3b [0,4194304] 0 2024-03-23T08:24:13.497 INFO:tasks.workunit.client.0.smithi070.stdout:1/982: mkdir d8/d15a 0 2024-03-23T08:24:13.501 INFO:tasks.workunit.client.0.smithi070.stdout:1/983: getdents d8/d42/d51/d11f 0 2024-03-23T08:24:13.506 INFO:tasks.workunit.client.0.smithi070.stdout:1/984: link d8/db/d2c/d3f/dd3/f149 d8/db/d100/d74/d5d/d70/f15b 0 2024-03-23T08:24:13.509 INFO:tasks.workunit.client.0.smithi070.stdout:1/985: mknod d8/db/d2c/d3f/dd3/d143/c15c 0 2024-03-23T08:24:13.510 INFO:tasks.workunit.client.0.smithi070.stdout:1/986: creat d8/d26/d49/d65/de8/f15d x:0 0 0 2024-03-23T08:24:13.512 INFO:tasks.workunit.client.0.smithi070.stdout:1/987: fdatasync d8/d1a/f1e 0 2024-03-23T08:24:13.517 INFO:tasks.workunit.client.0.smithi070.stdout:1/988: mkdir d8/d15a/d15e 0 2024-03-23T08:24:13.525 INFO:tasks.workunit.client.0.smithi070.stdout:1/989: write d8/db/d40/f84 [1602722,73024] 0 2024-03-23T08:24:13.533 INFO:tasks.workunit.client.0.smithi070.stdout:1/990: write d8/db/d2c/f3c [1734884,118] 0 2024-03-23T08:24:13.543 INFO:tasks.workunit.client.0.smithi070.stdout:1/991: dread d8/d11/f29 [0,4194304] 0 2024-03-23T08:24:13.543 INFO:tasks.workunit.client.0.smithi070.stdout:1/992: chown d8/db 173 1 2024-03-23T08:24:13.547 INFO:tasks.workunit.client.0.smithi070.stdout:1/993: dread d8/d26/d41/f92 [0,4194304] 0 2024-03-23T08:24:13.554 INFO:tasks.workunit.client.0.smithi070.stdout:5/949: sync 2024-03-23T08:24:13.559 INFO:tasks.workunit.client.0.smithi070.stdout:5/950: getdents dc/d1b/d1d/d25/da9/deb 0 2024-03-23T08:24:13.561 INFO:tasks.workunit.client.0.smithi070.stdout:5/951: fdatasync dc/d1b/d1d/d25/d36/d32/f87 0 2024-03-23T08:24:13.563 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:13 smithi070 ceph-mon[39699]: pgmap v46: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 58 MiB/s rd, 136 MiB/s wr, 450 op/s 2024-03-23T08:24:13.567 INFO:tasks.workunit.client.0.smithi070.stdout:5/952: write dc/d1b/d1d/d25/d38/d39/f79 [5261457,3826] 0 2024-03-23T08:24:13.578 INFO:tasks.workunit.client.0.smithi070.stdout:5/953: dwrite dc/d1b/d1d/d25/d38/d39/f10c [0,4194304] 0 2024-03-23T08:24:13.582 INFO:tasks.workunit.client.0.smithi070.stdout:5/954: write dc/fa6 [41258,109518] 0 2024-03-23T08:24:13.590 INFO:tasks.workunit.client.0.smithi070.stdout:5/955: write dc/d1b/d1d/d25/d68/d124/fef [1613592,83888] 0 2024-03-23T08:24:13.591 INFO:tasks.workunit.client.0.smithi070.stdout:5/956: fdatasync dc/d1b/d1d/d25/d38/d53/f9a 0 2024-03-23T08:24:13.592 INFO:tasks.workunit.client.0.smithi070.stdout:5/957: chown dc/d1b/d1d/d25/da9/deb/d107/dc8/caa 2 1 2024-03-23T08:24:13.596 INFO:tasks.workunit.client.0.smithi070.stdout:5/958: creat dc/d1b/d1d/d25/d68/d8a/d99/dd7/d12c/f136 x:0 0 0 2024-03-23T08:24:13.598 INFO:tasks.workunit.client.0.smithi070.stdout:5/959: unlink dc/d1b/d1d/d25/d38/la5 0 2024-03-23T08:24:13.599 INFO:tasks.workunit.client.0.smithi070.stdout:5/960: fsync dc/d1b/d1d/d25/d36/d32/d5c/fab 0 2024-03-23T08:24:13.600 INFO:tasks.workunit.client.0.smithi070.stdout:5/961: dread - dc/d1b/d1d/d25/d36/d32/d5c/dcb/f11f zero size 2024-03-23T08:24:13.601 INFO:tasks.workunit.client.0.smithi070.stdout:5/962: truncate dc/d1b/d1d/d25/d38/d53/f10d 222938 0 2024-03-23T08:24:13.794 INFO:tasks.workunit.client.0.smithi070.stdout:1/994: sync 2024-03-23T08:24:13.794 INFO:tasks.workunit.client.0.smithi070.stdout:1/995: write d8/d11/f3b [1880585,120163] 0 2024-03-23T08:24:13.798 INFO:tasks.workunit.client.0.smithi070.stdout:1/996: rename d8/db/d100/d74/l112 to d8/db/l15f 0 2024-03-23T08:24:13.801 INFO:tasks.workunit.client.0.smithi070.stdout:1/997: mknod d8/d15a/d15e/c160 0 2024-03-23T08:24:13.808 INFO:tasks.workunit.client.0.smithi070.stdout:1/998: dwrite d8/db/d40/d43/d64/d85/d97/dfd/f113 [0,4194304] 0 2024-03-23T08:24:13.811 INFO:tasks.workunit.client.0.smithi070.stdout:1/999: mknod d8/d26/d7b/d9c/c161 0 2024-03-23T08:24:13.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:13 smithi193 ceph-mon[48370]: pgmap v46: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 58 MiB/s rd, 136 MiB/s wr, 450 op/s 2024-03-23T08:24:13.892 INFO:tasks.workunit.client.0.smithi070.stdout:5/963: sync 2024-03-23T08:24:13.892 INFO:tasks.workunit.client.0.smithi070.stdout:5/964: chown dc/d1b/d1d/d25/d36/d32/d11a 0 1 2024-03-23T08:24:13.897 INFO:tasks.workunit.client.0.smithi070.stdout:5/965: dread dc/d1b/d1d/d25/d38/d39/f79 [4194304,4194304] 0 2024-03-23T08:24:13.898 INFO:tasks.workunit.client.0.smithi070.stdout:5/966: fsync dc/d1b/d1d/d25/d36/dfc/d11b/f12a 0 2024-03-23T08:24:13.899 INFO:tasks.workunit.client.0.smithi070.stdout:5/967: write dc/d1b/d1d/d25/d68/d124/f10e [361861,65191] 0 2024-03-23T08:24:13.902 INFO:tasks.workunit.client.0.smithi070.stdout:5/968: write dc/d1b/d1d/d25/d68/f10b [3218089,26465] 0 2024-03-23T08:24:13.912 INFO:tasks.workunit.client.0.smithi070.stdout:5/969: link dc/d1b/d1d/d25/d36/f2a dc/d1b/d1d/d25/d36/d32/d5c/dcb/dec/f137 0 2024-03-23T08:24:13.917 INFO:tasks.workunit.client.0.smithi070.stdout:5/970: mkdir dc/d1b/d1d/d25/d36/d32/d5c/d138 0 2024-03-23T08:24:13.925 INFO:tasks.workunit.client.0.smithi070.stdout:5/971: mknod dc/d1b/d1d/d25/d36/d32/d5c/c139 0 2024-03-23T08:24:13.925 INFO:tasks.workunit.client.0.smithi070.stdout:5/972: fsync dc/d1b/d1d/d25/d68/d8a/f8e 0 2024-03-23T08:24:13.929 INFO:tasks.workunit.client.0.smithi070.stdout:5/973: write dc/d1b/d1d/d25/d38/d39/f10c [4802123,59010] 0 2024-03-23T08:24:13.936 INFO:tasks.workunit.client.0.smithi070.stdout:5/974: mkdir dc/d1b/d1d/d25/d68/d8a/dad/df8/d13a 0 2024-03-23T08:24:13.940 INFO:tasks.workunit.client.0.smithi070.stdout:5/975: dread - dc/d1b/d1d/d25/d68/d124/de6/f102 zero size 2024-03-23T08:24:13.946 INFO:tasks.workunit.client.0.smithi070.stdout:5/976: chown dc/d1b/d1d/d25/d68/d8a/dad/df8/ffa 15 1 2024-03-23T08:24:13.951 INFO:tasks.workunit.client.0.smithi070.stdout:5/977: chown dc/d1b/d1d/d25/d68/d8a/d99/lc5 33580363 1 2024-03-23T08:24:13.960 INFO:tasks.workunit.client.0.smithi070.stdout:5/978: stat dc/d1b/d1d/d25/d36/d32/c81 0 2024-03-23T08:24:13.967 INFO:tasks.workunit.client.0.smithi070.stdout:5/979: mknod dc/d1b/d1d/d25/d36/d32/d5c/d10a/c13b 0 2024-03-23T08:24:13.976 INFO:tasks.workunit.client.0.smithi070.stdout:5/980: creat dc/d1b/d1d/d25/da9/deb/d107/dc8/f13c x:0 0 0 2024-03-23T08:24:13.987 INFO:tasks.workunit.client.0.smithi070.stdout:5/981: creat dc/d1b/d1d/d25/d68/d8a/f13d x:0 0 0 2024-03-23T08:24:13.989 INFO:tasks.workunit.client.0.smithi070.stdout:5/982: dread - dc/d1b/d1d/d25/d36/dfc/d11b/f12a zero size 2024-03-23T08:24:14.002 INFO:tasks.workunit.client.0.smithi070.stdout:5/983: dread dc/d1b/d1d/d25/d36/d32/d5c/fab [0,4194304] 0 2024-03-23T08:24:14.063 INFO:tasks.workunit.client.0.smithi070.stdout:5/984: symlink dc/d1b/d1d/d25/d36/dfc/l13e 0 2024-03-23T08:24:14.068 INFO:tasks.workunit.client.0.smithi070.stdout:5/985: creat dc/d1b/d1d/d25/da9/deb/d107/f13f x:0 0 0 2024-03-23T08:24:14.102 INFO:tasks.workunit.client.0.smithi070.stdout:5/986: truncate dc/d1b/d1d/d25/d36/d32/d5c/d10a/fff 1434544 0 2024-03-23T08:24:14.108 INFO:tasks.workunit.client.0.smithi070.stdout:5/987: dwrite dc/d1b/d1d/d25/d36/dfc/d11b/f12a [0,4194304] 0 2024-03-23T08:24:14.116 INFO:tasks.workunit.client.0.smithi070.stdout:5/988: dwrite dc/d1b/d1d/d25/d36/dfc/d11b/f12a [0,4194304] 0 2024-03-23T08:24:14.118 INFO:tasks.workunit.client.0.smithi070.stdout:5/989: fdatasync dc/d1b/d1d/d25/d36/d32/d5c/f65 0 2024-03-23T08:24:14.120 INFO:tasks.workunit.client.0.smithi070.stdout:5/990: read dc/d1b/d1d/d25/d68/d8a/d99/dd7/f116 [539562,46650] 0 2024-03-23T08:24:14.122 INFO:tasks.workunit.client.0.smithi070.stdout:5/991: creat dc/f140 x:0 0 0 2024-03-23T08:24:14.129 INFO:tasks.workunit.client.0.smithi070.stdout:5/992: link dc/d1b/d1d/d25/d68/cda dc/d1b/d1d/d25/d68/d8a/dad/df8/d13a/c141 0 2024-03-23T08:24:14.130 INFO:tasks.workunit.client.0.smithi070.stdout:5/993: dread - dc/d1b/d1d/d25/da9/deb/f10f zero size 2024-03-23T08:24:14.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 -- 172.21.15.70:0/3856872285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 msgr2=0x7f17c00fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 --2- 172.21.15.70:0/3856872285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 0x7f17c00fd400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f17b0004940 tx=0x7f17b0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 -- 172.21.15.70:0/3856872285 shutdown_connections 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 --2- 172.21.15.70:0/3856872285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 0x7f17c00fd400 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 --2- 172.21.15.70:0/3856872285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17c00fbce0 0x7f17c00fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 -- 172.21.15.70:0/3856872285 >> 172.21.15.70:0/3856872285 conn(0x7f17c00f7480 msgr2=0x7f17c00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 -- 172.21.15.70:0/3856872285 shutdown_connections 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 -- 172.21.15.70:0/3856872285 wait complete. 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 Processor -- start 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.328+0000 7f17c8759700 1 -- start start 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c8759700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17c00fbce0 0x7f17c00ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c8759700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 0x7f17c00fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c8759700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00fe810 con 0x7f17c00fcf90 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c8759700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00fe950 con 0x7f17c00fbce0 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c5cf4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 0x7f17c00fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c5cf4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 0x7f17c00fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50756/0 (socket says 172.21.15.70:50756) 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c5cf4700 1 -- 172.21.15.70:0/729075062 learned_addr learned my addr 172.21.15.70:0/729075062 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c64f5700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17c00fbce0 0x7f17c00ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c5cf4700 1 -- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17c00fbce0 msgr2=0x7f17c00ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c5cf4700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17c00fbce0 0x7f17c00ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c5cf4700 1 -- 172.21.15.70:0/729075062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0005040 con 0x7f17c00fcf90 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c64f5700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17c00fbce0 0x7f17c00ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c5cf4700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 0x7f17c00fe2d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f17b0007d40 tx=0x7f17b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17b6ffd700 1 -- 172.21.15.70:0/729075062 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b000e030 con 0x7f17c00fcf90 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17b6ffd700 1 -- 172.21.15.70:0/729075062 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17b00093f0 con 0x7f17c00fcf90 2024-03-23T08:24:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c00febd0 con 0x7f17c00fcf90 2024-03-23T08:24:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.329+0000 7f17b6ffd700 1 -- 172.21.15.70:0/729075062 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b000b410 con 0x7f17c00fcf90 2024-03-23T08:24:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.330+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c0115940 con 0x7f17c00fcf90 2024-03-23T08:24:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.331+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c0101b20 con 0x7f17c00fcf90 2024-03-23T08:24:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.331+0000 7f17b6ffd700 1 -- 172.21.15.70:0/729075062 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f17b001c400 con 0x7f17c00fcf90 2024-03-23T08:24:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.331+0000 7f17b6ffd700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f17ac06aec0 0x7f17ac06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.332+0000 7f17b6ffd700 1 -- 172.21.15.70:0/729075062 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f17b008ae10 con 0x7f17c00fcf90 2024-03-23T08:24:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.332+0000 7f17c64f5700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f17ac06aec0 0x7f17ac06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.332+0000 7f17c64f5700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f17ac06aec0 0x7f17ac06d370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f17c00ffaf0 tx=0x7f17bc00b040 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:14.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.338+0000 7f17b6ffd700 1 -- 172.21.15.70:0/729075062 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f17b0055380 con 0x7f17c00fcf90 2024-03-23T08:24:14.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.586+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17c01047f0 con 0x7f17ac06aec0 2024-03-23T08:24:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.588+0000 7f17b6ffd700 1 -- 172.21.15.70:0/729075062 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f17c01047f0 con 0x7f17ac06aec0 2024-03-23T08:24:14.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f17ac06aec0 msgr2=0x7f17ac06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f17ac06aec0 0x7f17ac06d370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f17c00ffaf0 tx=0x7f17bc00b040 comp rx=0 tx=0).stop 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 msgr2=0x7f17c00fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 0x7f17c00fe2d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f17b0007d40 tx=0x7f17b0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 shutdown_connections 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f17ac06aec0 0x7f17ac06d370 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17c00fbce0 0x7f17c00ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 --2- 172.21.15.70:0/729075062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17c00fcf90 0x7f17c00fe2d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 >> 172.21.15.70:0/729075062 conn(0x7f17c00f7480 msgr2=0x7f17c0066460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 shutdown_connections 2024-03-23T08:24:14.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.590+0000 7f17c8759700 1 -- 172.21.15.70:0/729075062 wait complete. 2024-03-23T08:24:14.601 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:24:14.621 INFO:tasks.workunit.client.0.smithi070.stdout:5/994: sync 2024-03-23T08:24:14.625 INFO:tasks.workunit.client.0.smithi070.stdout:5/995: creat dc/d1b/d1d/d25/d68/d124/de6/df0/f142 x:0 0 0 2024-03-23T08:24:14.626 INFO:tasks.workunit.client.0.smithi070.stdout:5/996: write dc/d1b/d1d/d25/d68/d8a/fa3 [265175,56296] 0 2024-03-23T08:24:14.628 INFO:tasks.workunit.client.0.smithi070.stdout:5/997: symlink dc/d1b/d1d/d25/dcd/l143 0 2024-03-23T08:24:14.629 INFO:tasks.workunit.client.0.smithi070.stdout:5/998: chown dc/d1b/d1d/d25/da9/deb/d107 23 1 2024-03-23T08:24:14.631 INFO:tasks.workunit.client.0.smithi070.stdout:5/999: mkdir dc/d1b/d1d/d25/d68/d8a/d99/d144 0 2024-03-23T08:24:14.635 INFO:tasks.workunit.client.0.smithi070.stderr:+ rm -rf -- ./tmp.r0PBGTPYZv 2024-03-23T08:24:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 -- 172.21.15.70:0/1621456849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9700fe400 msgr2=0x7fb9700fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 --2- 172.21.15.70:0/1621456849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9700fe400 0x7fb9700fe810 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb964004940 tx=0x7fb964009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 -- 172.21.15.70:0/1621456849 shutdown_connections 2024-03-23T08:24:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 --2- 172.21.15.70:0/1621456849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9700fed50 0x7fb9700fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 --2- 172.21.15.70:0/1621456849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9700fe400 0x7fb9700fe810 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 -- 172.21.15.70:0/1621456849 >> 172.21.15.70:0/1621456849 conn(0x7fb9700f6ba0 msgr2=0x7fb9700f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 -- 172.21.15.70:0/1621456849 shutdown_connections 2024-03-23T08:24:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 -- 172.21.15.70:0/1621456849 wait complete. 2024-03-23T08:24:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 Processor -- start 2024-03-23T08:24:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.722+0000 7fb976cf1700 1 -- start start 2024-03-23T08:24:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb976cf1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9700fe400 0x7fb9700fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb976cf1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9700fed50 0x7fb9700fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb976cf1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9700fc670 con 0x7fb9700fed50 2024-03-23T08:24:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb976cf1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9700fc7e0 con 0x7fb9700fe400 2024-03-23T08:24:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb96ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9700fed50 0x7fb9700fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb96ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9700fed50 0x7fb9700fc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50776/0 (socket says 172.21.15.70:50776) 2024-03-23T08:24:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb974a8d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9700fe400 0x7fb9700fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb96ffff700 1 -- 172.21.15.70:0/77227227 learned_addr learned my addr 172.21.15.70:0/77227227 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb974a8d700 1 -- 172.21.15.70:0/77227227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9700fed50 msgr2=0x7fb9700fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb974a8d700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9700fed50 0x7fb9700fc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb974a8d700 1 -- 172.21.15.70:0/77227227 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964005040 con 0x7fb9700fe400 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.723+0000 7fb974a8d700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9700fe400 0x7fb9700fdbf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c00 tx=0x7fb964009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.724+0000 7fb96d7fa700 1 -- 172.21.15.70:0/77227227 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb96400e070 con 0x7fb9700fe400 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.724+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9700fca60 con 0x7fb9700fe400 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.724+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9701158f0 con 0x7fb9700fe400 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.724+0000 7fb96d7fa700 1 -- 172.21.15.70:0/77227227 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb964026460 con 0x7fb9700fe400 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.724+0000 7fb96d7fa700 1 -- 172.21.15.70:0/77227227 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb96401c640 con 0x7fb9700fe400 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.725+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97005f5f0 con 0x7fb9700fe400 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.725+0000 7fb96d7fa700 1 -- 172.21.15.70:0/77227227 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fb964026a00 con 0x7fb9700fe400 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.726+0000 7fb96d7fa700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb96006aec0 0x7fb96006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.726+0000 7fb96d7fa700 1 -- 172.21.15.70:0/77227227 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb96408ad80 con 0x7fb9700fe400 2024-03-23T08:24:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.726+0000 7fb96ffff700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb96006aec0 0x7fb96006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.726+0000 7fb96ffff700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb96006aec0 0x7fb96006d370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0041d0 tx=0x7fb95c00b040 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:14.731+0000 7fb96d7fa700 1 -- 172.21.15.70:0/77227227 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fb964055290 con 0x7fb9700fe400 2024-03-23T08:24:15.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.030+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb970102540 con 0x7fb96006aec0 2024-03-23T08:24:15.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.031+0000 7fb96d7fa700 1 -- 172.21.15.70:0/77227227 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fb970102540 con 0x7fb96006aec0 2024-03-23T08:24:15.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb96006aec0 msgr2=0x7fb96006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb96006aec0 0x7fb96006d370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0041d0 tx=0x7fb95c00b040 comp rx=0 tx=0).stop 2024-03-23T08:24:15.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9700fe400 msgr2=0x7fb9700fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9700fe400 0x7fb9700fdbf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c00 tx=0x7fb964009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:15.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 shutdown_connections 2024-03-23T08:24:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb96006aec0 0x7fb96006d370 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9700fe400 0x7fb9700fdbf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 --2- 172.21.15.70:0/77227227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9700fed50 0x7fb9700fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 >> 172.21.15.70:0/77227227 conn(0x7fb9700f6ba0 msgr2=0x7fb9700f8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 shutdown_connections 2024-03-23T08:24:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.034+0000 7fb976cf1700 1 -- 172.21.15.70:0/77227227 wait complete. 2024-03-23T08:24:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.171+0000 7fbdc625c700 1 -- 172.21.15.70:0/2215314288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc00fce40 msgr2=0x7fbdc00fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.171+0000 7fbdc625c700 1 --2- 172.21.15.70:0/2215314288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc00fce40 0x7fbdc00fd250 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0004940 tx=0x7fbdb0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.171+0000 7fbdc625c700 1 -- 172.21.15.70:0/2215314288 shutdown_connections 2024-03-23T08:24:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.171+0000 7fbdc625c700 1 --2- 172.21.15.70:0/2215314288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc00fe040 0x7fbdc00fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.171+0000 7fbdc625c700 1 --2- 172.21.15.70:0/2215314288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc00fce40 0x7fbdc00fd250 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.171+0000 7fbdc625c700 1 -- 172.21.15.70:0/2215314288 >> 172.21.15.70:0/2215314288 conn(0x7fbdc00f8510 msgr2=0x7fbdc00fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.171+0000 7fbdc625c700 1 -- 172.21.15.70:0/2215314288 shutdown_connections 2024-03-23T08:24:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.171+0000 7fbdc625c700 1 -- 172.21.15.70:0/2215314288 wait complete. 2024-03-23T08:24:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdc625c700 1 Processor -- start 2024-03-23T08:24:15.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdc625c700 1 -- start start 2024-03-23T08:24:15.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdc625c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc00fce40 0x7fbdc010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:15.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdc625c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc00fe040 0x7fbdc010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:15.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdc625c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc010e4a0 con 0x7fbdc00fce40 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdc625c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc010eca0 con 0x7fbdc00fe040 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdbf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc00fce40 0x7fbdc010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdbeffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc00fe040 0x7fbdc010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdbeffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc00fe040 0x7fbdc010dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:59588/0 (socket says 172.21.15.70:59588) 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.172+0000 7fbdbeffd700 1 -- 172.21.15.70:0/6402572 learned_addr learned my addr 172.21.15.70:0/6402572 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbdbeffd700 1 -- 172.21.15.70:0/6402572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc00fce40 msgr2=0x7fbdc010d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbdbeffd700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc00fce40 0x7fbdc010d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbdbeffd700 1 -- 172.21.15.70:0/6402572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb0005040 con 0x7fbdc00fe040 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbdbf7fe700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc00fce40 0x7fbdc010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbdbeffd700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc00fe040 0x7fbdc010dea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4002a50 tx=0x7fbdb40046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbda7fff700 1 -- 172.21.15.70:0/6402572 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdb4007cc0 con 0x7fbdc00fe040 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc010ef80 con 0x7fbdc00fe040 2024-03-23T08:24:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc010f4c0 con 0x7fbdc00fe040 2024-03-23T08:24:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbda7fff700 1 -- 172.21.15.70:0/6402572 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbdb400f040 con 0x7fbdc00fe040 2024-03-23T08:24:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.173+0000 7fbda7fff700 1 -- 172.21.15.70:0/6402572 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdb400d9c0 con 0x7fbdc00fe040 2024-03-23T08:24:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.174+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdac0052e0 con 0x7fbdc00fe040 2024-03-23T08:24:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.175+0000 7fbda7fff700 1 -- 172.21.15.70:0/6402572 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fbdb400db40 con 0x7fbdc00fe040 2024-03-23T08:24:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.175+0000 7fbda7fff700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fbda806aec0 0x7fbda806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.176+0000 7fbda7fff700 1 -- 172.21.15.70:0/6402572 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fbdb40894b0 con 0x7fbdc00fe040 2024-03-23T08:24:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.176+0000 7fbdbf7fe700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fbda806aec0 0x7fbda806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.176+0000 7fbdbf7fe700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fbda806aec0 0x7fbda806d370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000b900 tx=0x7fbdb0009e30 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:15.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.180+0000 7fbda7fff700 1 -- 172.21.15.70:0/6402572 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fbdb4057400 con 0x7fbdc00fe040 2024-03-23T08:24:15.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.457+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdac000bf0 con 0x7fbda806aec0 2024-03-23T08:24:15.468 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.468+0000 7fbda7fff700 1 -- 172.21.15.70:0/6402572 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fbdac000bf0 con 0x7fbda806aec0 2024-03-23T08:24:15.468 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:24:15.468 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 starting - - - - 2024-03-23T08:24:15.468 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (9m) 73s ago 9m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:24:15.468 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (8m) 83s ago 8m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:24:15.468 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (7m) 73s ago 8m 92.4M - 9.4.7 954c08fa6188 87fc95a6ce8f 2024-03-23T08:24:15.468 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (3m) 73s ago 3m 107M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:24:15.468 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (3m) 73s ago 3m 11.2M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (3m) 83s ago 3m 69.7M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (3m) 83s ago 3m 12.9M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:9283 running (10m) 73s ago 10m 178M - 17.2.7-726-g9c4e0c34 837124da804f 85d0a92e59bc 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283 running (99s) 83s ago 8m 481M - 18.2.2-954-g82316b04 b24e1baba637 d2827fb230ee 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (10m) 73s ago 10m 51.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (8m) 83s ago 8m 39.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 starting - - - - 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (8m) 83s ago 8m 18.6M - 1.5.0 0da6a335fe13 4197d838b05a 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (7m) 73s ago 7m 94.5M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (6m) 73s ago 6m 102M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (6m) 73s ago 6m 89.0M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (5m) 83s ago 5m 111M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (5m) 83s ago 5m 95.2M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (5m) 83s ago 5m 94.1M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:24:15.469 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (7m) 73s ago 8m 58.2M - 2.43.0 a07b618ecd1d e842f03864f0 2024-03-23T08:24:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fbda806aec0 msgr2=0x7fbda806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fbda806aec0 0x7fbda806d370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000b900 tx=0x7fbdb0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc00fe040 msgr2=0x7fbdc010dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc00fe040 0x7fbdc010dea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4002a50 tx=0x7fbdb40046b0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 shutdown_connections 2024-03-23T08:24:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fbda806aec0 0x7fbda806d370 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc00fce40 0x7fbdc010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 --2- 172.21.15.70:0/6402572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc00fe040 0x7fbdc010dea0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 >> 172.21.15.70:0/6402572 conn(0x7fbdc00f8510 msgr2=0x7fbdc00f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 shutdown_connections 2024-03-23T08:24:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.471+0000 7fbdc625c700 1 -- 172.21.15.70:0/6402572 wait complete. 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.609+0000 7f8c075da700 1 -- 172.21.15.70:0/524775495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c00066b10 msgr2=0x7f8c000ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.609+0000 7f8c075da700 1 --2- 172.21.15.70:0/524775495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c00066b10 0x7f8c000ffab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc004940 tx=0x7f8bfc009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.609+0000 7f8c075da700 1 -- 172.21.15.70:0/524775495 shutdown_connections 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.609+0000 7f8c075da700 1 --2- 172.21.15.70:0/524775495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c000ffff0 0x7f8c001023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.609+0000 7f8c075da700 1 --2- 172.21.15.70:0/524775495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c00066b10 0x7f8c000ffab0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.609+0000 7f8c075da700 1 -- 172.21.15.70:0/524775495 >> 172.21.15.70:0/524775495 conn(0x7f8c000f74e0 msgr2=0x7f8c000f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.609+0000 7f8c075da700 1 -- 172.21.15.70:0/524775495 shutdown_connections 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.609+0000 7f8c075da700 1 -- 172.21.15.70:0/524775495 wait complete. 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c075da700 1 Processor -- start 2024-03-23T08:24:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c075da700 1 -- start start 2024-03-23T08:24:15.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c075da700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c00066b10 0x7f8c0010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:15.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c075da700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c000ffff0 0x7f8c0010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c075da700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0010e460 con 0x7f8c000ffff0 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c075da700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0010ecb0 con 0x7f8c00066b10 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c04b75700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c000ffff0 0x7f8c0010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c04b75700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c000ffff0 0x7f8c0010de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50808/0 (socket says 172.21.15.70:50808) 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c04b75700 1 -- 172.21.15.70:0/147999074 learned_addr learned my addr 172.21.15.70:0/147999074 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.610+0000 7f8c05376700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c00066b10 0x7f8c0010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8c04b75700 1 -- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c00066b10 msgr2=0x7f8c0010d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8c04b75700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c00066b10 0x7f8c0010d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8c04b75700 1 -- 172.21.15.70:0/147999074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc005040 con 0x7f8c000ffff0 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8c05376700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c00066b10 0x7f8c0010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8c04b75700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c000ffff0 0x7f8c0010de60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000eb40 tx=0x7f8bf000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8bf5ffb700 1 -- 172.21.15.70:0/147999074 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf0007cc0 con 0x7f8c000ffff0 2024-03-23T08:24:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8bf5ffb700 1 -- 172.21.15.70:0/147999074 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8bf000f040 con 0x7f8c000ffff0 2024-03-23T08:24:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0010ef30 con 0x7f8c000ffff0 2024-03-23T08:24:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8bf5ffb700 1 -- 172.21.15.70:0/147999074 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf000c990 con 0x7f8c000ffff0 2024-03-23T08:24:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.611+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0010f470 con 0x7f8c000ffff0 2024-03-23T08:24:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.613+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0005f5f0 con 0x7f8c000ffff0 2024-03-23T08:24:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.613+0000 7f8bf5ffb700 1 -- 172.21.15.70:0/147999074 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f8bf000cc00 con 0x7f8c000ffff0 2024-03-23T08:24:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.613+0000 7f8bf5ffb700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8bec06f2d0 0x7f8bec071780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.613+0000 7f8bf5ffb700 1 -- 172.21.15.70:0/147999074 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8bf0089490 con 0x7f8c000ffff0 2024-03-23T08:24:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.614+0000 7f8c05376700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8bec06f2d0 0x7f8bec071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.614+0000 7f8c05376700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8bec06f2d0 0x7f8bec071780 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0137d0 tx=0x7f8bfc009e30 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.618+0000 7f8bf5ffb700 1 -- 172.21.15.70:0/147999074 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f8bf004f560 con 0x7f8c000ffff0 2024-03-23T08:24:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:15 smithi070 ceph-mon[39699]: pgmap v47: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 46 MiB/s rd, 106 MiB/s wr, 373 op/s 2024-03-23T08:24:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:15 smithi070 ceph-mon[39699]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:15.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:15 smithi193 ceph-mon[48370]: pgmap v47: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 46 MiB/s rd, 106 MiB/s wr, 373 op/s 2024-03-23T08:24:15.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:15 smithi193 ceph-mon[48370]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:15.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.985+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8c00105790 con 0x7f8c000ffff0 2024-03-23T08:24:15.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.986+0000 7f8bf5ffb700 1 -- 172.21.15.70:0/147999074 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+848 (secure 0 0 0) 0x7f8bf00573a0 con 0x7f8c000ffff0 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:24:15.990 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:24:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:24:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:24:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:24:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:24:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:24:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-23T08:24:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:24:15.991 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:24:15.991 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:24:15.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.990+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8bec06f2d0 msgr2=0x7f8bec071780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.990+0000 7f8c075da700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8bec06f2d0 0x7f8bec071780 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0137d0 tx=0x7f8bfc009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:15.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.990+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c000ffff0 msgr2=0x7f8c0010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:15.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.990+0000 7f8c075da700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c000ffff0 0x7f8c0010de60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000eb40 tx=0x7f8bf000b040 comp rx=0 tx=0).stop 2024-03-23T08:24:15.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.991+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 shutdown_connections 2024-03-23T08:24:15.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.991+0000 7f8c075da700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7f8bec06f2d0 0x7f8bec071780 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.991+0000 7f8c075da700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c00066b10 0x7f8c0010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.991+0000 7f8c075da700 1 --2- 172.21.15.70:0/147999074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c000ffff0 0x7f8c0010de60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:15.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.991+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 >> 172.21.15.70:0/147999074 conn(0x7f8c000f74e0 msgr2=0x7f8c000f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:15.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.993+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 shutdown_connections 2024-03-23T08:24:15.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:15.994+0000 7f8c075da700 1 -- 172.21.15.70:0/147999074 wait complete. 2024-03-23T08:24:16.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.130+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/4178052321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00fff60 msgr2=0x7fb3e0102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.130+0000 7fb3e7e9d700 1 --2- 172.21.15.70:0/4178052321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00fff60 0x7fb3e0102340 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0004940 tx=0x7fb3d0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.130+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/4178052321 shutdown_connections 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.130+0000 7fb3e7e9d700 1 --2- 172.21.15.70:0/4178052321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00fff60 0x7fb3e0102340 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.130+0000 7fb3e7e9d700 1 --2- 172.21.15.70:0/4178052321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3e0066280 0x7fb3e00ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.130+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/4178052321 >> 172.21.15.70:0/4178052321 conn(0x7fb3e00f7430 msgr2=0x7fb3e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.130+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/4178052321 shutdown_connections 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/4178052321 wait complete. 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e7e9d700 1 Processor -- start 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e7e9d700 1 -- start start 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e7e9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e0066280 0x7fb3e00fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e7e9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3e00fff60 0x7fb3e00fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e7e9d700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00fbed0 con 0x7fb3e0066280 2024-03-23T08:24:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e7e9d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00fc040 con 0x7fb3e00fff60 2024-03-23T08:24:16.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e5c39700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e0066280 0x7fb3e00fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:16.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e5438700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3e00fff60 0x7fb3e00fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:16.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e5c39700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e0066280 0x7fb3e00fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50814/0 (socket says 172.21.15.70:50814) 2024-03-23T08:24:16.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.131+0000 7fb3e5c39700 1 -- 172.21.15.70:0/1863316016 learned_addr learned my addr 172.21.15.70:0/1863316016 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3e5438700 1 -- 172.21.15.70:0/1863316016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e0066280 msgr2=0x7fb3e00fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3e5438700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e0066280 0x7fb3e00fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3e5438700 1 -- 172.21.15.70:0/1863316016 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d0005040 con 0x7fb3e00fff60 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3e5c39700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e0066280 0x7fb3e00fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3e5438700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3e00fff60 0x7fb3e00fb920 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000b0d0 tx=0x7fb3d0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3d67fc700 1 -- 172.21.15.70:0/1863316016 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3d000e070 con 0x7fb3e00fff60 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e00fc2c0 con 0x7fb3e00fff60 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3d67fc700 1 -- 172.21.15.70:0/1863316016 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb3d001c400 con 0x7fb3e00fff60 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3d67fc700 1 -- 172.21.15.70:0/1863316016 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3d000b670 con 0x7fb3e00fff60 2024-03-23T08:24:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.132+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e0115910 con 0x7fb3e00fff60 2024-03-23T08:24:16.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.133+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e005f5f0 con 0x7fb3e00fff60 2024-03-23T08:24:16.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.134+0000 7fb3d67fc700 1 -- 172.21.15.70:0/1863316016 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fb3d000b7d0 con 0x7fb3e00fff60 2024-03-23T08:24:16.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.134+0000 7fb3d67fc700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb3cc06aec0 0x7fb3cc06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:16.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.134+0000 7fb3d67fc700 1 -- 172.21.15.70:0/1863316016 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb3d008ab40 con 0x7fb3e00fff60 2024-03-23T08:24:16.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.134+0000 7fb3e5c39700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb3cc06aec0 0x7fb3cc06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:16.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.135+0000 7fb3e5c39700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb3cc06aec0 0x7fb3cc06d370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00fd1e0 tx=0x7fb3dc00b040 comp rx=0 tx=0).ready entity=mgr.24333 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.139+0000 7fb3d67fc700 1 -- 172.21.15.70:0/1863316016 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fb3d0055040 con 0x7fb3e00fff60 2024-03-23T08:24:16.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.433+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 --> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3e00f8f70 con 0x7fb3cc06aec0 2024-03-23T08:24:16.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.435+0000 7fb3d67fc700 1 -- 172.21.15.70:0/1863316016 <== mgr.24333 v2:172.21.15.193:6828/758671107 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fb3e00f8f70 con 0x7fb3cc06aec0 2024-03-23T08:24:16.435 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:24:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:24:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:24:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-23T08:24:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-03-23T08:24:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "1/2 daemons upgraded", 2024-03-23T08:24:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-03-23T08:24:16.435 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:24:16.435 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:24:16.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb3cc06aec0 msgr2=0x7fb3cc06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:16.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb3cc06aec0 0x7fb3cc06d370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00fd1e0 tx=0x7fb3dc00b040 comp rx=0 tx=0).stop 2024-03-23T08:24:16.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3e00fff60 msgr2=0x7fb3e00fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:16.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3e00fff60 0x7fb3e00fb920 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000b0d0 tx=0x7fb3d0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:16.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 shutdown_connections 2024-03-23T08:24:16.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:6828/758671107,v1:172.21.15.193:6829/758671107] conn(0x7fb3cc06aec0 0x7fb3cc06d370 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:16.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e0066280 0x7fb3e00fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:16.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 --2- 172.21.15.70:0/1863316016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3e00fff60 0x7fb3e00fb920 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:16.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 >> 172.21.15.70:0/1863316016 conn(0x7fb3e00f7430 msgr2=0x7fb3e00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:16.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 shutdown_connections 2024-03-23T08:24:16.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:16.437+0000 7fb3e7e9d700 1 -- 172.21.15.70:0/1863316016 wait complete. 2024-03-23T08:24:16.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:16 smithi193 ceph-mon[48370]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:16.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:16 smithi193 ceph-mon[48370]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:16.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:16 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/147999074' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:24:16.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:16 smithi070 ceph-mon[39699]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:16.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:16 smithi070 ceph-mon[39699]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:16.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:16 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/147999074' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:24:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:17 smithi070 ceph-mon[39699]: pgmap v48: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 97 MiB/s wr, 352 op/s 2024-03-23T08:24:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:17 smithi070 ceph-mon[39699]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:17 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:24:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:17 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:17 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:17.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:17 smithi193 ceph-mon[48370]: pgmap v48: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 97 MiB/s wr, 352 op/s 2024-03-23T08:24:17.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:17 smithi193 ceph-mon[48370]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:17.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:17 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:24:17.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:17 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:17.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:17 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:18.682 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:18 smithi070 ceph-mon[39699]: mgrmap e25: smithi193.zgpdbu(active, since 91s), standbys: smithi070.jhsbch 2024-03-23T08:24:18.683 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:18 smithi070 ceph-mon[39699]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-03-23T08:24:18.683 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:18 smithi070 ceph-mon[39699]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-03-23T08:24:18.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:18 smithi193 ceph-mon[48370]: mgrmap e25: smithi193.zgpdbu(active, since 91s), standbys: smithi070.jhsbch 2024-03-23T08:24:18.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:18 smithi193 ceph-mon[48370]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-03-23T08:24:18.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:18 smithi193 ceph-mon[48370]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-03-23T08:24:19.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:19 smithi070 ceph-mon[39699]: pgmap v49: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 46 MiB/s rd, 104 MiB/s wr, 488 op/s 2024-03-23T08:24:19.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:19 smithi193 ceph-mon[48370]: pgmap v49: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 46 MiB/s rd, 104 MiB/s wr, 488 op/s 2024-03-23T08:24:21.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:21 smithi193 ceph-mon[48370]: pgmap v50: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 21 MiB/s rd, 50 MiB/s wr, 298 op/s 2024-03-23T08:24:21.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:21 smithi070 ceph-mon[39699]: pgmap v50: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 21 MiB/s rd, 50 MiB/s wr, 298 op/s 2024-03-23T08:24:23.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:23 smithi193 ceph-mon[48370]: pgmap v51: 65 pgs: 65 active+clean; 1.8 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 21 MiB/s rd, 50 MiB/s wr, 384 op/s 2024-03-23T08:24:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:23 smithi070 ceph-mon[39699]: pgmap v51: 65 pgs: 65 active+clean; 1.8 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 21 MiB/s rd, 50 MiB/s wr, 384 op/s 2024-03-23T08:24:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:24 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:24 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:24 smithi193 ceph-mon[48370]: Reconfiguring node-exporter.smithi193 (dependencies changed)... 2024-03-23T08:24:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:24 smithi193 ceph-mon[48370]: Reconfiguring daemon node-exporter.smithi193 on smithi193 2024-03-23T08:24:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:24 smithi193 ceph-mon[48370]: pgmap v52: 65 pgs: 65 active+clean; 1.7 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 311 op/s 2024-03-23T08:24:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:24 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:24 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:24 smithi070 ceph-mon[39699]: Reconfiguring node-exporter.smithi193 (dependencies changed)... 2024-03-23T08:24:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:24 smithi070 ceph-mon[39699]: Reconfiguring daemon node-exporter.smithi193 on smithi193 2024-03-23T08:24:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:24 smithi070 ceph-mon[39699]: pgmap v52: 65 pgs: 65 active+clean; 1.7 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 311 op/s 2024-03-23T08:24:27.178 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:27.178 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:27.178 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:24:27.178 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:24:27.178 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:24:27.178 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:24:27.178 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:24:27.179 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:24:27.179 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:27.179 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: Upgrade: Need to upgrade myself (mgr.smithi193.zgpdbu) 2024-03-23T08:24:27.179 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: Upgrade: Need to upgrade myself (mgr.smithi193.zgpdbu) 2024-03-23T08:24:27.179 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:27 smithi070 ceph-mon[39699]: pgmap v53: 65 pgs: 65 active+clean; 1.7 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 2.8 MiB/s rd, 8.0 MiB/s wr, 260 op/s 2024-03-23T08:24:27.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:27.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:27.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:24:27.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:24:27.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:24:27.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:24:27.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:24:27.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:24:27.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:27.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: Upgrade: Need to upgrade myself (mgr.smithi193.zgpdbu) 2024-03-23T08:24:27.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: Upgrade: Need to upgrade myself (mgr.smithi193.zgpdbu) 2024-03-23T08:24:27.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:27 smithi193 ceph-mon[48370]: pgmap v53: 65 pgs: 65 active+clean; 1.7 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 2.8 MiB/s rd, 8.0 MiB/s wr, 260 op/s 2024-03-23T08:24:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:29 smithi070 ceph-mon[39699]: Upgrade: Updating mgr.smithi070.jhsbch 2024-03-23T08:24:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:29 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:29 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.jhsbch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:24:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:29 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.jhsbch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:24:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:29 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:24:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:29 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:24:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:29 smithi070 ceph-mon[39699]: Deploying daemon mgr.smithi070.jhsbch on smithi070 2024-03-23T08:24:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:29 smithi070 ceph-mon[39699]: pgmap v54: 65 pgs: 65 active+clean; 1.3 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 445 op/s 2024-03-23T08:24:29.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:29 smithi193 ceph-mon[48370]: Upgrade: Updating mgr.smithi070.jhsbch 2024-03-23T08:24:29.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:29 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:29.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:29 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.jhsbch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:24:29.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:29 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.jhsbch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:24:29.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:29 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:24:29.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:29 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:24:29.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:29 smithi193 ceph-mon[48370]: Deploying daemon mgr.smithi070.jhsbch on smithi070 2024-03-23T08:24:29.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:29 smithi193 ceph-mon[48370]: pgmap v54: 65 pgs: 65 active+clean; 1.3 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 3.4 MiB/s rd, 8.7 MiB/s wr, 445 op/s 2024-03-23T08:24:31.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:31 smithi070 ceph-mon[39699]: pgmap v55: 65 pgs: 65 active+clean; 1.3 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 309 op/s 2024-03-23T08:24:31.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:31 smithi193 ceph-mon[48370]: pgmap v55: 65 pgs: 65 active+clean; 1.3 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 309 op/s 2024-03-23T08:24:32.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:32 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:24:32.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:32 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:32.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:32 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:32.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:32 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:32 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:24:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:32 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:32 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:32 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:33.937 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:33 smithi193 ceph-mon[48370]: pgmap v56: 65 pgs: 65 active+clean; 951 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 427 op/s 2024-03-23T08:24:33.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:33 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:33.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:33 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:33 smithi070 ceph-mon[39699]: pgmap v56: 65 pgs: 65 active+clean; 951 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 427 op/s 2024-03-23T08:24:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:33 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:33 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:35.044 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-23T08:24:35.044 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-23T08:24:35.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:35 smithi070 ceph-mon[39699]: pgmap v57: 65 pgs: 65 active+clean; 951 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 341 op/s 2024-03-23T08:24:36.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:35 smithi193 ceph-mon[48370]: pgmap v57: 65 pgs: 65 active+clean; 951 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 341 op/s 2024-03-23T08:24:36.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:36 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:36.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:36 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:37.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:36 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:37.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:36 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:37.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:37 smithi070 ceph-mon[39699]: pgmap v58: 65 pgs: 65 active+clean; 951 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 302 op/s 2024-03-23T08:24:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:37 smithi193 ceph-mon[48370]: pgmap v58: 65 pgs: 65 active+clean; 951 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 302 op/s 2024-03-23T08:24:39.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:39 smithi070 ceph-mon[39699]: pgmap v59: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 414 op/s 2024-03-23T08:24:39.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:39 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:39.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:39 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:39.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:39 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:39.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:39 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:39 smithi193 ceph-mon[48370]: pgmap v59: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 414 op/s 2024-03-23T08:24:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:39 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:39 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:39 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:39 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:41 smithi070 ceph-mon[39699]: pgmap v60: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 229 op/s 2024-03-23T08:24:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:41 smithi070 ceph-mon[39699]: Standby manager daemon smithi070.jhsbch restarted 2024-03-23T08:24:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:41 smithi070 ceph-mon[39699]: Standby manager daemon smithi070.jhsbch started 2024-03-23T08:24:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:41 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.jhsbch/crt"}]: dispatch 2024-03-23T08:24:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:41 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:24:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:41 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.jhsbch/key"}]: dispatch 2024-03-23T08:24:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:41 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:24:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:41 smithi193 ceph-mon[48370]: pgmap v60: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 229 op/s 2024-03-23T08:24:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:41 smithi193 ceph-mon[48370]: Standby manager daemon smithi070.jhsbch restarted 2024-03-23T08:24:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:41 smithi193 ceph-mon[48370]: Standby manager daemon smithi070.jhsbch started 2024-03-23T08:24:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:41 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.jhsbch/crt"}]: dispatch 2024-03-23T08:24:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:41 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:24:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:41 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.jhsbch/key"}]: dispatch 2024-03-23T08:24:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:41 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:24:42.887 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:42 smithi070 ceph-mon[39699]: mgrmap e26: smithi193.zgpdbu(active, since 115s), standbys: smithi070.jhsbch 2024-03-23T08:24:43.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:42 smithi193 ceph-mon[48370]: mgrmap e26: smithi193.zgpdbu(active, since 115s), standbys: smithi070.jhsbch 2024-03-23T08:24:43.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:43 smithi070 ceph-mon[39699]: pgmap v61: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 373 op/s 2024-03-23T08:24:44.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:43 smithi193 ceph-mon[48370]: pgmap v61: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 373 op/s 2024-03-23T08:24:45.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: pgmap v62: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 255 op/s 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr fail", "who": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr fail", "who": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: osdmap e50: 6 total, 6 up, 6 in 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd='[{"prefix": "mgr fail", "who": "smithi193.zgpdbu"}]': finished 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: mgrmap e27: smithi070.jhsbch(active, starting, since 0.00607269s) 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:24:45.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:24:45.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: Manager daemon smithi070.jhsbch is now available 2024-03-23T08:24:45.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:45.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:45.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:24:45.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:24:45.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/trash_purge_schedule"}]: dispatch 2024-03-23T08:24:45.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: pgmap v62: 65 pgs: 65 active+clean; 275 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 255 op/s 2024-03-23T08:24:45.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.24333 172.21.15.193:0/1152846131' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr fail", "who": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "mgr fail", "who": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: osdmap e50: 6 total, 6 up, 6 in 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.24333 ' entity='mgr.smithi193.zgpdbu' cmd='[{"prefix": "mgr fail", "who": "smithi193.zgpdbu"}]': finished 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: mgrmap e27: smithi070.jhsbch(active, starting, since 0.00607269s) 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:24:45.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:24:45.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: Manager daemon smithi070.jhsbch is now available 2024-03-23T08:24:45.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:45.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:24:45.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:24:45.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:24:45.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/trash_purge_schedule"}]: dispatch 2024-03-23T08:24:46.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.593+0000 7faec7513700 1 -- 172.21.15.70:0/3888852169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fcfc0 msgr2=0x7faec00fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:46.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.593+0000 7faec7513700 1 --2- 172.21.15.70:0/3888852169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fcfc0 0x7faec00fd430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faeb0004940 tx=0x7faeb0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:46.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.595+0000 7faec7513700 1 -- 172.21.15.70:0/3888852169 shutdown_connections 2024-03-23T08:24:46.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.595+0000 7faec7513700 1 --2- 172.21.15.70:0/3888852169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fcfc0 0x7faec00fd430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.595+0000 7faec7513700 1 --2- 172.21.15.70:0/3888852169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec00fbd10 0x7faec00fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.595+0000 7faec7513700 1 -- 172.21.15.70:0/3888852169 >> 172.21.15.70:0/3888852169 conn(0x7faec00f7490 msgr2=0x7faec00f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.595+0000 7faec7513700 1 -- 172.21.15.70:0/3888852169 shutdown_connections 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.595+0000 7faec7513700 1 -- 172.21.15.70:0/3888852169 wait complete. 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.596+0000 7faec7513700 1 Processor -- start 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.596+0000 7faec7513700 1 -- start start 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.596+0000 7faec7513700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fbd10 0x7faec010fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.596+0000 7faec7513700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec00fcfc0 0x7faec0110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.596+0000 7faec7513700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec0110690 con 0x7faec00fcfc0 2024-03-23T08:24:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.596+0000 7faec7513700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec0114fe0 con 0x7faec00fbd10 2024-03-23T08:24:46.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.597+0000 7faec52af700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fbd10 0x7faec010fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:46.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.597+0000 7faec52af700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fbd10 0x7faec010fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:49270/0 (socket says 172.21.15.70:49270) 2024-03-23T08:24:46.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.597+0000 7faec52af700 1 -- 172.21.15.70:0/126347078 learned_addr learned my addr 172.21.15.70:0/126347078 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:46.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.598+0000 7faec52af700 1 -- 172.21.15.70:0/126347078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec00fcfc0 msgr2=0x7faec0110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:46.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.598+0000 7faec52af700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec00fcfc0 0x7faec0110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:46.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.598+0000 7faec52af700 1 -- 172.21.15.70:0/126347078 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0005040 con 0x7faec00fbd10 2024-03-23T08:24:46.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.599+0000 7faec52af700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fbd10 0x7faec010fac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b790 tx=0x7faebc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.602+0000 7faeb5ffb700 1 -- 172.21.15.70:0/126347078 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faebc00f770 con 0x7faec00fbd10 2024-03-23T08:24:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.602+0000 7faeb5ffb700 1 -- 172.21.15.70:0/126347078 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faebc00fd80 con 0x7faec00fbd10 2024-03-23T08:24:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.602+0000 7faeb5ffb700 1 -- 172.21.15.70:0/126347078 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faebc00d310 con 0x7faec00fbd10 2024-03-23T08:24:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.602+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec0115240 con 0x7faec00fbd10 2024-03-23T08:24:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.602+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec01157b0 con 0x7faec00fbd10 2024-03-23T08:24:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.603+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec004c710 con 0x7faec00fbd10 2024-03-23T08:24:46.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.604+0000 7faeb5ffb700 1 -- 172.21.15.70:0/126347078 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 28) v1 ==== 45330+0+0 (secure 0 0 0) 0x7faebc010420 con 0x7faec00fbd10 2024-03-23T08:24:46.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.604+0000 7faeb5ffb700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7faeac0382c0 0x7faeac03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:46.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.604+0000 7faeb5ffb700 1 -- 172.21.15.70:0/126347078 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7faebc04c1b0 con 0x7faec00fbd10 2024-03-23T08:24:46.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.604+0000 7faec4aae700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7faeac0382c0 0x7faeac03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.605+0000 7faec4aae700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7faeac0382c0 0x7faeac03a770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faeb000b900 tx=0x7faeb0009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.613+0000 7faeb5ffb700 1 -- 172.21.15.70:0/126347078 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7faebc010740 con 0x7faec00fbd10 2024-03-23T08:24:46.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:46 smithi193 ceph-mon[48370]: [23/Mar/2024:08:24:45] ENGINE Bus STARTING 2024-03-23T08:24:46.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:46 smithi193 ceph-mon[48370]: [23/Mar/2024:08:24:45] ENGINE Serving on https://172.21.15.70:7150 2024-03-23T08:24:46.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:46 smithi193 ceph-mon[48370]: [23/Mar/2024:08:24:45] ENGINE Serving on http://172.21.15.70:8765 2024-03-23T08:24:46.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:46 smithi193 ceph-mon[48370]: [23/Mar/2024:08:24:45] ENGINE Bus STARTED 2024-03-23T08:24:46.665 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:46 smithi193 ceph-mon[48370]: mgrmap e28: smithi070.jhsbch(active, since 1.00816s) 2024-03-23T08:24:46.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:46 smithi070 ceph-mon[39699]: [23/Mar/2024:08:24:45] ENGINE Bus STARTING 2024-03-23T08:24:46.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:46 smithi070 ceph-mon[39699]: [23/Mar/2024:08:24:45] ENGINE Serving on https://172.21.15.70:7150 2024-03-23T08:24:46.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:46 smithi070 ceph-mon[39699]: [23/Mar/2024:08:24:45] ENGINE Serving on http://172.21.15.70:8765 2024-03-23T08:24:46.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:46 smithi070 ceph-mon[39699]: [23/Mar/2024:08:24:45] ENGINE Bus STARTED 2024-03-23T08:24:46.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:46 smithi070 ceph-mon[39699]: mgrmap e28: smithi070.jhsbch(active, since 1.00816s) 2024-03-23T08:24:46.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.935+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faec00f8f80 con 0x7faeac0382c0 2024-03-23T08:24:46.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.937+0000 7faeb5ffb700 1 -- 172.21.15.70:0/126347078 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7faec00f8f80 con 0x7faeac0382c0 2024-03-23T08:24:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7faeac0382c0 msgr2=0x7faeac03a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7faeac0382c0 0x7faeac03a770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faeb000b900 tx=0x7faeb0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fbd10 msgr2=0x7faec010fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fbd10 0x7faec010fac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b790 tx=0x7faebc009250 comp rx=0 tx=0).stop 2024-03-23T08:24:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 shutdown_connections 2024-03-23T08:24:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7faeac0382c0 0x7faeac03a770 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faec00fbd10 0x7faec010fac0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 --2- 172.21.15.70:0/126347078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec00fcfc0 0x7faec0110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 >> 172.21.15.70:0/126347078 conn(0x7faec00f7490 msgr2=0x7faec00f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 shutdown_connections 2024-03-23T08:24:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:46.940+0000 7faec7513700 1 -- 172.21.15.70:0/126347078 wait complete. 2024-03-23T08:24:46.951 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.061+0000 7f41e727c700 1 -- 172.21.15.70:0/3213018363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fcf80 msgr2=0x7f41e00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.061+0000 7f41e727c700 1 --2- 172.21.15.70:0/3213018363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fcf80 0x7f41e00fd3f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f41d0004940 tx=0x7f41d0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.061+0000 7f41e727c700 1 -- 172.21.15.70:0/3213018363 shutdown_connections 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.061+0000 7f41e727c700 1 --2- 172.21.15.70:0/3213018363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fcf80 0x7f41e00fd3f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.061+0000 7f41e727c700 1 --2- 172.21.15.70:0/3213018363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e00fbcd0 0x7f41e00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.061+0000 7f41e727c700 1 -- 172.21.15.70:0/3213018363 >> 172.21.15.70:0/3213018363 conn(0x7f41e00f7430 msgr2=0x7f41e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.061+0000 7f41e727c700 1 -- 172.21.15.70:0/3213018363 shutdown_connections 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.061+0000 7f41e727c700 1 -- 172.21.15.70:0/3213018363 wait complete. 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.061+0000 7f41e727c700 1 Processor -- start 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e727c700 1 -- start start 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e727c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fbcd0 0x7f41e00ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e727c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e00fcf80 0x7f41e00fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:47.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e727c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00fe850 con 0x7f41e00fbcd0 2024-03-23T08:24:47.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e727c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00fe990 con 0x7f41e00fcf80 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e5018700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fbcd0 0x7f41e00ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e5018700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fbcd0 0x7f41e00ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49200/0 (socket says 172.21.15.70:49200) 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e5018700 1 -- 172.21.15.70:0/4035146170 learned_addr learned my addr 172.21.15.70:0/4035146170 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e4817700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e00fcf80 0x7f41e00fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e5018700 1 -- 172.21.15.70:0/4035146170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e00fcf80 msgr2=0x7f41e00fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e5018700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e00fcf80 0x7f41e00fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e5018700 1 -- 172.21.15.70:0/4035146170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0005040 con 0x7f41e00fbcd0 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.062+0000 7f41e5018700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fbcd0 0x7f41e00ffda0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00eb40 tx=0x7f41dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.063+0000 7f41d5ffb700 1 -- 172.21.15.70:0/4035146170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41dc007cc0 con 0x7f41e00fbcd0 2024-03-23T08:24:47.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.063+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e00fec10 con 0x7f41e00fbcd0 2024-03-23T08:24:47.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.063+0000 7f41d5ffb700 1 -- 172.21.15.70:0/4035146170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f41dc00f040 con 0x7f41e00fbcd0 2024-03-23T08:24:47.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.063+0000 7f41d5ffb700 1 -- 172.21.15.70:0/4035146170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41dc009570 con 0x7f41e00fbcd0 2024-03-23T08:24:47.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.063+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e0115920 con 0x7f41e00fbcd0 2024-03-23T08:24:47.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.064+0000 7f41d5ffb700 1 -- 172.21.15.70:0/4035146170 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 28) v1 ==== 45330+0+0 (secure 0 0 0) 0x7f41dc0096d0 con 0x7f41e00fbcd0 2024-03-23T08:24:47.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.064+0000 7f41d5ffb700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f41cc038230 0x7f41cc03a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:47.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.064+0000 7f41d5ffb700 1 -- 172.21.15.70:0/4035146170 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f41dc04b920 con 0x7f41e00fbcd0 2024-03-23T08:24:47.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.064+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e0101b10 con 0x7f41e00fbcd0 2024-03-23T08:24:47.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.064+0000 7f41e4817700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f41cc038230 0x7f41cc03a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:47.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.064+0000 7f41e4817700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f41cc038230 0x7f41cc03a6e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41e0100380 tx=0x7f41d0009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:47.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.070+0000 7f41d5ffb700 1 -- 172.21.15.70:0/4035146170 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f41dc019070 con 0x7f41e00fbcd0 2024-03-23T08:24:47.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.419+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41e0104830 con 0x7f41cc038230 2024-03-23T08:24:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.425+0000 7f41d5ffb700 1 -- 172.21.15.70:0/4035146170 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f41d0007e40 con 0x7f41cc038230 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.431+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f41cc038230 msgr2=0x7f41cc03a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.431+0000 7f41e727c700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f41cc038230 0x7f41cc03a6e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f41e0100380 tx=0x7f41d0009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.431+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fbcd0 msgr2=0x7f41e00ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.431+0000 7f41e727c700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fbcd0 0x7f41e00ffda0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00eb40 tx=0x7f41dc00b040 comp rx=0 tx=0).stop 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.431+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 shutdown_connections 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.431+0000 7f41e727c700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f41cc038230 0x7f41cc03a6e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.431+0000 7f41e727c700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41e00fbcd0 0x7f41e00ffda0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.431+0000 7f41e727c700 1 --2- 172.21.15.70:0/4035146170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e00fcf80 0x7f41e00fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.431+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 >> 172.21.15.70:0/4035146170 conn(0x7f41e00f7430 msgr2=0x7f41e0065b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.432+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 shutdown_connections 2024-03-23T08:24:47.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.432+0000 7f41e727c700 1 -- 172.21.15.70:0/4035146170 wait complete. 2024-03-23T08:24:47.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.586+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/1335032950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0662e0 msgr2=0x7fe9bc0fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:47.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.586+0000 7fe9c0dcc700 1 --2- 172.21.15.70:0/1335032950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0662e0 0x7fe9bc0fd7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac004940 tx=0x7fe9ac009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:47.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.590+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/1335032950 shutdown_connections 2024-03-23T08:24:47.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.590+0000 7fe9c0dcc700 1 --2- 172.21.15.70:0/1335032950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0fdcf0 0x7fe9bc100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.590+0000 7fe9c0dcc700 1 --2- 172.21.15.70:0/1335032950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0662e0 0x7fe9bc0fd7b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.590+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/1335032950 >> 172.21.15.70:0/1335032950 conn(0x7fe9bc0f7430 msgr2=0x7fe9bc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:47.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.591+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/1335032950 shutdown_connections 2024-03-23T08:24:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.591+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/1335032950 wait complete. 2024-03-23T08:24:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.591+0000 7fe9c0dcc700 1 Processor -- start 2024-03-23T08:24:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.591+0000 7fe9c0dcc700 1 -- start start 2024-03-23T08:24:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.592+0000 7fe9c0dcc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0662e0 0x7fe9bc06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.592+0000 7fe9c0dcc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0fdcf0 0x7fe9bc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.592+0000 7fe9c0dcc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc06eee0 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.592+0000 7fe9c0dcc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc06f020 con 0x7fe9bc0662e0 2024-03-23T08:24:47.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.592+0000 7fe9b3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0fdcf0 0x7fe9bc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.592+0000 7fe9b3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0fdcf0 0x7fe9bc06e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49232/0 (socket says 172.21.15.70:49232) 2024-03-23T08:24:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.592+0000 7fe9b3fff700 1 -- 172.21.15.70:0/3275530582 learned_addr learned my addr 172.21.15.70:0/3275530582 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.593+0000 7fe9b3fff700 1 -- 172.21.15.70:0/3275530582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0662e0 msgr2=0x7fe9bc06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.593+0000 7fe9b3fff700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0662e0 0x7fe9bc06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.593+0000 7fe9b3fff700 1 -- 172.21.15.70:0/3275530582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac005040 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.593+0000 7fe9b3fff700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0fdcf0 0x7fe9bc06e9a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400b790 tx=0x7fe9a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.594+0000 7fe9b2ffd700 1 -- 172.21.15.70:0/3275530582 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9a400f770 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.594+0000 7fe9b2ffd700 1 -- 172.21.15.70:0/3275530582 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9a400fd80 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.594+0000 7fe9b2ffd700 1 -- 172.21.15.70:0/3275530582 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9a400d310 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.594+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc1221f0 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.594+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc1225c0 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.596+0000 7fe9b2ffd700 1 -- 172.21.15.70:0/3275530582 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 28) v1 ==== 45330+0+0 (secure 0 0 0) 0x7fe9a4010420 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.597+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc05f5f0 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.597+0000 7fe9b2ffd700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe9a8038280 0x7fe9a803a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:47.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.597+0000 7fe9b2ffd700 1 -- 172.21.15.70:0/3275530582 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe9a404b080 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.606+0000 7fe9b2ffd700 1 -- 172.21.15.70:0/3275530582 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fe9a400e8b0 con 0x7fe9bc0fdcf0 2024-03-23T08:24:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.606+0000 7fe9ba59c700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe9a8038280 0x7fe9a803a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:47.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.621+0000 7fe9ba59c700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe9a8038280 0x7fe9a803a730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00b3b0 tx=0x7fe9ac009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:47.687 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:47 smithi193 ceph-mon[48370]: pgmap v3: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:24:47.687 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:47 smithi193 ceph-mon[48370]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:47.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.968+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9bc0f8f70 con 0x7fe9a8038280 2024-03-23T08:24:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:47 smithi070 ceph-mon[39699]: pgmap v3: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:24:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:47 smithi070 ceph-mon[39699]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:47.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.984+0000 7fe9b2ffd700 1 -- 172.21.15.70:0/3275530582 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fe9bc0f8f70 con 0x7fe9a8038280 2024-03-23T08:24:47.985 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:24:47.985 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (48s) 9s ago 9m 17.0M - 0.25.0 c8568f914cd2 5b6e057163d5 2024-03-23T08:24:47.985 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (9m) 9s ago 9m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:24:47.985 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (8m) 12s ago 8m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:24:47.985 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (31s) 9s ago 9m 73.6M - 9.4.7 954c08fa6188 f8ec4927f107 2024-03-23T08:24:47.985 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (4m) 9s ago 4m 208M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:24:47.985 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (4m) 9s ago 4m 12.0M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:24:47.985 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (3m) 12s ago 3m 134M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:24:47.986 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (4m) 12s ago 4m 13.7M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:24:47.986 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (16s) 9s ago 11m 75.2M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:24:47.986 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283 running (2m) 12s ago 8m 499M - 18.2.2-954-g82316b04 b24e1baba637 d2827fb230ee 2024-03-23T08:24:47.986 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (11m) 9s ago 11m 54.2M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:24:47.986 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (8m) 12s ago 8m 38.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:24:47.986 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (95s) 9s ago 9m 9541k - 1.5.0 0da6a335fe13 c714e9921e5b 2024-03-23T08:24:47.987 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (22s) 12s ago 8m 7876k - 1.5.0 0da6a335fe13 8d14144e53f7 2024-03-23T08:24:47.987 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (7m) 9s ago 7m 198M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:24:47.987 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (7m) 9s ago 7m 207M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:24:47.987 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (6m) 9s ago 6m 175M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:24:47.987 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (6m) 12s ago 6m 244M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:24:47.987 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (6m) 12s ago 6m 219M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:24:47.987 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (5m) 12s ago 5m 211M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:24:47.987 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (24s) 9s ago 9m 43.6M - 2.43.0 a07b618ecd1d 77d8f0d5ca83 2024-03-23T08:24:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe9a8038280 msgr2=0x7fe9a803a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe9a8038280 0x7fe9a803a730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00b3b0 tx=0x7fe9ac009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0fdcf0 msgr2=0x7fe9bc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0fdcf0 0x7fe9bc06e9a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400b790 tx=0x7fe9a4009250 comp rx=0 tx=0).stop 2024-03-23T08:24:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 shutdown_connections 2024-03-23T08:24:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe9a8038280 0x7fe9a803a730 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9bc0662e0 0x7fe9bc06e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 --2- 172.21.15.70:0/3275530582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0fdcf0 0x7fe9bc06e9a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 >> 172.21.15.70:0/3275530582 conn(0x7fe9bc0f7430 msgr2=0x7fe9bc0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.986+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 shutdown_connections 2024-03-23T08:24:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:47.987+0000 7fe9c0dcc700 1 -- 172.21.15.70:0/3275530582 wait complete. 2024-03-23T08:24:48.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.111+0000 7fbc92255700 1 -- 172.21.15.70:0/1685731579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 msgr2=0x7fbc8c0faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:48.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.111+0000 7fbc92255700 1 --2- 172.21.15.70:0/1685731579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 0x7fbc8c0faed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbc80004940 tx=0x7fbc80009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:48.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.111+0000 7fbc92255700 1 -- 172.21.15.70:0/1685731579 shutdown_connections 2024-03-23T08:24:48.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.111+0000 7fbc92255700 1 --2- 172.21.15.70:0/1685731579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 0x7fbc8c0faed0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:48.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.111+0000 7fbc92255700 1 --2- 172.21.15.70:0/1685731579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc8c0fe440 0x7fbc8c0fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:48.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.111+0000 7fbc92255700 1 -- 172.21.15.70:0/1685731579 >> 172.21.15.70:0/1685731579 conn(0x7fbc8c0f6be0 msgr2=0x7fbc8c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:48.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.111+0000 7fbc92255700 1 -- 172.21.15.70:0/1685731579 shutdown_connections 2024-03-23T08:24:48.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.111+0000 7fbc92255700 1 -- 172.21.15.70:0/1685731579 wait complete. 2024-03-23T08:24:48.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc92255700 1 Processor -- start 2024-03-23T08:24:48.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc92255700 1 -- start start 2024-03-23T08:24:48.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc92255700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc8c0fe440 0x7fbc8c10f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:48.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc92255700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 0x7fbc8c10fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:48.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc92255700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c1104d0 con 0x7fbc8c0fed90 2024-03-23T08:24:48.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc92255700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c110640 con 0x7fbc8c0fe440 2024-03-23T08:24:48.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc8affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 0x7fbc8c10fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:48.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc8affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 0x7fbc8c10fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49246/0 (socket says 172.21.15.70:49246) 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc8affd700 1 -- 172.21.15.70:0/1640639245 learned_addr learned my addr 172.21.15.70:0/1640639245 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.112+0000 7fbc8b7fe700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc8c0fe440 0x7fbc8c10f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.113+0000 7fbc8affd700 1 -- 172.21.15.70:0/1640639245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc8c0fe440 msgr2=0x7fbc8c10f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.113+0000 7fbc8affd700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc8c0fe440 0x7fbc8c10f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.113+0000 7fbc8affd700 1 -- 172.21.15.70:0/1640639245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80005040 con 0x7fbc8c0fed90 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.113+0000 7fbc8affd700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 0x7fbc8c10fed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbc800137b0 tx=0x7fbc80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.113+0000 7fbc73fff700 1 -- 172.21.15.70:0/1640639245 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc8000e070 con 0x7fbc8c0fed90 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.113+0000 7fbc73fff700 1 -- 172.21.15.70:0/1640639245 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc8001c400 con 0x7fbc8c0fed90 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.113+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c115040 con 0x7fbc8c0fed90 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.113+0000 7fbc73fff700 1 -- 172.21.15.70:0/1640639245 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc8000b650 con 0x7fbc8c0fed90 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.113+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8c1155a0 con 0x7fbc8c0fed90 2024-03-23T08:24:48.115 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.114+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8c04c710 con 0x7fbc8c0fed90 2024-03-23T08:24:48.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.115+0000 7fbc73fff700 1 -- 172.21.15.70:0/1640639245 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 28) v1 ==== 45330+0+0 (secure 0 0 0) 0x7fbc8001c9e0 con 0x7fbc8c0fed90 2024-03-23T08:24:48.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.115+0000 7fbc73fff700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fbc74038220 0x7fbc7403a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:48.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.115+0000 7fbc73fff700 1 -- 172.21.15.70:0/1640639245 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbc8004db00 con 0x7fbc8c0fed90 2024-03-23T08:24:48.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.115+0000 7fbc8b7fe700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fbc74038220 0x7fbc7403a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:48.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.115+0000 7fbc8b7fe700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fbc74038220 0x7fbc7403a6d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009b80 tx=0x7fbc7c00b040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:48.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.123+0000 7fbc73fff700 1 -- 172.21.15.70:0/1640639245 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fbc8001c570 con 0x7fbc8c0fed90 2024-03-23T08:24:48.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.143+0000 7fbc73fff700 1 -- 172.21.15.70:0/1640639245 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 29) v1 ==== 45624+0+0 (secure 0 0 0) 0x7fbc8004b9f0 con 0x7fbc8c0fed90 2024-03-23T08:24:48.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.575+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbc8c1079b0 con 0x7fbc8c0fed90 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.575+0000 7fbc73fff700 1 -- 172.21.15.70:0/1640639245 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7fbc8001acc0 con 0x7fbc8c0fed90 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:24:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:24:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:24:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:24:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:24:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:24:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-23T08:24:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:24:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:24:48.577 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:24:48.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.578+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fbc74038220 msgr2=0x7fbc7403a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:48.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.578+0000 7fbc92255700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fbc74038220 0x7fbc7403a6d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009b80 tx=0x7fbc7c00b040 comp rx=0 tx=0).stop 2024-03-23T08:24:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.578+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 msgr2=0x7fbc8c10fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.578+0000 7fbc92255700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 0x7fbc8c10fed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbc800137b0 tx=0x7fbc80009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.578+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 shutdown_connections 2024-03-23T08:24:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.579+0000 7fbc92255700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fbc74038220 0x7fbc7403a6d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.579+0000 7fbc92255700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc8c0fe440 0x7fbc8c10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.579+0000 7fbc92255700 1 --2- 172.21.15.70:0/1640639245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0fed90 0x7fbc8c10fed0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.579+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 >> 172.21.15.70:0/1640639245 conn(0x7fbc8c0f6be0 msgr2=0x7fbc8c0f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.579+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 shutdown_connections 2024-03-23T08:24:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.579+0000 7fbc92255700 1 -- 172.21.15.70:0/1640639245 wait complete. 2024-03-23T08:24:48.679 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:48 smithi193 ceph-mon[48370]: pgmap v4: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:24:48.679 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:48 smithi193 ceph-mon[48370]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:48.679 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:48 smithi193 ceph-mon[48370]: mgrmap e29: smithi070.jhsbch(active, since 3s) 2024-03-23T08:24:48.679 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:48 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1640639245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:24:48.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3054885378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 msgr2=0x7f3a440fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:48.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 --2- 172.21.15.70:0/3054885378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 0x7f3a440fe850 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3a40004940 tx=0x7f3a40009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:48.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3054885378 shutdown_connections 2024-03-23T08:24:48.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 --2- 172.21.15.70:0/3054885378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a440fed90 0x7f3a440faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:48.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 --2- 172.21.15.70:0/3054885378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 0x7f3a440fe850 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:48.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3054885378 >> 172.21.15.70:0/3054885378 conn(0x7f3a440f6bc0 msgr2=0x7f3a440f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:48.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3054885378 shutdown_connections 2024-03-23T08:24:48.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3054885378 wait complete. 2024-03-23T08:24:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 Processor -- start 2024-03-23T08:24:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.718+0000 7f3a4b2bf700 1 -- start start 2024-03-23T08:24:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4b2bf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 0x7f3a440fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4b2bf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a440fed90 0x7f3a440fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4b2bf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440fe050 con 0x7f3a440fe440 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4b2bf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440fc640 con 0x7f3a440fed90 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4905b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 0x7f3a440fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4905b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 0x7f3a440fda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33832/0 (socket says 172.21.15.70:33832) 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4905b700 1 -- 172.21.15.70:0/3820274495 learned_addr learned my addr 172.21.15.70:0/3820274495 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4885a700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a440fed90 0x7f3a440fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4905b700 1 -- 172.21.15.70:0/3820274495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a440fed90 msgr2=0x7f3a440fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4905b700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a440fed90 0x7f3a440fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4905b700 1 -- 172.21.15.70:0/3820274495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a40005040 con 0x7f3a440fe440 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4905b700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 0x7f3a440fda50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3a400137b0 tx=0x7f3a40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a39ffb700 1 -- 172.21.15.70:0/3820274495 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4000e070 con 0x7f3a440fe440 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.719+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a440fc890 con 0x7f3a440fe440 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.720+0000 7f3a39ffb700 1 -- 172.21.15.70:0/3820274495 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a4001c400 con 0x7f3a440fe440 2024-03-23T08:24:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.720+0000 7f3a39ffb700 1 -- 172.21.15.70:0/3820274495 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4000b650 con 0x7f3a440fe440 2024-03-23T08:24:48.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.720+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a440fcdf0 con 0x7f3a440fe440 2024-03-23T08:24:48.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.721+0000 7f3a39ffb700 1 -- 172.21.15.70:0/3820274495 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 29) v1 ==== 45624+0+0 (secure 0 0 0) 0x7f3a4001c9e0 con 0x7f3a440fe440 2024-03-23T08:24:48.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.721+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4404c710 con 0x7f3a440fe440 2024-03-23T08:24:48.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.721+0000 7f3a39ffb700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f3a300385b0 0x7f3a3003aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:24:48.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.721+0000 7f3a39ffb700 1 -- 172.21.15.70:0/3820274495 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f3a4004d6e0 con 0x7f3a440fe440 2024-03-23T08:24:48.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.721+0000 7f3a4885a700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f3a300385b0 0x7f3a3003aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:24:48.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.722+0000 7f3a4885a700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f3a300385b0 0x7f3a3003aa60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009b80 tx=0x7f3a3400b040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:24:48.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:48.728+0000 7f3a39ffb700 1 -- 172.21.15.70:0/3820274495 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f3a4001c570 con 0x7f3a440fe440 2024-03-23T08:24:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:48 smithi070 ceph-mon[39699]: pgmap v4: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:24:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:48 smithi070 ceph-mon[39699]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:48 smithi070 ceph-mon[39699]: mgrmap e29: smithi070.jhsbch(active, since 3s) 2024-03-23T08:24:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:48 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1640639245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:24:49.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.023+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a44102520 con 0x7f3a300385b0 2024-03-23T08:24:49.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.025+0000 7f3a39ffb700 1 -- 172.21.15.70:0/3820274495 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f3a44102520 con 0x7f3a300385b0 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/2 daemons upgraded", 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:24:49.025 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:24:49.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f3a300385b0 msgr2=0x7f3a3003aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:49.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f3a300385b0 0x7f3a3003aa60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009b80 tx=0x7f3a3400b040 comp rx=0 tx=0).stop 2024-03-23T08:24:49.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 msgr2=0x7f3a440fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:24:49.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 0x7f3a440fda50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3a400137b0 tx=0x7f3a40009e30 comp rx=0 tx=0).stop 2024-03-23T08:24:49.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 shutdown_connections 2024-03-23T08:24:49.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f3a300385b0 0x7f3a3003aa60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:49.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a440fe440 0x7f3a440fda50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:49.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 --2- 172.21.15.70:0/3820274495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a440fed90 0x7f3a440fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:24:49.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 >> 172.21.15.70:0/3820274495 conn(0x7f3a440f6bc0 msgr2=0x7f3a440f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:24:49.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 shutdown_connections 2024-03-23T08:24:49.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:24:49.027+0000 7f3a4b2bf700 1 -- 172.21.15.70:0/3820274495 wait complete. 2024-03-23T08:24:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:49 smithi193 ceph-mon[48370]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:49 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:49 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:49 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:49 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:49 smithi070 ceph-mon[39699]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:49 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:49 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:49 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:49 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:50 smithi193 ceph-mon[48370]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:50 smithi193 ceph-mon[48370]: pgmap v5: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:24:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:50 smithi193 ceph-mon[48370]: mgrmap e30: smithi070.jhsbch(active, since 4s) 2024-03-23T08:24:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:50 smithi070 ceph-mon[39699]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:24:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:50 smithi070 ceph-mon[39699]: pgmap v5: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:24:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:50 smithi070 ceph-mon[39699]: mgrmap e30: smithi070.jhsbch(active, since 4s) 2024-03-23T08:24:52.972 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:52 smithi193 ceph-mon[48370]: pgmap v6: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:24:52.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:52.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:52.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:52.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:52 smithi070 ceph-mon[39699]: pgmap v6: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:24:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:24:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:53 smithi070 ceph-mon[39699]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:24:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:53 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/1306768237' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:24:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:53 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/1306768237' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:24:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:53 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/1306768237' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:24:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:53 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/1306768237' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:24:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:53 smithi193 ceph-mon[48370]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:24:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:53 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1306768237' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:24:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:53 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1306768237' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:24:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:53 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1306768237' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:24:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:53 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1306768237' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:24:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:54 smithi070 ceph-mon[39699]: pgmap v7: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1024 KiB/s rd, 1.0 MiB/s wr, 641 op/s 2024-03-23T08:24:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:54 smithi070 ceph-mon[39699]: mgrmap e31: smithi070.jhsbch(active, since 8s), standbys: smithi193.zgpdbu 2024-03-23T08:24:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:54 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi193.zgpdbu", "id": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:24:55.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:54 smithi193 ceph-mon[48370]: pgmap v7: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1024 KiB/s rd, 1.0 MiB/s wr, 641 op/s 2024-03-23T08:24:55.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:54 smithi193 ceph-mon[48370]: mgrmap e31: smithi070.jhsbch(active, since 8s), standbys: smithi193.zgpdbu 2024-03-23T08:24:55.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:54 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi193.zgpdbu", "id": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:24:57.218 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:56 smithi070 ceph-mon[39699]: pgmap v8: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 796 KiB/s rd, 817 KiB/s wr, 498 op/s 2024-03-23T08:24:57.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:56 smithi193 ceph-mon[48370]: pgmap v8: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 796 KiB/s rd, 817 KiB/s wr, 498 op/s 2024-03-23T08:24:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:24:58 smithi070 ceph-mon[39699]: pgmap v9: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 700 op/s 2024-03-23T08:24:59.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:24:58 smithi193 ceph-mon[48370]: pgmap v9: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 700 op/s 2024-03-23T08:25:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:00 smithi193 ceph-mon[48370]: pgmap v10: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 642 op/s 2024-03-23T08:25:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:00 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:00 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:00 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:25:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:00 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:25:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:00 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:25:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:00 smithi070 ceph-mon[39699]: pgmap v10: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 642 op/s 2024-03-23T08:25:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:00 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:00 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:00 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:25:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:00 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:25:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:00 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:25:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:25:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:25:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:25:02.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:25:02.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:25:02.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:25:02.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:25:02.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:25:02.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:02.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:02.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:02.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:02.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:01 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:25:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:25:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:25:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:25:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:25:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:25:02.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:25:02.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:25:02.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:02.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:02.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:02.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:02.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:01 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:03.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:02 smithi070 ceph-mon[39699]: pgmap v11: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 642 op/s 2024-03-23T08:25:03.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:02 smithi070 ceph-mon[39699]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-03-23T08:25:03.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:02 smithi070 ceph-mon[39699]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-03-23T08:25:03.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:02 smithi193 ceph-mon[48370]: pgmap v11: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 642 op/s 2024-03-23T08:25:03.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:02 smithi193 ceph-mon[48370]: Reconfiguring prometheus.smithi070 (dependencies changed)... 2024-03-23T08:25:03.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:02 smithi193 ceph-mon[48370]: Reconfiguring daemon prometheus.smithi070 on smithi070 2024-03-23T08:25:05.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:04 smithi070 ceph-mon[39699]: pgmap v12: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 869 op/s 2024-03-23T08:25:05.255 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:04 smithi193 ceph-mon[48370]: pgmap v12: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 869 op/s 2024-03-23T08:25:05.256 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-23T08:25:05.257 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-23T08:25:05.524 DEBUG:teuthology.parallel:result is None 2024-03-23T08:25:07.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:06 smithi070 ceph-mon[39699]: pgmap v13: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 495 op/s 2024-03-23T08:25:07.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:06 smithi193 ceph-mon[48370]: pgmap v13: 65 pgs: 65 active+clean; 255 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 495 op/s 2024-03-23T08:25:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:08 smithi070 ceph-mon[39699]: pgmap v14: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 714 op/s 2024-03-23T08:25:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:08 smithi193 ceph-mon[48370]: pgmap v14: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 714 op/s 2024-03-23T08:25:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:10 smithi193 ceph-mon[48370]: pgmap v15: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 445 op/s 2024-03-23T08:25:11.167 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:10 smithi070 ceph-mon[39699]: pgmap v15: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 445 op/s 2024-03-23T08:25:13.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:12 smithi070 ceph-mon[39699]: pgmap v16: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 445 op/s 2024-03-23T08:25:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:12 smithi193 ceph-mon[48370]: pgmap v16: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 445 op/s 2024-03-23T08:25:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:14 smithi070 ceph-mon[39699]: pgmap v17: 65 pgs: 65 active+clean; 233 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 605 op/s 2024-03-23T08:25:15.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:14 smithi193 ceph-mon[48370]: pgmap v17: 65 pgs: 65 active+clean; 233 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 605 op/s 2024-03-23T08:25:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:15 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:25:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:15 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:25:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:16 smithi070 ceph-mon[39699]: pgmap v18: 65 pgs: 65 active+clean; 233 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 378 op/s 2024-03-23T08:25:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:16 smithi193 ceph-mon[48370]: pgmap v18: 65 pgs: 65 active+clean; 233 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 378 op/s 2024-03-23T08:25:19.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.145+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/714816960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc080fce10 msgr2=0x7fcc080fd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.145+0000 7fcc0cfbf700 1 --2- 172.21.15.70:0/714816960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc080fce10 0x7fcc080fd220 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009ce0 tx=0x7fcbf8004d10 comp rx=0 tx=0).stop 2024-03-23T08:25:19.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.145+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/714816960 shutdown_connections 2024-03-23T08:25:19.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.145+0000 7fcc0cfbf700 1 --2- 172.21.15.70:0/714816960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc080fdf80 0x7fcc080fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.145+0000 7fcc0cfbf700 1 --2- 172.21.15.70:0/714816960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc080fce10 0x7fcc080fd220 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.145+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/714816960 >> 172.21.15.70:0/714816960 conn(0x7fcc080f84d0 msgr2=0x7fcc080fa900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:19.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.145+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/714816960 shutdown_connections 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.145+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/714816960 wait complete. 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc0cfbf700 1 Processor -- start 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc0cfbf700 1 -- start start 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc0cfbf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc080fce10 0x7fcc0810d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc0cfbf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc080fdf80 0x7fcc0810de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc0cfbf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0810e410 con 0x7fcc080fdf80 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc0cfbf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0810ec60 con 0x7fcc080fce10 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc0659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc080fce10 0x7fcc0810d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc05d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc080fdf80 0x7fcc0810de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc0659c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc080fce10 0x7fcc0810d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:54290/0 (socket says 172.21.15.70:54290) 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.146+0000 7fcc0659c700 1 -- 172.21.15.70:0/2227357910 learned_addr learned my addr 172.21.15.70:0/2227357910 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.147+0000 7fcc05d9b700 1 -- 172.21.15.70:0/2227357910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc080fce10 msgr2=0x7fcc0810d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.147+0000 7fcc05d9b700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc080fce10 0x7fcc0810d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.147+0000 7fcc05d9b700 1 -- 172.21.15.70:0/2227357910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf8009940 con 0x7fcc080fdf80 2024-03-23T08:25:19.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.147+0000 7fcc05d9b700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc080fdf80 0x7fcc0810de10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fd0 tx=0x7fcbf000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:19.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.147+0000 7fcbfeffd700 1 -- 172.21.15.70:0/2227357910 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf0004e90 con 0x7fcc080fdf80 2024-03-23T08:25:19.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.147+0000 7fcbfeffd700 1 -- 172.21.15.70:0/2227357910 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcbf000f040 con 0x7fcc080fdf80 2024-03-23T08:25:19.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.147+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0810eee0 con 0x7fcc080fdf80 2024-03-23T08:25:19.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.147+0000 7fcbfeffd700 1 -- 172.21.15.70:0/2227357910 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf000c9b0 con 0x7fcc080fdf80 2024-03-23T08:25:19.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.147+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0810f420 con 0x7fcc080fdf80 2024-03-23T08:25:19.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.148+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc080fc170 con 0x7fcc080fdf80 2024-03-23T08:25:19.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.149+0000 7fcbfeffd700 1 -- 172.21.15.70:0/2227357910 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fcbf00090d0 con 0x7fcc080fdf80 2024-03-23T08:25:19.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.149+0000 7fcbfeffd700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcbf406c9d0 0x7fcbf406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:19.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.149+0000 7fcbfeffd700 1 -- 172.21.15.70:0/2227357910 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fcbf008b7c0 con 0x7fcc080fdf80 2024-03-23T08:25:19.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.149+0000 7fcc0659c700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcbf406c9d0 0x7fcbf406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:19.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.150+0000 7fcc0659c700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcbf406c9d0 0x7fcbf406ee80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8004f50 tx=0x7fcbf8004d10 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:19.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.156+0000 7fcbfeffd700 1 -- 172.21.15.70:0/2227357910 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fcbf00595c0 con 0x7fcc080fdf80 2024-03-23T08:25:19.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:18 smithi070 ceph-mon[39699]: pgmap v19: 65 pgs: 65 active+clean; 226 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 516 op/s 2024-03-23T08:25:19.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:18 smithi193 ceph-mon[48370]: pgmap v19: 65 pgs: 65 active+clean; 226 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 516 op/s 2024-03-23T08:25:19.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.401+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc0810f750 con 0x7fcbf406c9d0 2024-03-23T08:25:19.421 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.402+0000 7fcbfeffd700 1 -- 172.21.15.70:0/2227357910 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fcc0810f750 con 0x7fcbf406c9d0 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.404+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcbf406c9d0 msgr2=0x7fcbf406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.404+0000 7fcc0cfbf700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcbf406c9d0 0x7fcbf406ee80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8004f50 tx=0x7fcbf8004d10 comp rx=0 tx=0).stop 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.404+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc080fdf80 msgr2=0x7fcc0810de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.404+0000 7fcc0cfbf700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc080fdf80 0x7fcc0810de10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009fd0 tx=0x7fcbf000b040 comp rx=0 tx=0).stop 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.404+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 shutdown_connections 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.404+0000 7fcc0cfbf700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcbf406c9d0 0x7fcbf406ee80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.404+0000 7fcc0cfbf700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc080fce10 0x7fcc0810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.404+0000 7fcc0cfbf700 1 --2- 172.21.15.70:0/2227357910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc080fdf80 0x7fcc0810de10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.404+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 >> 172.21.15.70:0/2227357910 conn(0x7fcc080f84d0 msgr2=0x7fcc080f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:19.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.405+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 shutdown_connections 2024-03-23T08:25:19.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.405+0000 7fcc0cfbf700 1 -- 172.21.15.70:0/2227357910 wait complete. 2024-03-23T08:25:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.517+0000 7f55721ab700 1 -- 172.21.15.70:0/1718073481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c066b10 msgr2=0x7f556c0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.517+0000 7f55721ab700 1 --2- 172.21.15.70:0/1718073481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c066b10 0x7f556c0ffa30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f555c009ce0 tx=0x7f555c004d10 comp rx=0 tx=0).stop 2024-03-23T08:25:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.517+0000 7f55721ab700 1 -- 172.21.15.70:0/1718073481 shutdown_connections 2024-03-23T08:25:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.517+0000 7f55721ab700 1 --2- 172.21.15.70:0/1718073481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f556c0fff70 0x7f556c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.517+0000 7f55721ab700 1 --2- 172.21.15.70:0/1718073481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c066b10 0x7f556c0ffa30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.517+0000 7f55721ab700 1 -- 172.21.15.70:0/1718073481 >> 172.21.15.70:0/1718073481 conn(0x7f556c0f7480 msgr2=0x7f556c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.517+0000 7f55721ab700 1 -- 172.21.15.70:0/1718073481 shutdown_connections 2024-03-23T08:25:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.517+0000 7f55721ab700 1 -- 172.21.15.70:0/1718073481 wait complete. 2024-03-23T08:25:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f55721ab700 1 Processor -- start 2024-03-23T08:25:19.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f55721ab700 1 -- start start 2024-03-23T08:25:19.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f55721ab700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f556c066b10 0x7f556c0fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f55721ab700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0fff70 0x7f556c0fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f55721ab700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0fbf70 con 0x7f556c0fff70 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f55721ab700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0fc0e0 con 0x7f556c066b10 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f556affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0fff70 0x7f556c0fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f556b7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f556c066b10 0x7f556c0fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f556affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0fff70 0x7f556c0fb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33802/0 (socket says 172.21.15.70:33802) 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.518+0000 7f556affd700 1 -- 172.21.15.70:0/326354048 learned_addr learned my addr 172.21.15.70:0/326354048 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.519+0000 7f556affd700 1 -- 172.21.15.70:0/326354048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f556c066b10 msgr2=0x7f556c0fd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.519+0000 7f556affd700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f556c066b10 0x7f556c0fd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.519+0000 7f556affd700 1 -- 172.21.15.70:0/326354048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c009940 con 0x7f556c0fff70 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.519+0000 7f556affd700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0fff70 0x7f556c0fb930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f555400b790 tx=0x7f5554009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.519+0000 7f5563fff700 1 -- 172.21.15.70:0/326354048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f555400f6f0 con 0x7f556c0fff70 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.519+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c0fc3c0 con 0x7f556c0fff70 2024-03-23T08:25:19.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.519+0000 7f5563fff700 1 -- 172.21.15.70:0/326354048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f555400fd00 con 0x7f556c0fff70 2024-03-23T08:25:19.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.519+0000 7f5563fff700 1 -- 172.21.15.70:0/326354048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55540103a0 con 0x7f556c0fff70 2024-03-23T08:25:19.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.519+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c115b90 con 0x7f556c0fff70 2024-03-23T08:25:19.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.520+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c05f5f0 con 0x7f556c0fff70 2024-03-23T08:25:19.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.521+0000 7f5563fff700 1 -- 172.21.15.70:0/326354048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f555400f850 con 0x7f556c0fff70 2024-03-23T08:25:19.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.521+0000 7f5563fff700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f555806c9c0 0x7f555806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:19.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.521+0000 7f5563fff700 1 -- 172.21.15.70:0/326354048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f555408ba70 con 0x7f556c0fff70 2024-03-23T08:25:19.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.521+0000 7f556b7fe700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f555806c9c0 0x7f555806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:19.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.522+0000 7f556b7fe700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f555806c9c0 0x7f555806ee70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f555c009cb0 tx=0x7f555c004d10 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:19.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.526+0000 7f5563fff700 1 -- 172.21.15.70:0/326354048 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f5554055ec0 con 0x7f556c0fff70 2024-03-23T08:25:19.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.820+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f556c0f8f70 con 0x7f555806c9c0 2024-03-23T08:25:19.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.823+0000 7f5563fff700 1 -- 172.21.15.70:0/326354048 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f556c0f8f70 con 0x7f555806c9c0 2024-03-23T08:25:19.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.828+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f555806c9c0 msgr2=0x7f555806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.828+0000 7f55721ab700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f555806c9c0 0x7f555806ee70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f555c009cb0 tx=0x7f555c004d10 comp rx=0 tx=0).stop 2024-03-23T08:25:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.828+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0fff70 msgr2=0x7f556c0fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.828+0000 7f55721ab700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0fff70 0x7f556c0fb930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f555400b790 tx=0x7f5554009250 comp rx=0 tx=0).stop 2024-03-23T08:25:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.828+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 shutdown_connections 2024-03-23T08:25:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.828+0000 7f55721ab700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f555806c9c0 0x7f555806ee70 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f555c009cb0 tx=0x7f555c004d10 comp rx=0 tx=0).stop 2024-03-23T08:25:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.828+0000 7f55721ab700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f556c066b10 0x7f556c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.828+0000 7f55721ab700 1 --2- 172.21.15.70:0/326354048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f556c0fff70 0x7f556c0fb930 secure :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f555400b790 tx=0x7f5554009250 comp rx=0 tx=0).stop 2024-03-23T08:25:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.828+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 >> 172.21.15.70:0/326354048 conn(0x7f556c0f7480 msgr2=0x7f556c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:19.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.830+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 shutdown_connections 2024-03-23T08:25:19.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.831+0000 7f55721ab700 1 -- 172.21.15.70:0/326354048 wait complete. 2024-03-23T08:25:19.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.964+0000 7f9d3e836700 1 -- 172.21.15.70:0/4164771850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d380fcf80 msgr2=0x7f9d380fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.964+0000 7f9d3e836700 1 --2- 172.21.15.70:0/4164771850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d380fcf80 0x7f9d380fd3f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c004940 tx=0x7f9d2c009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:19.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.964+0000 7f9d3e836700 1 -- 172.21.15.70:0/4164771850 shutdown_connections 2024-03-23T08:25:19.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.964+0000 7f9d3e836700 1 --2- 172.21.15.70:0/4164771850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d380fcf80 0x7f9d380fd3f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.964+0000 7f9d3e836700 1 --2- 172.21.15.70:0/4164771850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d380fbcd0 0x7f9d380fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.964+0000 7f9d3e836700 1 -- 172.21.15.70:0/4164771850 >> 172.21.15.70:0/4164771850 conn(0x7f9d380f7430 msgr2=0x7f9d380f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:19.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.964+0000 7f9d3e836700 1 -- 172.21.15.70:0/4164771850 shutdown_connections 2024-03-23T08:25:19.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.964+0000 7f9d3e836700 1 -- 172.21.15.70:0/4164771850 wait complete. 2024-03-23T08:25:19.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.965+0000 7f9d3e836700 1 Processor -- start 2024-03-23T08:25:19.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.965+0000 7f9d3e836700 1 -- start start 2024-03-23T08:25:19.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d3e836700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d380fbcd0 0x7f9d3810fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:19.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d3e836700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d380fcf80 0x7f9d38110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:19.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d3e836700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38110620 con 0x7f9d380fcf80 2024-03-23T08:25:19.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d3e836700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38110e70 con 0x7f9d380fbcd0 2024-03-23T08:25:19.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d37fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d380fbcd0 0x7f9d3810fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:19.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d37fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d380fbcd0 0x7f9d3810fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:54316/0 (socket says 172.21.15.70:54316) 2024-03-23T08:25:19.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d37fff700 1 -- 172.21.15.70:0/689854713 learned_addr learned my addr 172.21.15.70:0/689854713 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:19.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d377fe700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d380fcf80 0x7f9d38110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:19.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d37fff700 1 -- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d380fcf80 msgr2=0x7f9d38110020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:19.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d37fff700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d380fcf80 0x7f9d38110020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:19.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d37fff700 1 -- 172.21.15.70:0/689854713 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c005040 con 0x7f9d380fbcd0 2024-03-23T08:25:19.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.966+0000 7f9d37fff700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d380fbcd0 0x7f9d3810fae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800eb70 tx=0x7f9d2800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:19.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.967+0000 7f9d34ff9700 1 -- 172.21.15.70:0/689854713 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d28007cc0 con 0x7f9d380fbcd0 2024-03-23T08:25:19.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.967+0000 7f9d34ff9700 1 -- 172.21.15.70:0/689854713 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d2800f040 con 0x7f9d380fbcd0 2024-03-23T08:25:19.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.967+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d381110f0 con 0x7f9d380fbcd0 2024-03-23T08:25:19.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.967+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d38111630 con 0x7f9d380fbcd0 2024-03-23T08:25:19.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.967+0000 7f9d34ff9700 1 -- 172.21.15.70:0/689854713 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d2800c9f0 con 0x7f9d380fbcd0 2024-03-23T08:25:19.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.968+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d38101910 con 0x7f9d380fbcd0 2024-03-23T08:25:19.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.968+0000 7f9d34ff9700 1 -- 172.21.15.70:0/689854713 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 31) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f9d2800cb50 con 0x7f9d380fbcd0 2024-03-23T08:25:19.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.969+0000 7f9d34ff9700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9d2006c9c0 0x7f9d2006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:19.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.969+0000 7f9d34ff9700 1 -- 172.21.15.70:0/689854713 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9d2808b840 con 0x7f9d380fbcd0 2024-03-23T08:25:19.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.969+0000 7f9d377fe700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9d2006c9c0 0x7f9d2006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:19.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.969+0000 7f9d377fe700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9d2006c9c0 0x7f9d2006ee70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c004890 tx=0x7f9d2c009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:19.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:19.973+0000 7f9d34ff9700 1 -- 172.21.15.70:0/689854713 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f9d28059610 con 0x7f9d380fbcd0 2024-03-23T08:25:20.258 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.258+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d381118a0 con 0x7f9d2006c9c0 2024-03-23T08:25:20.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.268+0000 7f9d34ff9700 1 -- 172.21.15.70:0/689854713 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f9d381118a0 con 0x7f9d2006c9c0 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (80s) 28s ago 10m 17.1M - 0.25.0 c8568f914cd2 5b6e057163d5 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (10m) 28s ago 10m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (9m) 31s ago 9m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (63s) 28s ago 9m 82.2M - 9.4.7 954c08fa6188 f8ec4927f107 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (4m) 28s ago 4m 211M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (4m) 28s ago 4m 12.2M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (4m) 31s ago 4m 135M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (4m) 31s ago 4m 13.9M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (48s) 28s ago 11m 471M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283 running (2m) 31s ago 9m 62.0M - 18.2.2-954-g82316b04 b24e1baba637 d2827fb230ee 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (11m) 28s ago 11m 55.1M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (9m) 31s ago 9m 39.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (2m) 28s ago 10m 9.86M - 1.5.0 0da6a335fe13 c714e9921e5b 2024-03-23T08:25:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (54s) 31s ago 9m 8464k - 1.5.0 0da6a335fe13 8d14144e53f7 2024-03-23T08:25:20.270 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (8m) 28s ago 8m 200M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:25:20.270 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (7m) 28s ago 7m 208M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:25:20.270 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (7m) 28s ago 7m 181M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:25:20.270 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (7m) 31s ago 7m 244M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:25:20.270 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (6m) 31s ago 6m 216M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:25:20.270 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (6m) 31s ago 6m 202M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:25:20.270 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (57s) 28s ago 9m 44.1M - 2.43.0 a07b618ecd1d 77d8f0d5ca83 2024-03-23T08:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9d2006c9c0 msgr2=0x7f9d2006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9d2006c9c0 0x7f9d2006ee70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c004890 tx=0x7f9d2c009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d380fbcd0 msgr2=0x7f9d3810fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d380fbcd0 0x7f9d3810fae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800eb70 tx=0x7f9d2800b040 comp rx=0 tx=0).stop 2024-03-23T08:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 shutdown_connections 2024-03-23T08:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9d2006c9c0 0x7f9d2006ee70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d380fbcd0 0x7f9d3810fae0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 --2- 172.21.15.70:0/689854713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d380fcf80 0x7f9d38110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 >> 172.21.15.70:0/689854713 conn(0x7f9d380f7430 msgr2=0x7f9d38065cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:20.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 shutdown_connections 2024-03-23T08:25:20.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.271+0000 7f9d3e836700 1 -- 172.21.15.70:0/689854713 wait complete. 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.386+0000 7f2849b86700 1 -- 172.21.15.70:0/1622978501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fcf90 msgr2=0x7f28440fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.386+0000 7f2849b86700 1 --2- 172.21.15.70:0/1622978501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fcf90 0x7f28440fd400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2834004940 tx=0x7f2834009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.386+0000 7f2849b86700 1 -- 172.21.15.70:0/1622978501 shutdown_connections 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.386+0000 7f2849b86700 1 --2- 172.21.15.70:0/1622978501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fcf90 0x7f28440fd400 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.386+0000 7f2849b86700 1 --2- 172.21.15.70:0/1622978501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28440fbce0 0x7f28440fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.386+0000 7f2849b86700 1 -- 172.21.15.70:0/1622978501 >> 172.21.15.70:0/1622978501 conn(0x7f28440f7480 msgr2=0x7f28440f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.386+0000 7f2849b86700 1 -- 172.21.15.70:0/1622978501 shutdown_connections 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.386+0000 7f2849b86700 1 -- 172.21.15.70:0/1622978501 wait complete. 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f2849b86700 1 Processor -- start 2024-03-23T08:25:20.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f2849b86700 1 -- start start 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f2849b86700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fbce0 0x7f284410d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f2849b86700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28440fcf90 0x7f284410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f2849b86700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284410e5b0 con 0x7f28440fbce0 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f2849b86700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284410eda0 con 0x7f28440fcf90 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f28437fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fbce0 0x7f284410d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f28437fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fbce0 0x7f284410d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33824/0 (socket says 172.21.15.70:33824) 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f28437fe700 1 -- 172.21.15.70:0/1353097514 learned_addr learned my addr 172.21.15.70:0/1353097514 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f28437fe700 1 -- 172.21.15.70:0/1353097514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28440fcf90 msgr2=0x7f284410df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.387+0000 7f28437fe700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28440fcf90 0x7f284410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.388+0000 7f28437fe700 1 -- 172.21.15.70:0/1353097514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834005040 con 0x7f28440fbce0 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.388+0000 7f28437fe700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fbce0 0x7f284410d9e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f282c00b580 tx=0x7f282c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.388+0000 7f283bfff700 1 -- 172.21.15.70:0/1353097514 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f282c00f6f0 con 0x7f28440fbce0 2024-03-23T08:25:20.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.388+0000 7f283bfff700 1 -- 172.21.15.70:0/1353097514 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f282c00fd00 con 0x7f28440fbce0 2024-03-23T08:25:20.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.388+0000 7f283bfff700 1 -- 172.21.15.70:0/1353097514 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f282c0103a0 con 0x7f28440fbce0 2024-03-23T08:25:20.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.388+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284410f080 con 0x7f28440fbce0 2024-03-23T08:25:20.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.388+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284410f640 con 0x7f28440fbce0 2024-03-23T08:25:20.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.389+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284405f5f0 con 0x7f28440fbce0 2024-03-23T08:25:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.390+0000 7f283bfff700 1 -- 172.21.15.70:0/1353097514 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f282c00f850 con 0x7f28440fbce0 2024-03-23T08:25:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.390+0000 7f283bfff700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f283006c9c0 0x7f283006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.390+0000 7f283bfff700 1 -- 172.21.15.70:0/1353097514 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f282c08b900 con 0x7f28440fbce0 2024-03-23T08:25:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.390+0000 7f2842ffd700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f283006c9c0 0x7f283006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:20.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.391+0000 7f2842ffd700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f283006c9c0 0x7f283006ee70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2834000c00 tx=0x7f2834009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:20.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.394+0000 7f283bfff700 1 -- 172.21.15.70:0/1353097514 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f282c0597c0 con 0x7f28440fbce0 2024-03-23T08:25:20.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.737+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f284404bed0 con 0x7f28440fbce0 2024-03-23T08:25:20.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.737+0000 7f283bfff700 1 -- 172.21.15.70:0/1353097514 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f282c059350 con 0x7f28440fbce0 2024-03-23T08:25:20.738 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:25:20.738 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:25:20.738 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:25:20.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:25:20.739 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:25:20.739 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:25:20.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:25:20.739 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:25:20.739 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:25:20.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:25:20.740 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:25:20.740 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:25:20.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:25:20.740 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:25:20.740 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-23T08:25:20.740 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:25:20.741 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:25:20.741 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:25:20.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.740+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f283006c9c0 msgr2=0x7f283006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:20.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.740+0000 7f2849b86700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f283006c9c0 0x7f283006ee70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2834000c00 tx=0x7f2834009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:20.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.740+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fbce0 msgr2=0x7f284410d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:20.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.740+0000 7f2849b86700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fbce0 0x7f284410d9e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f282c00b580 tx=0x7f282c0091a0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.740+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 shutdown_connections 2024-03-23T08:25:20.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.740+0000 7f2849b86700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f283006c9c0 0x7f283006ee70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.740+0000 7f2849b86700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28440fbce0 0x7f284410d9e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.740+0000 7f2849b86700 1 --2- 172.21.15.70:0/1353097514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28440fcf90 0x7f284410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.740+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 >> 172.21.15.70:0/1353097514 conn(0x7f28440f7480 msgr2=0x7f28440f8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:20.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.741+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 shutdown_connections 2024-03-23T08:25:20.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.741+0000 7f2849b86700 1 -- 172.21.15.70:0/1353097514 wait complete. 2024-03-23T08:25:20.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.884+0000 7fc381f6b700 1 -- 172.21.15.70:0/2664166815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fc6f0 msgr2=0x7fc37c0fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:20.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.884+0000 7fc381f6b700 1 --2- 172.21.15.70:0/2664166815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fc6f0 0x7fc37c0fcb60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc370004940 tx=0x7fc370009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:20.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.884+0000 7fc381f6b700 1 -- 172.21.15.70:0/2664166815 shutdown_connections 2024-03-23T08:25:20.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.884+0000 7fc381f6b700 1 --2- 172.21.15.70:0/2664166815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fc6f0 0x7fc37c0fcb60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.884+0000 7fc381f6b700 1 --2- 172.21.15.70:0/2664166815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c0fb440 0x7fc37c0fb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.884+0000 7fc381f6b700 1 -- 172.21.15.70:0/2664166815 >> 172.21.15.70:0/2664166815 conn(0x7fc37c0f6ba0 msgr2=0x7fc37c0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:20.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.884+0000 7fc381f6b700 1 -- 172.21.15.70:0/2664166815 shutdown_connections 2024-03-23T08:25:20.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.884+0000 7fc381f6b700 1 -- 172.21.15.70:0/2664166815 wait complete. 2024-03-23T08:25:20.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.884+0000 7fc381f6b700 1 Processor -- start 2024-03-23T08:25:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.885+0000 7fc381f6b700 1 -- start start 2024-03-23T08:25:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.885+0000 7fc381f6b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fb440 0x7fc37c10b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.885+0000 7fc381f6b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c0fc6f0 0x7fc37c10bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.885+0000 7fc381f6b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c10c210 con 0x7fc37c0fb440 2024-03-23T08:25:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.885+0000 7fc381f6b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c10ca60 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc37b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fb440 0x7fc37c10b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc37b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fb440 0x7fc37c10b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33836/0 (socket says 172.21.15.70:33836) 2024-03-23T08:25:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc37b7fe700 1 -- 172.21.15.70:0/3380110046 learned_addr learned my addr 172.21.15.70:0/3380110046 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc37affd700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c0fc6f0 0x7fc37c10bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc37affd700 1 -- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fb440 msgr2=0x7fc37c10b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc37affd700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fb440 0x7fc37c10b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc37affd700 1 -- 172.21.15.70:0/3380110046 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc370005040 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc37b7fe700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fb440 0x7fc37c10b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T08:25:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc37affd700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c0fc6f0 0x7fc37c10bbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3700137b0 tx=0x7fc370009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:20.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc363fff700 1 -- 172.21.15.70:0/3380110046 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc370021070 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c10cc60 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc363fff700 1 -- 172.21.15.70:0/3380110046 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc37001c400 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc363fff700 1 -- 172.21.15.70:0/3380110046 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc37000e070 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.886+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c10d1f0 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.887+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c05f5f0 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.888+0000 7fc363fff700 1 -- 172.21.15.70:0/3380110046 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 31) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fc370013a30 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.888+0000 7fc363fff700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc36406c9c0 0x7fc36406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.888+0000 7fc363fff700 1 -- 172.21.15.70:0/3380110046 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc37008cd70 con 0x7fc37c0fc6f0 2024-03-23T08:25:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.889+0000 7fc37b7fe700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc36406c9c0 0x7fc36406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.889+0000 7fc37b7fe700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc36406c9c0 0x7fc36406ee70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009fd0 tx=0x7fc36c00c040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:20.893+0000 7fc363fff700 1 -- 172.21.15.70:0/3380110046 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fc3700570b0 con 0x7fc37c0fc6f0 2024-03-23T08:25:21.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.170+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc37c10d550 con 0x7fc36406c9c0 2024-03-23T08:25:21.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.171+0000 7fc363fff700 1 -- 172.21.15.70:0/3380110046 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fc37c10d550 con 0x7fc36406c9c0 2024-03-23T08:25:21.171 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:25:21.171 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:25:21.171 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:25:21.171 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-23T08:25:21.171 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:25:21.172 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-03-23T08:25:21.172 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:25:21.172 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/2 daemons upgraded", 2024-03-23T08:25:21.172 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-03-23T08:25:21.172 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:25:21.172 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.174+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc36406c9c0 msgr2=0x7fc36406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.174+0000 7fc381f6b700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc36406c9c0 0x7fc36406ee70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009fd0 tx=0x7fc36c00c040 comp rx=0 tx=0).stop 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.174+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c0fc6f0 msgr2=0x7fc37c10bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.174+0000 7fc381f6b700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c0fc6f0 0x7fc37c10bbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3700137b0 tx=0x7fc370009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.175+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 shutdown_connections 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.175+0000 7fc381f6b700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc36406c9c0 0x7fc36406ee70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.175+0000 7fc381f6b700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37c0fb440 0x7fc37c10b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.175+0000 7fc381f6b700 1 --2- 172.21.15.70:0/3380110046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc37c0fc6f0 0x7fc37c10bbc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.175+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 >> 172.21.15.70:0/3380110046 conn(0x7fc37c0f6ba0 msgr2=0x7fc37c0f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.175+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 shutdown_connections 2024-03-23T08:25:21.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:21.175+0000 7fc381f6b700 1 -- 172.21.15.70:0/3380110046 wait complete. 2024-03-23T08:25:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:20 smithi070 ceph-mon[39699]: pgmap v20: 65 pgs: 65 active+clean; 226 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 821 KiB/s rd, 907 KiB/s wr, 298 op/s 2024-03-23T08:25:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:20 smithi070 ceph-mon[39699]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:20 smithi070 ceph-mon[39699]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:20 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1353097514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:21.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:20 smithi193 ceph-mon[48370]: pgmap v20: 65 pgs: 65 active+clean; 226 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 821 KiB/s rd, 907 KiB/s wr, 298 op/s 2024-03-23T08:25:21.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:20 smithi193 ceph-mon[48370]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:20 smithi193 ceph-mon[48370]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:21.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:20 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1353097514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:21.928 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:21 smithi070 ceph-mon[39699]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:21 smithi193 ceph-mon[48370]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:22 smithi070 ceph-mon[39699]: pgmap v21: 65 pgs: 65 active+clean; 226 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 821 KiB/s rd, 907 KiB/s wr, 298 op/s 2024-03-23T08:25:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:22 smithi070 ceph-mon[39699]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:22 smithi193 ceph-mon[48370]: pgmap v21: 65 pgs: 65 active+clean; 226 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 821 KiB/s rd, 907 KiB/s wr, 298 op/s 2024-03-23T08:25:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:22 smithi193 ceph-mon[48370]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:24 smithi193 ceph-mon[48370]: pgmap v22: 65 pgs: 65 active+clean; 214 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 430 op/s 2024-03-23T08:25:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:24 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:24 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:24 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:25:25.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:24 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:24 smithi070 ceph-mon[39699]: pgmap v22: 65 pgs: 65 active+clean; 214 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 430 op/s 2024-03-23T08:25:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:24 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:24 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:24 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:25:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:24 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:26.092 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-23T08:25:26.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-23T08:25:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:25 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:25:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:25 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:25:26.359 DEBUG:teuthology.parallel:result is None 2024-03-23T08:25:26.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-23T08:25:26.395 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-23T08:25:26.395 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-23T08:25:26.431 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-23T08:25:26.431 DEBUG:teuthology.parallel:result is None 2024-03-23T08:25:27.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:26 smithi193 ceph-mon[48370]: pgmap v23: 65 pgs: 65 active+clean; 214 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 904 KiB/s rd, 897 KiB/s wr, 270 op/s 2024-03-23T08:25:27.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:26 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:25:27.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:26 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:25:27.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:26 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:25:27.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:26 smithi070 ceph-mon[39699]: pgmap v23: 65 pgs: 65 active+clean; 214 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 904 KiB/s rd, 897 KiB/s wr, 270 op/s 2024-03-23T08:25:27.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:26 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:25:27.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:26 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:25:27.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:26 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:25:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:27 smithi070 ceph-mon[39699]: Upgrade: Updating mgr.smithi193.zgpdbu 2024-03-23T08:25:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:27 smithi070 ceph-mon[39699]: Deploying daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:25:28.286 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:27 smithi193 ceph-mon[48370]: Upgrade: Updating mgr.smithi193.zgpdbu 2024-03-23T08:25:28.286 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:27 smithi193 ceph-mon[48370]: Deploying daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:25:29.221 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:28 smithi193 ceph-mon[48370]: pgmap v24: 65 pgs: 65 active+clean; 209 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 392 op/s 2024-03-23T08:25:29.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:28 smithi070 ceph-mon[39699]: pgmap v24: 65 pgs: 65 active+clean; 209 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 392 op/s 2024-03-23T08:25:30.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:30 smithi070 ceph-mon[39699]: pgmap v25: 65 pgs: 65 active+clean; 209 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 795 KiB/s rd, 895 KiB/s wr, 254 op/s 2024-03-23T08:25:30.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:30.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:30.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:30.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:25:30.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:30 smithi193 ceph-mon[48370]: pgmap v25: 65 pgs: 65 active+clean; 209 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 795 KiB/s rd, 895 KiB/s wr, 254 op/s 2024-03-23T08:25:30.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:30.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:30.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:30.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:25:32.920 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:32 smithi193 ceph-mon[48370]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 920 KiB/s rd, 918 KiB/s wr, 299 op/s 2024-03-23T08:25:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:32 smithi070 ceph-mon[39699]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 920 KiB/s rd, 918 KiB/s wr, 299 op/s 2024-03-23T08:25:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:33 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:33 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:33 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:33 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:33.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:33 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:33.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:33 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:33.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:33 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:33.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:33 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:34.900 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:34 smithi193 ceph-mon[48370]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 920 KiB/s rd, 918 KiB/s wr, 299 op/s 2024-03-23T08:25:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:34 smithi070 ceph-mon[39699]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 920 KiB/s rd, 918 KiB/s wr, 299 op/s 2024-03-23T08:25:36.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:36 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:36.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:36 smithi070 ceph-mon[39699]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 482 KiB/s rd, 486 KiB/s wr, 166 op/s 2024-03-23T08:25:36.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:36 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:36.534 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:36 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:36.534 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:36 smithi193 ceph-mon[48370]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 482 KiB/s rd, 486 KiB/s wr, 166 op/s 2024-03-23T08:25:36.534 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:36 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: Standby manager daemon smithi193.zgpdbu restarted 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/3212085665' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/3212085665' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/3212085665' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/3212085665' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:25:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.jhsbch"}]: dispatch 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.jhsbch"}]': finished 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.zgpdbu"}]: dispatch 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.zgpdbu"}]': finished 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:37 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: Standby manager daemon smithi193.zgpdbu restarted 2024-03-23T08:25:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:25:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/3212085665' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/3212085665' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/3212085665' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.? 172.21.15.193:0/3212085665' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.jhsbch"}]: dispatch 2024-03-23T08:25:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.jhsbch"}]': finished 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.zgpdbu"}]: dispatch 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.zgpdbu"}]': finished 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:37.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:37.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:37 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:38.303 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: Upgrade: Setting container_image for all mgr 2024-03-23T08:25:38.303 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: Upgrade: Setting container_image for all rgw 2024-03-23T08:25:38.303 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: Upgrade: Setting container_image for all rbd-mirror 2024-03-23T08:25:38.303 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-23T08:25:38.303 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: Upgrade: Setting container_image for all ceph-exporter 2024-03-23T08:25:38.303 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: Upgrade: Setting container_image for all iscsi 2024-03-23T08:25:38.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: Upgrade: Setting container_image for all nfs 2024-03-23T08:25:38.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: Upgrade: Setting container_image for all nvmeof 2024-03-23T08:25:38.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: pgmap v29: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 493 KiB/s rd, 497 KiB/s wr, 168 op/s 2024-03-23T08:25:38.304 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:38 smithi070 ceph-mon[39699]: mgrmap e32: smithi070.jhsbch(active, since 52s), standbys: smithi193.zgpdbu 2024-03-23T08:25:38.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: Upgrade: Setting container_image for all mgr 2024-03-23T08:25:38.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: Upgrade: Setting container_image for all rgw 2024-03-23T08:25:38.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: Upgrade: Setting container_image for all rbd-mirror 2024-03-23T08:25:38.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-23T08:25:38.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: Upgrade: Setting container_image for all ceph-exporter 2024-03-23T08:25:38.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: Upgrade: Setting container_image for all iscsi 2024-03-23T08:25:38.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: Upgrade: Setting container_image for all nfs 2024-03-23T08:25:38.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: Upgrade: Setting container_image for all nvmeof 2024-03-23T08:25:38.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: pgmap v29: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 493 KiB/s rd, 497 KiB/s wr, 168 op/s 2024-03-23T08:25:38.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:38 smithi193 ceph-mon[48370]: mgrmap e32: smithi070.jhsbch(active, since 52s), standbys: smithi193.zgpdbu 2024-03-23T08:25:39.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:39 smithi070 ceph-mon[39699]: Upgrade: Updating node-exporter.smithi070 (1/2) 2024-03-23T08:25:39.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:39 smithi070 ceph-mon[39699]: Deploying daemon node-exporter.smithi070 on smithi070 2024-03-23T08:25:39.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:39 smithi193 ceph-mon[48370]: Upgrade: Updating node-exporter.smithi070 (1/2) 2024-03-23T08:25:39.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:39 smithi193 ceph-mon[48370]: Deploying daemon node-exporter.smithi070 on smithi070 2024-03-23T08:25:40.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:40 smithi070 ceph-mon[39699]: pgmap v30: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 136 KiB/s rd, 33 KiB/s wr, 46 op/s 2024-03-23T08:25:40.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:40 smithi193 ceph-mon[48370]: pgmap v30: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 136 KiB/s rd, 33 KiB/s wr, 46 op/s 2024-03-23T08:25:42.729 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:42 smithi070 ceph-mon[39699]: pgmap v31: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 137 KiB/s rd, 33 KiB/s wr, 47 op/s 2024-03-23T08:25:42.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:42 smithi193 ceph-mon[48370]: pgmap v31: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 137 KiB/s rd, 33 KiB/s wr, 47 op/s 2024-03-23T08:25:44.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:44 smithi193 ceph-mon[48370]: pgmap v32: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-03-23T08:25:44.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:44 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:44.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:44 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:44.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:44 smithi070 ceph-mon[39699]: pgmap v32: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-03-23T08:25:44.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:44 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:44.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:44 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:45.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:25:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:25:46.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:46 smithi070 ceph-mon[39699]: pgmap v33: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-03-23T08:25:46.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:46 smithi070 ceph-mon[39699]: Upgrade: Updating node-exporter.smithi193 (2/2) 2024-03-23T08:25:46.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:46 smithi070 ceph-mon[39699]: Deploying daemon node-exporter.smithi193 on smithi193 2024-03-23T08:25:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:46 smithi193 ceph-mon[48370]: pgmap v33: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-03-23T08:25:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:46 smithi193 ceph-mon[48370]: Upgrade: Updating node-exporter.smithi193 (2/2) 2024-03-23T08:25:47.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:46 smithi193 ceph-mon[48370]: Deploying daemon node-exporter.smithi193 on smithi193 2024-03-23T08:25:48.659 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:48 smithi070 ceph-mon[39699]: pgmap v34: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 11 KiB/s wr, 3 op/s 2024-03-23T08:25:48.659 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:48 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:48.659 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:48 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:48.659 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:48 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:48.918 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:48 smithi193 ceph-mon[48370]: pgmap v34: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 11 KiB/s wr, 3 op/s 2024-03-23T08:25:48.918 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:48 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:48.918 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:48 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:48.918 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:48 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:50 smithi070 ceph-mon[39699]: pgmap v35: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-03-23T08:25:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:50 smithi193 ceph-mon[48370]: pgmap v35: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-03-23T08:25:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.282+0000 7fe832f31700 1 -- 172.21.15.70:0/191994884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fcf80 msgr2=0x7fe82c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.282+0000 7fe832f31700 1 --2- 172.21.15.70:0/191994884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fcf80 0x7fe82c0fd3f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe818004940 tx=0x7fe818009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.282+0000 7fe832f31700 1 -- 172.21.15.70:0/191994884 shutdown_connections 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.282+0000 7fe832f31700 1 --2- 172.21.15.70:0/191994884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fcf80 0x7fe82c0fd3f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.282+0000 7fe832f31700 1 --2- 172.21.15.70:0/191994884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe82c0fbcd0 0x7fe82c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.282+0000 7fe832f31700 1 -- 172.21.15.70:0/191994884 >> 172.21.15.70:0/191994884 conn(0x7fe82c0f7430 msgr2=0x7fe82c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.282+0000 7fe832f31700 1 -- 172.21.15.70:0/191994884 shutdown_connections 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.282+0000 7fe832f31700 1 -- 172.21.15.70:0/191994884 wait complete. 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe832f31700 1 Processor -- start 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe832f31700 1 -- start start 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe832f31700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fbcd0 0x7fe82c110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe832f31700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe82c0fcf80 0x7fe82c112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe832f31700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c066350 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe832f31700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c066490 con 0x7fe82c0fcf80 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe830ccd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fbcd0 0x7fe82c110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe830ccd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fbcd0 0x7fe82c110670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41310/0 (socket says 172.21.15.70:41310) 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe830ccd700 1 -- 172.21.15.70:0/3068781424 learned_addr learned my addr 172.21.15.70:0/3068781424 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe82bfff700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe82c0fcf80 0x7fe82c112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe830ccd700 1 -- 172.21.15.70:0/3068781424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe82c0fcf80 msgr2=0x7fe82c112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe830ccd700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe82c0fcf80 0x7fe82c112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.283+0000 7fe830ccd700 1 -- 172.21.15.70:0/3068781424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe818005040 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.284+0000 7fe830ccd700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fbcd0 0x7fe82c110670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe820002a50 tx=0x7fe8200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.284+0000 7fe8297fa700 1 -- 172.21.15.70:0/3068781424 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe820007cc0 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.284+0000 7fe8297fa700 1 -- 172.21.15.70:0/3068781424 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe82000f040 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.284+0000 7fe8297fa700 1 -- 172.21.15.70:0/3068781424 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82000d9c0 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.284+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c113160 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.284+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82c1136d0 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.285+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82c05f5f0 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.286+0000 7fe8297fa700 1 -- 172.21.15.70:0/3068781424 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fe82000db20 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.286+0000 7fe8297fa700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe81c06c9c0 0x7fe81c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.286+0000 7fe8297fa700 1 -- 172.21.15.70:0/3068781424 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe82008ba40 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.286+0000 7fe82bfff700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe81c06c9c0 0x7fe81c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.287+0000 7fe82bfff700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe81c06c9c0 0x7fe81c06ee70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe81800b900 tx=0x7fe818009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.295+0000 7fe8297fa700 1 -- 172.21.15.70:0/3068781424 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fe820055d00 con 0x7fe82c0fbcd0 2024-03-23T08:25:51.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.532+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe82c101380 con 0x7fe81c06c9c0 2024-03-23T08:25:51.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.534+0000 7fe8297fa700 1 -- 172.21.15.70:0/3068781424 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fe82c101380 con 0x7fe81c06c9c0 2024-03-23T08:25:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe81c06c9c0 msgr2=0x7fe81c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe81c06c9c0 0x7fe81c06ee70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe81800b900 tx=0x7fe818009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fbcd0 msgr2=0x7fe82c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fbcd0 0x7fe82c110670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe820002a50 tx=0x7fe8200046b0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 shutdown_connections 2024-03-23T08:25:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe81c06c9c0 0x7fe81c06ee70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe82c0fbcd0 0x7fe82c110670 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 --2- 172.21.15.70:0/3068781424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe82c0fcf80 0x7fe82c112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 >> 172.21.15.70:0/3068781424 conn(0x7fe82c0f7430 msgr2=0x7fe82c0f8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:51.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 shutdown_connections 2024-03-23T08:25:51.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.536+0000 7fe832f31700 1 -- 172.21.15.70:0/3068781424 wait complete. 2024-03-23T08:25:51.547 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:25:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.640+0000 7fc2d0559700 1 -- 172.21.15.70:0/3099119489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 msgr2=0x7fc2c8102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.640+0000 7fc2d0559700 1 --2- 172.21.15.70:0/3099119489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 0x7fc2c8102320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8004940 tx=0x7fc2b8009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.640+0000 7fc2d0559700 1 -- 172.21.15.70:0/3099119489 shutdown_connections 2024-03-23T08:25:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.640+0000 7fc2d0559700 1 --2- 172.21.15.70:0/3099119489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 0x7fc2c8102320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.640+0000 7fc2d0559700 1 --2- 172.21.15.70:0/3099119489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2c80662e0 0x7fc2c80ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.640+0000 7fc2d0559700 1 -- 172.21.15.70:0/3099119489 >> 172.21.15.70:0/3099119489 conn(0x7fc2c80f7430 msgr2=0x7fc2c80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.640+0000 7fc2d0559700 1 -- 172.21.15.70:0/3099119489 shutdown_connections 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.640+0000 7fc2d0559700 1 -- 172.21.15.70:0/3099119489 wait complete. 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2d0559700 1 Processor -- start 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2d0559700 1 -- start start 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2d0559700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2c80662e0 0x7fc2c810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2d0559700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 0x7fc2c810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2d0559700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c810e4a0 con 0x7fc2c80fff00 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2d0559700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c810ecf0 con 0x7fc2c80662e0 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2cdaf4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 0x7fc2c810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2cdaf4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 0x7fc2c810dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41322/0 (socket says 172.21.15.70:41322) 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2cdaf4700 1 -- 172.21.15.70:0/1588999964 learned_addr learned my addr 172.21.15.70:0/1588999964 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.641+0000 7fc2ce2f5700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2c80662e0 0x7fc2c810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.642+0000 7fc2cdaf4700 1 -- 172.21.15.70:0/1588999964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2c80662e0 msgr2=0x7fc2c810d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.642+0000 7fc2cdaf4700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2c80662e0 0x7fc2c810d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.642+0000 7fc2cdaf4700 1 -- 172.21.15.70:0/1588999964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b8005040 con 0x7fc2c80fff00 2024-03-23T08:25:51.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.642+0000 7fc2cdaf4700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 0x7fc2c810dea0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800b3b0 tx=0x7fc2b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:51.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.642+0000 7fc2beffd700 1 -- 172.21.15.70:0/1588999964 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2b800e040 con 0x7fc2c80fff00 2024-03-23T08:25:51.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.642+0000 7fc2beffd700 1 -- 172.21.15.70:0/1588999964 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc2b800bc50 con 0x7fc2c80fff00 2024-03-23T08:25:51.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.642+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c810ef70 con 0x7fc2c80fff00 2024-03-23T08:25:51.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.642+0000 7fc2beffd700 1 -- 172.21.15.70:0/1588999964 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2b801c630 con 0x7fc2c80fff00 2024-03-23T08:25:51.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.642+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c810f4b0 con 0x7fc2c80fff00 2024-03-23T08:25:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.643+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c805f5f0 con 0x7fc2c80fff00 2024-03-23T08:25:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.643+0000 7fc2beffd700 1 -- 172.21.15.70:0/1588999964 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7fc2b8013420 con 0x7fc2c80fff00 2024-03-23T08:25:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.644+0000 7fc2beffd700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc2b406c9c0 0x7fc2b406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.644+0000 7fc2beffd700 1 -- 172.21.15.70:0/1588999964 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc2b808c910 con 0x7fc2c80fff00 2024-03-23T08:25:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.644+0000 7fc2ce2f5700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc2b406c9c0 0x7fc2b406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.644+0000 7fc2ce2f5700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc2b406c9c0 0x7fc2b406ee70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8066020 tx=0x7fc2c400b040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:51.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.648+0000 7fc2beffd700 1 -- 172.21.15.70:0/1588999964 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fc2b805a6e0 con 0x7fc2c80fff00 2024-03-23T08:25:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.908+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2c810f7e0 con 0x7fc2b406c9c0 2024-03-23T08:25:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.909+0000 7fc2beffd700 1 -- 172.21.15.70:0/1588999964 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fc2c810f7e0 con 0x7fc2b406c9c0 2024-03-23T08:25:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.911+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc2b406c9c0 msgr2=0x7fc2b406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.911+0000 7fc2d0559700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc2b406c9c0 0x7fc2b406ee70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8066020 tx=0x7fc2c400b040 comp rx=0 tx=0).stop 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.911+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 msgr2=0x7fc2c810dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.911+0000 7fc2d0559700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 0x7fc2c810dea0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800b3b0 tx=0x7fc2b8009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.911+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 shutdown_connections 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.911+0000 7fc2d0559700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc2b406c9c0 0x7fc2b406ee70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.911+0000 7fc2d0559700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2c80662e0 0x7fc2c810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.911+0000 7fc2d0559700 1 --2- 172.21.15.70:0/1588999964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2c80fff00 0x7fc2c810dea0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.911+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 >> 172.21.15.70:0/1588999964 conn(0x7fc2c80f7430 msgr2=0x7fc2c80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.912+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 shutdown_connections 2024-03-23T08:25:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:51.912+0000 7fc2d0559700 1 -- 172.21.15.70:0/1588999964 wait complete. 2024-03-23T08:25:52.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.013+0000 7f026dcdb700 1 -- 172.21.15.70:0/1905705907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 msgr2=0x7f02680fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.013+0000 7f026dcdb700 1 --2- 172.21.15.70:0/1905705907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 0x7f02680fd250 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f025c004940 tx=0x7f025c009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 -- 172.21.15.70:0/1905705907 shutdown_connections 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 --2- 172.21.15.70:0/1905705907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02680fe040 0x7f02680fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 --2- 172.21.15.70:0/1905705907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 0x7f02680fd250 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 -- 172.21.15.70:0/1905705907 >> 172.21.15.70:0/1905705907 conn(0x7f02680f8570 msgr2=0x7f02680fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 -- 172.21.15.70:0/1905705907 shutdown_connections 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 -- 172.21.15.70:0/1905705907 wait complete. 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 Processor -- start 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 -- start start 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 0x7f026810d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02680fe040 0x7f026810de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026810e420 con 0x7f02680fce40 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.014+0000 7f026dcdb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026810ec70 con 0x7f02680fe040 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f02677fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 0x7f026810d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:52.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f02677fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 0x7f026810d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41342/0 (socket says 172.21.15.70:41342) 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f02677fe700 1 -- 172.21.15.70:0/2816767998 learned_addr learned my addr 172.21.15.70:0/2816767998 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f0266ffd700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02680fe040 0x7f026810de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f02677fe700 1 -- 172.21.15.70:0/2816767998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02680fe040 msgr2=0x7f026810de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f02677fe700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02680fe040 0x7f026810de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f02677fe700 1 -- 172.21.15.70:0/2816767998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c005040 con 0x7f02680fce40 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f02677fe700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 0x7f026810d8e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f025c0137b0 tx=0x7f025c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f024ffff700 1 -- 172.21.15.70:0/2816767998 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025c021070 con 0x7f02680fce40 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f024ffff700 1 -- 172.21.15.70:0/2816767998 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f025c01c400 con 0x7f02680fce40 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f024ffff700 1 -- 172.21.15.70:0/2816767998 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025c00e070 con 0x7f02680fce40 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026810eef0 con 0x7f02680fce40 2024-03-23T08:25:52.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.015+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026810f430 con 0x7f02680fce40 2024-03-23T08:25:52.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.016+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026805f5f0 con 0x7f02680fce40 2024-03-23T08:25:52.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.018+0000 7f024ffff700 1 -- 172.21.15.70:0/2816767998 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f025c01c9c0 con 0x7f02680fce40 2024-03-23T08:25:52.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.018+0000 7f024ffff700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f025006c9c0 0x7f025006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:52.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.018+0000 7f024ffff700 1 -- 172.21.15.70:0/2816767998 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f025c08d4b0 con 0x7f02680fce40 2024-03-23T08:25:52.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.018+0000 7f0266ffd700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f025006c9c0 0x7f025006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:52.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.019+0000 7f0266ffd700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f025006c9c0 0x7f025006ee70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0258009b80 tx=0x7f025800b040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:52.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.022+0000 7f024ffff700 1 -- 172.21.15.70:0/2816767998 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f025c05b320 con 0x7f02680fce40 2024-03-23T08:25:52.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.233+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f026810f820 con 0x7f025006c9c0 2024-03-23T08:25:52.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.242+0000 7f024ffff700 1 -- 172.21.15.70:0/2816767998 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f026810f820 con 0x7f025006c9c0 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (112s) 17s ago 10m 17.1M - 0.25.0 c8568f914cd2 5b6e057163d5 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (10m) 17s ago 10m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (9m) 0s ago 9m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (95s) 17s ago 10m 82.2M - 9.4.7 954c08fa6188 f8ec4927f107 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (5m) 17s ago 5m 162M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (5m) 17s ago 5m 12.5M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (5m) 0s ago 5m 97.0M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (5m) 0s ago 5m 14.3M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (80s) 17s ago 12m 490M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:25:52.243 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (23s) 0s ago 9m 449M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (12m) 17s ago 12m 57.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (9m) 0s ago 9m 41.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 starting - - - - 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (5s) 0s ago 9m 3561k - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (8m) 17s ago 8m 216M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (8m) 17s ago 8m 231M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (7m) 17s ago 7m 200M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (7m) 0s ago 7m 276M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (7m) 0s ago 7m 241M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (6m) 0s ago 6m 223M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:25:52.244 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (30s) 17s ago 10m 37.8M - 2.43.0 a07b618ecd1d c71fa5f1b97c 2024-03-23T08:25:52.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.244+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f025006c9c0 msgr2=0x7f025006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f025006c9c0 0x7f025006ee70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0258009b80 tx=0x7f025800b040 comp rx=0 tx=0).stop 2024-03-23T08:25:52.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 msgr2=0x7f026810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 0x7f026810d8e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f025c0137b0 tx=0x7f025c009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:52.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 shutdown_connections 2024-03-23T08:25:52.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f025006c9c0 0x7f025006ee70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02680fce40 0x7f026810d8e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 --2- 172.21.15.70:0/2816767998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02680fe040 0x7f026810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 >> 172.21.15.70:0/2816767998 conn(0x7f02680f8570 msgr2=0x7f02680f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:52.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 shutdown_connections 2024-03-23T08:25:52.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.245+0000 7f026dcdb700 1 -- 172.21.15.70:0/2816767998 wait complete. 2024-03-23T08:25:52.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.345+0000 7f0439618700 1 -- 172.21.15.70:0/1088973658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fe400 msgr2=0x7f04340fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.345+0000 7f0439618700 1 --2- 172.21.15.70:0/1088973658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fe400 0x7f04340fe810 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0424004940 tx=0x7f0424009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:52.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.345+0000 7f0439618700 1 -- 172.21.15.70:0/1088973658 shutdown_connections 2024-03-23T08:25:52.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.345+0000 7f0439618700 1 --2- 172.21.15.70:0/1088973658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04340fed50 0x7f04340fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 --2- 172.21.15.70:0/1088973658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fe400 0x7f04340fe810 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 -- 172.21.15.70:0/1088973658 >> 172.21.15.70:0/1088973658 conn(0x7f04340f6ba0 msgr2=0x7f04340f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:52.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 -- 172.21.15.70:0/1088973658 shutdown_connections 2024-03-23T08:25:52.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 -- 172.21.15.70:0/1088973658 wait complete. 2024-03-23T08:25:52.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 Processor -- start 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 -- start start 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04340fe400 0x7f04340fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fed50 0x7f04340fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04340fc700 con 0x7f04340fed50 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.346+0000 7f0439618700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04340fc870 con 0x7f04340fe400 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f04327fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fed50 0x7f04340fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f04327fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fed50 0x7f04340fc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41360/0 (socket says 172.21.15.70:41360) 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f04327fc700 1 -- 172.21.15.70:0/914305483 learned_addr learned my addr 172.21.15.70:0/914305483 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f0432ffd700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04340fe400 0x7f04340fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f04327fc700 1 -- 172.21.15.70:0/914305483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04340fe400 msgr2=0x7f04340fdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f04327fc700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04340fe400 0x7f04340fdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f04327fc700 1 -- 172.21.15.70:0/914305483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0424005040 con 0x7f04340fed50 2024-03-23T08:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f04327fc700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fed50 0x7f04340fc0c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b790 tx=0x7f041c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f042b7fe700 1 -- 172.21.15.70:0/914305483 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f041c00f6f0 con 0x7f04340fed50 2024-03-23T08:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f042b7fe700 1 -- 172.21.15.70:0/914305483 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f041c00fd00 con 0x7f04340fed50 2024-03-23T08:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04340fcb50 con 0x7f04340fed50 2024-03-23T08:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f042b7fe700 1 -- 172.21.15.70:0/914305483 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f041c0103a0 con 0x7f04340fed50 2024-03-23T08:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.347+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04341159f0 con 0x7f04340fed50 2024-03-23T08:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.348+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043405f5f0 con 0x7f04340fed50 2024-03-23T08:25:52.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.349+0000 7f042b7fe700 1 -- 172.21.15.70:0/914305483 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7f041c00f850 con 0x7f04340fed50 2024-03-23T08:25:52.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.349+0000 7f042b7fe700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f042006c9c0 0x7f042006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:52.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.349+0000 7f042b7fe700 1 -- 172.21.15.70:0/914305483 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f041c08ba80 con 0x7f04340fed50 2024-03-23T08:25:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.349+0000 7f0432ffd700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f042006c9c0 0x7f042006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.350+0000 7f0432ffd700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f042006c9c0 0x7f042006ee70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f042400b900 tx=0x7f0424009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:52.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.354+0000 7f042b7fe700 1 -- 172.21.15.70:0/914305483 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f041c055ed0 con 0x7f04340fed50 2024-03-23T08:25:52.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:52 smithi070 ceph-mon[39699]: pgmap v36: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-03-23T08:25:52.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:52.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:52.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:52 smithi070 ceph-mon[39699]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:52.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:52 smithi070 ceph-mon[39699]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:52.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:52.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:52.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:52 smithi193 ceph-mon[48370]: pgmap v36: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-03-23T08:25:52.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:52.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:52.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:52 smithi193 ceph-mon[48370]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:52.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:52 smithi193 ceph-mon[48370]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:52.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:52.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.638+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f043404bed0 con 0x7f04340fed50 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.638+0000 7f042b7fe700 1 -- 172.21.15.70:0/914305483 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f041c0594f0 con 0x7f04340fed50 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:25:52.639 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:25:52.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:25:52.640 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:25:52.640 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:25:52.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:25:52.640 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:25:52.640 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-23T08:25:52.640 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:25:52.640 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:25:52.640 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:25:52.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f042006c9c0 msgr2=0x7f042006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f042006c9c0 0x7f042006ee70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f042400b900 tx=0x7f0424009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:52.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fed50 msgr2=0x7f04340fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fed50 0x7f04340fc0c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b790 tx=0x7f041c009250 comp rx=0 tx=0).stop 2024-03-23T08:25:52.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 shutdown_connections 2024-03-23T08:25:52.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f042006c9c0 0x7f042006ee70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04340fe400 0x7f04340fdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 --2- 172.21.15.70:0/914305483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04340fed50 0x7f04340fc0c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 >> 172.21.15.70:0/914305483 conn(0x7f04340f6ba0 msgr2=0x7f04340f8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 shutdown_connections 2024-03-23T08:25:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.641+0000 7f0439618700 1 -- 172.21.15.70:0/914305483 wait complete. 2024-03-23T08:25:52.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.751+0000 7ffb483cb700 1 -- 172.21.15.70:0/3062587216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 msgr2=0x7ffb401046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.751+0000 7ffb483cb700 1 --2- 172.21.15.70:0/3062587216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 0x7ffb401046f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffb30004970 tx=0x7ffb30009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:52.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.751+0000 7ffb483cb700 1 -- 172.21.15.70:0/3062587216 shutdown_connections 2024-03-23T08:25:52.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.751+0000 7ffb483cb700 1 --2- 172.21.15.70:0/3062587216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 0x7ffb401046f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.751+0000 7ffb483cb700 1 --2- 172.21.15.70:0/3062587216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb400fd450 0x7ffb400fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.751+0000 7ffb483cb700 1 -- 172.21.15.70:0/3062587216 >> 172.21.15.70:0/3062587216 conn(0x7ffb400f6bc0 msgr2=0x7ffb400f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:52.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.751+0000 7ffb483cb700 1 -- 172.21.15.70:0/3062587216 shutdown_connections 2024-03-23T08:25:52.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.751+0000 7ffb483cb700 1 -- 172.21.15.70:0/3062587216 wait complete. 2024-03-23T08:25:52.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb483cb700 1 Processor -- start 2024-03-23T08:25:52.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb483cb700 1 -- start start 2024-03-23T08:25:52.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb483cb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb400fd450 0x7ffb4010fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:52.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb483cb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 0x7ffb4010ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:52.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb483cb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40110580 con 0x7ffb400fdda0 2024-03-23T08:25:52.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb483cb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb401106f0 con 0x7ffb400fd450 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb45966700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 0x7ffb4010ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb45966700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 0x7ffb4010ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41366/0 (socket says 172.21.15.70:41366) 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb45966700 1 -- 172.21.15.70:0/1732659754 learned_addr learned my addr 172.21.15.70:0/1732659754 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.752+0000 7ffb46167700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb400fd450 0x7ffb4010fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.753+0000 7ffb45966700 1 -- 172.21.15.70:0/1732659754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb400fd450 msgr2=0x7ffb4010fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.753+0000 7ffb45966700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb400fd450 0x7ffb4010fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.753+0000 7ffb45966700 1 -- 172.21.15.70:0/1732659754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb30005040 con 0x7ffb400fdda0 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.753+0000 7ffb45966700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 0x7ffb4010ff80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009fd0 tx=0x7ffb30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.753+0000 7ffb36ffd700 1 -- 172.21.15.70:0/1732659754 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3000e040 con 0x7ffb400fdda0 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.753+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb401150a0 con 0x7ffb400fdda0 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.753+0000 7ffb36ffd700 1 -- 172.21.15.70:0/1732659754 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb3000bc10 con 0x7ffb400fdda0 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.753+0000 7ffb36ffd700 1 -- 172.21.15.70:0/1732659754 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3001c630 con 0x7ffb400fdda0 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.753+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb40115660 con 0x7ffb400fdda0 2024-03-23T08:25:52.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.754+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4004c710 con 0x7ffb400fdda0 2024-03-23T08:25:52.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.755+0000 7ffb36ffd700 1 -- 172.21.15.70:0/1732659754 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) v1 ==== 90805+0+0 (secure 0 0 0) 0x7ffb30013960 con 0x7ffb400fdda0 2024-03-23T08:25:52.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.755+0000 7ffb36ffd700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ffb2c06c9d0 0x7ffb2c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:25:52.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.755+0000 7ffb36ffd700 1 -- 172.21.15.70:0/1732659754 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ffb3008d0f0 con 0x7ffb400fdda0 2024-03-23T08:25:52.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.756+0000 7ffb46167700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ffb2c06c9d0 0x7ffb2c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:25:52.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.756+0000 7ffb46167700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ffb2c06c9d0 0x7ffb2c06ee80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c007fd0 tx=0x7ffb3c009040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:25:52.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.760+0000 7ffb36ffd700 1 -- 172.21.15.70:0/1732659754 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7ffb3005af70 con 0x7ffb400fdda0 2024-03-23T08:25:52.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.982+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb400f8720 con 0x7ffb2c06c9d0 2024-03-23T08:25:52.983 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.983+0000 7ffb36ffd700 1 -- 172.21.15.70:0/1732659754 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7ffb400f8720 con 0x7ffb2c06c9d0 2024-03-23T08:25:52.983 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:25:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:25:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:25:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-23T08:25:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:25:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-03-23T08:25:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:25:52.983 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/2 daemons upgraded", 2024-03-23T08:25:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-23T08:25:52.984 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:25:52.984 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:25:52.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ffb2c06c9d0 msgr2=0x7ffb2c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ffb2c06c9d0 0x7ffb2c06ee80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c007fd0 tx=0x7ffb3c009040 comp rx=0 tx=0).stop 2024-03-23T08:25:52.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 msgr2=0x7ffb4010ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:25:52.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 0x7ffb4010ff80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009fd0 tx=0x7ffb30009e30 comp rx=0 tx=0).stop 2024-03-23T08:25:52.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 shutdown_connections 2024-03-23T08:25:52.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ffb2c06c9d0 0x7ffb2c06ee80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb400fd450 0x7ffb4010fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 --2- 172.21.15.70:0/1732659754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb400fdda0 0x7ffb4010ff80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:25:52.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 >> 172.21.15.70:0/1732659754 conn(0x7ffb400f6bc0 msgr2=0x7ffb400f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:25:52.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 shutdown_connections 2024-03-23T08:25:52.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:25:52.985+0000 7ffb483cb700 1 -- 172.21.15.70:0/1732659754 wait complete. 2024-03-23T08:25:53.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:53 smithi193 ceph-mon[48370]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:53.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:53 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/914305483' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:53.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:53 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:53.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:53 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:53 smithi070 ceph-mon[39699]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:53 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/914305483' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:53 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:53.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:53 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:54.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:54 smithi070 ceph-mon[39699]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:54.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:54 smithi070 ceph-mon[39699]: pgmap v37: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:25:54.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:54 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:54.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:54 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:54.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:54 smithi193 ceph-mon[48370]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:25:54.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:54 smithi193 ceph-mon[48370]: pgmap v37: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:25:54.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:54 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:54.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:54 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:56.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:56 smithi070 ceph-mon[39699]: pgmap v38: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:25:56.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:56 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:56.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:56 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:56.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:56 smithi193 ceph-mon[48370]: pgmap v38: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:25:56.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:56 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:56.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:56 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:58.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:58.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:58.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:25:58.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:25:58.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:58.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: pgmap v39: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:25:58.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:58.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:25:58 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:58.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:58.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:25:58.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:25:58.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: pgmap v39: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:25:58.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:25:58 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:00.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:00 smithi070 ceph-mon[39699]: pgmap v40: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:00.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:00 smithi193 ceph-mon[48370]: pgmap v40: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:01.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:01 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:26:01.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:01 smithi070 ceph-mon[39699]: Upgrade: Updating prometheus.smithi070 2024-03-23T08:26:01.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:01 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:26:01.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:01 smithi193 ceph-mon[48370]: Upgrade: Updating prometheus.smithi070 2024-03-23T08:26:02.445 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:02 smithi070 ceph-mon[39699]: pgmap v41: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:02.445 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:02 smithi070 ceph-mon[39699]: Deploying daemon prometheus.smithi070 on smithi070 2024-03-23T08:26:02.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:02 smithi193 ceph-mon[48370]: pgmap v41: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:02.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:02 smithi193 ceph-mon[48370]: Deploying daemon prometheus.smithi070 on smithi070 2024-03-23T08:26:04.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:04 smithi070 ceph-mon[39699]: pgmap v42: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:04.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:04 smithi193 ceph-mon[48370]: pgmap v42: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:06.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:06 smithi070 ceph-mon[39699]: pgmap v43: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:06 smithi193 ceph-mon[48370]: pgmap v43: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:07.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:07 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:07.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:07 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:07.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:07 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:07.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:07 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:07.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:07 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:07 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:07 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:07 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:07 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:07 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:08 smithi070 ceph-mon[39699]: pgmap v44: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:08 smithi193 ceph-mon[48370]: pgmap v44: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:11.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:10 smithi070 ceph-mon[39699]: pgmap v45: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:10 smithi193 ceph-mon[48370]: pgmap v45: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:13.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:12 smithi193 ceph-mon[48370]: pgmap v46: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:13.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:12 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:13.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:12 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:13.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:12 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:13.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:12 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:13.184 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:12 smithi070 ceph-mon[39699]: pgmap v46: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:13.184 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:12 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:13.184 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:12 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:13.184 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:12 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:13.184 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:12 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:15.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:14 smithi193 ceph-mon[48370]: pgmap v47: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:14 smithi070 ceph-mon[39699]: pgmap v47: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:15 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:26:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:15 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:26:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: pgmap v48: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: mgrmap e33: smithi070.jhsbch(active, since 90s), standbys: smithi193.zgpdbu 2024-03-23T08:26:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.077 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: pgmap v48: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:17.077 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: mgrmap e33: smithi070.jhsbch(active, since 90s), standbys: smithi193.zgpdbu 2024-03-23T08:26:17.077 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:17.077 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:17.077 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:17.077 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:17.077 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:17.077 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:17.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:18.065 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:17 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:26:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:17 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-23T08:26:18.975 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:18 smithi070 ceph-mon[39699]: pgmap v49: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-23T08:26:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:18 smithi193 ceph-mon[48370]: pgmap v49: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-23T08:26:19.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:19 smithi070 ceph-mon[39699]: Upgrade: Updating alertmanager.smithi070 2024-03-23T08:26:19.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:19 smithi070 ceph-mon[39699]: Deploying daemon alertmanager.smithi070 on smithi070 2024-03-23T08:26:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:19 smithi193 ceph-mon[48370]: Upgrade: Updating alertmanager.smithi070 2024-03-23T08:26:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:19 smithi193 ceph-mon[48370]: Deploying daemon alertmanager.smithi070 on smithi070 2024-03-23T08:26:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:20 smithi070 ceph-mon[39699]: pgmap v50: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-23T08:26:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:20 smithi193 ceph-mon[48370]: pgmap v50: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-23T08:26:22.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:22 smithi070 ceph-mon[39699]: pgmap v51: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-23T08:26:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:22 smithi193 ceph-mon[48370]: pgmap v51: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-23T08:26:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.092+0000 7fc47426a700 1 -- 172.21.15.70:0/434388596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 msgr2=0x7fc46c0fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.092+0000 7fc47426a700 1 --2- 172.21.15.70:0/434388596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 0x7fc46c0fd300 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc468004940 tx=0x7fc468009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.092+0000 7fc47426a700 1 -- 172.21.15.70:0/434388596 shutdown_connections 2024-03-23T08:26:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.092+0000 7fc47426a700 1 --2- 172.21.15.70:0/434388596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc46c0fe060 0x7fc46c0fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.092+0000 7fc47426a700 1 --2- 172.21.15.70:0/434388596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 0x7fc46c0fd300 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.092+0000 7fc47426a700 1 -- 172.21.15.70:0/434388596 >> 172.21.15.70:0/434388596 conn(0x7fc46c0f85b0 msgr2=0x7fc46c0fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.092+0000 7fc47426a700 1 -- 172.21.15.70:0/434388596 shutdown_connections 2024-03-23T08:26:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.092+0000 7fc47426a700 1 -- 172.21.15.70:0/434388596 wait complete. 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc47426a700 1 Processor -- start 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc47426a700 1 -- start start 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc47426a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 0x7fc46c10fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc47426a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc46c0fe060 0x7fc46c110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc47426a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1106f0 con 0x7fc46c0fcef0 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc47426a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c110ee0 con 0x7fc46c0fe060 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc472006700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 0x7fc46c10fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc472006700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 0x7fc46c10fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42190/0 (socket says 172.21.15.70:42190) 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc472006700 1 -- 172.21.15.70:0/1084041562 learned_addr learned my addr 172.21.15.70:0/1084041562 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc471805700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc46c0fe060 0x7fc46c110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc472006700 1 -- 172.21.15.70:0/1084041562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc46c0fe060 msgr2=0x7fc46c110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc472006700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc46c0fe060 0x7fc46c110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc472006700 1 -- 172.21.15.70:0/1084041562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468005040 con 0x7fc46c0fcef0 2024-03-23T08:26:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.093+0000 7fc472006700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 0x7fc46c10fb20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc46800b3b0 tx=0x7fc468009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:23.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.094+0000 7fc462ffd700 1 -- 172.21.15.70:0/1084041562 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46800e040 con 0x7fc46c0fcef0 2024-03-23T08:26:23.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.094+0000 7fc462ffd700 1 -- 172.21.15.70:0/1084041562 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc46800bc50 con 0x7fc46c0fcef0 2024-03-23T08:26:23.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.094+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c111160 con 0x7fc46c0fcef0 2024-03-23T08:26:23.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.094+0000 7fc462ffd700 1 -- 172.21.15.70:0/1084041562 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46801c630 con 0x7fc46c0fcef0 2024-03-23T08:26:23.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.094+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c111720 con 0x7fc46c0fcef0 2024-03-23T08:26:23.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.095+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c1019f0 con 0x7fc46c0fcef0 2024-03-23T08:26:23.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.095+0000 7fc462ffd700 1 -- 172.21.15.70:0/1084041562 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fc468013420 con 0x7fc46c0fcef0 2024-03-23T08:26:23.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.096+0000 7fc462ffd700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc45806c8a0 0x7fc45806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:23.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.096+0000 7fc462ffd700 1 -- 172.21.15.70:0/1084041562 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc46808ca10 con 0x7fc46c0fcef0 2024-03-23T08:26:23.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.096+0000 7fc471805700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc45806c8a0 0x7fc45806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:23.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.096+0000 7fc471805700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc45806c8a0 0x7fc45806ed50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009bf0 tx=0x7fc45c0091a0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:23.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.100+0000 7fc462ffd700 1 -- 172.21.15.70:0/1084041562 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fc46805aaa0 con 0x7fc46c0fcef0 2024-03-23T08:26:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.325+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc46c111990 con 0x7fc45806c8a0 2024-03-23T08:26:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.326+0000 7fc462ffd700 1 -- 172.21.15.70:0/1084041562 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fc46c111990 con 0x7fc45806c8a0 2024-03-23T08:26:23.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc45806c8a0 msgr2=0x7fc45806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc45806c8a0 0x7fc45806ed50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009bf0 tx=0x7fc45c0091a0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 msgr2=0x7fc46c10fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 0x7fc46c10fb20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc46800b3b0 tx=0x7fc468009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 shutdown_connections 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc45806c8a0 0x7fc45806ed50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc46c0fcef0 0x7fc46c10fb20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 --2- 172.21.15.70:0/1084041562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc46c0fe060 0x7fc46c110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 >> 172.21.15.70:0/1084041562 conn(0x7fc46c0f85b0 msgr2=0x7fc46c0f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.328+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 shutdown_connections 2024-03-23T08:26:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.329+0000 7fc47426a700 1 -- 172.21.15.70:0/1084041562 wait complete. 2024-03-23T08:26:23.339 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:26:23.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 -- 172.21.15.70:0/195676967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fbcd0 msgr2=0x7f30a00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 --2- 172.21.15.70:0/195676967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fbcd0 0x7f30a00fc0e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f309c004940 tx=0x7f309c009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:23.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 -- 172.21.15.70:0/195676967 shutdown_connections 2024-03-23T08:26:23.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 --2- 172.21.15.70:0/195676967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a00fcf80 0x7f30a00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 --2- 172.21.15.70:0/195676967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fbcd0 0x7f30a00fc0e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 -- 172.21.15.70:0/195676967 >> 172.21.15.70:0/195676967 conn(0x7f30a00f7450 msgr2=0x7f30a00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:23.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 -- 172.21.15.70:0/195676967 shutdown_connections 2024-03-23T08:26:23.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 -- 172.21.15.70:0/195676967 wait complete. 2024-03-23T08:26:23.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 Processor -- start 2024-03-23T08:26:23.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.436+0000 7f30a7610700 1 -- start start 2024-03-23T08:26:23.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a7610700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a00fbcd0 0x7f30a010c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:23.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a7610700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fcf80 0x7f30a010cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:23.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a7610700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a010d300 con 0x7f30a00fcf80 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a7610700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a0111ca0 con 0x7f30a00fbcd0 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a4bab700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fcf80 0x7f30a010cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a4bab700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fcf80 0x7f30a010cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42198/0 (socket says 172.21.15.70:42198) 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a4bab700 1 -- 172.21.15.70:0/2544706375 learned_addr learned my addr 172.21.15.70:0/2544706375 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a53ac700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a00fbcd0 0x7f30a010c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a4bab700 1 -- 172.21.15.70:0/2544706375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a00fbcd0 msgr2=0x7f30a010c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a4bab700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a00fbcd0 0x7f30a010c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a4bab700 1 -- 172.21.15.70:0/2544706375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c005040 con 0x7f30a00fcf80 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a4bab700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fcf80 0x7f30a010cc70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f309000daf0 tx=0x7f309000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f3095ffb700 1 -- 172.21.15.70:0/2544706375 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f309000ecc0 con 0x7f30a00fcf80 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f3095ffb700 1 -- 172.21.15.70:0/2544706375 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f309000ee20 con 0x7f30a00fcf80 2024-03-23T08:26:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a00662d0 con 0x7f30a00fcf80 2024-03-23T08:26:23.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.437+0000 7f3095ffb700 1 -- 172.21.15.70:0/2544706375 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3090016840 con 0x7f30a00fcf80 2024-03-23T08:26:23.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.438+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a0066840 con 0x7f30a00fcf80 2024-03-23T08:26:23.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.438+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a004c710 con 0x7f30a00fcf80 2024-03-23T08:26:23.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.439+0000 7f3095ffb700 1 -- 172.21.15.70:0/2544706375 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f3090014070 con 0x7f30a00fcf80 2024-03-23T08:26:23.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.439+0000 7f3095ffb700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f308c06c8a0 0x7f308c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:23.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.439+0000 7f3095ffb700 1 -- 172.21.15.70:0/2544706375 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f309008dbf0 con 0x7f30a00fcf80 2024-03-23T08:26:23.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.439+0000 7f30a53ac700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f308c06c8a0 0x7f308c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:23.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.440+0000 7f30a53ac700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f308c06c8a0 0x7f308c06ed50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f309c00b900 tx=0x7f309c009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:23.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.443+0000 7f3095ffb700 1 -- 172.21.15.70:0/2544706375 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f309005bbe0 con 0x7f30a00fcf80 2024-03-23T08:26:23.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.671+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30a0066cf0 con 0x7f308c06c8a0 2024-03-23T08:26:23.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.672+0000 7f3095ffb700 1 -- 172.21.15.70:0/2544706375 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f30a0066cf0 con 0x7f308c06c8a0 2024-03-23T08:26:23.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f308c06c8a0 msgr2=0x7f308c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f308c06c8a0 0x7f308c06ed50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f309c00b900 tx=0x7f309c009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:23.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fcf80 msgr2=0x7f30a010cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fcf80 0x7f30a010cc70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f309000daf0 tx=0x7f309000b4f0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 shutdown_connections 2024-03-23T08:26:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f308c06c8a0 0x7f308c06ed50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a00fbcd0 0x7f30a010c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 --2- 172.21.15.70:0/2544706375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a00fcf80 0x7f30a010cc70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 >> 172.21.15.70:0/2544706375 conn(0x7f30a00f7450 msgr2=0x7f30a00f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 shutdown_connections 2024-03-23T08:26:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.675+0000 7f30a7610700 1 -- 172.21.15.70:0/2544706375 wait complete. 2024-03-23T08:26:23.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.780+0000 7f44e8a01700 1 -- 172.21.15.70:0/505685742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fbb20 msgr2=0x7f44e40fbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.780+0000 7f44e8a01700 1 --2- 172.21.15.70:0/505685742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fbb20 0x7f44e40fbf30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f44cc004940 tx=0x7f44cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:23.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.780+0000 7f44e8a01700 1 -- 172.21.15.70:0/505685742 shutdown_connections 2024-03-23T08:26:23.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.780+0000 7f44e8a01700 1 --2- 172.21.15.70:0/505685742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e40fcdd0 0x7f44e40fd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.780+0000 7f44e8a01700 1 --2- 172.21.15.70:0/505685742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fbb20 0x7f44e40fbf30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.780+0000 7f44e8a01700 1 -- 172.21.15.70:0/505685742 >> 172.21.15.70:0/505685742 conn(0x7f44e40f72a0 msgr2=0x7f44e40f96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:23.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.780+0000 7f44e8a01700 1 -- 172.21.15.70:0/505685742 shutdown_connections 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.780+0000 7f44e8a01700 1 -- 172.21.15.70:0/505685742 wait complete. 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e8a01700 1 Processor -- start 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e8a01700 1 -- start start 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e8a01700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e40fbb20 0x7f44e4112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e8a01700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fcdd0 0x7f44e4114de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e8a01700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4066ad0 con 0x7f44e40fcdd0 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e8a01700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4066c10 con 0x7f44e40fbb20 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fcdd0 0x7f44e4114de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fcdd0 0x7f44e4114de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42220/0 (socket says 172.21.15.70:42220) 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e259c700 1 -- 172.21.15.70:0/3225807787 learned_addr learned my addr 172.21.15.70:0/3225807787 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e2d9d700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e40fbb20 0x7f44e4112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e259c700 1 -- 172.21.15.70:0/3225807787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e40fbb20 msgr2=0x7f44e4112890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e259c700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e40fbb20 0x7f44e4112890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:23.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.781+0000 7f44e259c700 1 -- 172.21.15.70:0/3225807787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc005040 con 0x7f44e40fcdd0 2024-03-23T08:26:23.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.782+0000 7f44e259c700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fcdd0 0x7f44e4114de0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f44d4002a50 tx=0x7f44d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:23.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.782+0000 7f44db7fe700 1 -- 172.21.15.70:0/3225807787 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44d4007cc0 con 0x7f44e40fcdd0 2024-03-23T08:26:23.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.782+0000 7f44db7fe700 1 -- 172.21.15.70:0/3225807787 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44d400f040 con 0x7f44e40fcdd0 2024-03-23T08:26:23.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.782+0000 7f44db7fe700 1 -- 172.21.15.70:0/3225807787 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44d400d9c0 con 0x7f44e40fcdd0 2024-03-23T08:26:23.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.782+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e4115320 con 0x7f44e40fcdd0 2024-03-23T08:26:23.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.782+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e4115740 con 0x7f44e40fcdd0 2024-03-23T08:26:23.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.783+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e405f5f0 con 0x7f44e40fcdd0 2024-03-23T08:26:23.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.783+0000 7f44db7fe700 1 -- 172.21.15.70:0/3225807787 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f44d400db20 con 0x7f44e40fcdd0 2024-03-23T08:26:23.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.784+0000 7f44db7fe700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f44d006c8a0 0x7f44d006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:23.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.784+0000 7f44db7fe700 1 -- 172.21.15.70:0/3225807787 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f44d408b7b0 con 0x7f44e40fcdd0 2024-03-23T08:26:23.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.784+0000 7f44e2d9d700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f44d006c8a0 0x7f44d006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:23.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.784+0000 7f44e2d9d700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f44d006c8a0 0x7f44d006ed50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f44cc004890 tx=0x7f44cc009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:23.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:23.788+0000 7f44db7fe700 1 -- 172.21.15.70:0/3225807787 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f44d4055c90 con 0x7f44e40fcdd0 2024-03-23T08:26:24.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.024+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f44e41011e0 con 0x7f44d006c8a0 2024-03-23T08:26:24.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.034+0000 7f44db7fe700 1 -- 172.21.15.70:0/3225807787 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f44e41011e0 con 0x7f44d006c8a0 2024-03-23T08:26:24.034 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:26:24.034 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 starting - - - - 2024-03-23T08:26:24.034 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (11m) 12s ago 11m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:26:24.034 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (10m) 32s ago 10m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:26:24.034 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (2m) 12s ago 10m 82.3M - 9.4.7 954c08fa6188 f8ec4927f107 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (5m) 12s ago 5m 148M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (5m) 12s ago 5m 12.7M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (5m) 32s ago 5m 97.0M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (5m) 32s ago 5m 14.3M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (112s) 12s ago 12m 495M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (55s) 32s ago 10m 449M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (12m) 12s ago 12m 55.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (10m) 32s ago 10m 41.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (41s) 12s ago 11m 8753k - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (37s) 32s ago 10m 3561k - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (9m) 12s ago 9m 216M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (8m) 12s ago 8m 231M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (8m) 12s ago 8m 201M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (8m) 32s ago 8m 276M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:26:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (7m) 32s ago 7m 241M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:26:24.036 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (7m) 32s ago 7m 223M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:26:24.036 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (17s) 12s ago 10m 36.9M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:26:24.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f44d006c8a0 msgr2=0x7f44d006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f44d006c8a0 0x7f44d006ed50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f44cc004890 tx=0x7f44cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fcdd0 msgr2=0x7f44e4114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fcdd0 0x7f44e4114de0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f44d4002a50 tx=0x7f44d40046b0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 shutdown_connections 2024-03-23T08:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f44d006c8a0 0x7f44d006ed50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e40fbb20 0x7f44e4112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 --2- 172.21.15.70:0/3225807787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44e40fcdd0 0x7f44e4114de0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 >> 172.21.15.70:0/3225807787 conn(0x7f44e40f72a0 msgr2=0x7f44e40f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 shutdown_connections 2024-03-23T08:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.036+0000 7f44e8a01700 1 -- 172.21.15.70:0/3225807787 wait complete. 2024-03-23T08:26:24.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.135+0000 7fc7047bc700 1 -- 172.21.15.70:0/761210992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 msgr2=0x7fc6fc0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.135+0000 7fc7047bc700 1 --2- 172.21.15.70:0/761210992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 0x7fc6fc0fc0e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0004940 tx=0x7fc6f0009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:24.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.135+0000 7fc7047bc700 1 -- 172.21.15.70:0/761210992 shutdown_connections 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.135+0000 7fc7047bc700 1 --2- 172.21.15.70:0/761210992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6fc0fcf80 0x7fc6fc0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.135+0000 7fc7047bc700 1 --2- 172.21.15.70:0/761210992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 0x7fc6fc0fc0e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.135+0000 7fc7047bc700 1 -- 172.21.15.70:0/761210992 >> 172.21.15.70:0/761210992 conn(0x7fc6fc0f7430 msgr2=0x7fc6fc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.135+0000 7fc7047bc700 1 -- 172.21.15.70:0/761210992 shutdown_connections 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.135+0000 7fc7047bc700 1 -- 172.21.15.70:0/761210992 wait complete. 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc7047bc700 1 Processor -- start 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc7047bc700 1 -- start start 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc7047bc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 0x7fc6fc10c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc7047bc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6fc0fcf80 0x7fc6fc10cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc7047bc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc10d370 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc7047bc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc10dbc0 con 0x7fc6fc0fcf80 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc702558700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 0x7fc6fc10c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc702558700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 0x7fc6fc10c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42238/0 (socket says 172.21.15.70:42238) 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc702558700 1 -- 172.21.15.70:0/3343422276 learned_addr learned my addr 172.21.15.70:0/3343422276 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc701d57700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6fc0fcf80 0x7fc6fc10cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc702558700 1 -- 172.21.15.70:0/3343422276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6fc0fcf80 msgr2=0x7fc6fc10cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc702558700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6fc0fcf80 0x7fc6fc10cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc702558700 1 -- 172.21.15.70:0/3343422276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f0005040 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.136+0000 7fc702558700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 0x7fc6fc10c830 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000b3b0 tx=0x7fc6f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.137+0000 7fc6eeffd700 1 -- 172.21.15.70:0/3343422276 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6f000e040 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.137+0000 7fc6eeffd700 1 -- 172.21.15.70:0/3343422276 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6f000bd20 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.137+0000 7fc6eeffd700 1 -- 172.21.15.70:0/3343422276 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6f001c630 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.137+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc10de40 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.137+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc10e380 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.138+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6fc05f5f0 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.139+0000 7fc6eeffd700 1 -- 172.21.15.70:0/3343422276 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fc6f0013420 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.139+0000 7fc6eeffd700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc6e806c8a0 0x7fc6e806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:24.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.139+0000 7fc6eeffd700 1 -- 172.21.15.70:0/3343422276 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc6f008cdb0 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.139+0000 7fc701d57700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc6e806c8a0 0x7fc6e806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:24.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.139+0000 7fc701d57700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc6e806c8a0 0x7fc6e806ed50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8004c40 tx=0x7fc6f8009040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:24.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.143+0000 7fc6eeffd700 1 -- 172.21.15.70:0/3343422276 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fc6f005ae40 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.427+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc6fc104680 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.427+0000 7fc6eeffd700 1 -- 172.21.15.70:0/3343422276 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7fc6f005a9d0 con 0x7fc6fc0fbcd0 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:26:24.428 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:26:24.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:26:24.429 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:26:24.429 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-23T08:26:24.429 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:26:24.429 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:26:24.429 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.429+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc6e806c8a0 msgr2=0x7fc6e806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.429+0000 7fc7047bc700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc6e806c8a0 0x7fc6e806ed50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8004c40 tx=0x7fc6f8009040 comp rx=0 tx=0).stop 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.429+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 msgr2=0x7fc6fc10c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.429+0000 7fc7047bc700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 0x7fc6fc10c830 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000b3b0 tx=0x7fc6f0009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.430+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 shutdown_connections 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.430+0000 7fc7047bc700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc6e806c8a0 0x7fc6e806ed50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.430+0000 7fc7047bc700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6fc0fbcd0 0x7fc6fc10c830 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.430+0000 7fc7047bc700 1 --2- 172.21.15.70:0/3343422276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6fc0fcf80 0x7fc6fc10cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.430+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 >> 172.21.15.70:0/3343422276 conn(0x7fc6fc0f7430 msgr2=0x7fc6fc0f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.430+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 shutdown_connections 2024-03-23T08:26:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.430+0000 7fc7047bc700 1 -- 172.21.15.70:0/3343422276 wait complete. 2024-03-23T08:26:24.535 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.533+0000 7f66e4f5e700 1 -- 172.21.15.70:0/371189425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fbcd0 msgr2=0x7f66e00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.535 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.533+0000 7f66e4f5e700 1 --2- 172.21.15.70:0/371189425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fbcd0 0x7f66e00fc0e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f66d0004940 tx=0x7f66d0009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 -- 172.21.15.70:0/371189425 shutdown_connections 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 --2- 172.21.15.70:0/371189425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e00fcf80 0x7f66e00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 --2- 172.21.15.70:0/371189425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fbcd0 0x7f66e00fc0e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 -- 172.21.15.70:0/371189425 >> 172.21.15.70:0/371189425 conn(0x7f66e00f7430 msgr2=0x7f66e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 -- 172.21.15.70:0/371189425 shutdown_connections 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 -- 172.21.15.70:0/371189425 wait complete. 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 Processor -- start 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 -- start start 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e00fbcd0 0x7f66e010c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fcf80 0x7f66e010cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e010d370 con 0x7f66e00fcf80 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66e4f5e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e010dbc0 con 0x7f66e00fbcd0 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66ddd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fcf80 0x7f66e010cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66ddd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fcf80 0x7f66e010cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42260/0 (socket says 172.21.15.70:42260) 2024-03-23T08:26:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66ddd9b700 1 -- 172.21.15.70:0/3304724349 learned_addr learned my addr 172.21.15.70:0/3304724349 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.534+0000 7f66de59c700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e00fbcd0 0x7f66e010c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.535+0000 7f66ddd9b700 1 -- 172.21.15.70:0/3304724349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e00fbcd0 msgr2=0x7f66e010c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.535+0000 7f66ddd9b700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e00fbcd0 0x7f66e010c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.535+0000 7f66ddd9b700 1 -- 172.21.15.70:0/3304724349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d0005040 con 0x7f66e00fcf80 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.535+0000 7f66ddd9b700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fcf80 0x7f66e010cd70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f66d400c8c0 tx=0x7f66d40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.535+0000 7f66ceffd700 1 -- 172.21.15.70:0/3304724349 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d400a760 con 0x7f66e00fcf80 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.535+0000 7f66ceffd700 1 -- 172.21.15.70:0/3304724349 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66d400ad70 con 0x7f66e00fcf80 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.535+0000 7f66ceffd700 1 -- 172.21.15.70:0/3304724349 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d40113a0 con 0x7f66e00fcf80 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.535+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e010dea0 con 0x7f66e00fcf80 2024-03-23T08:26:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.535+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e010e3e0 con 0x7f66e00fcf80 2024-03-23T08:26:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.536+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e005f5f0 con 0x7f66e00fcf80 2024-03-23T08:26:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.536+0000 7f66ceffd700 1 -- 172.21.15.70:0/3304724349 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f66d4012400 con 0x7f66e00fcf80 2024-03-23T08:26:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.537+0000 7f66ceffd700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f66c806c8a0 0x7f66c806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.537+0000 7f66ceffd700 1 -- 172.21.15.70:0/3304724349 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f66d408b9d0 con 0x7f66e00fcf80 2024-03-23T08:26:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.537+0000 7f66de59c700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f66c806c8a0 0x7f66c806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.537+0000 7f66de59c700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f66c806c8a0 0x7f66c806ed50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66d000b3b0 tx=0x7f66d0009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.541+0000 7f66ceffd700 1 -- 172.21.15.70:0/3304724349 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f66d4059a60 con 0x7f66e00fcf80 2024-03-23T08:26:24.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:24 smithi070 ceph-mon[39699]: pgmap v52: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-23T08:26:24.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:24 smithi070 ceph-mon[39699]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:24.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:24 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:24.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:24 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:24.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:24 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:24.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:24 smithi070 ceph-mon[39699]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:24.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:24 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/3343422276' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.759+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66e010e820 con 0x7f66c806c8a0 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.761+0000 7f66ceffd700 1 -- 172.21.15.70:0/3304724349 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f66e010e820 con 0x7f66c806c8a0 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/2 daemons upgraded", 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading alertmanager daemons", 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:26:24.761 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:26:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f66c806c8a0 msgr2=0x7f66c806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f66c806c8a0 0x7f66c806ed50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66d000b3b0 tx=0x7f66d0009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fcf80 msgr2=0x7f66e010cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fcf80 0x7f66e010cd70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f66d400c8c0 tx=0x7f66d40044c0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 shutdown_connections 2024-03-23T08:26:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f66c806c8a0 0x7f66c806ed50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e00fbcd0 0x7f66e010c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 --2- 172.21.15.70:0/3304724349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e00fcf80 0x7f66e010cd70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 >> 172.21.15.70:0/3304724349 conn(0x7f66e00f7430 msgr2=0x7f66e00f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 shutdown_connections 2024-03-23T08:26:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:24.763+0000 7f66e4f5e700 1 -- 172.21.15.70:0/3304724349 wait complete. 2024-03-23T08:26:24.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:24 smithi193 ceph-mon[48370]: pgmap v52: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-23T08:26:24.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:24 smithi193 ceph-mon[48370]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:24.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:24 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:24.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:24 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:24.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:24 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:24.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:24 smithi193 ceph-mon[48370]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:24.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:24 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/3343422276' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:25.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:25 smithi070 ceph-mon[39699]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:25.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:25 smithi070 ceph-mon[39699]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:25 smithi193 ceph-mon[48370]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:25 smithi193 ceph-mon[48370]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:26 smithi070 ceph-mon[39699]: pgmap v53: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-23T08:26:27.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:26 smithi193 ceph-mon[48370]: pgmap v53: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-23T08:26:28.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:28 smithi070 ceph-mon[39699]: pgmap v54: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-23T08:26:29.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:28 smithi193 ceph-mon[48370]: pgmap v54: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-23T08:26:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:29 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:29 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:29 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:29 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:30.133 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:29 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:30.133 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:29 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:30.133 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:29 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:30.133 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:29 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:30.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:30 smithi070 ceph-mon[39699]: pgmap v55: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:30.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:26:31.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:30 smithi193 ceph-mon[48370]: pgmap v55: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:31.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:26:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:32 smithi193 ceph-mon[48370]: pgmap v56: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:33.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:32 smithi070 ceph-mon[39699]: pgmap v56: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:35.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:34 smithi193 ceph-mon[48370]: pgmap v57: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:34 smithi070 ceph-mon[39699]: pgmap v57: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:37.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:36 smithi193 ceph-mon[48370]: pgmap v58: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:37.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:36 smithi070 ceph-mon[39699]: pgmap v58: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:39.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:38 smithi193 ceph-mon[48370]: pgmap v59: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:39.109 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:38 smithi070 ceph-mon[39699]: pgmap v59: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:40.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: pgmap v60: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:40 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:40.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:40.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: pgmap v60: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:40.577 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:40 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:41.414 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:41 smithi070 ceph-mon[39699]: Upgrade: Updating grafana.smithi070 2024-03-23T08:26:41.414 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:41 smithi070 ceph-mon[39699]: Deploying daemon grafana.smithi070 on smithi070 2024-03-23T08:26:41.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:41 smithi193 ceph-mon[48370]: Upgrade: Updating grafana.smithi070 2024-03-23T08:26:41.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:41 smithi193 ceph-mon[48370]: Deploying daemon grafana.smithi070 on smithi070 2024-03-23T08:26:42.349 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:42 smithi070 ceph-mon[39699]: pgmap v61: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:42.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:42 smithi193 ceph-mon[48370]: pgmap v61: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:44.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:44 smithi070 ceph-mon[39699]: pgmap v62: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:44.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:44 smithi193 ceph-mon[48370]: pgmap v62: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:45.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:45 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:26:45.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:45 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:26:46.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:46 smithi070 ceph-mon[39699]: pgmap v63: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:46.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:46 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:46.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:46 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:46.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:46 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:46 smithi193 ceph-mon[48370]: pgmap v63: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:46 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:46 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:47.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:46 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:48 smithi070 ceph-mon[39699]: pgmap v64: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:49.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:48 smithi193 ceph-mon[48370]: pgmap v64: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:50 smithi070 ceph-mon[39699]: pgmap v65: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:50 smithi193 ceph-mon[48370]: pgmap v65: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:52 smithi193 ceph-mon[48370]: pgmap v66: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:52.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:52 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:52.408 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:52.408 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:52.408 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:52 smithi070 ceph-mon[39699]: pgmap v66: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:52.408 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:52.408 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:52 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:54.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:54 smithi070 ceph-mon[39699]: pgmap v67: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:54.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:54 smithi193 ceph-mon[48370]: pgmap v67: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.880+0000 7f53f9094700 1 -- 172.21.15.70:0/3814594178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 msgr2=0x7f53f40fc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.880+0000 7f53f9094700 1 --2- 172.21.15.70:0/3814594178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 0x7f53f40fc760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f53e4004940 tx=0x7f53e4009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.880+0000 7f53f9094700 1 -- 172.21.15.70:0/3814594178 shutdown_connections 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.880+0000 7f53f9094700 1 --2- 172.21.15.70:0/3814594178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f40fcca0 0x7f53f40fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.880+0000 7f53f9094700 1 --2- 172.21.15.70:0/3814594178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 0x7f53f40fc760 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.880+0000 7f53f9094700 1 -- 172.21.15.70:0/3814594178 >> 172.21.15.70:0/3814594178 conn(0x7f53f40f6be0 msgr2=0x7f53f40f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.880+0000 7f53f9094700 1 -- 172.21.15.70:0/3814594178 shutdown_connections 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.880+0000 7f53f9094700 1 -- 172.21.15.70:0/3814594178 wait complete. 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.880+0000 7f53f9094700 1 Processor -- start 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f9094700 1 -- start start 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f9094700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 0x7f53f410c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f9094700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f40fcca0 0x7f53f410ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:54.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f9094700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f410d2b0 con 0x7f53f40fc350 2024-03-23T08:26:54.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f9094700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f410d420 con 0x7f53f40fcca0 2024-03-23T08:26:54.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f2d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 0x7f53f410c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:54.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f2d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 0x7f53f410c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43550/0 (socket says 172.21.15.70:43550) 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f2d9d700 1 -- 172.21.15.70:0/2861550782 learned_addr learned my addr 172.21.15.70:0/2861550782 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53ea59c700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f40fcca0 0x7f53f410ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f2d9d700 1 -- 172.21.15.70:0/2861550782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f40fcca0 msgr2=0x7f53f410ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f2d9d700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f40fcca0 0x7f53f410ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f2d9d700 1 -- 172.21.15.70:0/2861550782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e4005040 con 0x7f53f40fc350 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53ea59c700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f40fcca0 0x7f53f410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f2d9d700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 0x7f53f410c770 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f53e400b3b0 tx=0x7f53e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53ebfff700 1 -- 172.21.15.70:0/2861550782 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53e400e040 con 0x7f53f40fc350 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f4111e20 con 0x7f53f40fc350 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53ebfff700 1 -- 172.21.15.70:0/2861550782 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f53e400bd20 con 0x7f53f40fc350 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53ebfff700 1 -- 172.21.15.70:0/2861550782 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53e401c630 con 0x7f53f40fc350 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.881+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f4112350 con 0x7f53f40fc350 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.883+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f404c710 con 0x7f53f40fc350 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.883+0000 7f53ebfff700 1 -- 172.21.15.70:0/2861550782 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f53e4013420 con 0x7f53f40fc350 2024-03-23T08:26:54.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.884+0000 7f53ebfff700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f53e006c8a0 0x7f53e006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:54.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.884+0000 7f53ea59c700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f53e006c8a0 0x7f53e006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:54.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.884+0000 7f53ebfff700 1 -- 172.21.15.70:0/2861550782 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f53e408c670 con 0x7f53f40fc350 2024-03-23T08:26:54.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.884+0000 7f53ea59c700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f53e006c8a0 0x7f53e006ed50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0041d0 tx=0x7f53dc009040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:54.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:54.888+0000 7f53ebfff700 1 -- 172.21.15.70:0/2861550782 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f53e405a660 con 0x7f53f40fc350 2024-03-23T08:26:55.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.081+0000 7f53ebfff700 1 -- 172.21.15.70:0/2861550782 <== mon.0 v2:172.21.15.70:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f53e405a660 con 0x7f53f40fc350 2024-03-23T08:26:55.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.123+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53d8000d50 con 0x7f53e006c8a0 2024-03-23T08:26:55.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.124+0000 7f53ebfff700 1 -- 172.21.15.70:0/2861550782 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f53d8000d50 con 0x7f53e006c8a0 2024-03-23T08:26:55.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f53e006c8a0 msgr2=0x7f53e006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:55.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f53e006c8a0 0x7f53e006ed50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0041d0 tx=0x7f53dc009040 comp rx=0 tx=0).stop 2024-03-23T08:26:55.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 msgr2=0x7f53f410c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:55.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 0x7f53f410c770 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f53e400b3b0 tx=0x7f53e4009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:55.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 shutdown_connections 2024-03-23T08:26:55.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f53e006c8a0 0x7f53e006ed50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:55.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40fc350 0x7f53f410c770 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:55.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 --2- 172.21.15.70:0/2861550782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53f40fcca0 0x7f53f410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:55.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 >> 172.21.15.70:0/2861550782 conn(0x7f53f40f6be0 msgr2=0x7f53f40f8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:55.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 shutdown_connections 2024-03-23T08:26:55.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:55.127+0000 7f53f9094700 1 -- 172.21.15.70:0/2861550782 wait complete. 2024-03-23T08:26:55.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-23T08:26:55.906 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:55.906 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.907 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-23T08:26:55.908 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-23T08:26:55.909 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-23T08:26:55.909 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:55.909 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-23T08:26:55.909 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-23T08:26:55.909 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-23T08:26:55.909 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-23T08:26:55.909 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:55.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:55 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:56.045 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:26:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-23T08:26:56.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-23T08:26:56.328 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-23T08:26:56.329 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-23T08:26:56.330 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:56.330 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:56.330 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:56.330 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:56.330 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:56.330 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:26:56.330 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:26:56.330 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:26:56.330 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:55 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:26:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:56 smithi070 ceph-mon[39699]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:26:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:56 smithi070 ceph-mon[39699]: Upgrade: Finalizing container_image settings 2024-03-23T08:26:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:56 smithi070 ceph-mon[39699]: Upgrade: Complete! 2024-03-23T08:26:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:56 smithi070 ceph-mon[39699]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:56 smithi070 ceph-mon[39699]: pgmap v68: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:57.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:56 smithi193 ceph-mon[48370]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-23T08:26:57.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:56 smithi193 ceph-mon[48370]: Upgrade: Finalizing container_image settings 2024-03-23T08:26:57.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:56 smithi193 ceph-mon[48370]: Upgrade: Complete! 2024-03-23T08:26:57.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:56 smithi193 ceph-mon[48370]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:26:57.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:56 smithi193 ceph-mon[48370]: pgmap v68: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.888+0000 7f08579ed700 1 -- 172.21.15.70:0/2294494532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 msgr2=0x7f08500fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.888+0000 7f08579ed700 1 --2- 172.21.15.70:0/2294494532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 0x7f08500fce30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f084c004940 tx=0x7f084c009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.888+0000 7f08579ed700 1 -- 172.21.15.70:0/2294494532 shutdown_connections 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.888+0000 7f08579ed700 1 --2- 172.21.15.70:0/2294494532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 0x7f08500fce30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.888+0000 7f08579ed700 1 --2- 172.21.15.70:0/2294494532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0850102e20 0x7f08501031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.888+0000 7f08579ed700 1 -- 172.21.15.70:0/2294494532 >> 172.21.15.70:0/2294494532 conn(0x7f08500f8970 msgr2=0x7f08500fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.888+0000 7f08579ed700 1 -- 172.21.15.70:0/2294494532 shutdown_connections 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.888+0000 7f08579ed700 1 -- 172.21.15.70:0/2294494532 wait complete. 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f08579ed700 1 Processor -- start 2024-03-23T08:26:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f08579ed700 1 -- start start 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f08579ed700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0850102e20 0x7f08500702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f08579ed700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 0x7f0850070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f08579ed700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0850070d90 con 0x7f08501037c0 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f08579ed700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0850070ed0 con 0x7f0850102e20 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f0854f88700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 0x7f0850070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f0854f88700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 0x7f0850070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43568/0 (socket says 172.21.15.70:43568) 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f0854f88700 1 -- 172.21.15.70:0/2563517360 learned_addr learned my addr 172.21.15.70:0/2563517360 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f0855789700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0850102e20 0x7f08500702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f0854f88700 1 -- 172.21.15.70:0/2563517360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0850102e20 msgr2=0x7f08500702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f0854f88700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0850102e20 0x7f08500702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.889+0000 7f0854f88700 1 -- 172.21.15.70:0/2563517360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c005040 con 0x7f08501037c0 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.890+0000 7f0854f88700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 0x7f0850070800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f084c009380 tx=0x7f084c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.890+0000 7f0841ffb700 1 -- 172.21.15.70:0/2563517360 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084c00e040 con 0x7f08501037c0 2024-03-23T08:26:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.890+0000 7f0841ffb700 1 -- 172.21.15.70:0/2563517360 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f084c013420 con 0x7f08501037c0 2024-03-23T08:26:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.890+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0850071120 con 0x7f08501037c0 2024-03-23T08:26:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.890+0000 7f0841ffb700 1 -- 172.21.15.70:0/2563517360 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084c01c630 con 0x7f08501037c0 2024-03-23T08:26:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.890+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085011a1f0 con 0x7f08501037c0 2024-03-23T08:26:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.891+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08500fe570 con 0x7f08501037c0 2024-03-23T08:26:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.891+0000 7f0841ffb700 1 -- 172.21.15.70:0/2563517360 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f084c0139b0 con 0x7f08501037c0 2024-03-23T08:26:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.892+0000 7f0841ffb700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f083c06c8a0 0x7f083c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:26:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.892+0000 7f0841ffb700 1 -- 172.21.15.70:0/2563517360 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f084c08cbf0 con 0x7f08501037c0 2024-03-23T08:26:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.892+0000 7f0855789700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f083c06c8a0 0x7f083c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:26:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.892+0000 7f0855789700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f083c06c8a0 0x7f083c06ed50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0844004f40 tx=0x7f084400b040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:26:57.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:57.896+0000 7f0841ffb700 1 -- 172.21.15.70:0/2563517360 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f084c057150 con 0x7f08501037c0 2024-03-23T08:26:58.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.168+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f08500fe570 con 0x7f08501037c0 2024-03-23T08:26:58.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.168+0000 7f0841ffb700 1 -- 172.21.15.70:0/2563517360 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f084c05a770 con 0x7f08501037c0 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.170+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f083c06c8a0 msgr2=0x7f083c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.170+0000 7f08579ed700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f083c06c8a0 0x7f083c06ed50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0844004f40 tx=0x7f084400b040 comp rx=0 tx=0).stop 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.170+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 msgr2=0x7f0850070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.170+0000 7f08579ed700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 0x7f0850070800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f084c009380 tx=0x7f084c009e30 comp rx=0 tx=0).stop 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.171+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 shutdown_connections 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.171+0000 7f08579ed700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f083c06c8a0 0x7f083c06ed50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.171+0000 7f08579ed700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0850102e20 0x7f08500702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.171+0000 7f08579ed700 1 --2- 172.21.15.70:0/2563517360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08501037c0 0x7f0850070800 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.171+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 >> 172.21.15.70:0/2563517360 conn(0x7f08500f8970 msgr2=0x7f08500f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.171+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 shutdown_connections 2024-03-23T08:26:58.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:26:58.171+0000 7f08579ed700 1 -- 172.21.15.70:0/2563517360 wait complete. 2024-03-23T08:26:58.181 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:26:58.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-23T08:26:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:58 smithi070 ceph-mon[39699]: pgmap v69: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:26:58 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2563517360' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:26:59.019 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:26:59.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:58 smithi193 ceph-mon[48370]: pgmap v69: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:26:59.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:26:58 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2563517360' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:00.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.857+0000 7fd46e278700 1 -- 172.21.15.70:0/1026979456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 msgr2=0x7fd4680fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.857+0000 7fd46e278700 1 --2- 172.21.15.70:0/1026979456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 0x7fd4680fc5b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd45c004940 tx=0x7fd45c009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.857+0000 7fd46e278700 1 -- 172.21.15.70:0/1026979456 shutdown_connections 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.857+0000 7fd46e278700 1 --2- 172.21.15.70:0/1026979456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 0x7fd4680fc5b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.857+0000 7fd46e278700 1 --2- 172.21.15.70:0/1026979456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd468101d50 0x7fd468102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.857+0000 7fd46e278700 1 -- 172.21.15.70:0/1026979456 >> 172.21.15.70:0/1026979456 conn(0x7fd4680714c0 msgr2=0x7fd4680718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.858+0000 7fd46e278700 1 -- 172.21.15.70:0/1026979456 shutdown_connections 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.858+0000 7fd46e278700 1 -- 172.21.15.70:0/1026979456 wait complete. 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.858+0000 7fd46e278700 1 Processor -- start 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.858+0000 7fd46e278700 1 -- start start 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.858+0000 7fd46e278700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd468101d50 0x7fd468113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.858+0000 7fd46e278700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 0x7fd4681140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.858+0000 7fd46e278700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd468114790 con 0x7fd468102660 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.858+0000 7fd46e278700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46810daf0 con 0x7fd468101d50 2024-03-23T08:27:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd4677fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 0x7fd4681140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd4677fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 0x7fd4681140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60734/0 (socket says 172.21.15.70:60734) 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd4677fe700 1 -- 172.21.15.70:0/343780867 learned_addr learned my addr 172.21.15.70:0/343780867 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd467fff700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd468101d50 0x7fd468113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd4677fe700 1 -- 172.21.15.70:0/343780867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd468101d50 msgr2=0x7fd468113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd4677fe700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd468101d50 0x7fd468113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd4677fe700 1 -- 172.21.15.70:0/343780867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c005040 con 0x7fd468102660 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd4677fe700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 0x7fd4681140b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009180 tx=0x7fd45c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd464ff9700 1 -- 172.21.15.70:0/343780867 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45c00e040 con 0x7fd468102660 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46810dd70 con 0x7fd468102660 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd464ff9700 1 -- 172.21.15.70:0/343780867 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd45c013420 con 0x7fd468102660 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd464ff9700 1 -- 172.21.15.70:0/343780867 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45c01c630 con 0x7fd468102660 2024-03-23T08:27:00.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.859+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46810e2b0 con 0x7fd468102660 2024-03-23T08:27:00.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.860+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4680fd5c0 con 0x7fd468102660 2024-03-23T08:27:00.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.861+0000 7fd464ff9700 1 -- 172.21.15.70:0/343780867 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fd45c0139e0 con 0x7fd468102660 2024-03-23T08:27:00.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.861+0000 7fd464ff9700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd45006c8a0 0x7fd45006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:00.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.861+0000 7fd464ff9700 1 -- 172.21.15.70:0/343780867 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd45c08cd50 con 0x7fd468102660 2024-03-23T08:27:00.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.861+0000 7fd467fff700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd45006c8a0 0x7fd45006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:00.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.862+0000 7fd467fff700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd45006c8a0 0x7fd45006ed50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd458006fd0 tx=0x7fd458008040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:00.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:00.866+0000 7fd464ff9700 1 -- 172.21.15.70:0/343780867 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fd45c05ad40 con 0x7fd468102660 2024-03-23T08:27:01.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.140+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd46805f5f0 con 0x7fd468102660 2024-03-23T08:27:01.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.140+0000 7fd464ff9700 1 -- 172.21.15.70:0/343780867 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7fd45c05a8d0 con 0x7fd468102660 2024-03-23T08:27:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.142+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd45006c8a0 msgr2=0x7fd45006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.142+0000 7fd46e278700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd45006c8a0 0x7fd45006ed50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd458006fd0 tx=0x7fd458008040 comp rx=0 tx=0).stop 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.142+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 msgr2=0x7fd4681140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.142+0000 7fd46e278700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 0x7fd4681140b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009180 tx=0x7fd45c009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.142+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 shutdown_connections 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.142+0000 7fd46e278700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd45006c8a0 0x7fd45006ed50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.142+0000 7fd46e278700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd468101d50 0x7fd468113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.142+0000 7fd46e278700 1 --2- 172.21.15.70:0/343780867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd468102660 0x7fd4681140b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.142+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 >> 172.21.15.70:0/343780867 conn(0x7fd4680714c0 msgr2=0x7fd4680fb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.143+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 shutdown_connections 2024-03-23T08:27:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:01.143+0000 7fd46e278700 1 -- 172.21.15.70:0/343780867 wait complete. 2024-03-23T08:27:01.153 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)" 2024-03-23T08:27:01.163 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:00 smithi070 ceph-mon[39699]: pgmap v70: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:01.163 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:00 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:27:01.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:00 smithi193 ceph-mon[48370]: pgmap v70: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:01.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:00 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:27:01.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-23T08:27:02.132 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:27:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:01 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/343780867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:01 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/343780867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:03.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:02 smithi070 ceph-mon[39699]: pgmap v71: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:03.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:02 smithi193 ceph-mon[48370]: pgmap v71: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:04.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.134+0000 7f6db4c25700 1 -- 172.21.15.70:0/3971595750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fe830 msgr2=0x7f6db0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:04.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.134+0000 7f6db4c25700 1 --2- 172.21.15.70:0/3971595750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fe830 0x7f6db0106f00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6d98004940 tx=0x7f6d98009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:04.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.134+0000 7f6db4c25700 1 -- 172.21.15.70:0/3971595750 shutdown_connections 2024-03-23T08:27:04.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.134+0000 7f6db4c25700 1 --2- 172.21.15.70:0/3971595750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fe830 0x7f6db0106f00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.134+0000 7f6db4c25700 1 --2- 172.21.15.70:0/3971595750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6db00fde90 0x7f6db00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.134+0000 7f6db4c25700 1 -- 172.21.15.70:0/3971595750 >> 172.21.15.70:0/3971595750 conn(0x7f6db0072bc0 msgr2=0x7f6db0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.134+0000 7f6db4c25700 1 -- 172.21.15.70:0/3971595750 shutdown_connections 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.134+0000 7f6db4c25700 1 -- 172.21.15.70:0/3971595750 wait complete. 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6db4c25700 1 Processor -- start 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6db4c25700 1 -- start start 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6db4c25700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fde90 0x7f6db0113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6db4c25700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6db00fe830 0x7f6db0113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6db4c25700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01146d0 con 0x7f6db00fde90 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6db4c25700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db010da30 con 0x7f6db00fe830 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6dae59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fde90 0x7f6db0113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6dae59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fde90 0x7f6db0113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60762/0 (socket says 172.21.15.70:60762) 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6dae59c700 1 -- 172.21.15.70:0/2089705187 learned_addr learned my addr 172.21.15.70:0/2089705187 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6dae59c700 1 -- 172.21.15.70:0/2089705187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6db00fe830 msgr2=0x7f6db0113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6dadd9b700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6db00fe830 0x7f6db0113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6dae59c700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6db00fe830 0x7f6db0113ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6dae59c700 1 -- 172.21.15.70:0/2089705187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98005040 con 0x7f6db00fde90 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6dadd9b700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6db00fe830 0x7f6db0113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.135+0000 7f6dae59c700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fde90 0x7f6db0113ab0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6da000b790 tx=0x7f6da0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.136+0000 7f6da6ffd700 1 -- 172.21.15.70:0/2089705187 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6da000f770 con 0x7f6db00fde90 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.136+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db010dd10 con 0x7f6db00fde90 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.136+0000 7f6da6ffd700 1 -- 172.21.15.70:0/2089705187 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6da000fd80 con 0x7f6db00fde90 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.136+0000 7f6da6ffd700 1 -- 172.21.15.70:0/2089705187 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6da000d310 con 0x7f6db00fde90 2024-03-23T08:27:04.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.136+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db010e250 con 0x7f6db00fde90 2024-03-23T08:27:04.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.137+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db0065e30 con 0x7f6db00fde90 2024-03-23T08:27:04.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.137+0000 7f6da6ffd700 1 -- 172.21.15.70:0/2089705187 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f6da0010420 con 0x7f6db00fde90 2024-03-23T08:27:04.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.137+0000 7f6da6ffd700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f6d9c06c8a0 0x7f6d9c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:04.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.138+0000 7f6da6ffd700 1 -- 172.21.15.70:0/2089705187 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6da008ba90 con 0x7f6db00fde90 2024-03-23T08:27:04.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.138+0000 7f6dadd9b700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f6d9c06c8a0 0x7f6d9c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:04.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.138+0000 7f6dadd9b700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f6d9c06c8a0 0x7f6d9c06ed50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800b3b0 tx=0x7f6d98009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:04.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.142+0000 7f6da6ffd700 1 -- 172.21.15.70:0/2089705187 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f6da0059a80 con 0x7f6db00fde90 2024-03-23T08:27:04.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.413+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6db005f5f0 con 0x7f6db00fde90 2024-03-23T08:27:04.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.413+0000 7f6da6ffd700 1 -- 172.21.15.70:0/2089705187 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7f6da0059610 con 0x7f6db00fde90 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.415+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f6d9c06c8a0 msgr2=0x7f6d9c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.415+0000 7f6db4c25700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f6d9c06c8a0 0x7f6d9c06ed50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800b3b0 tx=0x7f6d98009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.416+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fde90 msgr2=0x7f6db0113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.416+0000 7f6db4c25700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fde90 0x7f6db0113ab0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6da000b790 tx=0x7f6da0009250 comp rx=0 tx=0).stop 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.416+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 shutdown_connections 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.416+0000 7f6db4c25700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f6d9c06c8a0 0x7f6d9c06ed50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.416+0000 7f6db4c25700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db00fde90 0x7f6db0113ab0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.416+0000 7f6db4c25700 1 --2- 172.21.15.70:0/2089705187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6db00fe830 0x7f6db0113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.416+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 >> 172.21.15.70:0/2089705187 conn(0x7f6db0072bc0 msgr2=0x7f6db01037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.416+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 shutdown_connections 2024-03-23T08:27:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:04.416+0000 7f6db4c25700 1 -- 172.21.15.70:0/2089705187 wait complete. 2024-03-23T08:27:04.426 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:27:05.132 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:04 smithi070 ceph-mon[39699]: pgmap v72: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:05.132 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:04 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2089705187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:05.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-23T08:27:05.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:04 smithi193 ceph-mon[48370]: pgmap v72: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:05.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:04 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2089705187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:05.420 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:27:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.091+0000 7fe2bda9f700 1 -- 172.21.15.70:0/2365973222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 msgr2=0x7fe2b80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.091+0000 7fe2bda9f700 1 --2- 172.21.15.70:0/2365973222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 0x7fe2b80fcdf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac004940 tx=0x7fe2ac009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 -- 172.21.15.70:0/2365973222 shutdown_connections 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 --2- 172.21.15.70:0/2365973222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 0x7fe2b80fcdf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 --2- 172.21.15.70:0/2365973222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8102de0 0x7fe2b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 -- 172.21.15.70:0/2365973222 >> 172.21.15.70:0/2365973222 conn(0x7fe2b80f8950 msgr2=0x7fe2b80fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 -- 172.21.15.70:0/2365973222 shutdown_connections 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 -- 172.21.15.70:0/2365973222 wait complete. 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 Processor -- start 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 -- start start 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8102de0 0x7fe2b8113850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 0x7fe2b8113d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8114470 con 0x7fe2b8103780 2024-03-23T08:27:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.092+0000 7fe2bda9f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b810da20 con 0x7fe2b8102de0 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 0x7fe2b8113d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2b6ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 0x7fe2b8113d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60768/0 (socket says 172.21.15.70:60768) 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2b6ffd700 1 -- 172.21.15.70:0/864302759 learned_addr learned my addr 172.21.15.70:0/864302759 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2b77fe700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8102de0 0x7fe2b8113850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2b6ffd700 1 -- 172.21.15.70:0/864302759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8102de0 msgr2=0x7fe2b8113850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2b6ffd700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8102de0 0x7fe2b8113850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2b6ffd700 1 -- 172.21.15.70:0/864302759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac005040 con 0x7fe2b8103780 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2b6ffd700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 0x7fe2b8113d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0093b0 tx=0x7fe2ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe29ffff700 1 -- 172.21.15.70:0/864302759 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2ac00e040 con 0x7fe2b8103780 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b810dca0 con 0x7fe2b8103780 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe29ffff700 1 -- 172.21.15.70:0/864302759 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2ac013420 con 0x7fe2b8103780 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe29ffff700 1 -- 172.21.15.70:0/864302759 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2ac01c630 con 0x7fe2b8103780 2024-03-23T08:27:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.093+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b810e260 con 0x7fe2b8103780 2024-03-23T08:27:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.094+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b80fe530 con 0x7fe2b8103780 2024-03-23T08:27:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.095+0000 7fe29ffff700 1 -- 172.21.15.70:0/864302759 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fe2ac0139e0 con 0x7fe2b8103780 2024-03-23T08:27:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.095+0000 7fe29ffff700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe2a006c8a0 0x7fe2a006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.095+0000 7fe29ffff700 1 -- 172.21.15.70:0/864302759 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe2ac08cb80 con 0x7fe2b8103780 2024-03-23T08:27:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.095+0000 7fe2b77fe700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe2a006c8a0 0x7fe2a006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:07.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.096+0000 7fe2b77fe700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe2a006c8a0 0x7fe2a006ed50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009bf0 tx=0x7fe2a80091a0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.100+0000 7fe29ffff700 1 -- 172.21.15.70:0/864302759 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fe2ac05ab70 con 0x7fe2b8103780 2024-03-23T08:27:07.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:06 smithi070 ceph-mon[39699]: pgmap v73: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:07.310 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:07.310+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2b8109e60 con 0x7fe2a006c8a0 2024-03-23T08:27:07.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:06 smithi193 ceph-mon[48370]: pgmap v73: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:08 smithi070 ceph-mon[39699]: pgmap v74: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:09.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:08 smithi070 ceph-mon[39699]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:08 smithi193 ceph-mon[48370]: pgmap v74: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:08 smithi193 ceph-mon[48370]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:10.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.584+0000 7fe29ffff700 1 -- 172.21.15.70:0/864302759 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5093 (secure 0 0 0) 0x7fe2b8109e60 con 0x7fe2a006c8a0 2024-03-23T08:27:10.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe2a006c8a0 msgr2=0x7fe2a006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:10.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe2a006c8a0 0x7fe2a006ed50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009bf0 tx=0x7fe2a80091a0 comp rx=0 tx=0).stop 2024-03-23T08:27:10.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 msgr2=0x7fe2b8113d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:10.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 0x7fe2b8113d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0093b0 tx=0x7fe2ac009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:10.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 shutdown_connections 2024-03-23T08:27:10.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe2a006c8a0 0x7fe2a006ed50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:10.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b8102de0 0x7fe2b8113850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:10.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 --2- 172.21.15.70:0/864302759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2b8103780 0x7fe2b8113d90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:10.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 >> 172.21.15.70:0/864302759 conn(0x7fe2b80f8950 msgr2=0x7fe2b80f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:10.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 shutdown_connections 2024-03-23T08:27:10.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:10.586+0000 7fe2bda9f700 1 -- 172.21.15.70:0/864302759 wait complete. 2024-03-23T08:27:10.598 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:27:11.228 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:10 smithi070 ceph-mon[39699]: pgmap v75: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:11.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph orch ps' 2024-03-23T08:27:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:10 smithi193 ceph-mon[48370]: pgmap v75: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:11.523 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:27:13.090 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:12 smithi070 ceph-mon[39699]: pgmap v76: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:12 smithi193 ceph-mon[48370]: pgmap v76: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 -- 172.21.15.70:0/508390640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c1026c0 msgr2=0x7fe31c0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 --2- 172.21.15.70:0/508390640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c1026c0 0x7fe31c0fbd30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe30c004940 tx=0x7fe30c009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 -- 172.21.15.70:0/508390640 shutdown_connections 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 --2- 172.21.15.70:0/508390640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c1026c0 0x7fe31c0fbd30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 --2- 172.21.15.70:0/508390640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31c101d20 0x7fe31c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 -- 172.21.15.70:0/508390640 >> 172.21.15.70:0/508390640 conn(0x7fe31c0f78d0 msgr2=0x7fe31c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 -- 172.21.15.70:0/508390640 shutdown_connections 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 -- 172.21.15.70:0/508390640 wait complete. 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 Processor -- start 2024-03-23T08:27:13.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.386+0000 7fe3238c7700 1 -- start start 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe3238c7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c101d20 0x7fe31c10ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe3238c7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31c1026c0 0x7fe31c10f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe3238c7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c10f6c0 con 0x7fe31c101d20 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe3238c7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c10f800 con 0x7fe31c1026c0 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe321663700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c101d20 0x7fe31c10ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe321663700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c101d20 0x7fe31c10ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33092/0 (socket says 172.21.15.70:33092) 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe321663700 1 -- 172.21.15.70:0/1679516742 learned_addr learned my addr 172.21.15.70:0/1679516742 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe320e62700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31c1026c0 0x7fe31c10f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe321663700 1 -- 172.21.15.70:0/1679516742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31c1026c0 msgr2=0x7fe31c10f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe321663700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31c1026c0 0x7fe31c10f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe321663700 1 -- 172.21.15.70:0/1679516742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c005040 con 0x7fe31c101d20 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe321663700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c101d20 0x7fe31c10ec40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe318002a50 tx=0x7fe3180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe311ffb700 1 -- 172.21.15.70:0/1679516742 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe318007cc0 con 0x7fe31c101d20 2024-03-23T08:27:13.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c117af0 con 0x7fe31c101d20 2024-03-23T08:27:13.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe311ffb700 1 -- 172.21.15.70:0/1679516742 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe31800f040 con 0x7fe31c101d20 2024-03-23T08:27:13.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.387+0000 7fe311ffb700 1 -- 172.21.15.70:0/1679516742 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe318009540 con 0x7fe31c101d20 2024-03-23T08:27:13.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.388+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c118080 con 0x7fe31c101d20 2024-03-23T08:27:13.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.389+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c0fd470 con 0x7fe31c101d20 2024-03-23T08:27:13.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.389+0000 7fe311ffb700 1 -- 172.21.15.70:0/1679516742 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fe3180096e0 con 0x7fe31c101d20 2024-03-23T08:27:13.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.389+0000 7fe311ffb700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe30806c8a0 0x7fe30806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:13.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.389+0000 7fe311ffb700 1 -- 172.21.15.70:0/1679516742 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe31808b8a0 con 0x7fe31c101d20 2024-03-23T08:27:13.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.389+0000 7fe320e62700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe30806c8a0 0x7fe30806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:13.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.390+0000 7fe320e62700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe30806c8a0 0x7fe30806ed50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00b3b0 tx=0x7fe30c009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:13.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.394+0000 7fe311ffb700 1 -- 172.21.15.70:0/1679516742 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fe318055d80 con 0x7fe31c101d20 2024-03-23T08:27:13.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.599+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe31c066660 con 0x7fe30806c8a0 2024-03-23T08:27:13.608 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.608+0000 7fe311ffb700 1 -- 172.21.15.70:0/1679516742 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe31c066660 con 0x7fe30806c8a0 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (50s) 22s ago 11m 16.7M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (12m) 22s ago 12m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (11m) 82s ago 11m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (28s) 22s ago 11m 45.4M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (6m) 22s ago 6m 148M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (6m) 22s ago 6m 12.9M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (6m) 82s ago 6m 97.0M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (6m) 82s ago 6m 14.3M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (2m) 22s ago 13m 497M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:27:13.609 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (104s) 82s ago 11m 449M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (13m) 22s ago 13m 56.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (11m) 82s ago 11m 41.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (90s) 22s ago 12m 9332k - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (86s) 82s ago 11m 3561k - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (10m) 22s ago 10m 217M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (9m) 22s ago 9m 233M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (9m) 22s ago 9m 202M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (8m) 82s ago 8m 276M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (8m) 82s ago 8m 241M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (8m) 82s ago 8m 223M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:27:13.610 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (67s) 22s ago 11m 46.4M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:27:13.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.610+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe30806c8a0 msgr2=0x7fe30806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:13.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.610+0000 7fe3238c7700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe30806c8a0 0x7fe30806ed50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00b3b0 tx=0x7fe30c009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:13.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.610+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c101d20 msgr2=0x7fe31c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:13.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.610+0000 7fe3238c7700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c101d20 0x7fe31c10ec40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe318002a50 tx=0x7fe3180046b0 comp rx=0 tx=0).stop 2024-03-23T08:27:13.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.610+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 shutdown_connections 2024-03-23T08:27:13.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.610+0000 7fe3238c7700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fe30806c8a0 0x7fe30806ed50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:13.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.610+0000 7fe3238c7700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31c101d20 0x7fe31c10ec40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:13.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.610+0000 7fe3238c7700 1 --2- 172.21.15.70:0/1679516742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe31c1026c0 0x7fe31c10f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:13.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.611+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 >> 172.21.15.70:0/1679516742 conn(0x7fe31c0f78d0 msgr2=0x7fe31c0f89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:13.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.611+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 shutdown_connections 2024-03-23T08:27:13.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:13.611+0000 7fe3238c7700 1 -- 172.21.15.70:0/1679516742 wait complete. 2024-03-23T08:27:14.318 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-23T08:27:14.319 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:27:14.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-23T08:27:14.592 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:27:15.118 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:14 smithi070 ceph-mon[39699]: pgmap v77: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:15.118 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:14 smithi070 ceph-mon[39699]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:15.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:14 smithi193 ceph-mon[48370]: pgmap v77: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:15.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:14 smithi193 ceph-mon[48370]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:16.137 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:15 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:27:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:15 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:27:16.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.418+0000 7f0fe5247700 1 -- 172.21.15.70:0/86597978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0102e10 msgr2=0x7f0fe01031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:16.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.418+0000 7f0fe5247700 1 --2- 172.21.15.70:0/86597978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0102e10 0x7f0fe01031e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4004940 tx=0x7f0fd4009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.418+0000 7f0fe5247700 1 -- 172.21.15.70:0/86597978 shutdown_connections 2024-03-23T08:27:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.418+0000 7f0fe5247700 1 --2- 172.21.15.70:0/86597978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fe01037b0 0x7f0fe00fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.418+0000 7f0fe5247700 1 --2- 172.21.15.70:0/86597978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0102e10 0x7f0fe01031e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.418+0000 7f0fe5247700 1 -- 172.21.15.70:0/86597978 >> 172.21.15.70:0/86597978 conn(0x7f0fe00f8980 msgr2=0x7f0fe00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.418+0000 7f0fe5247700 1 -- 172.21.15.70:0/86597978 shutdown_connections 2024-03-23T08:27:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.418+0000 7f0fe5247700 1 -- 172.21.15.70:0/86597978 wait complete. 2024-03-23T08:27:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fe5247700 1 Processor -- start 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fe5247700 1 -- start start 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fe5247700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fe0102e10 0x7f0fe0113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fe5247700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe01037b0 0x7f0fe0114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fe5247700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe0114660 con 0x7f0fe01037b0 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fe5247700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe010da50 con 0x7f0fe0102e10 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fde59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe01037b0 0x7f0fe0114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fde59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe01037b0 0x7f0fe0114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33108/0 (socket says 172.21.15.70:33108) 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fde59c700 1 -- 172.21.15.70:0/1195971780 learned_addr learned my addr 172.21.15.70:0/1195971780 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fded9d700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fe0102e10 0x7f0fe0113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fde59c700 1 -- 172.21.15.70:0/1195971780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fe0102e10 msgr2=0x7f0fe0113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fde59c700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fe0102e10 0x7f0fe0113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fde59c700 1 -- 172.21.15.70:0/1195971780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4005040 con 0x7f0fe01037b0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fde59c700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe01037b0 0x7f0fe0114010 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000c800 tx=0x7f0fd0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.419+0000 7f0fc77fe700 1 -- 172.21.15.70:0/1195971780 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fd000a760 con 0x7f0fe01037b0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.420+0000 7f0fc77fe700 1 -- 172.21.15.70:0/1195971780 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0fd000ad70 con 0x7f0fe01037b0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.420+0000 7f0fc77fe700 1 -- 172.21.15.70:0/1195971780 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fd00113a0 con 0x7f0fe01037b0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.420+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe010dd30 con 0x7f0fe01037b0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.420+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe010e2f0 con 0x7f0fe01037b0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.421+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe00fe560 con 0x7f0fe01037b0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.421+0000 7f0fc77fe700 1 -- 172.21.15.70:0/1195971780 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f0fd0012400 con 0x7f0fe01037b0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.421+0000 7f0fc77fe700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f0fc806c8b0 0x7f0fc806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.422+0000 7f0fc77fe700 1 -- 172.21.15.70:0/1195971780 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0fd008bac0 con 0x7f0fe01037b0 2024-03-23T08:27:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.422+0000 7f0fded9d700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f0fc806c8b0 0x7f0fc806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:16.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.422+0000 7f0fded9d700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f0fc806c8b0 0x7f0fc806ed60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400b3b0 tx=0x7f0fd4009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:16.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.426+0000 7f0fc77fe700 1 -- 172.21.15.70:0/1195971780 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f0fd0059a90 con 0x7f0fe01037b0 2024-03-23T08:27:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.622+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f0fe0109dd0 con 0x7f0fe01037b0 2024-03-23T08:27:16.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.627+0000 7f0fc77fe700 1 -- 172.21.15.70:0/1195971780 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v47)=0 v47) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0fd0059620 con 0x7f0fe01037b0 2024-03-23T08:27:16.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f0fc806c8b0 msgr2=0x7f0fc806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:16.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f0fc806c8b0 0x7f0fc806ed60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400b3b0 tx=0x7f0fd4009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:16.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe01037b0 msgr2=0x7f0fe0114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:16.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe01037b0 0x7f0fe0114010 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000c800 tx=0x7f0fd0004470 comp rx=0 tx=0).stop 2024-03-23T08:27:16.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 shutdown_connections 2024-03-23T08:27:16.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f0fc806c8b0 0x7f0fc806ed60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:16.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fe0102e10 0x7f0fe0113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:16.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 --2- 172.21.15.70:0/1195971780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe01037b0 0x7f0fe0114010 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:16.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 >> 172.21.15.70:0/1195971780 conn(0x7f0fe00f8980 msgr2=0x7f0fe00f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:16.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 shutdown_connections 2024-03-23T08:27:16.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:16.629+0000 7f0fe5247700 1 -- 172.21.15.70:0/1195971780 wait complete. 2024-03-23T08:27:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:16 smithi070 ceph-mon[39699]: pgmap v78: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:16 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1195971780' entity='client.admin' 2024-03-23T08:27:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:27:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:27:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:16 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:16 smithi193 ceph-mon[48370]: pgmap v78: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:16 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1195971780' entity='client.admin' 2024-03-23T08:27:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:27:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:27:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:16 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:17.383 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-23T08:27:17.383 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:27:17.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-23T08:27:17.653 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:27:19.168 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:18 smithi070 ceph-mon[39699]: pgmap v79: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:19.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:18 smithi193 ceph-mon[48370]: pgmap v79: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:19.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.412+0000 7fc4c176b700 1 -- 172.21.15.70:0/3048027045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 msgr2=0x7fc4bc1092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.412+0000 7fc4c176b700 1 --2- 172.21.15.70:0/3048027045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 0x7fc4bc1092f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0004940 tx=0x7fc4b0009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.412+0000 7fc4c176b700 1 -- 172.21.15.70:0/3048027045 shutdown_connections 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.412+0000 7fc4c176b700 1 --2- 172.21.15.70:0/3048027045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 0x7fc4bc1092f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.412+0000 7fc4c176b700 1 --2- 172.21.15.70:0/3048027045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4bc0fbc80 0x7fc4bc0fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.412+0000 7fc4c176b700 1 -- 172.21.15.70:0/3048027045 >> 172.21.15.70:0/3048027045 conn(0x7fc4bc072bc0 msgr2=0x7fc4bc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.412+0000 7fc4c176b700 1 -- 172.21.15.70:0/3048027045 shutdown_connections 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.412+0000 7fc4c176b700 1 -- 172.21.15.70:0/3048027045 wait complete. 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4c176b700 1 Processor -- start 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4c176b700 1 -- start start 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4c176b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4bc0fbc80 0x7fc4bc115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4c176b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 0x7fc4bc1161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4c176b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc116830 con 0x7fc4bc0fc590 2024-03-23T08:27:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4c176b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc10fc50 con 0x7fc4bc0fbc80 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4ba7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 0x7fc4bc1161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4ba7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 0x7fc4bc1161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39656/0 (socket says 172.21.15.70:39656) 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4ba7fc700 1 -- 172.21.15.70:0/2417259512 learned_addr learned my addr 172.21.15.70:0/2417259512 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4baffd700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4bc0fbc80 0x7fc4bc115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4ba7fc700 1 -- 172.21.15.70:0/2417259512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4bc0fbc80 msgr2=0x7fc4bc115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4ba7fc700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4bc0fbc80 0x7fc4bc115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.413+0000 7fc4ba7fc700 1 -- 172.21.15.70:0/2417259512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0005040 con 0x7fc4bc0fc590 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.414+0000 7fc4ba7fc700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 0x7fc4bc1161e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000b3b0 tx=0x7fc4b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.414+0000 7fc4a37fe700 1 -- 172.21.15.70:0/2417259512 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4b000e040 con 0x7fc4bc0fc590 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.414+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc10fed0 con 0x7fc4bc0fc590 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.414+0000 7fc4a37fe700 1 -- 172.21.15.70:0/2417259512 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4b0013420 con 0x7fc4bc0fc590 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.414+0000 7fc4a37fe700 1 -- 172.21.15.70:0/2417259512 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4b001c610 con 0x7fc4bc0fc590 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.414+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc110490 con 0x7fc4bc0fc590 2024-03-23T08:27:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.415+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc065e30 con 0x7fc4bc0fc590 2024-03-23T08:27:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.415+0000 7fc4a37fe700 1 -- 172.21.15.70:0/2417259512 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fc4b0013a00 con 0x7fc4bc0fc590 2024-03-23T08:27:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.415+0000 7fc4a37fe700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc4a406c8a0 0x7fc4a406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.416+0000 7fc4a37fe700 1 -- 172.21.15.70:0/2417259512 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc4b008cc00 con 0x7fc4bc0fc590 2024-03-23T08:27:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.416+0000 7fc4baffd700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc4a406c8a0 0x7fc4a406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.416+0000 7fc4baffd700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc4a406c8a0 0x7fc4a406ed50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4007a40 tx=0x7fc4b4009da0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.420+0000 7fc4a37fe700 1 -- 172.21.15.70:0/2417259512 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fc4b0092020 con 0x7fc4bc0fc590 2024-03-23T08:27:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.611+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fc4bc05f5f0 con 0x7fc4bc0fc590 2024-03-23T08:27:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.612+0000 7fc4a37fe700 1 -- 172.21.15.70:0/2417259512 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v47)=0 v47) v1 ==== 155+0+0 (secure 0 0 0) 0x7fc4b005abf0 con 0x7fc4bc0fc590 2024-03-23T08:27:19.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc4a406c8a0 msgr2=0x7fc4a406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:19.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc4a406c8a0 0x7fc4a406ed50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4007a40 tx=0x7fc4b4009da0 comp rx=0 tx=0).stop 2024-03-23T08:27:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 msgr2=0x7fc4bc1161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 0x7fc4bc1161e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000b3b0 tx=0x7fc4b0009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 shutdown_connections 2024-03-23T08:27:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fc4a406c8a0 0x7fc4a406ed50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4bc0fbc80 0x7fc4bc115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 --2- 172.21.15.70:0/2417259512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4bc0fc590 0x7fc4bc1161e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 >> 172.21.15.70:0/2417259512 conn(0x7fc4bc072bc0 msgr2=0x7fc4bc105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 shutdown_connections 2024-03-23T08:27:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:19.614+0000 7fc4c176b700 1 -- 172.21.15.70:0/2417259512 wait complete. 2024-03-23T08:27:20.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-23T08:27:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:27:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:20 smithi070 ceph-mon[39699]: pgmap v80: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:21.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:20 smithi193 ceph-mon[48370]: pgmap v80: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 -- 172.21.15.70:0/2935329503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c58101d20 msgr2=0x7f9c581020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 --2- 172.21.15.70:0/2935329503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c58101d20 0x7f9c581020f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9c54004940 tx=0x7f9c54009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 -- 172.21.15.70:0/2935329503 shutdown_connections 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 --2- 172.21.15.70:0/2935329503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c581026c0 0x7f9c580fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 --2- 172.21.15.70:0/2935329503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c58101d20 0x7f9c581020f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 -- 172.21.15.70:0/2935329503 >> 172.21.15.70:0/2935329503 conn(0x7f9c580f78d0 msgr2=0x7f9c580f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 -- 172.21.15.70:0/2935329503 shutdown_connections 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 -- 172.21.15.70:0/2935329503 wait complete. 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 Processor -- start 2024-03-23T08:27:22.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.337+0000 7f9c5f86a700 1 -- start start 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5f86a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c58101d20 0x7f9c5810ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5f86a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c581026c0 0x7f9c5810ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5f86a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5810f590 con 0x7f9c581026c0 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5f86a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5810f6d0 con 0x7f9c58101d20 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5ce05700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c581026c0 0x7f9c5810ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5ce05700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c581026c0 0x7f9c5810ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39674/0 (socket says 172.21.15.70:39674) 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5ce05700 1 -- 172.21.15.70:0/1135636428 learned_addr learned my addr 172.21.15.70:0/1135636428 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5d606700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c58101d20 0x7f9c5810ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5ce05700 1 -- 172.21.15.70:0/1135636428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c58101d20 msgr2=0x7f9c5810ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5ce05700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c58101d20 0x7f9c5810ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5ce05700 1 -- 172.21.15.70:0/1135636428 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c54005040 con 0x7f9c581026c0 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5ce05700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c581026c0 0x7f9c5810ef40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800eb40 tx=0x7f9c4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c4dffb700 1 -- 172.21.15.70:0/1135636428 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c48007cc0 con 0x7f9c581026c0 2024-03-23T08:27:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c4dffb700 1 -- 172.21.15.70:0/1135636428 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9c4800f040 con 0x7f9c581026c0 2024-03-23T08:27:22.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c58117a70 con 0x7f9c581026c0 2024-03-23T08:27:22.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.338+0000 7f9c4dffb700 1 -- 172.21.15.70:0/1135636428 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c4800c8f0 con 0x7f9c581026c0 2024-03-23T08:27:22.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.339+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c58117f80 con 0x7f9c581026c0 2024-03-23T08:27:22.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.339+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c580fd470 con 0x7f9c581026c0 2024-03-23T08:27:22.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.340+0000 7f9c4dffb700 1 -- 172.21.15.70:0/1135636428 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f9c4800cb60 con 0x7f9c581026c0 2024-03-23T08:27:22.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.340+0000 7f9c4dffb700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9c4406c8a0 0x7f9c4406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:22.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.340+0000 7f9c4dffb700 1 -- 172.21.15.70:0/1135636428 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9c4808b4d0 con 0x7f9c581026c0 2024-03-23T08:27:22.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.340+0000 7f9c5d606700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9c4406c8a0 0x7f9c4406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:22.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.341+0000 7f9c5d606700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9c4406c8a0 0x7f9c4406ed50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400b0d0 tx=0x7f9c54009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:22.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.345+0000 7f9c4dffb700 1 -- 172.21.15.70:0/1135636428 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f9c48055a60 con 0x7f9c581026c0 2024-03-23T08:27:22.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.536+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f9c5805f5f0 con 0x7f9c581026c0 2024-03-23T08:27:22.536 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.537+0000 7f9c4dffb700 1 -- 172.21.15.70:0/1135636428 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v47)=0 v47) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9c48059080 con 0x7f9c581026c0 2024-03-23T08:27:22.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9c4406c8a0 msgr2=0x7f9c4406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:22.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9c4406c8a0 0x7f9c4406ed50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400b0d0 tx=0x7f9c54009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:22.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c581026c0 msgr2=0x7f9c5810ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:22.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c581026c0 0x7f9c5810ef40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800eb40 tx=0x7f9c4800b040 comp rx=0 tx=0).stop 2024-03-23T08:27:22.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 shutdown_connections 2024-03-23T08:27:22.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f9c4406c8a0 0x7f9c4406ed50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:22.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c58101d20 0x7f9c5810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:22.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 --2- 172.21.15.70:0/1135636428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c581026c0 0x7f9c5810ef40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:22.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 >> 172.21.15.70:0/1135636428 conn(0x7f9c580f78d0 msgr2=0x7f9c580f8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:22.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 shutdown_connections 2024-03-23T08:27:22.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:22.539+0000 7f9c5f86a700 1 -- 172.21.15.70:0/1135636428 wait complete. 2024-03-23T08:27:23.122 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:22 smithi070 ceph-mon[39699]: pgmap v81: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:23.141 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-23T08:27:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:22 smithi193 ceph-mon[48370]: pgmap v81: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:23.412 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:27:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 -- 172.21.15.70:0/2724707334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 msgr2=0x7fd97c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 --2- 172.21.15.70:0/2724707334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 0x7fd97c0fe260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd96c004940 tx=0x7fd96c009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 -- 172.21.15.70:0/2724707334 shutdown_connections 2024-03-23T08:27:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 --2- 172.21.15.70:0/2724707334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97c0fe830 0x7fd97c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 --2- 172.21.15.70:0/2724707334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 0x7fd97c0fe260 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 -- 172.21.15.70:0/2724707334 >> 172.21.15.70:0/2724707334 conn(0x7fd97c072bc0 msgr2=0x7fd97c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 -- 172.21.15.70:0/2724707334 shutdown_connections 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 -- 172.21.15.70:0/2724707334 wait complete. 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 Processor -- start 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.183+0000 7fd980ca5700 1 -- start start 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd980ca5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 0x7fd97c113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd980ca5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97c0fe830 0x7fd97c114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd980ca5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c114670 con 0x7fd97c0fde90 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd980ca5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c10da60 con 0x7fd97c0fe830 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd97a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 0x7fd97c113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd97a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 0x7fd97c113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39684/0 (socket says 172.21.15.70:39684) 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd97a59c700 1 -- 172.21.15.70:0/1513981894 learned_addr learned my addr 172.21.15.70:0/1513981894 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:25.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd979d9b700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97c0fe830 0x7fd97c114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd97a59c700 1 -- 172.21.15.70:0/1513981894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97c0fe830 msgr2=0x7fd97c114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd97a59c700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97c0fe830 0x7fd97c114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd97a59c700 1 -- 172.21.15.70:0/1513981894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c005040 con 0x7fd97c0fde90 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd979d9b700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97c0fe830 0x7fd97c114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd97a59c700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 0x7fd97c113ae0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b3b0 tx=0x7fd96c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd96affd700 1 -- 172.21.15.70:0/1513981894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96c00e040 con 0x7fd97c0fde90 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c10dce0 con 0x7fd97c0fde90 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd96affd700 1 -- 172.21.15.70:0/1513981894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd96c013420 con 0x7fd97c0fde90 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd96affd700 1 -- 172.21.15.70:0/1513981894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96c01c5d0 con 0x7fd97c0fde90 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.184+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c10e220 con 0x7fd97c0fde90 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.185+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c065e30 con 0x7fd97c0fde90 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.186+0000 7fd96affd700 1 -- 172.21.15.70:0/1513981894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fd96c013590 con 0x7fd97c0fde90 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.186+0000 7fd96affd700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd96406c8a0 0x7fd96406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.186+0000 7fd96affd700 1 -- 172.21.15.70:0/1513981894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd96c08cd20 con 0x7fd97c0fde90 2024-03-23T08:27:25.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.186+0000 7fd979d9b700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd96406c8a0 0x7fd96406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:25.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.187+0000 7fd979d9b700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd96406c8a0 0x7fd96406ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd970009fd0 tx=0x7fd970009f20 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:25.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.191+0000 7fd96affd700 1 -- 172.21.15.70:0/1513981894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fd96c05ac90 con 0x7fd97c0fde90 2024-03-23T08:27:25.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:24 smithi070 ceph-mon[39699]: pgmap v82: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:25.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:24 smithi193 ceph-mon[48370]: pgmap v82: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:25.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.382+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fd97c05f5f0 con 0x7fd97c0fde90 2024-03-23T08:27:25.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.382+0000 7fd96affd700 1 -- 172.21.15.70:0/1513981894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v47)=0 v47) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd96c05a820 con 0x7fd97c0fde90 2024-03-23T08:27:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.384+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd96406c8a0 msgr2=0x7fd96406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.384+0000 7fd980ca5700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd96406c8a0 0x7fd96406ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd970009fd0 tx=0x7fd970009f20 comp rx=0 tx=0).stop 2024-03-23T08:27:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.384+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 msgr2=0x7fd97c113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.384+0000 7fd980ca5700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 0x7fd97c113ae0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b3b0 tx=0x7fd96c009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.385+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 shutdown_connections 2024-03-23T08:27:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.385+0000 7fd980ca5700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fd96406c8a0 0x7fd96406ed50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.385+0000 7fd980ca5700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c0fde90 0x7fd97c113ae0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.385+0000 7fd980ca5700 1 --2- 172.21.15.70:0/1513981894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97c0fe830 0x7fd97c114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.385+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 >> 172.21.15.70:0/1513981894 conn(0x7fd97c072bc0 msgr2=0x7fd97c1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.385+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 shutdown_connections 2024-03-23T08:27:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:25.385+0000 7fd980ca5700 1 -- 172.21.15.70:0/1513981894 wait complete. 2024-03-23T08:27:26.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-23T08:27:26.356 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:27:27.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:26 smithi070 ceph-mon[39699]: pgmap v83: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:27.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:26 smithi193 ceph-mon[48370]: pgmap v83: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.178+0000 7fefaadac700 1 -- 172.21.15.70:0/754472124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4102710 msgr2=0x7fefa40fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.178+0000 7fefaadac700 1 --2- 172.21.15.70:0/754472124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4102710 0x7fefa40fbd80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fef90004940 tx=0x7fef90009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.178+0000 7fefaadac700 1 -- 172.21.15.70:0/754472124 shutdown_connections 2024-03-23T08:27:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.178+0000 7fefaadac700 1 --2- 172.21.15.70:0/754472124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4102710 0x7fefa40fbd80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.178+0000 7fefaadac700 1 --2- 172.21.15.70:0/754472124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa4101d70 0x7fefa4102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.178+0000 7fefaadac700 1 -- 172.21.15.70:0/754472124 >> 172.21.15.70:0/754472124 conn(0x7fefa4072bc0 msgr2=0x7fefa4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.178+0000 7fefaadac700 1 -- 172.21.15.70:0/754472124 shutdown_connections 2024-03-23T08:27:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.178+0000 7fefaadac700 1 -- 172.21.15.70:0/754472124 wait complete. 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefaadac700 1 Processor -- start 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefaadac700 1 -- start start 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefaadac700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4101d70 0x7fefa4112960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefaadac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa4102710 0x7fefa4112ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefaadac700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa41134f0 con 0x7fefa4101d70 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefaadac700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa410c8e0 con 0x7fefa4102710 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefa8b48700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4101d70 0x7fefa4112960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefa8b48700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4101d70 0x7fefa4112960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39702/0 (socket says 172.21.15.70:39702) 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefa8b48700 1 -- 172.21.15.70:0/3932000518 learned_addr learned my addr 172.21.15.70:0/3932000518 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.179+0000 7fefa3fff700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa4102710 0x7fefa4112ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.180+0000 7fefa8b48700 1 -- 172.21.15.70:0/3932000518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa4102710 msgr2=0x7fefa4112ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.180+0000 7fefa8b48700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa4102710 0x7fefa4112ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.180+0000 7fefa8b48700 1 -- 172.21.15.70:0/3932000518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef90005040 con 0x7fefa4101d70 2024-03-23T08:27:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.180+0000 7fefa8b48700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4101d70 0x7fefa4112960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fef98002a50 tx=0x7fef980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:28.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.180+0000 7fefa17fa700 1 -- 172.21.15.70:0/3932000518 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef98007cc0 con 0x7fefa4101d70 2024-03-23T08:27:28.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.180+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa410cbc0 con 0x7fefa4101d70 2024-03-23T08:27:28.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.180+0000 7fefa17fa700 1 -- 172.21.15.70:0/3932000518 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef9800f040 con 0x7fefa4101d70 2024-03-23T08:27:28.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.180+0000 7fefa17fa700 1 -- 172.21.15.70:0/3932000518 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef9800d9c0 con 0x7fefa4101d70 2024-03-23T08:27:28.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.180+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa410d150 con 0x7fefa4101d70 2024-03-23T08:27:28.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.181+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa40fd4c0 con 0x7fefa4101d70 2024-03-23T08:27:28.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.181+0000 7fefa17fa700 1 -- 172.21.15.70:0/3932000518 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fef9800db40 con 0x7fefa4101d70 2024-03-23T08:27:28.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.182+0000 7fefa17fa700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fef9406eab0 0x7fef94070f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:28.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.182+0000 7fefa17fa700 1 -- 172.21.15.70:0/3932000518 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fef9808b7b0 con 0x7fefa4101d70 2024-03-23T08:27:28.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.182+0000 7fefa3fff700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fef9406eab0 0x7fef94070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:28.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.182+0000 7fefa3fff700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fef9406eab0 0x7fef94070f60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef90004890 tx=0x7fef90009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:28.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.186+0000 7fefa17fa700 1 -- 172.21.15.70:0/3932000518 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fef980597d0 con 0x7fefa4101d70 2024-03-23T08:27:28.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.396+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837", "target": ["mon-mgr", ""]}) v1 -- 0x7fefa4108d10 con 0x7fef9406eab0 2024-03-23T08:27:28.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.401+0000 7fefa17fa700 1 -- 172.21.15.70:0/3932000518 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fefa4108d10 con 0x7fef9406eab0 2024-03-23T08:27:28.401 INFO:teuthology.orchestra.run.smithi070.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:27:28.403 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.403+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fef9406eab0 msgr2=0x7fef94070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.403+0000 7fefaadac700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fef9406eab0 0x7fef94070f60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef90004890 tx=0x7fef90009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.403+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4101d70 msgr2=0x7fefa4112960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.403+0000 7fefaadac700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4101d70 0x7fefa4112960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fef98002a50 tx=0x7fef980046b0 comp rx=0 tx=0).stop 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.403+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 shutdown_connections 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.404+0000 7fefaadac700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fef9406eab0 0x7fef94070f60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.404+0000 7fefaadac700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa4101d70 0x7fefa4112960 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.404+0000 7fefaadac700 1 --2- 172.21.15.70:0/3932000518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa4102710 0x7fefa4112ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.404+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 >> 172.21.15.70:0/3932000518 conn(0x7fefa4072bc0 msgr2=0x7fefa40fb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.404+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 shutdown_connections 2024-03-23T08:27:28.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:28.404+0000 7fefaadac700 1 -- 172.21.15.70:0/3932000518 wait complete. 2024-03-23T08:27:29.134 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:28 smithi070 ceph-mon[39699]: pgmap v84: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:29.134 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:28 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:29.134 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:28 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:29.134 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:28 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:27:29.134 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:28 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:27:29.134 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:28 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:29.153 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-23T08:27:29.153 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:27:29.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-23T08:27:29.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:28 smithi193 ceph-mon[48370]: pgmap v84: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:29.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:28 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:29.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:28 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:29.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:28 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:27:29.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:28 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:27:29.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:28 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:29.830 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:27:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:29 smithi070 ceph-mon[39699]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:29 smithi070 ceph-mon[39699]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:27:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:29 smithi070 ceph-mon[39699]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:27:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:29 smithi193 ceph-mon[48370]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:29 smithi193 ceph-mon[48370]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:27:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:29 smithi193 ceph-mon[48370]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:82316b04821a6c2c1d458a881920454bbadb8837 2024-03-23T08:27:31.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:30 smithi070 ceph-mon[39699]: pgmap v85: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:31.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:27:31.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:31.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:31.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:31.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:31.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:31.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:30 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-23T08:27:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:30 smithi193 ceph-mon[48370]: pgmap v85: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:27:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:30 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-23T08:27:32.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 -- 172.21.15.70:0/45206028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 msgr2=0x7ff9a40fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 --2- 172.21.15.70:0/45206028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 0x7ff9a40fc050 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0004940 tx=0x7ff9a0009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 -- 172.21.15.70:0/45206028 shutdown_connections 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 --2- 172.21.15.70:0/45206028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9a40fc590 0x7ff9a41092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 --2- 172.21.15.70:0/45206028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 0x7ff9a40fc050 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 -- 172.21.15.70:0/45206028 >> 172.21.15.70:0/45206028 conn(0x7ff9a4072bc0 msgr2=0x7ff9a4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 -- 172.21.15.70:0/45206028 shutdown_connections 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 -- 172.21.15.70:0/45206028 wait complete. 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 Processor -- start 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.168+0000 7ff9ac624700 1 -- start start 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9ac624700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 0x7ff9a4115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:32.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9ac624700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9a40fc590 0x7ff9a4116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9ac624700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41168c0 con 0x7ff9a40fbc80 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9ac624700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a410fc90 con 0x7ff9a40fc590 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9aa3c0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 0x7ff9a4115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9aa3c0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 0x7ff9a4115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44396/0 (socket says 172.21.15.70:44396) 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9aa3c0700 1 -- 172.21.15.70:0/1184658922 learned_addr learned my addr 172.21.15.70:0/1184658922 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9a9bbf700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9a40fc590 0x7ff9a4116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9aa3c0700 1 -- 172.21.15.70:0/1184658922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9a40fc590 msgr2=0x7ff9a4116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9aa3c0700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9a40fc590 0x7ff9a4116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9aa3c0700 1 -- 172.21.15.70:0/1184658922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0005040 con 0x7ff9a40fbc80 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9aa3c0700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 0x7ff9a4115d30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009180 tx=0x7ff9a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff99affd700 1 -- 172.21.15.70:0/1184658922 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9a000e040 con 0x7ff9a40fbc80 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a410ff10 con 0x7ff9a40fbc80 2024-03-23T08:27:32.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff99affd700 1 -- 172.21.15.70:0/1184658922 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff9a0004290 con 0x7ff9a40fbc80 2024-03-23T08:27:32.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff99affd700 1 -- 172.21.15.70:0/1184658922 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9a0013e60 con 0x7ff9a40fbc80 2024-03-23T08:27:32.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.169+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a4110450 con 0x7ff9a40fbc80 2024-03-23T08:27:32.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.170+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4065e30 con 0x7ff9a40fbc80 2024-03-23T08:27:32.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.171+0000 7ff99affd700 1 -- 172.21.15.70:0/1184658922 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7ff9a00136b0 con 0x7ff9a40fbc80 2024-03-23T08:27:32.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.171+0000 7ff99affd700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff99006c830 0x7ff99006ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:32.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.171+0000 7ff99affd700 1 -- 172.21.15.70:0/1184658922 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff9a008d740 con 0x7ff9a40fbc80 2024-03-23T08:27:32.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.171+0000 7ff9a9bbf700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff99006c830 0x7ff99006ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:32.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.172+0000 7ff9a9bbf700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff99006c830 0x7ff99006ece0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9a41115d0 tx=0x7ff994009040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:32.176 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.176+0000 7ff99affd700 1 -- 172.21.15.70:0/1184658922 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7ff9a005b7d0 con 0x7ff9a40fbc80 2024-03-23T08:27:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:31 smithi070 ceph-mon[39699]: Upgrade: Target is version 18.2.2-954-g82316b04 (reef) 2024-03-23T08:27:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:31 smithi070 ceph-mon[39699]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f, digests ['quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f'] 2024-03-23T08:27:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:31 smithi070 ceph-mon[39699]: Upgrade: Setting container_image for all mgr 2024-03-23T08:27:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:31 smithi193 ceph-mon[48370]: Upgrade: Target is version 18.2.2-954-g82316b04 (reef) 2024-03-23T08:27:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:31 smithi193 ceph-mon[48370]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f, digests ['quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f'] 2024-03-23T08:27:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:31 smithi193 ceph-mon[48370]: Upgrade: Setting container_image for all mgr 2024-03-23T08:27:32.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.395+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9a404bed0 con 0x7ff99006c830 2024-03-23T08:27:32.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.397+0000 7ff99affd700 1 -- 172.21.15.70:0/1184658922 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7ff9a404bed0 con 0x7ff99006c830 2024-03-23T08:27:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff99006c830 msgr2=0x7ff99006ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff99006c830 0x7ff99006ece0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9a41115d0 tx=0x7ff994009040 comp rx=0 tx=0).stop 2024-03-23T08:27:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 msgr2=0x7ff9a4115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 0x7ff9a4115d30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009180 tx=0x7ff9a0009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 shutdown_connections 2024-03-23T08:27:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff99006c830 0x7ff99006ece0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9a40fbc80 0x7ff9a4115d30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 --2- 172.21.15.70:0/1184658922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9a40fc590 0x7ff9a4116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 >> 172.21.15.70:0/1184658922 conn(0x7ff9a4072bc0 msgr2=0x7ff9a4105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:32.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 shutdown_connections 2024-03-23T08:27:32.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.399+0000 7ff9ac624700 1 -- 172.21.15.70:0/1184658922 wait complete. 2024-03-23T08:27:32.409 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 -- 172.21.15.70:0/98376243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 msgr2=0x7f73b4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 --2- 172.21.15.70:0/98376243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 0x7f73b4106f00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f73a4004940 tx=0x7f73a4009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 -- 172.21.15.70:0/98376243 shutdown_connections 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 --2- 172.21.15.70:0/98376243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 0x7f73b4106f00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 --2- 172.21.15.70:0/98376243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73b40fde90 0x7f73b40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 -- 172.21.15.70:0/98376243 >> 172.21.15.70:0/98376243 conn(0x7f73b4072bc0 msgr2=0x7f73b4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 -- 172.21.15.70:0/98376243 shutdown_connections 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 -- 172.21.15.70:0/98376243 wait complete. 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 Processor -- start 2024-03-23T08:27:32.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 -- start start 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.502+0000 7f73ba0d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73b40fde90 0x7f73b4113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73ba0d5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 0x7f73b4114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73ba0d5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b4114670 con 0x7f73b40fe830 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73ba0d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b410da60 con 0x7f73b40fde90 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73b2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 0x7f73b4114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73b2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 0x7f73b4114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44424/0 (socket says 172.21.15.70:44424) 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73b2ffd700 1 -- 172.21.15.70:0/450708228 learned_addr learned my addr 172.21.15.70:0/450708228 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73b37fe700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73b40fde90 0x7f73b4113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73b2ffd700 1 -- 172.21.15.70:0/450708228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73b40fde90 msgr2=0x7f73b4113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73b2ffd700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73b40fde90 0x7f73b4113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73b2ffd700 1 -- 172.21.15.70:0/450708228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a4005040 con 0x7f73b40fe830 2024-03-23T08:27:32.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73b2ffd700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 0x7f73b4114020 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009360 tx=0x7f73a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:32.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f739bfff700 1 -- 172.21.15.70:0/450708228 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73a400e040 con 0x7f73b40fe830 2024-03-23T08:27:32.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f739bfff700 1 -- 172.21.15.70:0/450708228 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f73a400bed0 con 0x7f73b40fe830 2024-03-23T08:27:32.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b410dce0 con 0x7f73b40fe830 2024-03-23T08:27:32.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f739bfff700 1 -- 172.21.15.70:0/450708228 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73a401c3f0 con 0x7f73b40fe830 2024-03-23T08:27:32.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.503+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b410e220 con 0x7f73b40fe830 2024-03-23T08:27:32.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.504+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b4065e30 con 0x7f73b40fe830 2024-03-23T08:27:32.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.505+0000 7f739bfff700 1 -- 172.21.15.70:0/450708228 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f73a4013790 con 0x7f73b40fe830 2024-03-23T08:27:32.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.505+0000 7f739bfff700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f739c06c8a0 0x7f739c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:32.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.505+0000 7f739bfff700 1 -- 172.21.15.70:0/450708228 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f73a408ce50 con 0x7f73b40fe830 2024-03-23T08:27:32.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.505+0000 7f73b37fe700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f739c06c8a0 0x7f739c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:32.507 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.506+0000 7f73b37fe700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f739c06c8a0 0x7f739c06ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73a8004f40 tx=0x7f73a800b040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:32.509 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.509+0000 7f739bfff700 1 -- 172.21.15.70:0/450708228 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f73a405adc0 con 0x7f73b40fe830 2024-03-23T08:27:32.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.726+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73b4109e40 con 0x7f739c06c8a0 2024-03-23T08:27:32.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.727+0000 7f739bfff700 1 -- 172.21.15.70:0/450708228 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f73b4109e40 con 0x7f739c06c8a0 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.729+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f739c06c8a0 msgr2=0x7f739c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.729+0000 7f73ba0d5700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f739c06c8a0 0x7f739c06ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73a8004f40 tx=0x7f73a800b040 comp rx=0 tx=0).stop 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.729+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 msgr2=0x7f73b4114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.729+0000 7f73ba0d5700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 0x7f73b4114020 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009360 tx=0x7f73a4009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.729+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 shutdown_connections 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.729+0000 7f73ba0d5700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f739c06c8a0 0x7f739c06ed50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.729+0000 7f73ba0d5700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73b40fde90 0x7f73b4113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.729+0000 7f73ba0d5700 1 --2- 172.21.15.70:0/450708228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40fe830 0x7f73b4114020 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.729+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 >> 172.21.15.70:0/450708228 conn(0x7f73b4072bc0 msgr2=0x7f73b4103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:32.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.730+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 shutdown_connections 2024-03-23T08:27:32.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.730+0000 7f73ba0d5700 1 -- 172.21.15.70:0/450708228 wait complete. 2024-03-23T08:27:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.833+0000 7f5d048b6700 1 -- 172.21.15.70:0/537071824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 msgr2=0x7f5d000fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.833+0000 7f5d048b6700 1 --2- 172.21.15.70:0/537071824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 0x7f5d000fe260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0004940 tx=0x7f5cf0009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.833+0000 7f5d048b6700 1 -- 172.21.15.70:0/537071824 shutdown_connections 2024-03-23T08:27:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.833+0000 7f5d048b6700 1 --2- 172.21.15.70:0/537071824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d000fe830 0x7f5d00106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.833+0000 7f5d048b6700 1 --2- 172.21.15.70:0/537071824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 0x7f5d000fe260 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.833+0000 7f5d048b6700 1 -- 172.21.15.70:0/537071824 >> 172.21.15.70:0/537071824 conn(0x7f5d00072bc0 msgr2=0x7f5d00072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.833+0000 7f5d048b6700 1 -- 172.21.15.70:0/537071824 shutdown_connections 2024-03-23T08:27:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.833+0000 7f5d048b6700 1 -- 172.21.15.70:0/537071824 wait complete. 2024-03-23T08:27:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.833+0000 7f5d048b6700 1 Processor -- start 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5d048b6700 1 -- start start 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5d048b6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 0x7f5d00113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5d048b6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d000fe830 0x7f5d00114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5d048b6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00114690 con 0x7f5d000fde90 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5d048b6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0010da80 con 0x7f5d000fe830 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cfe59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 0x7f5d00113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cfe59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 0x7f5d00113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44436/0 (socket says 172.21.15.70:44436) 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cfe59c700 1 -- 172.21.15.70:0/2323066743 learned_addr learned my addr 172.21.15.70:0/2323066743 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cfdd9b700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d000fe830 0x7f5d00114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cfe59c700 1 -- 172.21.15.70:0/2323066743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d000fe830 msgr2=0x7f5d00114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cfe59c700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d000fe830 0x7f5d00114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cfe59c700 1 -- 172.21.15.70:0/2323066743 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0005040 con 0x7f5d000fde90 2024-03-23T08:27:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cfe59c700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 0x7f5d00113b00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000b3b0 tx=0x7f5cf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:32.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cf6ffd700 1 -- 172.21.15.70:0/2323066743 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf000e040 con 0x7f5d000fde90 2024-03-23T08:27:32.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cf6ffd700 1 -- 172.21.15.70:0/2323066743 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5cf0013420 con 0x7f5d000fde90 2024-03-23T08:27:32.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0010dd00 con 0x7f5d000fde90 2024-03-23T08:27:32.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5cf6ffd700 1 -- 172.21.15.70:0/2323066743 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cf001c5d0 con 0x7f5d000fde90 2024-03-23T08:27:32.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.834+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0010e240 con 0x7f5d000fde90 2024-03-23T08:27:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.835+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d00065e30 con 0x7f5d000fde90 2024-03-23T08:27:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.836+0000 7f5cf6ffd700 1 -- 172.21.15.70:0/2323066743 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7f5cf0013590 con 0x7f5d000fde90 2024-03-23T08:27:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.836+0000 7f5cf6ffd700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f5cec06c8a0 0x7f5cec06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.836+0000 7f5cf6ffd700 1 -- 172.21.15.70:0/2323066743 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5cf008cbd0 con 0x7f5d000fde90 2024-03-23T08:27:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.836+0000 7f5cfdd9b700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f5cec06c8a0 0x7f5cec06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.837+0000 7f5cfdd9b700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f5cec06c8a0 0x7f5cec06ed50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009fd0 tx=0x7f5ce8009f20 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:32.841+0000 7f5cf6ffd700 1 -- 172.21.15.70:0/2323066743 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7f5cf005abc0 con 0x7f5d000fde90 2024-03-23T08:27:33.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.050+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d00109e60 con 0x7f5cec06c8a0 2024-03-23T08:27:33.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.060+0000 7f5cf6ffd700 1 -- 172.21.15.70:0/2323066743 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5d00109e60 con 0x7f5cec06c8a0 2024-03-23T08:27:33.060 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:27:33.060 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (70s) 42s ago 12m 16.7M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:27:33.060 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (12m) 42s ago 12m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:27:33.060 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (11m) 101s ago 11m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:27:33.060 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (48s) 42s ago 12m 45.4M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:27:33.060 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (6m) 42s ago 6m 148M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:27:33.060 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (6m) 42s ago 6m 12.9M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:27:33.060 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (6m) 101s ago 6m 97.0M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (6m) 101s ago 6m 14.3M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (3m) 42s ago 13m 497M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (2m) 101s ago 11m 449M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (13m) 42s ago 13m 56.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f dfce3cd82b23 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (11m) 101s ago 11m 41.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f e0bad724d320 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (110s) 42s ago 12m 9332k - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (106s) 101s ago 11m 3561k - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (10m) 42s ago 10m 217M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (10m) 42s ago 10m 233M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (9m) 42s ago 9m 202M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (9m) 101s ago 9m 276M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (8m) 101s ago 8m 241M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (8m) 101s ago 8m 223M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:27:33.061 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (86s) 42s ago 11m 46.4M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:27:33.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f5cec06c8a0 msgr2=0x7f5cec06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f5cec06c8a0 0x7f5cec06ed50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009fd0 tx=0x7f5ce8009f20 comp rx=0 tx=0).stop 2024-03-23T08:27:33.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 msgr2=0x7f5d00113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 0x7f5d00113b00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000b3b0 tx=0x7f5cf0009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 shutdown_connections 2024-03-23T08:27:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7f5cec06c8a0 0x7f5cec06ed50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d000fde90 0x7f5d00113b00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 --2- 172.21.15.70:0/2323066743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d000fe830 0x7f5d00114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 >> 172.21.15.70:0/2323066743 conn(0x7f5d00072bc0 msgr2=0x7f5d00103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 shutdown_connections 2024-03-23T08:27:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.062+0000 7f5d048b6700 1 -- 172.21.15.70:0/2323066743 wait complete. 2024-03-23T08:27:33.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.161+0000 7fb19ff27700 1 -- 172.21.15.70:0/4206000647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb198101d10 msgr2=0x7fb1981020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.161+0000 7fb19ff27700 1 --2- 172.21.15.70:0/4206000647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb198101d10 0x7fb1981020e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb194004940 tx=0x7fb194009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:33.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.161+0000 7fb19ff27700 1 -- 172.21.15.70:0/4206000647 shutdown_connections 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.161+0000 7fb19ff27700 1 --2- 172.21.15.70:0/4206000647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1981026b0 0x7fb1980fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.161+0000 7fb19ff27700 1 --2- 172.21.15.70:0/4206000647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb198101d10 0x7fb1981020e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.161+0000 7fb19ff27700 1 -- 172.21.15.70:0/4206000647 >> 172.21.15.70:0/4206000647 conn(0x7fb1980f7880 msgr2=0x7fb1980f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19ff27700 1 -- 172.21.15.70:0/4206000647 shutdown_connections 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19ff27700 1 -- 172.21.15.70:0/4206000647 wait complete. 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19ff27700 1 Processor -- start 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19ff27700 1 -- start start 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19ff27700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb198101d10 0x7fb198112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19ff27700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1981026b0 0x7fb198113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19ff27700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1981138f0 con 0x7fb1981026b0 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19ff27700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19810cc30 con 0x7fb198101d10 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19d4c2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1981026b0 0x7fb198113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19d4c2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1981026b0 0x7fb198113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44464/0 (socket says 172.21.15.70:44464) 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.162+0000 7fb19d4c2700 1 -- 172.21.15.70:0/2017062807 learned_addr learned my addr 172.21.15.70:0/2017062807 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:33.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb19dcc3700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb198101d10 0x7fb198112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb19d4c2700 1 -- 172.21.15.70:0/2017062807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb198101d10 msgr2=0x7fb198112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb19d4c2700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb198101d10 0x7fb198112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb19d4c2700 1 -- 172.21.15.70:0/2017062807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb194005040 con 0x7fb1981026b0 2024-03-23T08:27:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb19d4c2700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1981026b0 0x7fb198113210 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb18800b790 tx=0x7fb188009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb18e7fc700 1 -- 172.21.15.70:0/2017062807 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb18800f6f0 con 0x7fb1981026b0 2024-03-23T08:27:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19810ceb0 con 0x7fb1981026b0 2024-03-23T08:27:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb18e7fc700 1 -- 172.21.15.70:0/2017062807 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb18800fd00 con 0x7fb1981026b0 2024-03-23T08:27:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb18e7fc700 1 -- 172.21.15.70:0/2017062807 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1880103a0 con 0x7fb1981026b0 2024-03-23T08:27:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.163+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19810d470 con 0x7fb1981026b0 2024-03-23T08:27:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.164+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1980fd460 con 0x7fb1981026b0 2024-03-23T08:27:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.164+0000 7fb18e7fc700 1 -- 172.21.15.70:0/2017062807 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fb18800f850 con 0x7fb1981026b0 2024-03-23T08:27:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.165+0000 7fb18e7fc700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb18406c8a0 0x7fb18406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.165+0000 7fb18e7fc700 1 -- 172.21.15.70:0/2017062807 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb18808b930 con 0x7fb1981026b0 2024-03-23T08:27:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.165+0000 7fb19dcc3700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb18406c8a0 0x7fb18406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:33.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.165+0000 7fb19dcc3700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb18406c8a0 0x7fb18406ed50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb194004f90 tx=0x7fb194009e30 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:33.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.169+0000 7fb18e7fc700 1 -- 172.21.15.70:0/2017062807 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fb1880598a0 con 0x7fb1981026b0 2024-03-23T08:27:33.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:32 smithi070 ceph-mon[39699]: pgmap v86: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:33.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:32 smithi193 ceph-mon[48370]: pgmap v86: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:33.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.473+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb19805f5f0 con 0x7fb1981026b0 2024-03-23T08:27:33.474 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.474+0000 7fb18e7fc700 1 -- 172.21.15.70:0/2017062807 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+741 (secure 0 0 0) 0x7fb188059430 con 0x7fb1981026b0 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:27:33.475 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-23T08:27:33.476 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:27:33.476 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:27:33.476 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:27:33.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb18406c8a0 msgr2=0x7fb18406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb18406c8a0 0x7fb18406ed50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb194004f90 tx=0x7fb194009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:33.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1981026b0 msgr2=0x7fb198113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1981026b0 0x7fb198113210 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb18800b790 tx=0x7fb188009220 comp rx=0 tx=0).stop 2024-03-23T08:27:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 shutdown_connections 2024-03-23T08:27:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb18406c8a0 0x7fb18406ed50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb198101d10 0x7fb198112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 --2- 172.21.15.70:0/2017062807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1981026b0 0x7fb198113210 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 >> 172.21.15.70:0/2017062807 conn(0x7fb1980f7880 msgr2=0x7fb1980f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 shutdown_connections 2024-03-23T08:27:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.476+0000 7fb19ff27700 1 -- 172.21.15.70:0/2017062807 wait complete. 2024-03-23T08:27:33.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.579+0000 7fb42e6ae700 1 -- 172.21.15.70:0/3957192579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 msgr2=0x7fb4280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.579+0000 7fb42e6ae700 1 --2- 172.21.15.70:0/3957192579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 0x7fb4280706b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb418004940 tx=0x7fb418009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:33.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.579+0000 7fb42e6ae700 1 -- 172.21.15.70:0/3957192579 shutdown_connections 2024-03-23T08:27:33.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.579+0000 7fb42e6ae700 1 --2- 172.21.15.70:0/3957192579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb428070bf0 0x7fb4281091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.579+0000 7fb42e6ae700 1 --2- 172.21.15.70:0/3957192579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 0x7fb4280706b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.579+0000 7fb42e6ae700 1 -- 172.21.15.70:0/3957192579 >> 172.21.15.70:0/3957192579 conn(0x7fb4280f89a0 msgr2=0x7fb4280fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:33.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.580+0000 7fb42e6ae700 1 -- 172.21.15.70:0/3957192579 shutdown_connections 2024-03-23T08:27:33.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.580+0000 7fb42e6ae700 1 -- 172.21.15.70:0/3957192579 wait complete. 2024-03-23T08:27:33.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.580+0000 7fb42e6ae700 1 Processor -- start 2024-03-23T08:27:33.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.580+0000 7fb42e6ae700 1 -- start start 2024-03-23T08:27:33.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb42e6ae700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 0x7fb428115ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:33.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb42e6ae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb428070bf0 0x7fb428116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:33.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb42e6ae700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb428116870 con 0x7fb4280702e0 2024-03-23T08:27:33.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb42e6ae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42810fc40 con 0x7fb428070bf0 2024-03-23T08:27:33.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb427fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 0x7fb428115ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:33.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb427fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 0x7fb428115ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44492/0 (socket says 172.21.15.70:44492) 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb427fff700 1 -- 172.21.15.70:0/194773023 learned_addr learned my addr 172.21.15.70:0/194773023 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb4277fe700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb428070bf0 0x7fb428116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb427fff700 1 -- 172.21.15.70:0/194773023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb428070bf0 msgr2=0x7fb428116220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb427fff700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb428070bf0 0x7fb428116220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb427fff700 1 -- 172.21.15.70:0/194773023 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb418005040 con 0x7fb4280702e0 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb4277fe700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb428070bf0 0x7fb428116220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb427fff700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 0x7fb428115ce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb41800b3b0 tx=0x7fb418009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb424ff9700 1 -- 172.21.15.70:0/194773023 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41800e040 con 0x7fb4280702e0 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42810fec0 con 0x7fb4280702e0 2024-03-23T08:27:33.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb424ff9700 1 -- 172.21.15.70:0/194773023 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb418013420 con 0x7fb4280702e0 2024-03-23T08:27:33.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.581+0000 7fb424ff9700 1 -- 172.21.15.70:0/194773023 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb41801c630 con 0x7fb4280702e0 2024-03-23T08:27:33.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.582+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb428110400 con 0x7fb4280702e0 2024-03-23T08:27:33.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.582+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb428103c20 con 0x7fb4280702e0 2024-03-23T08:27:33.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.583+0000 7fb424ff9700 1 -- 172.21.15.70:0/194773023 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fb418013970 con 0x7fb4280702e0 2024-03-23T08:27:33.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.583+0000 7fb424ff9700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb41006c8a0 0x7fb41006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:33.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.583+0000 7fb424ff9700 1 -- 172.21.15.70:0/194773023 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb41808ccf0 con 0x7fb4280702e0 2024-03-23T08:27:33.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.584+0000 7fb4277fe700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb41006c8a0 0x7fb41006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:33.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.584+0000 7fb4277fe700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb41006c8a0 0x7fb41006ed50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009fd0 tx=0x7fb41c009ee0 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:33.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.588+0000 7fb424ff9700 1 -- 172.21.15.70:0/194773023 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fb41805ace0 con 0x7fb4280702e0 2024-03-23T08:27:33.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.843+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb428103c20 con 0x7fb4280702e0 2024-03-23T08:27:33.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.844+0000 7fb424ff9700 1 -- 172.21.15.70:0/194773023 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1839 (secure 0 0 0) 0x7fb41805a870 con 0x7fb4280702e0 2024-03-23T08:27:33.844 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:27:33.844 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:27:33.844 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:27:33.844 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:27:33.844 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:27:33.844 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:27:33.845 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:27:33.846 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:27:33.847 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:27:33.847 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:27:33.847 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:27:33.847 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:27:33.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb41006c8a0 msgr2=0x7fb41006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb41006c8a0 0x7fb41006ed50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009fd0 tx=0x7fb41c009ee0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 msgr2=0x7fb428115ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 0x7fb428115ce0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb41800b3b0 tx=0x7fb418009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:33.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 shutdown_connections 2024-03-23T08:27:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fb41006c8a0 0x7fb41006ed50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4280702e0 0x7fb428115ce0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 --2- 172.21.15.70:0/194773023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb428070bf0 0x7fb428116220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 >> 172.21.15.70:0/194773023 conn(0x7fb4280f89a0 msgr2=0x7fb428101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 shutdown_connections 2024-03-23T08:27:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.847+0000 7fb42e6ae700 1 -- 172.21.15.70:0/194773023 wait complete. 2024-03-23T08:27:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.955+0000 7ff426737700 1 -- 172.21.15.70:0/1270172310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 msgr2=0x7ff420102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.955+0000 7ff426737700 1 --2- 172.21.15.70:0/1270172310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 0x7ff420102090 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff414004940 tx=0x7ff414009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.955+0000 7ff426737700 1 -- 172.21.15.70:0/1270172310 shutdown_connections 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.955+0000 7ff426737700 1 --2- 172.21.15.70:0/1270172310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4201025d0 0x7ff4200fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.955+0000 7ff426737700 1 --2- 172.21.15.70:0/1270172310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 0x7ff420102090 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.955+0000 7ff426737700 1 -- 172.21.15.70:0/1270172310 >> 172.21.15.70:0/1270172310 conn(0x7ff420071450 msgr2=0x7ff420071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.955+0000 7ff426737700 1 -- 172.21.15.70:0/1270172310 shutdown_connections 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.955+0000 7ff426737700 1 -- 172.21.15.70:0/1270172310 wait complete. 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff426737700 1 Processor -- start 2024-03-23T08:27:33.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff426737700 1 -- start start 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff426737700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 0x7ff42010eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff426737700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4201025d0 0x7ff42010efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff426737700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42010f630 con 0x7ff420101cc0 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff426737700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42010f7a0 con 0x7ff4201025d0 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff41ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 0x7ff42010eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff41ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 0x7ff42010eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44502/0 (socket says 172.21.15.70:44502) 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff41ffff700 1 -- 172.21.15.70:0/4213975200 learned_addr learned my addr 172.21.15.70:0/4213975200 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff41f7fe700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4201025d0 0x7ff42010efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff41ffff700 1 -- 172.21.15.70:0/4213975200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4201025d0 msgr2=0x7ff42010efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff41ffff700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4201025d0 0x7ff42010efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff41ffff700 1 -- 172.21.15.70:0/4213975200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff414005040 con 0x7ff420101cc0 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.956+0000 7ff41ffff700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 0x7ff42010eaa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b3b0 tx=0x7ff414009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.957+0000 7ff41cff9700 1 -- 172.21.15.70:0/4213975200 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff41400e040 con 0x7ff420101cc0 2024-03-23T08:27:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.957+0000 7ff41cff9700 1 -- 172.21.15.70:0/4213975200 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff414013420 con 0x7ff420101cc0 2024-03-23T08:27:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.957+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff420117b10 con 0x7ff420101cc0 2024-03-23T08:27:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.957+0000 7ff41cff9700 1 -- 172.21.15.70:0/4213975200 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff41401c610 con 0x7ff420101cc0 2024-03-23T08:27:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.957+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff420117f70 con 0x7ff420101cc0 2024-03-23T08:27:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.958+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4200fd550 con 0x7ff420101cc0 2024-03-23T08:27:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.958+0000 7ff41cff9700 1 -- 172.21.15.70:0/4213975200 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7ff414013a00 con 0x7ff420101cc0 2024-03-23T08:27:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.958+0000 7ff41cff9700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff40806c8a0 0x7ff40806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.959+0000 7ff41cff9700 1 -- 172.21.15.70:0/4213975200 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff41408cb10 con 0x7ff420101cc0 2024-03-23T08:27:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.959+0000 7ff41f7fe700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff40806c8a0 0x7ff40806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.959+0000 7ff41f7fe700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff40806c8a0 0x7ff40806ed50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4200fbde0 tx=0x7ff41000b040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:33.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:33.966+0000 7ff41cff9700 1 -- 172.21.15.70:0/4213975200 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7ff414057110 con 0x7ff420101cc0 2024-03-23T08:27:34.122 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:33 smithi070 ceph-mon[39699]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:34.123 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:33 smithi070 ceph-mon[39699]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:34.123 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:33 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:34.123 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:33 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:27:34.123 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:33 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:27:34.123 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:33 smithi070 ceph-mon[39699]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:27:34.123 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:33 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/2017062807' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:34.123 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:33 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/194773023' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:27:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.191+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 --> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff42004bf70 con 0x7ff40806c8a0 2024-03-23T08:27:34.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.192+0000 7ff41cff9700 1 -- 172.21.15.70:0/4213975200 <== mgr.14676 v2:172.21.15.70:6800/244960840 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7ff41000f040 con 0x7ff40806c8a0 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "2/21 daemons upgraded", 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mon daemons", 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:27:34.193 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:27:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.194+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff40806c8a0 msgr2=0x7ff40806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.194+0000 7ff426737700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff40806c8a0 0x7ff40806ed50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4200fbde0 tx=0x7ff41000b040 comp rx=0 tx=0).stop 2024-03-23T08:27:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.194+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 msgr2=0x7ff42010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.194+0000 7ff426737700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 0x7ff42010eaa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b3b0 tx=0x7ff414009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.194+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 shutdown_connections 2024-03-23T08:27:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.194+0000 7ff426737700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7ff40806c8a0 0x7ff40806ed50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.194+0000 7ff426737700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff420101cc0 0x7ff42010eaa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.194+0000 7ff426737700 1 --2- 172.21.15.70:0/4213975200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4201025d0 0x7ff42010efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.194+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 >> 172.21.15.70:0/4213975200 conn(0x7ff420071450 msgr2=0x7ff4201049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.195+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 shutdown_connections 2024-03-23T08:27:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.195+0000 7ff426737700 1 -- 172.21.15.70:0/4213975200 wait complete. 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.295+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1579476362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 msgr2=0x7fcc180fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.295+0000 7fcc1ca87700 1 --2- 172.21.15.70:0/1579476362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 0x7fcc180fe1e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcc08004940 tx=0x7fcc08009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.295+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1579476362 shutdown_connections 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.295+0000 7fcc1ca87700 1 --2- 172.21.15.70:0/1579476362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc180fe7b0 0x7fcc18106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.295+0000 7fcc1ca87700 1 --2- 172.21.15.70:0/1579476362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 0x7fcc180fe1e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.295+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1579476362 >> 172.21.15.70:0/1579476362 conn(0x7fcc180f7880 msgr2=0x7fcc180f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.295+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1579476362 shutdown_connections 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.295+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1579476362 wait complete. 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1ca87700 1 Processor -- start 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1ca87700 1 -- start start 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1ca87700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 0x7fcc180fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1ca87700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc180fe7b0 0x7fcc180fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:34.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1ca87700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc180fc710 con 0x7fcc180fde10 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1ca87700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc180fc850 con 0x7fcc180fe7b0 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 0x7fcc180fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 0x7fcc180fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44520/0 (socket says 172.21.15.70:44520) 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1659c700 1 -- 172.21.15.70:0/1305672284 learned_addr learned my addr 172.21.15.70:0/1305672284 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc15d9b700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc180fe7b0 0x7fcc180fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1659c700 1 -- 172.21.15.70:0/1305672284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc180fe7b0 msgr2=0x7fcc180fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1659c700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc180fe7b0 0x7fcc180fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1659c700 1 -- 172.21.15.70:0/1305672284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc08005040 con 0x7fcc180fde10 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.296+0000 7fcc1659c700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 0x7fcc180fbc40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009380 tx=0x7fcc08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.297+0000 7fcc0effd700 1 -- 172.21.15.70:0/1305672284 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc0800e040 con 0x7fcc180fde10 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.297+0000 7fcc0effd700 1 -- 172.21.15.70:0/1305672284 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc08004290 con 0x7fcc180fde10 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.297+0000 7fcc0effd700 1 -- 172.21.15.70:0/1305672284 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc0801c3f0 con 0x7fcc180fde10 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.297+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc180fcaa0 con 0x7fcc180fde10 2024-03-23T08:27:34.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.297+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1811a470 con 0x7fcc180fde10 2024-03-23T08:27:34.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.298+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc18066770 con 0x7fcc180fde10 2024-03-23T08:27:34.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.298+0000 7fcc0effd700 1 -- 172.21.15.70:0/1305672284 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 90703+0+0 (secure 0 0 0) 0x7fcc08013790 con 0x7fcc180fde10 2024-03-23T08:27:34.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.298+0000 7fcc0effd700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcc0406c8a0 0x7fcc0406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:27:34.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.299+0000 7fcc0effd700 1 -- 172.21.15.70:0/1305672284 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fcc0808cde0 con 0x7fcc180fde10 2024-03-23T08:27:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.299+0000 7fcc15d9b700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcc0406c8a0 0x7fcc0406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:27:34.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.299+0000 7fcc15d9b700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcc0406c8a0 0x7fcc0406ed50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc00007fd0 tx=0x7fcc00009040 comp rx=0 tx=0).ready entity=mgr.14676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:27:34.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.303+0000 7fcc0effd700 1 -- 172.21.15.70:0/1305672284 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181579 (secure 0 0 0) 0x7fcc08057340 con 0x7fcc180fde10 2024-03-23T08:27:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:33 smithi193 ceph-mon[48370]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:33 smithi193 ceph-mon[48370]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:33 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:33 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:27:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:33 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:27:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:33 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:27:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:33 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/2017062807' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:33 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/194773023' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:27:34.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.580+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcc1804bf00 con 0x7fcc180fde10 2024-03-23T08:27:34.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.580+0000 7fcc0effd700 1 -- 172.21.15.70:0/1305672284 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fcc08013420 con 0x7fcc180fde10 2024-03-23T08:27:34.580 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-03-23T08:27:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcc0406c8a0 msgr2=0x7fcc0406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcc0406c8a0 0x7fcc0406ed50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc00007fd0 tx=0x7fcc00009040 comp rx=0 tx=0).stop 2024-03-23T08:27:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 msgr2=0x7fcc180fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:27:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 0x7fcc180fbc40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009380 tx=0x7fcc08009e30 comp rx=0 tx=0).stop 2024-03-23T08:27:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 shutdown_connections 2024-03-23T08:27:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:6800/244960840,v1:172.21.15.70:6801/244960840] conn(0x7fcc0406c8a0 0x7fcc0406ed50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:34.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc180fde10 0x7fcc180fbc40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:34.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 --2- 172.21.15.70:0/1305672284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc180fe7b0 0x7fcc180fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:27:34.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 >> 172.21.15.70:0/1305672284 conn(0x7fcc180f7880 msgr2=0x7fcc180f9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:27:34.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 shutdown_connections 2024-03-23T08:27:34.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:27:34.582+0000 7fcc1ca87700 1 -- 172.21.15.70:0/1305672284 wait complete. 2024-03-23T08:27:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:34 smithi070 ceph-mon[39699]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:34 smithi070 ceph-mon[39699]: pgmap v87: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:34 smithi070 ceph-mon[39699]: Upgrade: Updating mon.smithi070 2024-03-23T08:27:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:34 smithi070 ceph-mon[39699]: Deploying daemon mon.smithi070 on smithi070 2024-03-23T08:27:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:34 smithi070 ceph-mon[39699]: from='client.? 172.21.15.70:0/1305672284' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:27:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:35 smithi070 systemd[1]: Stopping Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:27:35.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:34 smithi193 ceph-mon[48370]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:27:35.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:34 smithi193 ceph-mon[48370]: pgmap v87: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:27:35.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:34 smithi193 ceph-mon[48370]: Upgrade: Updating mon.smithi070 2024-03-23T08:27:35.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:34 smithi193 ceph-mon[48370]: Deploying daemon mon.smithi070 on smithi070 2024-03-23T08:27:35.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:34 smithi193 ceph-mon[48370]: from='client.? 172.21.15.70:0/1305672284' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:27:35.621 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:35 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070[39676]: 2024-03-23T08:27:35.276+0000 7f5adb277700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi070 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:27:35.621 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:35 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070[39676]: 2024-03-23T08:27:35.276+0000 7f5adb277700 -1 mon.smithi070@0(leader) e2 *** Got Signal Terminated *** 2024-03-23T08:27:35.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:35 smithi070 bash[145664]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070 2024-03-23T08:27:36.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 systemd[1]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi070.service: Succeeded. 2024-03-23T08:27:36.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 systemd[1]: Stopped Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:27:36.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 systemd[1]: Starting Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:27:36.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable), process ceph-mon, pid 7 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: pidfile_write: ignore empty --pid-file 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: load: jerasure load: lrc 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: RocksDB version: 7.9.2 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Git sha 0 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Compile date 2024-03-21 21:38:23 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: DB SUMMARY 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: DB Session ID: 8IBPQ72GRI68V3FYBJF6 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: CURRENT file: CURRENT 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1159 Bytes 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi070/store.db dir, Total Num: 1, files: 000033.sst 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi070/store.db: 000031.log size: 1895451 ; 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.error_if_exists: 0 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.create_if_missing: 0 2024-03-23T08:27:36.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.paranoid_checks: 1 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.env: 0x563f99b5e720 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.fs: PosixFileSystem 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.info_log: 0x563f9bd7b380 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.statistics: (nil) 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.use_fsync: 0 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_log_file_size: 0 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T08:27:36.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.allow_fallocate: 1 2024-03-23T08:27:36.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T08:27:36.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T08:27:36.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.use_direct_reads: 0 2024-03-23T08:27:36.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T08:27:36.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T08:27:36.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.db_log_dir: 2024-03-23T08:27:36.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.wal_dir: 2024-03-23T08:27:36.987 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.write_buffer_manager: 0x563f9aff2140 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T08:27:36.988 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.unordered_write: 0 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.row_cache: None 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.wal_filter: None 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.two_write_queues: 0 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.wal_compression: 0 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.atomic_flush: 0 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T08:27:36.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.log_readahead_size: 0 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_background_jobs: 2 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_background_compactions: -1 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_subcompactions: 1 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T08:27:36.990 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_open_files: -1 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_background_flushes: -1 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Compression algorithms supported: 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: kZSTD supported: 0 2024-03-23T08:27:36.991 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: kXpressCompression supported: 0 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: kZlibCompression supported: 1 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: kSnappyCompression supported: 1 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: kLZ4Compression supported: 1 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: kBZip2Compression supported: 0 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000015 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.merge_operator: 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_filter: None 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T08:27:36.992 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563f9bd7b4a0) 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks: 1 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_top_level_index_and_filter: 1 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_type: 0 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_index_type: 0 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_shortening: 1 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: checksum: 4 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: no_block_cache: 0 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache: 0x563f9b0931f0 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_name: BinnedLRUCache 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_options: 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: capacity : 536870912 2024-03-23T08:27:36.993 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_shard_bits : 4 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: strict_capacity_limit : 0 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: high_pri_pool_ratio: 0.000 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_compressed: (nil) 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: persistent_cache: (nil) 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size: 4096 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size_deviation: 10 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_restart_interval: 16 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_block_restart_interval: 1 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: metadata_block_size: 4096 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: partition_filters: 0 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: use_delta_encoding: 1 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: filter_policy: bloomfilter 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: whole_key_filtering: 1 2024-03-23T08:27:36.994 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: verify_compression: 0 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: read_amp_bytes_per_bit: 0 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: format_version: 5 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_index_compression: 1 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_align: 0 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_auto_readahead_size: 262144 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: prepopulate_block_cache: 0 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: initial_auto_readahead_size: 8192 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression: NoCompression 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.num_levels: 7 2024-03-23T08:27:36.995 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T08:27:36.996 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T08:27:36.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.arena_block_size: 1048576 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T08:27:36.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.table_properties_collectors: 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.inplace_update_support: 0 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.bloom_locality: 0 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.max_successive_merges: 0 2024-03-23T08:27:36.999 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.ttl: 2592000 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.enable_blob_files: false 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.min_blob_size: 0 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-23T08:27:37.000 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.blob_file_starting_level: 0 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 11969, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e145644b-fd3d-41f4-945b-9a9ae6535b27 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711182456714635, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711182456720990, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1676913, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11963, "largest_seqno": 13427, "table_properties": {"data_size": 1670094, "index_size": 4140, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1669, "raw_key_size": 15948, "raw_average_key_size": 23, "raw_value_size": 1655974, "raw_average_value_size": 2490, "num_data_blocks": 189, "num_entries": 665, "num_filter_entries": 665, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711182456, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e145644b-fd3d-41f4-945b-9a9ae6535b27", "db_session_id": "8IBPQ72GRI68V3FYBJF6", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711182456721093, "job": 1, "event": "recovery_finished"} 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi070/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T08:27:37.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563f9b130000 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: DB pointer 0x563f9b11e000 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** DB Stats ** 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-03-23T08:27:37.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L0 1/0 1.60 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 502.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L6 1/0 6.75 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Sum 2/0 8.35 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 502.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 502.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 502.6 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:27:37.003 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative compaction: 0.00 GB write, 185.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval compaction: 0.00 GB write, 185.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache BinnedLRUCache@0x563f9b0931f0#7 capacity: 512.00 MB usage: 6.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.58 KB,0.000873208%) Misc(1,0.00 KB,0%) 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: starting mon.smithi070 rank 0 at public addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] at bind addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi070 fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: mon.smithi070@-1(???) e2 preinit fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: mon.smithi070@-1(???).mds e17 new map 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: mon.smithi070@-1(???).mds e17 print_map 2024-03-23T08:27:37.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: e17 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: legacy client fscid: 1 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Filesystem 'cephfs' (1) 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: fs_name cephfs 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: epoch 17 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: tableserver 0 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: root 0 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: session_timeout 60 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: session_autoclose 300 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_file_size 1099511627776 2024-03-23T08:27:37.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_xattr_size 65536 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: required_client_features {} 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: last_failure 0 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: last_failure_osd_epoch 48 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_mds 1 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: in 0 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: up {0=14462} 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: failed 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: damaged 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: stopped 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_pools [3] 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: metadata_pool 2 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: inline_data disabled 2024-03-23T08:27:37.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: balancer 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: bal_rank_mask -1 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: standby_count_wanted 1 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Standby daemons: 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: mon.smithi070@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: mon.smithi070@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: mon.smithi070@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: mon.smithi070@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-23T08:27:37.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 ceph-mon[145869]: mon.smithi070@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-03-23T08:27:37.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 bash[145831]: a020dc442bcf25753330befa3535ae3ee56afeffd4ec805e263dcd6ecc71ebca 2024-03-23T08:27:37.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:36 smithi070 systemd[1]: Started Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:27:40.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:27:40.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:27:40.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: pgmap v89: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:40.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: mon.smithi070 calling monitor election 2024-03-23T08:27:40.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: mon.smithi070 is new leader, mons smithi070,smithi193 in quorum (ranks 0,1) 2024-03-23T08:27:40.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: monmap e2: 2 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],smithi193=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T08:27:40.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:27:40.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: osdmap e50: 6 total, 6 up, 6 in 2024-03-23T08:27:40.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: mgrmap e33: smithi070.jhsbch(active, since 2m), standbys: smithi193.zgpdbu 2024-03-23T08:27:40.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: mgrmap e34: smithi070.jhsbch(active, since 2m), standbys: smithi193.zgpdbu 2024-03-23T08:27:40.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: overall HEALTH_OK 2024-03-23T08:27:40.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:39 smithi193 ceph-mon[48370]: from='mgr.14676 ' entity='' 2024-03-23T08:27:40.095 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: from='mgr.14676 172.21.15.70:0/3962483649' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: pgmap v89: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: mon.smithi070 calling monitor election 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: mon.smithi070 is new leader, mons smithi070,smithi193 in quorum (ranks 0,1) 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: monmap e2: 2 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],smithi193=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: osdmap e50: 6 total, 6 up, 6 in 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: mgrmap e33: smithi070.jhsbch(active, since 2m), standbys: smithi193.zgpdbu 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: mgrmap e34: smithi070.jhsbch(active, since 2m), standbys: smithi193.zgpdbu 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: overall HEALTH_OK 2024-03-23T08:27:40.096 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:39 smithi070 ceph-mon[145869]: from='mgr.14676 ' entity='' 2024-03-23T08:27:46.665 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:46 smithi070 ceph-mon[145869]: Standby manager daemon smithi193.zgpdbu restarted 2024-03-23T08:27:46.665 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:46 smithi070 ceph-mon[145869]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:27:46.665 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:46 smithi070 ceph-mon[145869]: from='mgr.? 172.21.15.193:0/1933648160' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:27:46.665 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:46 smithi070 ceph-mon[145869]: from='mgr.? 172.21.15.193:0/1933648160' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:27:46.666 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:46 smithi070 ceph-mon[145869]: from='mgr.? 172.21.15.193:0/1933648160' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:27:46.666 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:46 smithi070 ceph-mon[145869]: from='mgr.? 172.21.15.193:0/1933648160' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:27:46.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:46 smithi193 ceph-mon[48370]: Standby manager daemon smithi193.zgpdbu restarted 2024-03-23T08:27:46.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:46 smithi193 ceph-mon[48370]: Standby manager daemon smithi193.zgpdbu started 2024-03-23T08:27:46.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:46 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1933648160' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/crt"}]: dispatch 2024-03-23T08:27:46.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:46 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1933648160' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-23T08:27:46.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:46 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1933648160' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.zgpdbu/key"}]: dispatch 2024-03-23T08:27:46.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:46 smithi193 ceph-mon[48370]: from='mgr.? 172.21.15.193:0/1933648160' entity='mgr.smithi193.zgpdbu' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-23T08:27:47.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: mgrmap e35: smithi070.jhsbch(active, since 3m), standbys: smithi193.zgpdbu 2024-03-23T08:27:47.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: Active manager daemon smithi070.jhsbch restarted 2024-03-23T08:27:47.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: Activating manager daemon smithi070.jhsbch 2024-03-23T08:27:47.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: osdmap e51: 6 total, 6 up, 6 in 2024-03-23T08:27:47.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: mgrmap e36: smithi070.jhsbch(active, starting, since 0.00787794s), standbys: smithi193.zgpdbu 2024-03-23T08:27:47.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi193.zgpdbu", "id": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:27:47.827 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:27:47.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:27:47.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:27:47.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:27:47.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: Manager daemon smithi070.jhsbch is now available 2024-03-23T08:27:47.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:47.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:47.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:27:47.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:27:47.828 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:47 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/trash_purge_schedule"}]: dispatch 2024-03-23T08:27:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: mgrmap e35: smithi070.jhsbch(active, since 3m), standbys: smithi193.zgpdbu 2024-03-23T08:27:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: Active manager daemon smithi070.jhsbch restarted 2024-03-23T08:27:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: Activating manager daemon smithi070.jhsbch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: osdmap e51: 6 total, 6 up, 6 in 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: mgrmap e36: smithi070.jhsbch(active, starting, since 0.00787794s), standbys: smithi193.zgpdbu 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi070.jhsbch", "id": "smithi070.jhsbch"}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr metadata", "who": "smithi193.zgpdbu", "id": "smithi193.zgpdbu"}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:27:47.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: Manager daemon smithi070.jhsbch is now available 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/mirror_snapshot_schedule"}]: dispatch 2024-03-23T08:27:47.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.jhsbch/trash_purge_schedule"}]: dispatch 2024-03-23T08:27:48.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:48 smithi070 ceph-mon[145869]: [23/Mar/2024:08:27:47] ENGINE Bus STARTING 2024-03-23T08:27:48.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:48 smithi070 ceph-mon[145869]: [23/Mar/2024:08:27:47] ENGINE Serving on http://172.21.15.70:8765 2024-03-23T08:27:48.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:48 smithi070 ceph-mon[145869]: [23/Mar/2024:08:27:47] ENGINE Serving on https://172.21.15.70:7150 2024-03-23T08:27:48.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:48 smithi070 ceph-mon[145869]: [23/Mar/2024:08:27:47] ENGINE Bus STARTED 2024-03-23T08:27:48.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:48 smithi070 ceph-mon[145869]: mgrmap e37: smithi070.jhsbch(active, since 1.01276s), standbys: smithi193.zgpdbu 2024-03-23T08:27:48.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:48 smithi070 ceph-mon[145869]: pgmap v3: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:27:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:48 smithi193 ceph-mon[48370]: [23/Mar/2024:08:27:47] ENGINE Bus STARTING 2024-03-23T08:27:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:48 smithi193 ceph-mon[48370]: [23/Mar/2024:08:27:47] ENGINE Serving on http://172.21.15.70:8765 2024-03-23T08:27:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:48 smithi193 ceph-mon[48370]: [23/Mar/2024:08:27:47] ENGINE Serving on https://172.21.15.70:7150 2024-03-23T08:27:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:48 smithi193 ceph-mon[48370]: [23/Mar/2024:08:27:47] ENGINE Bus STARTED 2024-03-23T08:27:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:48 smithi193 ceph-mon[48370]: mgrmap e37: smithi070.jhsbch(active, since 1.01276s), standbys: smithi193.zgpdbu 2024-03-23T08:27:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:48 smithi193 ceph-mon[48370]: pgmap v3: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:27:49.565 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:49 smithi193 ceph-mon[48370]: pgmap v4: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:27:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:49 smithi070 ceph-mon[145869]: pgmap v4: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:27:50.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:50 smithi193 ceph-mon[48370]: mgrmap e38: smithi070.jhsbch(active, since 3s), standbys: smithi193.zgpdbu 2024-03-23T08:27:50.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:50 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:50.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:50 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:50.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:50 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:50.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:50 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:50.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:50 smithi070 ceph-mon[145869]: mgrmap e38: smithi070.jhsbch(active, since 3s), standbys: smithi193.zgpdbu 2024-03-23T08:27:50.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:50.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:50.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:50.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:51.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:51 smithi070 ceph-mon[145869]: pgmap v5: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:27:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:51 smithi193 ceph-mon[48370]: pgmap v5: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:27:52.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:52 smithi070 ceph-mon[145869]: mgrmap e39: smithi070.jhsbch(active, since 5s), standbys: smithi193.zgpdbu 2024-03-23T08:27:52.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:52 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:52.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:52 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:53.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:52 smithi193 ceph-mon[48370]: mgrmap e39: smithi070.jhsbch(active, since 5s), standbys: smithi193.zgpdbu 2024-03-23T08:27:53.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:52 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:53.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:52 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:53.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:53 smithi070 ceph-mon[145869]: pgmap v6: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:27:53.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:53 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:53.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:53 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:53 smithi193 ceph-mon[48370]: pgmap v6: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-03-23T08:27:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:53 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:53 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:55.957 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:55 smithi070 ceph-mon[145869]: pgmap v7: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-23T08:27:56.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:55 smithi193 ceph-mon[48370]: pgmap v7: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-23T08:27:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:57 smithi070 ceph-mon[145869]: pgmap v8: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-23T08:27:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:57 smithi070 ceph-mon[145869]: Detected new or changed devices on smithi070 2024-03-23T08:27:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:27:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:27:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:57 smithi070 ceph-mon[145869]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:27:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:57 smithi070 ceph-mon[145869]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:27:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:57 smithi193 ceph-mon[48370]: pgmap v8: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-23T08:27:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:57 smithi193 ceph-mon[48370]: Detected new or changed devices on smithi070 2024-03-23T08:27:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:57 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:57 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:57 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:27:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:57 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:27:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:57 smithi193 ceph-mon[48370]: Updating smithi070:/etc/ceph/ceph.conf 2024-03-23T08:27:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:57 smithi193 ceph-mon[48370]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-23T08:27:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:27:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:27:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:27:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:27:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:27:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:27:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:58.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:58.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: Updating smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: Updating smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.993 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.994 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:27:58.994 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:27:58.994 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:27:58.994 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:58 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-23T08:28:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:59 smithi193 ceph-mon[48370]: pgmap v9: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-23T08:28:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:59 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:59 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:28:00.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:59 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:28:00.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:27:59 smithi193 ceph-mon[48370]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:59 smithi070 ceph-mon[145869]: pgmap v9: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-23T08:28:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:28:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:28:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:27:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:00.889 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:00 smithi193 ceph-mon[48370]: Upgrade: Updating mon.smithi193 2024-03-23T08:28:00.889 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:00 smithi193 ceph-mon[48370]: Deploying daemon mon.smithi193 on smithi193 2024-03-23T08:28:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:00 smithi070 ceph-mon[145869]: Upgrade: Updating mon.smithi193 2024-03-23T08:28:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:00 smithi070 ceph-mon[145869]: Deploying daemon mon.smithi193 on smithi193 2024-03-23T08:28:01.308 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:01 smithi193 systemd[1]: Stopping Ceph mon.smithi193 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:28:01.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:01 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi193[48335]: 2024-03-23T08:28:01.307+0000 7f2726897700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi193 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:28:01.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:01 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi193[48335]: 2024-03-23T08:28:01.307+0000 7f2726897700 -1 mon.smithi193@1(peon) e2 *** Got Signal Terminated *** 2024-03-23T08:28:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:01 smithi193 bash[108344]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi193 2024-03-23T08:28:02.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 systemd[1]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi193.service: Succeeded. 2024-03-23T08:28:02.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 systemd[1]: Stopped Ceph mon.smithi193 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:28:02.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 systemd[1]: Starting Ceph mon.smithi193 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:28:03.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: set uid:gid to 167:167 (ceph:ceph) 2024-03-23T08:28:03.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable), process ceph-mon, pid 7 2024-03-23T08:28:03.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: pidfile_write: ignore empty --pid-file 2024-03-23T08:28:03.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: load: jerasure load: lrc 2024-03-23T08:28:03.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: RocksDB version: 7.9.2 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Git sha 0 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Compile date 2024-03-21 21:38:23 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: DB SUMMARY 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: DB Session ID: 40ACR0ZWFB5TXXQLPPDN 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: CURRENT file: CURRENT 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: IDENTITY file: IDENTITY 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi193/store.db dir, Total Num: 1, files: 000027.sst 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi193/store.db: 000025.log size: 5672952 ; 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.error_if_exists: 0 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.create_if_missing: 0 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.paranoid_checks: 1 2024-03-23T08:28:03.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.env: 0x55f7d2e30720 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.fs: PosixFileSystem 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.info_log: 0x55f7d5d55380 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_file_opening_threads: 16 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.statistics: (nil) 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.use_fsync: 0 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_log_file_size: 0 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.keep_log_file_num: 1000 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.recycle_log_file_num: 0 2024-03-23T08:28:03.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.allow_fallocate: 1 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.allow_mmap_reads: 0 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.allow_mmap_writes: 0 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.use_direct_reads: 0 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.create_missing_column_families: 0 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.db_log_dir: 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.wal_dir: 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.advise_random_on_open: 1 2024-03-23T08:28:03.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.db_write_buffer_size: 0 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.write_buffer_manager: 0x55f7d4fcc140 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.rate_limiter: (nil) 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.wal_recovery_mode: 2 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.enable_thread_tracking: 0 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.enable_pipelined_write: 0 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.unordered_write: 0 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-23T08:28:03.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.row_cache: None 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.wal_filter: None 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.allow_ingest_behind: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.two_write_queues: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.manual_wal_flush: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.wal_compression: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.atomic_flush: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.log_readahead_size: 0 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-23T08:28:03.082 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.best_efforts_recovery: 0 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.allow_data_in_errors: 0 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.db_host_id: __hostname__ 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_background_jobs: 2 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_background_compactions: -1 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_subcompactions: 1 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_total_wal_size: 0 2024-03-23T08:28:03.083 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_open_files: -1 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bytes_per_sync: 0 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_readahead_size: 0 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_background_flushes: -1 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Compression algorithms supported: 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: kZSTD supported: 0 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: kXpressCompression supported: 0 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: kLZ4HCCompression supported: 1 2024-03-23T08:28:03.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: kZlibCompression supported: 1 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: kSnappyCompression supported: 1 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: kLZ4Compression supported: 1 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: kBZip2Compression supported: 0 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000009 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.merge_operator: 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_filter: None 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_filter_factory: None 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.sst_partitioner_factory: None 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-23T08:28:03.085 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f7d5d554a0) 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_type: 0 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_index_type: 0 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_shortening: 1 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: checksum: 4 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: no_block_cache: 0 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache: 0x55f7d506d1f0 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-03-23T08:28:03.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_options: 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: capacity : 536870912 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: num_shard_bits : 4 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: strict_capacity_limit : 0 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_compressed: (nil) 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: persistent_cache: (nil) 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size: 4096 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size_deviation: 10 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_restart_interval: 16 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_block_restart_interval: 1 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: metadata_block_size: 4096 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: partition_filters: 0 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: use_delta_encoding: 1 2024-03-23T08:28:03.087 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: filter_policy: bloomfilter 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: whole_key_filtering: 1 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: verify_compression: 0 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: format_version: 5 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_index_compression: 1 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_align: 0 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_auto_readahead_size: 262144 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: prepopulate_block_cache: 0 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: initial_auto_readahead_size: 8192 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.write_buffer_size: 33554432 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_write_buffer_number: 2 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression: NoCompression 2024-03-23T08:28:03.088 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression: Disabled 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.prefix_extractor: nullptr 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.num_levels: 7 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-23T08:28:03.089 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression_opts.level: 32767 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression_opts.strategy: 0 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression_opts.enabled: false 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.target_file_size_base: 67108864 2024-03-23T08:28:03.090 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.arena_block_size: 1048576 2024-03-23T08:28:03.091 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.disable_auto_compactions: 0 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.table_properties_collectors: 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.inplace_update_support: 0 2024-03-23T08:28:03.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.bloom_locality: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.max_successive_merges: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.paranoid_file_checks: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.force_consistency_checks: 1 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.report_bg_io_stats: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.ttl: 2592000 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.enable_blob_files: false 2024-03-23T08:28:03.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.min_blob_size: 0 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.blob_file_size: 268435456 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.blob_file_starting_level: 0 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12256, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 276f4683-f710-41a0-adb9-72ebf177de1a 2024-03-23T08:28:03.094 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711182482758048, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711182482770677, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3463033, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12253, "largest_seqno": 14416, "table_properties": {"data_size": 3454091, "index_size": 5682, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2245, "raw_key_size": 20987, "raw_average_key_size": 23, "raw_value_size": 3435280, "raw_average_value_size": 3890, "num_data_blocks": 258, "num_entries": 883, "num_filter_entries": 883, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711182482, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "276f4683-f710-41a0-adb9-72ebf177de1a", "db_session_id": "40ACR0ZWFB5TXXQLPPDN", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711182482770800, "job": 1, "event": "recovery_finished"} 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi193/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f7d510a000 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: DB pointer 0x55f7d50f8000 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** DB Stats ** 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-23T08:28:03.095 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: L0 1/0 3.30 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 713.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: L6 1/0 6.75 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Sum 2/0 10.06 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 713.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 713.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-03-23T08:28:03.096 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 713.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative compaction: 0.00 GB write, 171.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval compaction: 0.00 GB write, 171.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-23T08:28:03.097 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Block cache BinnedLRUCache@0x55f7d506d1f0#7 capacity: 512.00 MB usage: 8.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.28 KB,0.000435114%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: starting mon.smithi193 rank 1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi193 fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: mon.smithi193@-1(???) e2 preinit fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: mon.smithi193@-1(???).mds e17 new map 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: mon.smithi193@-1(???).mds e17 print_map 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: e17 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: legacy client fscid: 1 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Filesystem 'cephfs' (1) 2024-03-23T08:28:03.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: fs_name cephfs 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: epoch 17 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: tableserver 0 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: root 0 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: session_timeout 60 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: session_autoclose 300 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_file_size 1099511627776 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_xattr_size 65536 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: required_client_features {} 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: last_failure 0 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: last_failure_osd_epoch 48 2024-03-23T08:28:03.099 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_mds 1 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: in 0 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: up {0=14462} 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: failed 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: damaged 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: stopped 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_pools [3] 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: metadata_pool 2 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: inline_data disabled 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: balancer 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: bal_rank_mask -1 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: standby_count_wanted 1 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:03.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Standby daemons: 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: mon.smithi193@-1(???).osd e51 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: mon.smithi193@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: mon.smithi193@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: mon.smithi193@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 ceph-mon[108533]: mon.smithi193@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 bash[108496]: e49914f9c03ed1d285bc7c1993e46f4d81e6e1598484e13aa65b246f63d8fda1 2024-03-23T08:28:03.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:02 smithi193 systemd[1]: Started Ceph mon.smithi193 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: overall HEALTH_OK 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: mon.smithi070 calling monitor election 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: mon.smithi193 calling monitor election 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: mon.smithi070 is new leader, mons smithi070,smithi193 in quorum (ranks 0,1) 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: monmap e3: 2 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],smithi193=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: osdmap e51: 6 total, 6 up, 6 in 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: mgrmap e39: smithi070.jhsbch(active, since 16s), standbys: smithi193.zgpdbu 2024-03-23T08:28:04.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: overall HEALTH_OK 2024-03-23T08:28:04.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:04.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:04.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:28:04.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: overall HEALTH_OK 2024-03-23T08:28:04.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-03-23T08:28:04.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: mon.smithi070 calling monitor election 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: mon.smithi193 calling monitor election 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: mon.smithi070 is new leader, mons smithi070,smithi193 in quorum (ranks 0,1) 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: monmap e3: 2 mons at {smithi070=[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0],smithi193=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: osdmap e51: 6 total, 6 up, 6 in 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: mgrmap e39: smithi070.jhsbch(active, since 16s), standbys: smithi193.zgpdbu 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: overall HEALTH_OK 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:04.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:28:04.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.697+0000 7fe809c8f700 1 -- 172.21.15.70:0/3805273123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 msgr2=0x7fe804103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.697+0000 7fe809c8f700 1 --2- 172.21.15.70:0/3805273123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 0x7fe804103340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4004940 tx=0x7fe7f4009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.697+0000 7fe809c8f700 1 -- 172.21.15.70:0/3805273123 shutdown_connections 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.697+0000 7fe809c8f700 1 --2- 172.21.15.70:0/3805273123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8040fb3b0 0x7fe8040fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.697+0000 7fe809c8f700 1 --2- 172.21.15.70:0/3805273123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 0x7fe804103340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.698+0000 7fe809c8f700 1 -- 172.21.15.70:0/3805273123 >> 172.21.15.70:0/3805273123 conn(0x7fe804072310 msgr2=0x7fe804072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.698+0000 7fe809c8f700 1 -- 172.21.15.70:0/3805273123 shutdown_connections 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.698+0000 7fe809c8f700 1 -- 172.21.15.70:0/3805273123 wait complete. 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.698+0000 7fe809c8f700 1 Processor -- start 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.698+0000 7fe809c8f700 1 -- start start 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe809c8f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8040fb3b0 0x7fe8041137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe809c8f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 0x7fe804113d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe809c8f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe804114350 con 0x7fe804102f70 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe809c8f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80410d990 con 0x7fe8040fb3b0 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe802ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 0x7fe804113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe802ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 0x7fe804113d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49874/0 (socket says 172.21.15.70:49874) 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe802ffd700 1 -- 172.21.15.70:0/2894762967 learned_addr learned my addr 172.21.15.70:0/2894762967 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe8037fe700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8040fb3b0 0x7fe8041137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe802ffd700 1 -- 172.21.15.70:0/2894762967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8040fb3b0 msgr2=0x7fe8041137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe802ffd700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8040fb3b0 0x7fe8041137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe802ffd700 1 -- 172.21.15.70:0/2894762967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f4005040 con 0x7fe804102f70 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe802ffd700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 0x7fe804113d00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec002a50 tx=0x7fe7ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:04.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe7fbfff700 1 -- 172.21.15.70:0/2894762967 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7ec007cc0 con 0x7fe804102f70 2024-03-23T08:28:04.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80410dc70 con 0x7fe804102f70 2024-03-23T08:28:04.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.699+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80410e230 con 0x7fe804102f70 2024-03-23T08:28:04.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.700+0000 7fe7fbfff700 1 -- 172.21.15.70:0/2894762967 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7ec00f040 con 0x7fe804102f70 2024-03-23T08:28:04.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.700+0000 7fe7fbfff700 1 -- 172.21.15.70:0/2894762967 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7ec009510 con 0x7fe804102f70 2024-03-23T08:28:04.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.701+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e40052e0 con 0x7fe804102f70 2024-03-23T08:28:04.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.701+0000 7fe7fbfff700 1 -- 172.21.15.70:0/2894762967 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fe7ec009670 con 0x7fe804102f70 2024-03-23T08:28:04.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.701+0000 7fe7fbfff700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cd50 0x7fe7f006f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:04.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.701+0000 7fe7fbfff700 1 -- 172.21.15.70:0/2894762967 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe7ec08be80 con 0x7fe804102f70 2024-03-23T08:28:04.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.701+0000 7fe8037fe700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cd50 0x7fe7f006f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:04.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.702+0000 7fe8037fe700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cd50 0x7fe7f006f200 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4005010 tx=0x7fe7f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:04.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.706+0000 7fe7fbfff700 1 -- 172.21.15.70:0/2894762967 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe7ec0597d0 con 0x7fe804102f70 2024-03-23T08:28:04.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.917+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7e4000bf0 con 0x7fe7f006cd50 2024-03-23T08:28:04.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.919+0000 7fe7fbfff700 1 -- 172.21.15.70:0/2894762967 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe7e4000bf0 con 0x7fe7f006cd50 2024-03-23T08:28:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cd50 msgr2=0x7fe7f006f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cd50 0x7fe7f006f200 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4005010 tx=0x7fe7f4009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 msgr2=0x7fe804113d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 0x7fe804113d00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec002a50 tx=0x7fe7ec0046b0 comp rx=0 tx=0).stop 2024-03-23T08:28:04.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 shutdown_connections 2024-03-23T08:28:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cd50 0x7fe7f006f200 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8040fb3b0 0x7fe8041137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 --2- 172.21.15.70:0/2894762967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102f70 0x7fe804113d00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 >> 172.21.15.70:0/2894762967 conn(0x7fe804072310 msgr2=0x7fe8040f9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 shutdown_connections 2024-03-23T08:28:04.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:04.921+0000 7fe809c8f700 1 -- 172.21.15.70:0/2894762967 wait complete. 2024-03-23T08:28:04.931 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:28:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 -- 172.21.15.70:0/3242397095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0102710 msgr2=0x7f41f00fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 --2- 172.21.15.70:0/3242397095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0102710 0x7f41f00fbd80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f41ec004940 tx=0x7f41ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 -- 172.21.15.70:0/3242397095 shutdown_connections 2024-03-23T08:28:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 --2- 172.21.15.70:0/3242397095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0102710 0x7f41f00fbd80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 --2- 172.21.15.70:0/3242397095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0101d70 0x7f41f0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 -- 172.21.15.70:0/3242397095 >> 172.21.15.70:0/3242397095 conn(0x7f41f0072bc0 msgr2=0x7f41f0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 -- 172.21.15.70:0/3242397095 shutdown_connections 2024-03-23T08:28:05.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 -- 172.21.15.70:0/3242397095 wait complete. 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 Processor -- start 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.021+0000 7f41f75e0700 1 -- start start 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f75e0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0101d70 0x7f41f01129f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f75e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0102710 0x7f41f0112f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f75e0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f0113610 con 0x7f41f0101d70 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f75e0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f010c970 con 0x7f41f0102710 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f537c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0101d70 0x7f41f01129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f537c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0101d70 0x7f41f01129f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49896/0 (socket says 172.21.15.70:49896) 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f537c700 1 -- 172.21.15.70:0/682115812 learned_addr learned my addr 172.21.15.70:0/682115812 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f4b7b700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0102710 0x7f41f0112f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f4b7b700 1 -- 172.21.15.70:0/682115812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0102710 msgr2=0x7f41f0112f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f4b7b700 1 -- 172.21.15.70:0/682115812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0102710 msgr2=0x7f41f0112f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f4b7b700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0102710 0x7f41f0112f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f4b7b700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0102710 0x7f41f0112f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:28:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f537c700 1 -- 172.21.15.70:0/682115812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0102710 msgr2=0x7f41f0112f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:28:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f537c700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0102710 0x7f41f0112f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f537c700 1 -- 172.21.15.70:0/682115812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec005040 con 0x7f41f0101d70 2024-03-23T08:28:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f537c700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0101d70 0x7f41f01129f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b790 tx=0x7f41e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41e5ffb700 1 -- 172.21.15.70:0/682115812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e000f770 con 0x7f41f0101d70 2024-03-23T08:28:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f010cc50 con 0x7f41f0101d70 2024-03-23T08:28:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41e5ffb700 1 -- 172.21.15.70:0/682115812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f41e000fd80 con 0x7f41f0101d70 2024-03-23T08:28:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.022+0000 7f41e5ffb700 1 -- 172.21.15.70:0/682115812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41e000d310 con 0x7f41f0101d70 2024-03-23T08:28:05.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.023+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f010d210 con 0x7f41f0101d70 2024-03-23T08:28:05.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.023+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f00ff080 con 0x7f41f0101d70 2024-03-23T08:28:05.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.024+0000 7f41e5ffb700 1 -- 172.21.15.70:0/682115812 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f41e0010420 con 0x7f41f0101d70 2024-03-23T08:28:05.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.025+0000 7f41e5ffb700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f41dc06cd40 0x7f41dc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:05.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.025+0000 7f41e5ffb700 1 -- 172.21.15.70:0/682115812 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f41e008c770 con 0x7f41f0101d70 2024-03-23T08:28:05.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.025+0000 7f41f4b7b700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f41dc06cd40 0x7f41dc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:05.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.025+0000 7f41f4b7b700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f41dc06cd40 0x7f41dc06f1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f41f010db30 tx=0x7f41ec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:05.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.029+0000 7f41e5ffb700 1 -- 172.21.15.70:0/682115812 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f41e005a270 con 0x7f41f0101d70 2024-03-23T08:28:05.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.239+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41f0108d80 con 0x7f41dc06cd40 2024-03-23T08:28:05.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.240+0000 7f41e5ffb700 1 -- 172.21.15.70:0/682115812 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f41f0108d80 con 0x7f41dc06cd40 2024-03-23T08:28:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f41dc06cd40 msgr2=0x7f41dc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f41dc06cd40 0x7f41dc06f1f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f41f010db30 tx=0x7f41ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0101d70 msgr2=0x7f41f01129f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0101d70 0x7f41f01129f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b790 tx=0x7f41e0009250 comp rx=0 tx=0).stop 2024-03-23T08:28:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 shutdown_connections 2024-03-23T08:28:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f41dc06cd40 0x7f41dc06f1f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41f0101d70 0x7f41f01129f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 --2- 172.21.15.70:0/682115812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41f0102710 0x7f41f0112f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 >> 172.21.15.70:0/682115812 conn(0x7f41f0072bc0 msgr2=0x7f41f00fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 shutdown_connections 2024-03-23T08:28:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.242+0000 7f41f75e0700 1 -- 172.21.15.70:0/682115812 wait complete. 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2341659441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc102e70 msgr2=0x7fc8cc103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 --2- 172.21.15.70:0/2341659441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc102e70 0x7fc8cc103240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc004940 tx=0x7fc8bc009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2341659441 shutdown_connections 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 --2- 172.21.15.70:0/2341659441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8cc0fb4a0 0x7fc8cc0fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 --2- 172.21.15.70:0/2341659441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc102e70 0x7fc8cc103240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2341659441 >> 172.21.15.70:0/2341659441 conn(0x7fc8cc0714c0 msgr2=0x7fc8cc0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2341659441 shutdown_connections 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2341659441 wait complete. 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 Processor -- start 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.341+0000 7fc8d47f9700 1 -- start start 2024-03-23T08:28:05.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d47f9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc0fb4a0 0x7fc8cc115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d47f9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8cc102e70 0x7fc8cc1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d47f9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc116810 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d47f9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc10fbe0 con 0x7fc8cc102e70 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d2595700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc0fb4a0 0x7fc8cc115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d2595700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc0fb4a0 0x7fc8cc115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49910/0 (socket says 172.21.15.70:49910) 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d2595700 1 -- 172.21.15.70:0/2251096503 learned_addr learned my addr 172.21.15.70:0/2251096503 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d1d94700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8cc102e70 0x7fc8cc1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d2595700 1 -- 172.21.15.70:0/2251096503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8cc102e70 msgr2=0x7fc8cc1161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d2595700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8cc102e70 0x7fc8cc1161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d2595700 1 -- 172.21.15.70:0/2251096503 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc005040 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d2595700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc0fb4a0 0x7fc8cc115c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00b3b0 tx=0x7fc8bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8c2ffd700 1 -- 172.21.15.70:0/2251096503 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8bc00e040 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8cc10fe60 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8c2ffd700 1 -- 172.21.15.70:0/2251096503 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8bc013420 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8c2ffd700 1 -- 172.21.15.70:0/2251096503 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8bc01c630 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.342+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc110420 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.343+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8cc107270 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.344+0000 7fc8c2ffd700 1 -- 172.21.15.70:0/2251096503 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fc8bc013970 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.344+0000 7fc8c2ffd700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc8b806cd40 0x7fc8b806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:05.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.344+0000 7fc8c2ffd700 1 -- 172.21.15.70:0/2251096503 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc8bc08d450 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.344+0000 7fc8d1d94700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc8b806cd40 0x7fc8b806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:05.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.345+0000 7fc8d1d94700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc8b806cd40 0x7fc8b806f1f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc111520 tx=0x7fc8c800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:05.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.349+0000 7fc8c2ffd700 1 -- 172.21.15.70:0/2251096503 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc8bc05ada0 con 0x7fc8cc0fb4a0 2024-03-23T08:28:05.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:05 smithi070 ceph-mon[145869]: pgmap v12: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-23T08:28:05.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:05 smithi193 ceph-mon[108533]: pgmap v12: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-23T08:28:05.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.549+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8cc109e00 con 0x7fc8b806cd40 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.559+0000 7fc8c2ffd700 1 -- 172.21.15.70:0/2251096503 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc8cc109e00 con 0x7fc8b806cd40 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (102s) 13s ago 12m 16.9M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (13m) 13s ago 13m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (12m) 15s ago 12m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (80s) 13s ago 12m 80.0M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (7m) 13s ago 7m 148M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (7m) 13s ago 7m 13.6M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (7m) 15s ago 7m 97.2M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (7m) 15s ago 7m 15.0M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (3m) 13s ago 14m 470M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:28:05.559 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (2m) 15s ago 11m 445M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (29s) 13s ago 14m 30.2M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 starting - - - 2048M 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (2m) 13s ago 12m 18.2M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (2m) 15s ago 12m 16.2M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (11m) 13s ago 11m 218M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (10m) 13s ago 10m 234M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (10m) 13s ago 10m 202M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (9m) 15s ago 9m 278M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (9m) 15s ago 9m 242M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (8m) 15s ago 8m 225M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:28:05.560 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (119s) 13s ago 12m 50.2M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc8b806cd40 msgr2=0x7fc8b806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc8b806cd40 0x7fc8b806f1f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc111520 tx=0x7fc8c800b040 comp rx=0 tx=0).stop 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc0fb4a0 msgr2=0x7fc8cc115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc0fb4a0 0x7fc8cc115c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00b3b0 tx=0x7fc8bc009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 shutdown_connections 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc8b806cd40 0x7fc8b806f1f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8cc0fb4a0 0x7fc8cc115c80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 --2- 172.21.15.70:0/2251096503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8cc102e70 0x7fc8cc1161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 >> 172.21.15.70:0/2251096503 conn(0x7fc8cc0714c0 msgr2=0x7fc8cc0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 shutdown_connections 2024-03-23T08:28:05.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.561+0000 7fc8d47f9700 1 -- 172.21.15.70:0/2251096503 wait complete. 2024-03-23T08:28:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 -- 172.21.15.70:0/2564756286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 msgr2=0x7f01c40fba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 --2- 172.21.15.70:0/2564756286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 0x7f01c40fba70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f01b8004940 tx=0x7f01b8009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 -- 172.21.15.70:0/2564756286 shutdown_connections 2024-03-23T08:28:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 --2- 172.21.15.70:0/2564756286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 0x7f01c40fba70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 --2- 172.21.15.70:0/2564756286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 0x7f01c4103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 -- 172.21.15.70:0/2564756286 >> 172.21.15.70:0/2564756286 conn(0x7f01c40714c0 msgr2=0x7f01c40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 -- 172.21.15.70:0/2564756286 shutdown_connections 2024-03-23T08:28:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 -- 172.21.15.70:0/2564756286 wait complete. 2024-03-23T08:28:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 Processor -- start 2024-03-23T08:28:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.659+0000 7f01c9572700 1 -- start start 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c9572700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 0x7f01c40fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c9572700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 0x7f01c40fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c9572700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40fe730 con 0x7f01c40fb600 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c9572700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40fd260 con 0x7f01c4102e80 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 0x7f01c40fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c2ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 0x7f01c40fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49922/0 (socket says 172.21.15.70:49922) 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c2ffd700 1 -- 172.21.15.70:0/1075217049 learned_addr learned my addr 172.21.15.70:0/1075217049 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c27fc700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 0x7f01c40fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c27fc700 1 -- 172.21.15.70:0/1075217049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 msgr2=0x7f01c40fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c27fc700 1 -- 172.21.15.70:0/1075217049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 msgr2=0x7f01c40fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c27fc700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 0x7f01c40fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-23T08:28:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c27fc700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 0x7f01c40fcd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c2ffd700 1 -- 172.21.15.70:0/1075217049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 msgr2=0x7f01c40fcd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c2ffd700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 0x7f01c40fcd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c2ffd700 1 -- 172.21.15.70:0/1075217049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b8005040 con 0x7f01c40fb600 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c2ffd700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 0x7f01c40fc7e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01b4002a80 tx=0x7f01b4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01ab7fe700 1 -- 172.21.15.70:0/1075217049 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01b4007cc0 con 0x7f01c40fb600 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01ab7fe700 1 -- 172.21.15.70:0/1075217049 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01b400f040 con 0x7f01c40fb600 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c40fd540 con 0x7f01c40fb600 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01ab7fe700 1 -- 172.21.15.70:0/1075217049 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01b400da20 con 0x7f01c40fb600 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.660+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c411a030 con 0x7f01c40fb600 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.661+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c4065ec0 con 0x7f01c40fb600 2024-03-23T08:28:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.662+0000 7f01ab7fe700 1 -- 172.21.15.70:0/1075217049 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f01b400db80 con 0x7f01c40fb600 2024-03-23T08:28:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.662+0000 7f01ab7fe700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f01ac06cd50 0x7f01ac06f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.662+0000 7f01ab7fe700 1 -- 172.21.15.70:0/1075217049 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f01b408bfe0 con 0x7f01c40fb600 2024-03-23T08:28:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.662+0000 7f01c27fc700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f01ac06cd50 0x7f01ac06f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.663+0000 7f01c27fc700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f01ac06cd50 0x7f01ac06f200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009fd0 tx=0x7f01b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:05.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.667+0000 7f01ab7fe700 1 -- 172.21.15.70:0/1075217049 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f01b4055e20 con 0x7f01c40fb600 2024-03-23T08:28:05.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.943+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f01c404c710 con 0x7f01c40fb600 2024-03-23T08:28:05.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.943+0000 7f01ab7fe700 1 -- 172.21.15.70:0/1075217049 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f01b4059440 con 0x7f01c40fb600 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:28:05.944 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:28:05.945 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-23T08:28:05.945 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 4 2024-03-23T08:28:05.945 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:28:05.945 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:28:05.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f01ac06cd50 msgr2=0x7f01ac06f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f01ac06cd50 0x7f01ac06f200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009fd0 tx=0x7f01b8009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 msgr2=0x7f01c40fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 0x7f01c40fc7e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01b4002a80 tx=0x7f01b4004600 comp rx=0 tx=0).stop 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 shutdown_connections 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f01ac06cd50 0x7f01ac06f200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01c40fb600 0x7f01c40fc7e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 --2- 172.21.15.70:0/1075217049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01c4102e80 0x7f01c40fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 >> 172.21.15.70:0/1075217049 conn(0x7f01c40714c0 msgr2=0x7f01c4105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 shutdown_connections 2024-03-23T08:28:05.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:05.945+0000 7f01c9572700 1 -- 172.21.15.70:0/1075217049 wait complete. 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.046+0000 7f0680df5700 1 -- 172.21.15.70:0/2705486771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fde90 msgr2=0x7f067c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.046+0000 7f0680df5700 1 --2- 172.21.15.70:0/2705486771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fde90 0x7f067c0fe260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0674009ce0 tx=0x7f0674004d10 comp rx=0 tx=0).stop 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.046+0000 7f0680df5700 1 -- 172.21.15.70:0/2705486771 shutdown_connections 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.046+0000 7f0680df5700 1 --2- 172.21.15.70:0/2705486771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fe830 0x7f067c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.046+0000 7f0680df5700 1 --2- 172.21.15.70:0/2705486771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fde90 0x7f067c0fe260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.046+0000 7f0680df5700 1 -- 172.21.15.70:0/2705486771 >> 172.21.15.70:0/2705486771 conn(0x7f067c072bc0 msgr2=0x7f067c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.046+0000 7f0680df5700 1 -- 172.21.15.70:0/2705486771 shutdown_connections 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.046+0000 7f0680df5700 1 -- 172.21.15.70:0/2705486771 wait complete. 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0680df5700 1 Processor -- start 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0680df5700 1 -- start start 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0680df5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fde90 0x7f067c1139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:06.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0680df5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fe830 0x7f067c113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0680df5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c114580 con 0x7f067c0fe830 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0680df5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c10da00 con 0x7f067c0fde90 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0679d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fe830 0x7f067c113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0679d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fe830 0x7f067c113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49946/0 (socket says 172.21.15.70:49946) 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0679d9b700 1 -- 172.21.15.70:0/1588712394 learned_addr learned my addr 172.21.15.70:0/1588712394 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f067a59c700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fde90 0x7f067c1139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f067a59c700 1 -- 172.21.15.70:0/1588712394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fde90 msgr2=0x7f067c1139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f067a59c700 1 -- 172.21.15.70:0/1588712394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fde90 msgr2=0x7f067c1139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f067a59c700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fde90 0x7f067c1139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f067a59c700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fde90 0x7f067c1139f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0679d9b700 1 -- 172.21.15.70:0/1588712394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fde90 msgr2=0x7f067c1139f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0679d9b700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fde90 0x7f067c1139f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0679d9b700 1 -- 172.21.15.70:0/1588712394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0674009940 con 0x7f067c0fe830 2024-03-23T08:28:06.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.047+0000 7f0679d9b700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fe830 0x7f067c113f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06640041d0 tx=0x7f06640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:06.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.048+0000 7f0672ffd700 1 -- 172.21.15.70:0/1588712394 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0664007cc0 con 0x7f067c0fe830 2024-03-23T08:28:06.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.048+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c10dce0 con 0x7f067c0fe830 2024-03-23T08:28:06.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.048+0000 7f0672ffd700 1 -- 172.21.15.70:0/1588712394 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f066400f040 con 0x7f067c0fe830 2024-03-23T08:28:06.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.048+0000 7f0672ffd700 1 -- 172.21.15.70:0/1588712394 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066400dae0 con 0x7f067c0fe830 2024-03-23T08:28:06.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.048+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c10e220 con 0x7f067c0fe830 2024-03-23T08:28:06.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.048+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067c065e30 con 0x7f067c0fe830 2024-03-23T08:28:06.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.049+0000 7f0672ffd700 1 -- 172.21.15.70:0/1588712394 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f066400dc40 con 0x7f067c0fe830 2024-03-23T08:28:06.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.049+0000 7f0672ffd700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f066806cd40 0x7f066806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:06.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.049+0000 7f0672ffd700 1 -- 172.21.15.70:0/1588712394 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f066408b910 con 0x7f067c0fe830 2024-03-23T08:28:06.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.049+0000 7f067a59c700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f066806cd40 0x7f066806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:06.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.050+0000 7f067a59c700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f066806cd40 0x7f066806f1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f067400b740 tx=0x7f0674004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:06.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.054+0000 7f0672ffd700 1 -- 172.21.15.70:0/1588712394 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0664059260 con 0x7f067c0fe830 2024-03-23T08:28:06.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.302+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f067c10f0a0 con 0x7f067c0fe830 2024-03-23T08:28:06.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.302+0000 7f0672ffd700 1 -- 172.21.15.70:0/1588712394 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f0664058df0 con 0x7f067c0fe830 2024-03-23T08:28:06.303 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:28:06.303 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:28:06.303 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:28:06.303 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:28:06.303 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:28:06.303 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:28:06.304 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:28:06.305 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:06.306 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:06.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f066806cd40 msgr2=0x7f066806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:06.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f066806cd40 0x7f066806f1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f067400b740 tx=0x7f0674004d10 comp rx=0 tx=0).stop 2024-03-23T08:28:06.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fe830 msgr2=0x7f067c113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:06.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fe830 0x7f067c113f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06640041d0 tx=0x7f06640046b0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 shutdown_connections 2024-03-23T08:28:06.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f066806cd40 0x7f066806f1f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f067c0fde90 0x7f067c1139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 --2- 172.21.15.70:0/1588712394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f067c0fe830 0x7f067c113f30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 >> 172.21.15.70:0/1588712394 conn(0x7f067c072bc0 msgr2=0x7f067c103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:06.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.304+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 shutdown_connections 2024-03-23T08:28:06.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.305+0000 7f0680df5700 1 -- 172.21.15.70:0/1588712394 wait complete. 2024-03-23T08:28:06.308 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:28:06.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 -- 172.21.15.70:0/2344844829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 msgr2=0x7fcfd01092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:06.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 --2- 172.21.15.70:0/2344844829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 0x7fcfd01092f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0004940 tx=0x7fcfc0009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:06.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 -- 172.21.15.70:0/2344844829 shutdown_connections 2024-03-23T08:28:06.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 --2- 172.21.15.70:0/2344844829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 0x7fcfd01092f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.406 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 --2- 172.21.15.70:0/2344844829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 0x7fcfd00fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 -- 172.21.15.70:0/2344844829 >> 172.21.15.70:0/2344844829 conn(0x7fcfd0072bc0 msgr2=0x7fcfd0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:06.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 -- 172.21.15.70:0/2344844829 shutdown_connections 2024-03-23T08:28:06.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 -- 172.21.15.70:0/2344844829 wait complete. 2024-03-23T08:28:06.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 Processor -- start 2024-03-23T08:28:06.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.406+0000 7fcfd80a5700 1 -- start start 2024-03-23T08:28:06.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd80a5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 0x7fcfd0115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:06.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd80a5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 0x7fcfd01161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:06.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd80a5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0116800 con 0x7fcfd00fc590 2024-03-23T08:28:06.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd80a5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd010fbd0 con 0x7fcfd00fbc80 2024-03-23T08:28:06.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5640700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 0x7fcfd01161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5640700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 0x7fcfd01161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49956/0 (socket says 172.21.15.70:49956) 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5640700 1 -- 172.21.15.70:0/482465147 learned_addr learned my addr 172.21.15.70:0/482465147 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5e41700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 0x7fcfd0115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5e41700 1 -- 172.21.15.70:0/482465147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 msgr2=0x7fcfd0115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5e41700 1 -- 172.21.15.70:0/482465147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 msgr2=0x7fcfd0115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5e41700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 0x7fcfd0115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5e41700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 0x7fcfd0115c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5640700 1 -- 172.21.15.70:0/482465147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 msgr2=0x7fcfd0115c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5640700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 0x7fcfd0115c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5640700 1 -- 172.21.15.70:0/482465147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc0005040 con 0x7fcfd00fc590 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd5640700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 0x7fcfd01161b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009360 tx=0x7fcfc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfc67fc700 1 -- 172.21.15.70:0/482465147 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc000e040 con 0x7fcfd00fc590 2024-03-23T08:28:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfc67fc700 1 -- 172.21.15.70:0/482465147 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfc0013420 con 0x7fcfd00fc590 2024-03-23T08:28:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd010fe50 con 0x7fcfd00fc590 2024-03-23T08:28:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfc67fc700 1 -- 172.21.15.70:0/482465147 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc001c5f0 con 0x7fcfd00fc590 2024-03-23T08:28:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.407+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd0110410 con 0x7fcfd00fc590 2024-03-23T08:28:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.408+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd0065e30 con 0x7fcfd00fc590 2024-03-23T08:28:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.409+0000 7fcfc67fc700 1 -- 172.21.15.70:0/482465147 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fcfc0013590 con 0x7fcfd00fc590 2024-03-23T08:28:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.409+0000 7fcfc67fc700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcfbc06cd40 0x7fcfbc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.410+0000 7fcfd5e41700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcfbc06cd40 0x7fcfbc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.410+0000 7fcfc67fc700 1 -- 172.21.15.70:0/482465147 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcfc008d360 con 0x7fcfd00fc590 2024-03-23T08:28:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.410+0000 7fcfd5e41700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcfbc06cd40 0x7fcfbc06f1f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009e50 tx=0x7fcfcc009f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.414+0000 7fcfc67fc700 1 -- 172.21.15.70:0/482465147 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcfc005ad50 con 0x7fcfd00fc590 2024-03-23T08:28:06.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:06 smithi070 ceph-mon[145869]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:06.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:06 smithi070 ceph-mon[145869]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:06.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:06 smithi070 ceph-mon[145869]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:06.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:06 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1075217049' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:06.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:06 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:06.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:06 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:06 smithi193 ceph-mon[108533]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:06 smithi193 ceph-mon[108533]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:06 smithi193 ceph-mon[108533]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:06 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1075217049' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:06 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:06.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:06 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:06.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.629+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfd010c090 con 0x7fcfbc06cd40 2024-03-23T08:28:06.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.630+0000 7fcfc67fc700 1 -- 172.21.15.70:0/482465147 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fcfd010c090 con 0x7fcfbc06cd40 2024-03-23T08:28:06.630 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "4/21 daemons upgraded", 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mon daemons", 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:28:06.631 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.632+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcfbc06cd40 msgr2=0x7fcfbc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.632+0000 7fcfd80a5700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcfbc06cd40 0x7fcfbc06f1f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009e50 tx=0x7fcfcc009f10 comp rx=0 tx=0).stop 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.632+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 msgr2=0x7fcfd01161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.632+0000 7fcfd80a5700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 0x7fcfd01161b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009360 tx=0x7fcfc0009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.632+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 shutdown_connections 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.632+0000 7fcfd80a5700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcfbc06cd40 0x7fcfbc06f1f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.632+0000 7fcfd80a5700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd00fbc80 0x7fcfd0115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.632+0000 7fcfd80a5700 1 --2- 172.21.15.70:0/482465147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfd00fc590 0x7fcfd01161b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.632+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 >> 172.21.15.70:0/482465147 conn(0x7fcfd0072bc0 msgr2=0x7fcfd0105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.633+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 shutdown_connections 2024-03-23T08:28:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.633+0000 7fcfd80a5700 1 -- 172.21.15.70:0/482465147 wait complete. 2024-03-23T08:28:06.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3261966132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fe830 msgr2=0x7f6dc0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:06.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 --2- 172.21.15.70:0/3261966132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fe830 0x7f6dc0106f00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6db4004940 tx=0x7f6db4009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:06.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3261966132 shutdown_connections 2024-03-23T08:28:06.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 --2- 172.21.15.70:0/3261966132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fe830 0x7f6dc0106f00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 --2- 172.21.15.70:0/3261966132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fde90 0x7f6dc00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3261966132 >> 172.21.15.70:0/3261966132 conn(0x7f6dc0072bc0 msgr2=0x7f6dc0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:06.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3261966132 shutdown_connections 2024-03-23T08:28:06.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3261966132 wait complete. 2024-03-23T08:28:06.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 Processor -- start 2024-03-23T08:28:06.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.734+0000 7f6dc62b5700 1 -- start start 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dc62b5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fde90 0x7f6dc0113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dc62b5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fe830 0x7f6dc0113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dc62b5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0114640 con 0x7f6dc00fde90 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dc62b5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc010da80 con 0x7f6dc00fe830 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fde90 0x7f6dc0113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fde90 0x7f6dc0113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49978/0 (socket says 172.21.15.70:49978) 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbffff700 1 -- 172.21.15.70:0/3490765500 learned_addr learned my addr 172.21.15.70:0/3490765500 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbf7fe700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fe830 0x7f6dc0113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbf7fe700 1 -- 172.21.15.70:0/3490765500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fe830 msgr2=0x7f6dc0113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbf7fe700 1 -- 172.21.15.70:0/3490765500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fe830 msgr2=0x7f6dc0113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbf7fe700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fe830 0x7f6dc0113ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-23T08:28:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbf7fe700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fe830 0x7f6dc0113ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbffff700 1 -- 172.21.15.70:0/3490765500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fe830 msgr2=0x7f6dc0113ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbffff700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fe830 0x7f6dc0113ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbffff700 1 -- 172.21.15.70:0/3490765500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db4005040 con 0x7f6dc00fde90 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbffff700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fde90 0x7f6dc0113ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6db00041d0 tx=0x7f6db00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbcff9700 1 -- 172.21.15.70:0/3490765500 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6db0007cc0 con 0x7f6dc00fde90 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbcff9700 1 -- 172.21.15.70:0/3490765500 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6db000f040 con 0x7f6dc00fde90 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dbcff9700 1 -- 172.21.15.70:0/3490765500 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6db000dae0 con 0x7f6dc00fde90 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.735+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc010dd60 con 0x7f6dc00fde90 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.736+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc010e270 con 0x7f6dc00fde90 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.736+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc0065e30 con 0x7f6dc00fde90 2024-03-23T08:28:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.737+0000 7f6dbcff9700 1 -- 172.21.15.70:0/3490765500 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f6db000dc40 con 0x7f6dc00fde90 2024-03-23T08:28:06.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.737+0000 7f6dbcff9700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6da806cd40 0x7f6da806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:06.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.737+0000 7f6dbcff9700 1 -- 172.21.15.70:0/3490765500 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6db008b9e0 con 0x7f6dc00fde90 2024-03-23T08:28:06.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.737+0000 7f6dbf7fe700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6da806cd40 0x7f6da806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:06.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.738+0000 7f6dbf7fe700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6da806cd40 0x7f6da806f1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6dc010ed10 tx=0x7f6db4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:06.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:06.744+0000 7f6dbcff9700 1 -- 172.21.15.70:0/3490765500 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6db00592b0 con 0x7f6dc00fde90 2024-03-23T08:28:07.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.013+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6dc005f5f0 con 0x7f6dc00fde90 2024-03-23T08:28:07.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.013+0000 7f6dbcff9700 1 -- 172.21.15.70:0/3490765500 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6db0058e40 con 0x7f6dc00fde90 2024-03-23T08:28:07.014 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-03-23T08:28:07.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6da806cd40 msgr2=0x7f6da806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:07.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6da806cd40 0x7f6da806f1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6dc010ed10 tx=0x7f6db4009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:07.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fde90 msgr2=0x7f6dc0113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fde90 0x7f6dc0113ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6db00041d0 tx=0x7f6db00046b0 comp rx=0 tx=0).stop 2024-03-23T08:28:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 shutdown_connections 2024-03-23T08:28:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6da806cd40 0x7f6da806f1f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dc00fde90 0x7f6dc0113ab0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 --2- 172.21.15.70:0/3490765500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6dc00fe830 0x7f6dc0113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 >> 172.21.15.70:0/3490765500 conn(0x7f6dc0072bc0 msgr2=0x7f6dc0103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 shutdown_connections 2024-03-23T08:28:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:07.015+0000 7f6dc62b5700 1 -- 172.21.15.70:0/3490765500 wait complete. 2024-03-23T08:28:07.464 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:07 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1588712394' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:28:07.464 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:07 smithi193 ceph-mon[108533]: pgmap v13: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:07.464 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:07 smithi193 ceph-mon[108533]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:07.464 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:07 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:07.464 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:07 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:07.464 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:07 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3490765500' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:28:07.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:07 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1588712394' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:28:07.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:07 smithi070 ceph-mon[145869]: pgmap v13: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:07.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:07 smithi070 ceph-mon[145869]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:07.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:07 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:07.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:07 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:07.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:07 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3490765500' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:28:09.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:09 smithi193 ceph-mon[108533]: pgmap v14: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:09 smithi070 ceph-mon[145869]: pgmap v14: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: Detected new or changed devices on smithi193 2024-03-23T08:28:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:28:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: Reconfiguring mon.smithi070 (monmap changed)... 2024-03-23T08:28:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:28:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:28:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: Reconfiguring daemon mon.smithi070 on smithi070 2024-03-23T08:28:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:11 smithi193 ceph-mon[108533]: pgmap v15: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: Detected new or changed devices on smithi193 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: Reconfiguring mon.smithi070 (monmap changed)... 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:28:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:11.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: Reconfiguring daemon mon.smithi070 on smithi070 2024-03-23T08:28:11.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:11 smithi070 ceph-mon[145869]: pgmap v15: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:13.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:12 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:13.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:12 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:13.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:12 smithi070 ceph-mon[145869]: Reconfiguring mgr.smithi070.jhsbch (monmap changed)... 2024-03-23T08:28:13.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:12 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.jhsbch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:28:13.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:12 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:28:13.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:12 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:13.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:12 smithi070 ceph-mon[145869]: Reconfiguring daemon mgr.smithi070.jhsbch on smithi070 2024-03-23T08:28:13.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:12 smithi070 ceph-mon[145869]: pgmap v16: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:12 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:12 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:12 smithi193 ceph-mon[108533]: Reconfiguring mgr.smithi070.jhsbch (monmap changed)... 2024-03-23T08:28:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:12 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.jhsbch", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:28:13.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:12 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:28:13.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:12 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:13.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:12 smithi193 ceph-mon[108533]: Reconfiguring daemon mgr.smithi070.jhsbch on smithi070 2024-03-23T08:28:13.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:12 smithi193 ceph-mon[108533]: pgmap v16: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:14.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:14 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:14.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:14 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:14.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:14 smithi070 ceph-mon[145869]: Reconfiguring crash.smithi070 (monmap changed)... 2024-03-23T08:28:14.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:14 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:28:14.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:14 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:14.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:14 smithi070 ceph-mon[145869]: Reconfiguring daemon crash.smithi070 on smithi070 2024-03-23T08:28:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:14 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:14 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:14 smithi193 ceph-mon[108533]: Reconfiguring crash.smithi070 (monmap changed)... 2024-03-23T08:28:14.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:14 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:28:14.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:14 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:14.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:14 smithi193 ceph-mon[108533]: Reconfiguring daemon crash.smithi070 on smithi070 2024-03-23T08:28:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:15 smithi070 ceph-mon[145869]: pgmap v17: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T08:28:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:15.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:15 smithi193 ceph-mon[108533]: pgmap v17: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:15.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:15.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:15.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T08:28:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:16.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:16 smithi070 ceph-mon[145869]: Reconfiguring osd.0 (monmap changed)... 2024-03-23T08:28:16.667 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:16 smithi070 ceph-mon[145869]: Reconfiguring daemon osd.0 on smithi070 2024-03-23T08:28:16.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:16 smithi193 ceph-mon[108533]: Reconfiguring osd.0 (monmap changed)... 2024-03-23T08:28:16.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:16 smithi193 ceph-mon[108533]: Reconfiguring daemon osd.0 on smithi070 2024-03-23T08:28:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:17 smithi070 ceph-mon[145869]: pgmap v18: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:28:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:17 smithi070 ceph-mon[145869]: Reconfiguring osd.1 (monmap changed)... 2024-03-23T08:28:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T08:28:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:17.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:17 smithi070 ceph-mon[145869]: Reconfiguring daemon osd.1 on smithi070 2024-03-23T08:28:17.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:17 smithi193 ceph-mon[108533]: pgmap v18: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:17.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:28:17.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:17.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:17.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:17 smithi193 ceph-mon[108533]: Reconfiguring osd.1 (monmap changed)... 2024-03-23T08:28:17.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T08:28:17.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:17.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:17 smithi193 ceph-mon[108533]: Reconfiguring daemon osd.1 on smithi070 2024-03-23T08:28:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:19 smithi070 ceph-mon[145869]: Reconfiguring osd.2 (monmap changed)... 2024-03-23T08:28:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T08:28:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:19 smithi070 ceph-mon[145869]: Reconfiguring daemon osd.2 on smithi070 2024-03-23T08:28:19.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:19 smithi070 ceph-mon[145869]: pgmap v19: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:19.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:19.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:19.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:19 smithi193 ceph-mon[108533]: Reconfiguring osd.2 (monmap changed)... 2024-03-23T08:28:19.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T08:28:19.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:19.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:19 smithi193 ceph-mon[108533]: Reconfiguring daemon osd.2 on smithi070 2024-03-23T08:28:19.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:19 smithi193 ceph-mon[108533]: pgmap v19: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:20 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:20 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:20 smithi070 ceph-mon[145869]: Reconfiguring mds.cephfs.smithi070.iziydh (monmap changed)... 2024-03-23T08:28:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:20 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.iziydh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:28:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:20 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:20.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:20 smithi070 ceph-mon[145869]: Reconfiguring daemon mds.cephfs.smithi070.iziydh on smithi070 2024-03-23T08:28:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:20 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:20 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:20 smithi193 ceph-mon[108533]: Reconfiguring mds.cephfs.smithi070.iziydh (monmap changed)... 2024-03-23T08:28:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:20 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.iziydh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:28:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:20 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:20 smithi193 ceph-mon[108533]: Reconfiguring daemon mds.cephfs.smithi070.iziydh on smithi070 2024-03-23T08:28:21.973 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:21 smithi070 ceph-mon[145869]: pgmap v20: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:21.973 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:21.973 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:21.974 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.aymzqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:28:21.974 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:21 smithi193 ceph-mon[108533]: pgmap v20: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.aymzqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:28:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:22.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:22 smithi070 ceph-mon[145869]: Reconfiguring mds.cephfs.smithi070.aymzqb (monmap changed)... 2024-03-23T08:28:22.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:22 smithi070 ceph-mon[145869]: Reconfiguring daemon mds.cephfs.smithi070.aymzqb on smithi070 2024-03-23T08:28:22.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:22.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:22 smithi193 ceph-mon[108533]: Reconfiguring mds.cephfs.smithi070.aymzqb (monmap changed)... 2024-03-23T08:28:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:22 smithi193 ceph-mon[108533]: Reconfiguring daemon mds.cephfs.smithi070.aymzqb on smithi070 2024-03-23T08:28:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:24.029 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:23 smithi193 ceph-mon[108533]: pgmap v21: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:24.030 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:23 smithi193 ceph-mon[108533]: Reconfiguring crash.smithi193 (monmap changed)... 2024-03-23T08:28:24.030 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:28:24.030 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:24.030 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:23 smithi193 ceph-mon[108533]: Reconfiguring daemon crash.smithi193 on smithi193 2024-03-23T08:28:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:23 smithi070 ceph-mon[145869]: pgmap v21: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:23 smithi070 ceph-mon[145869]: Reconfiguring crash.smithi193 (monmap changed)... 2024-03-23T08:28:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:28:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:23 smithi070 ceph-mon[145869]: Reconfiguring daemon crash.smithi193 on smithi193 2024-03-23T08:28:25.222 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:25.223 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:25.223 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:25 smithi193 ceph-mon[108533]: Reconfiguring mgr.smithi193.zgpdbu (monmap changed)... 2024-03-23T08:28:25.223 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:28:25.223 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:28:25.223 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:25.223 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:25 smithi193 ceph-mon[108533]: Reconfiguring daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:28:25.223 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:25 smithi193 ceph-mon[108533]: pgmap v22: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:25.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:25.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:25.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:25 smithi070 ceph-mon[145869]: Reconfiguring mgr.smithi193.zgpdbu (monmap changed)... 2024-03-23T08:28:25.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.zgpdbu", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-23T08:28:25.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-23T08:28:25.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:25.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:25 smithi070 ceph-mon[145869]: Reconfiguring daemon mgr.smithi193.zgpdbu on smithi193 2024-03-23T08:28:25.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:25 smithi070 ceph-mon[145869]: pgmap v22: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:26.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:26 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:26.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:26 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:26.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:26 smithi193 ceph-mon[108533]: Reconfiguring mon.smithi193 (monmap changed)... 2024-03-23T08:28:26.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:26 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:28:26.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:26 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:28:26.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:26 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:26.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:26 smithi193 ceph-mon[108533]: Reconfiguring daemon mon.smithi193 on smithi193 2024-03-23T08:28:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:26 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:26 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:26 smithi070 ceph-mon[145869]: Reconfiguring mon.smithi193 (monmap changed)... 2024-03-23T08:28:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:26 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-23T08:28:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:26 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-23T08:28:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:26 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:26 smithi070 ceph-mon[145869]: Reconfiguring daemon mon.smithi193 on smithi193 2024-03-23T08:28:27.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:27 smithi193 ceph-mon[108533]: pgmap v23: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:27.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:27 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:27.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:27 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:27.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:27 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T08:28:27.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:27 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:27 smithi070 ceph-mon[145869]: pgmap v23: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:27 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:27 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:27 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T08:28:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:27 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:28.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:28 smithi070 ceph-mon[145869]: Reconfiguring osd.3 (monmap changed)... 2024-03-23T08:28:28.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:28 smithi070 ceph-mon[145869]: Reconfiguring daemon osd.3 on smithi193 2024-03-23T08:28:28.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:28 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:28.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:28 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:28.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:28 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T08:28:28.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:28 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:28.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:28 smithi193 ceph-mon[108533]: Reconfiguring osd.3 (monmap changed)... 2024-03-23T08:28:28.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:28 smithi193 ceph-mon[108533]: Reconfiguring daemon osd.3 on smithi193 2024-03-23T08:28:28.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:28 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:28.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:28 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:28.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:28 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T08:28:28.995 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:28 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:29.923 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:29 smithi193 ceph-mon[108533]: Reconfiguring osd.4 (monmap changed)... 2024-03-23T08:28:29.923 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:29 smithi193 ceph-mon[108533]: Reconfiguring daemon osd.4 on smithi193 2024-03-23T08:28:29.923 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:29 smithi193 ceph-mon[108533]: pgmap v24: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:29 smithi070 ceph-mon[145869]: Reconfiguring osd.4 (monmap changed)... 2024-03-23T08:28:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:29 smithi070 ceph-mon[145869]: Reconfiguring daemon osd.4 on smithi193 2024-03-23T08:28:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:29 smithi070 ceph-mon[145869]: pgmap v24: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:31 smithi193 ceph-mon[108533]: Reconfiguring osd.5 (monmap changed)... 2024-03-23T08:28:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T08:28:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:31 smithi193 ceph-mon[108533]: Reconfiguring daemon osd.5 on smithi193 2024-03-23T08:28:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:31 smithi193 ceph-mon[108533]: pgmap v25: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:31 smithi070 ceph-mon[145869]: Reconfiguring osd.5 (monmap changed)... 2024-03-23T08:28:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T08:28:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:31 smithi070 ceph-mon[145869]: Reconfiguring daemon osd.5 on smithi193 2024-03-23T08:28:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:31 smithi070 ceph-mon[145869]: pgmap v25: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:32 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:32 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:32 smithi193 ceph-mon[108533]: Reconfiguring mds.cephfs.smithi193.sdlbdd (monmap changed)... 2024-03-23T08:28:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:32 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sdlbdd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:28:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:32 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:32 smithi193 ceph-mon[108533]: Reconfiguring daemon mds.cephfs.smithi193.sdlbdd on smithi193 2024-03-23T08:28:32.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:32 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:28:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:32 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:32 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:32 smithi070 ceph-mon[145869]: Reconfiguring mds.cephfs.smithi193.sdlbdd (monmap changed)... 2024-03-23T08:28:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:32 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sdlbdd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:28:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:32 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:32 smithi070 ceph-mon[145869]: Reconfiguring daemon mds.cephfs.smithi193.sdlbdd on smithi193 2024-03-23T08:28:32.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:32 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:28:33.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:33 smithi070 ceph-mon[145869]: pgmap v26: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-23T08:28:33.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:33 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:33.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:33 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:33.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:33 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.nucaub", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:28:33.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:33 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:33.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:33 smithi193 ceph-mon[108533]: pgmap v26: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-23T08:28:33.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:33 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:33.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:33 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:33.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:33 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.nucaub", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:28:33.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:33 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:34.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:34 smithi193 ceph-mon[108533]: Reconfiguring mds.cephfs.smithi193.nucaub (monmap changed)... 2024-03-23T08:28:34.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:34 smithi193 ceph-mon[108533]: Reconfiguring daemon mds.cephfs.smithi193.nucaub on smithi193 2024-03-23T08:28:34.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:34 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:34.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:34 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:34.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:34 smithi070 ceph-mon[145869]: Reconfiguring mds.cephfs.smithi193.nucaub (monmap changed)... 2024-03-23T08:28:34.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:34 smithi070 ceph-mon[145869]: Reconfiguring daemon mds.cephfs.smithi193.nucaub on smithi193 2024-03-23T08:28:34.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:34 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:34.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:34 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:35.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:28:35.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:35.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:35.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all mon 2024-03-23T08:28:35.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:35.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]: dispatch 2024-03-23T08:28:35.717 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]': finished 2024-03-23T08:28:35.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: pgmap v27: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:35.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]: dispatch 2024-03-23T08:28:35.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:35 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]': finished 2024-03-23T08:28:35.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:28:35.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:35.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:35.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all mon 2024-03-23T08:28:35.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:35.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]: dispatch 2024-03-23T08:28:35.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]': finished 2024-03-23T08:28:35.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: pgmap v27: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:35.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]: dispatch 2024-03-23T08:28:35.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:35 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]': finished 2024-03-23T08:28:37.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.121+0000 7fc62c1d4700 1 -- 172.21.15.70:0/4234915985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 msgr2=0x7fc6241031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.121+0000 7fc62c1d4700 1 --2- 172.21.15.70:0/4234915985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 0x7fc6241031f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc614004940 tx=0x7fc614009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 -- 172.21.15.70:0/4234915985 shutdown_connections 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 --2- 172.21.15.70:0/4234915985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6241037c0 0x7fc6240fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 --2- 172.21.15.70:0/4234915985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 0x7fc6241031f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 -- 172.21.15.70:0/4234915985 >> 172.21.15.70:0/4234915985 conn(0x7fc6240f89d0 msgr2=0x7fc6240fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 -- 172.21.15.70:0/4234915985 shutdown_connections 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 -- 172.21.15.70:0/4234915985 wait complete. 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 Processor -- start 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 -- start start 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 0x7fc6240702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6241037c0 0x7fc624070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:37.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc624070e20 con 0x7fc624102e20 2024-03-23T08:28:37.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.122+0000 7fc62c1d4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc624070f60 con 0x7fc6241037c0 2024-03-23T08:28:37.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc629f70700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 0x7fc6240702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:37.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc629f70700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 0x7fc6240702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41194/0 (socket says 172.21.15.70:41194) 2024-03-23T08:28:37.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc629f70700 1 -- 172.21.15.70:0/483476627 learned_addr learned my addr 172.21.15.70:0/483476627 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc629f70700 1 -- 172.21.15.70:0/483476627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6241037c0 msgr2=0x7fc624070800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc629f70700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6241037c0 0x7fc624070800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc629f70700 1 -- 172.21.15.70:0/483476627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc614005040 con 0x7fc624102e20 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc629f70700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 0x7fc6240702c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc61400b3b0 tx=0x7fc614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc61a7fc700 1 -- 172.21.15.70:0/483476627 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61400e040 con 0x7fc624102e20 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc61a7fc700 1 -- 172.21.15.70:0/483476627 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc614013420 con 0x7fc624102e20 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc624119df0 con 0x7fc624102e20 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc61a7fc700 1 -- 172.21.15.70:0/483476627 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61401c630 con 0x7fc624102e20 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.123+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62411a2d0 con 0x7fc624102e20 2024-03-23T08:28:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.124+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6240fe570 con 0x7fc624102e20 2024-03-23T08:28:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.125+0000 7fc61a7fc700 1 -- 172.21.15.70:0/483476627 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fc614013990 con 0x7fc624102e20 2024-03-23T08:28:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.125+0000 7fc61a7fc700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc61006cd40 0x7fc61006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.125+0000 7fc61a7fc700 1 -- 172.21.15.70:0/483476627 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc61408d320 con 0x7fc624102e20 2024-03-23T08:28:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.125+0000 7fc62976f700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc61006cd40 0x7fc61006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.126+0000 7fc62976f700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc61006cd40 0x7fc61006f1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc620006fd0 tx=0x7fc620006d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:37.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.130+0000 7fc61a7fc700 1 -- 172.21.15.70:0/483476627 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc614057280 con 0x7fc624102e20 2024-03-23T08:28:37.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:37 smithi193 ceph-mon[108533]: Upgrade: Updating crash.smithi070 (1/2) 2024-03-23T08:28:37.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:37 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:37.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:37 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:28:37.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:37 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:37.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:37 smithi193 ceph-mon[108533]: Deploying daemon crash.smithi070 on smithi070 2024-03-23T08:28:37.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:37 smithi193 ceph-mon[108533]: pgmap v28: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:28:37.343 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.343+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc62404bed0 con 0x7fc61006cd40 2024-03-23T08:28:37.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.344+0000 7fc61a7fc700 1 -- 172.21.15.70:0/483476627 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fc62404bed0 con 0x7fc61006cd40 2024-03-23T08:28:37.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc61006cd40 msgr2=0x7fc61006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc61006cd40 0x7fc61006f1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc620006fd0 tx=0x7fc620006d30 comp rx=0 tx=0).stop 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 msgr2=0x7fc6240702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 0x7fc6240702c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc61400b3b0 tx=0x7fc614009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 shutdown_connections 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc61006cd40 0x7fc61006f1f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc624102e20 0x7fc6240702c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 --2- 172.21.15.70:0/483476627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6241037c0 0x7fc624070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 >> 172.21.15.70:0/483476627 conn(0x7fc6240f89d0 msgr2=0x7fc6240f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 shutdown_connections 2024-03-23T08:28:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.346+0000 7fc62c1d4700 1 -- 172.21.15.70:0/483476627 wait complete. 2024-03-23T08:28:37.356 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:28:37.421 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:37 smithi070 ceph-mon[145869]: Upgrade: Updating crash.smithi070 (1/2) 2024-03-23T08:28:37.421 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:37 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:37.421 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:37 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:28:37.422 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:37 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:37.422 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:37 smithi070 ceph-mon[145869]: Deploying daemon crash.smithi070 on smithi070 2024-03-23T08:28:37.422 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:37 smithi070 ceph-mon[145869]: pgmap v28: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:28:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.451+0000 7f0b929c0700 1 -- 172.21.15.70:0/3591407253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 msgr2=0x7f0b8c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.451+0000 7f0b929c0700 1 --2- 172.21.15.70:0/3591407253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 0x7f0b8c102140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c004940 tx=0x7f0b7c009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.451+0000 7f0b929c0700 1 -- 172.21.15.70:0/3591407253 shutdown_connections 2024-03-23T08:28:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.451+0000 7f0b929c0700 1 --2- 172.21.15.70:0/3591407253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102710 0x7f0b8c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.451+0000 7f0b929c0700 1 --2- 172.21.15.70:0/3591407253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 0x7f0b8c102140 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.451+0000 7f0b929c0700 1 -- 172.21.15.70:0/3591407253 >> 172.21.15.70:0/3591407253 conn(0x7f0b8c072bc0 msgr2=0x7f0b8c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.451+0000 7f0b929c0700 1 -- 172.21.15.70:0/3591407253 shutdown_connections 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.451+0000 7f0b929c0700 1 -- 172.21.15.70:0/3591407253 wait complete. 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b929c0700 1 Processor -- start 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b929c0700 1 -- start start 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b929c0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 0x7f0b8c113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b929c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102710 0x7f0b8c114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b929c0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c114650 con 0x7f0b8c101d70 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b929c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c10da40 con 0x7f0b8c102710 2024-03-23T08:28:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b8bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 0x7f0b8c113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b8bfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 0x7f0b8c113ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41208/0 (socket says 172.21.15.70:41208) 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b8bfff700 1 -- 172.21.15.70:0/4205137429 learned_addr learned my addr 172.21.15.70:0/4205137429 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b8b7fe700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102710 0x7f0b8c114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b8bfff700 1 -- 172.21.15.70:0/4205137429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102710 msgr2=0x7f0b8c114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b8bfff700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102710 0x7f0b8c114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b8bfff700 1 -- 172.21.15.70:0/4205137429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c005040 con 0x7f0b8c101d70 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b8b7fe700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102710 0x7f0b8c114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.452+0000 7f0b8bfff700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 0x7f0b8c113ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00b3b0 tx=0x7f0b7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.453+0000 7f0b88ff9700 1 -- 172.21.15.70:0/4205137429 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b7c00e040 con 0x7f0b8c101d70 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.453+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c10dcc0 con 0x7f0b8c101d70 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.453+0000 7f0b88ff9700 1 -- 172.21.15.70:0/4205137429 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b7c013420 con 0x7f0b8c101d70 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.453+0000 7f0b88ff9700 1 -- 172.21.15.70:0/4205137429 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b7c01c630 con 0x7f0b8c101d70 2024-03-23T08:28:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.453+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c10e280 con 0x7f0b8c101d70 2024-03-23T08:28:37.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.454+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c0fd4c0 con 0x7f0b8c101d70 2024-03-23T08:28:37.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.454+0000 7f0b88ff9700 1 -- 172.21.15.70:0/4205137429 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f0b7c013970 con 0x7f0b8c101d70 2024-03-23T08:28:37.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.454+0000 7f0b88ff9700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0b7406cd40 0x7f0b7406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:37.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.454+0000 7f0b88ff9700 1 -- 172.21.15.70:0/4205137429 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0b7c08d4f0 con 0x7f0b8c101d70 2024-03-23T08:28:37.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.455+0000 7f0b8b7fe700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0b7406cd40 0x7f0b7406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:37.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.455+0000 7f0b8b7fe700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0b7406cd40 0x7f0b7406f1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b80007fd0 tx=0x7f0b80009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:37.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.459+0000 7f0b88ff9700 1 -- 172.21.15.70:0/4205137429 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0b7c05ada0 con 0x7f0b8c101d70 2024-03-23T08:28:37.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.695+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b8c108da0 con 0x7f0b7406cd40 2024-03-23T08:28:37.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.697+0000 7f0b88ff9700 1 -- 172.21.15.70:0/4205137429 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f0b8c108da0 con 0x7f0b7406cd40 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.699+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0b7406cd40 msgr2=0x7f0b7406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.699+0000 7f0b929c0700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0b7406cd40 0x7f0b7406f1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b80007fd0 tx=0x7f0b80009040 comp rx=0 tx=0).stop 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.699+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 msgr2=0x7f0b8c113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.699+0000 7f0b929c0700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 0x7f0b8c113ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00b3b0 tx=0x7f0b7c009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.699+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 shutdown_connections 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.699+0000 7f0b929c0700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0b7406cd40 0x7f0b7406f1f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.699+0000 7f0b929c0700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b8c101d70 0x7f0b8c113ac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.699+0000 7f0b929c0700 1 --2- 172.21.15.70:0/4205137429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c102710 0x7f0b8c114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.699+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 >> 172.21.15.70:0/4205137429 conn(0x7f0b8c072bc0 msgr2=0x7f0b8c0659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.700+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 shutdown_connections 2024-03-23T08:28:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.700+0000 7f0b929c0700 1 -- 172.21.15.70:0/4205137429 wait complete. 2024-03-23T08:28:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 -- 172.21.15.70:0/2408663017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c102710 msgr2=0x7f1b1c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 --2- 172.21.15.70:0/2408663017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c102710 0x7f1b1c0fbd80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b08004940 tx=0x7f1b08009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 -- 172.21.15.70:0/2408663017 shutdown_connections 2024-03-23T08:28:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 --2- 172.21.15.70:0/2408663017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c102710 0x7f1b1c0fbd80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 --2- 172.21.15.70:0/2408663017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c101d70 0x7f1b1c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 -- 172.21.15.70:0/2408663017 >> 172.21.15.70:0/2408663017 conn(0x7f1b1c072bc0 msgr2=0x7f1b1c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 -- 172.21.15.70:0/2408663017 shutdown_connections 2024-03-23T08:28:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 -- 172.21.15.70:0/2408663017 wait complete. 2024-03-23T08:28:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 Processor -- start 2024-03-23T08:28:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.810+0000 7f1b22b73700 1 -- start start 2024-03-23T08:28:37.812 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b22b73700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c101d70 0x7f1b1c10ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b22b73700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c102710 0x7f1b1c10efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b22b73700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c10f5f0 con 0x7f1b1c101d70 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b22b73700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c10f730 con 0x7f1b1c102710 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b2090f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c101d70 0x7f1b1c10ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b2090f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c101d70 0x7f1b1c10ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41216/0 (socket says 172.21.15.70:41216) 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b2090f700 1 -- 172.21.15.70:0/1491599471 learned_addr learned my addr 172.21.15.70:0/1491599471 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b1bfff700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c102710 0x7f1b1c10efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b2090f700 1 -- 172.21.15.70:0/1491599471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c102710 msgr2=0x7f1b1c10efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b2090f700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c102710 0x7f1b1c10efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b2090f700 1 -- 172.21.15.70:0/1491599471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b08005040 con 0x7f1b1c101d70 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b1bfff700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c102710 0x7f1b1c10efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b2090f700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c101d70 0x7f1b1c10ea60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b10002a50 tx=0x7f1b100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b197fa700 1 -- 172.21.15.70:0/1491599471 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b10007cc0 con 0x7f1b1c101d70 2024-03-23T08:28:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b197fa700 1 -- 172.21.15.70:0/1491599471 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b1000f040 con 0x7f1b1c101d70 2024-03-23T08:28:37.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b197fa700 1 -- 172.21.15.70:0/1491599471 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b10009540 con 0x7f1b1c101d70 2024-03-23T08:28:37.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c117ad0 con 0x7f1b1c101d70 2024-03-23T08:28:37.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.811+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c118030 con 0x7f1b1c101d70 2024-03-23T08:28:37.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.812+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c0ff080 con 0x7f1b1c101d70 2024-03-23T08:28:37.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.813+0000 7f1b197fa700 1 -- 172.21.15.70:0/1491599471 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f1b100096a0 con 0x7f1b1c101d70 2024-03-23T08:28:37.814 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.813+0000 7f1b197fa700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1b0c06cd40 0x7f1b0c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:37.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.813+0000 7f1b197fa700 1 -- 172.21.15.70:0/1491599471 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1b1008c010 con 0x7f1b1c101d70 2024-03-23T08:28:37.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.813+0000 7f1b1bfff700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1b0c06cd40 0x7f1b0c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:37.815 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.814+0000 7f1b1bfff700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1b0c06cd40 0x7f1b0c06f1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b08000c00 tx=0x7f1b08009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:37.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:37.818+0000 7f1b197fa700 1 -- 172.21.15.70:0/1491599471 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1b10055e50 con 0x7f1b1c101d70 2024-03-23T08:28:38.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.029+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b1c108db0 con 0x7f1b0c06cd40 2024-03-23T08:28:38.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.039+0000 7f1b197fa700 1 -- 172.21.15.70:0/1491599471 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1b0800b5c0 con 0x7f1b0c06cd40 2024-03-23T08:28:38.039 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (2m) 45s ago 13m 16.9M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (13m) 45s ago 13m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 0294e6e3fa0d 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (12m) 31s ago 12m 6656k - 17.2.7-726-g9c4e0c34 837124da804f 5986c5f2d0fe 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (112s) 45s ago 13m 80.0M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (7m) 45s ago 7m 148M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (7m) 45s ago 7m 13.6M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (7m) 31s ago 7m 97.4M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (7m) 31s ago 7m 15.1M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (4m) 45s ago 14m 470M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (3m) 31s ago 12m 446M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (61s) 45s ago 15m 30.2M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (35s) 31s ago 12m 25.1M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (2m) 45s ago 13m 18.2M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:28:38.040 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (2m) 31s ago 12m 16.1M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:28:38.041 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (11m) 45s ago 11m 218M 1568M 17.2.7-726-g9c4e0c34 837124da804f 5d4473f7201b 2024-03-23T08:28:38.041 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (11m) 45s ago 11m 234M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:28:38.041 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (10m) 45s ago 10m 202M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:28:38.041 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (10m) 31s ago 10m 278M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:28:38.041 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (9m) 31s ago 9m 242M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:28:38.041 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (9m) 31s ago 9m 225M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:28:38.041 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (2m) 45s ago 12m 50.2M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.041+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1b0c06cd40 msgr2=0x7f1b0c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.041+0000 7f1b22b73700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1b0c06cd40 0x7f1b0c06f1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b08000c00 tx=0x7f1b08009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.041+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c101d70 msgr2=0x7f1b1c10ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.041+0000 7f1b22b73700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c101d70 0x7f1b1c10ea60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b10002a50 tx=0x7f1b100046b0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.041+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 shutdown_connections 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.041+0000 7f1b22b73700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1b0c06cd40 0x7f1b0c06f1f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.041+0000 7f1b22b73700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c101d70 0x7f1b1c10ea60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.041+0000 7f1b22b73700 1 --2- 172.21.15.70:0/1491599471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b1c102710 0x7f1b1c10efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.041+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 >> 172.21.15.70:0/1491599471 conn(0x7f1b1c072bc0 msgr2=0x7f1b1c0fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.042+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 shutdown_connections 2024-03-23T08:28:38.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.042+0000 7f1b22b73700 1 -- 172.21.15.70:0/1491599471 wait complete. 2024-03-23T08:28:38.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 -- 172.21.15.70:0/580397374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60102710 msgr2=0x7fce600fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 --2- 172.21.15.70:0/580397374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60102710 0x7fce600fbd80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fce4c004940 tx=0x7fce4c009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:38.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 -- 172.21.15.70:0/580397374 shutdown_connections 2024-03-23T08:28:38.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 --2- 172.21.15.70:0/580397374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60102710 0x7fce600fbd80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 --2- 172.21.15.70:0/580397374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce60101d70 0x7fce60102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 -- 172.21.15.70:0/580397374 >> 172.21.15.70:0/580397374 conn(0x7fce60072bc0 msgr2=0x7fce60072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:38.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 -- 172.21.15.70:0/580397374 shutdown_connections 2024-03-23T08:28:38.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 -- 172.21.15.70:0/580397374 wait complete. 2024-03-23T08:28:38.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 Processor -- start 2024-03-23T08:28:38.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.142+0000 7fce67022700 1 -- start start 2024-03-23T08:28:38.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce67022700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60101d70 0x7fce6010ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:38.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce67022700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce60102710 0x7fce6010efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:38.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce67022700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6010f5f0 con 0x7fce60101d70 2024-03-23T08:28:38.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce67022700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6010f730 con 0x7fce60102710 2024-03-23T08:28:38.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce64dbe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60101d70 0x7fce6010ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:38.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce64dbe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60101d70 0x7fce6010ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41230/0 (socket says 172.21.15.70:41230) 2024-03-23T08:28:38.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce64dbe700 1 -- 172.21.15.70:0/2178601408 learned_addr learned my addr 172.21.15.70:0/2178601408 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:38.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce5ffff700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce60102710 0x7fce6010efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:38.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce64dbe700 1 -- 172.21.15.70:0/2178601408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce60102710 msgr2=0x7fce6010efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce64dbe700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce60102710 0x7fce6010efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce64dbe700 1 -- 172.21.15.70:0/2178601408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c005040 con 0x7fce60101d70 2024-03-23T08:28:38.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce64dbe700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60101d70 0x7fce6010ea60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c8c0 tx=0x7fce540044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:38.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce5d7fa700 1 -- 172.21.15.70:0/2178601408 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce5400a980 con 0x7fce60101d70 2024-03-23T08:28:38.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce5d7fa700 1 -- 172.21.15.70:0/2178601408 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fce5400ce60 con 0x7fce60101d70 2024-03-23T08:28:38.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce5d7fa700 1 -- 172.21.15.70:0/2178601408 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce540113a0 con 0x7fce60101d70 2024-03-23T08:28:38.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.143+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce60117ad0 con 0x7fce60101d70 2024-03-23T08:28:38.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.144+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce60118030 con 0x7fce60101d70 2024-03-23T08:28:38.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.145+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce600ff080 con 0x7fce60101d70 2024-03-23T08:28:38.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.145+0000 7fce5d7fa700 1 -- 172.21.15.70:0/2178601408 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fce54012400 con 0x7fce60101d70 2024-03-23T08:28:38.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.145+0000 7fce5d7fa700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce5006cd40 0x7fce5006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:38.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.145+0000 7fce5d7fa700 1 -- 172.21.15.70:0/2178601408 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fce5408c0f0 con 0x7fce60101d70 2024-03-23T08:28:38.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.145+0000 7fce5ffff700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce5006cd40 0x7fce5006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:38.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.146+0000 7fce5ffff700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce5006cd40 0x7fce5006f1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00b3b0 tx=0x7fce4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:38.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.150+0000 7fce5d7fa700 1 -- 172.21.15.70:0/2178601408 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fce54056160 con 0x7fce60101d70 2024-03-23T08:28:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:38 smithi070 ceph-mon[145869]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:38 smithi070 ceph-mon[145869]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:38 smithi193 ceph-mon[108533]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:38 smithi193 ceph-mon[108533]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.442+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fce6004bed0 con 0x7fce60101d70 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.442+0000 7fce5d7fa700 1 -- 172.21.15.70:0/2178601408 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fce54059780 con 0x7fce60101d70 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:28:38.443 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 4 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:28:38.444 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:28:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.444+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce5006cd40 msgr2=0x7fce5006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.444+0000 7fce67022700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce5006cd40 0x7fce5006f1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00b3b0 tx=0x7fce4c009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.444+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60101d70 msgr2=0x7fce6010ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.444+0000 7fce67022700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60101d70 0x7fce6010ea60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c8c0 tx=0x7fce540044c0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.445+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 shutdown_connections 2024-03-23T08:28:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.445+0000 7fce67022700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce5006cd40 0x7fce5006f1f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.445+0000 7fce67022700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce60101d70 0x7fce6010ea60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.445+0000 7fce67022700 1 --2- 172.21.15.70:0/2178601408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce60102710 0x7fce6010efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.445+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 >> 172.21.15.70:0/2178601408 conn(0x7fce60072bc0 msgr2=0x7fce600fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.445+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 shutdown_connections 2024-03-23T08:28:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.445+0000 7fce67022700 1 -- 172.21.15.70:0/2178601408 wait complete. 2024-03-23T08:28:38.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.545+0000 7fc12e26b700 1 -- 172.21.15.70:0/818079240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128101d70 msgr2=0x7fc128102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.545+0000 7fc12e26b700 1 --2- 172.21.15.70:0/818079240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128101d70 0x7fc128102140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc110004940 tx=0x7fc110009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:38.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.545+0000 7fc12e26b700 1 -- 172.21.15.70:0/818079240 shutdown_connections 2024-03-23T08:28:38.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.545+0000 7fc12e26b700 1 --2- 172.21.15.70:0/818079240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128102710 0x7fc1280fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.545+0000 7fc12e26b700 1 --2- 172.21.15.70:0/818079240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128101d70 0x7fc128102140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.545+0000 7fc12e26b700 1 -- 172.21.15.70:0/818079240 >> 172.21.15.70:0/818079240 conn(0x7fc128072bc0 msgr2=0x7fc128072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:38.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.545+0000 7fc12e26b700 1 -- 172.21.15.70:0/818079240 shutdown_connections 2024-03-23T08:28:38.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.545+0000 7fc12e26b700 1 -- 172.21.15.70:0/818079240 wait complete. 2024-03-23T08:28:38.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc12e26b700 1 Processor -- start 2024-03-23T08:28:38.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc12e26b700 1 -- start start 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc12e26b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128101d70 0x7fc1281129a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc12e26b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128102710 0x7fc128112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc12e26b700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc128113530 con 0x7fc128102710 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc12e26b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12810c920 con 0x7fc128101d70 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc1277fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128102710 0x7fc128112ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc1277fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128102710 0x7fc128112ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54586/0 (socket says 172.21.15.70:54586) 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc1277fe700 1 -- 172.21.15.70:0/402945202 learned_addr learned my addr 172.21.15.70:0/402945202 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc127fff700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128101d70 0x7fc1281129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc1277fe700 1 -- 172.21.15.70:0/402945202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128101d70 msgr2=0x7fc1281129a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.546+0000 7fc1277fe700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128101d70 0x7fc1281129a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.547+0000 7fc1277fe700 1 -- 172.21.15.70:0/402945202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110005040 con 0x7fc128102710 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.547+0000 7fc1277fe700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128102710 0x7fc128112ee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc118002a50 tx=0x7fc118004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.547+0000 7fc124ff9700 1 -- 172.21.15.70:0/402945202 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc118007cc0 con 0x7fc128102710 2024-03-23T08:28:38.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.547+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12810cc00 con 0x7fc128102710 2024-03-23T08:28:38.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.547+0000 7fc124ff9700 1 -- 172.21.15.70:0/402945202 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc11800f040 con 0x7fc128102710 2024-03-23T08:28:38.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.547+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12810d1c0 con 0x7fc128102710 2024-03-23T08:28:38.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.547+0000 7fc124ff9700 1 -- 172.21.15.70:0/402945202 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11800d920 con 0x7fc128102710 2024-03-23T08:28:38.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.548+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1080052b0 con 0x7fc128102710 2024-03-23T08:28:38.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.549+0000 7fc124ff9700 1 -- 172.21.15.70:0/402945202 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fc11800dbc0 con 0x7fc128102710 2024-03-23T08:28:38.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.549+0000 7fc124ff9700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc11406cd40 0x7fc11406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:38.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.549+0000 7fc124ff9700 1 -- 172.21.15.70:0/402945202 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc11808bf30 con 0x7fc128102710 2024-03-23T08:28:38.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.549+0000 7fc127fff700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc11406cd40 0x7fc11406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:38.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.550+0000 7fc127fff700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc11406cd40 0x7fc11406f1f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc110004f90 tx=0x7fc110009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:38.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.554+0000 7fc124ff9700 1 -- 172.21.15.70:0/402945202 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc1180bf9f0 con 0x7fc128102710 2024-03-23T08:28:38.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.836+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc108005830 con 0x7fc128102710 2024-03-23T08:28:38.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.836+0000 7fc124ff9700 1 -- 172.21.15.70:0/402945202 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fc118059800 con 0x7fc128102710 2024-03-23T08:28:38.837 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:28:38.837 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:28:38.837 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:28:38.838 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:28:38.839 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:28:38.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc11406cd40 msgr2=0x7fc11406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc11406cd40 0x7fc11406f1f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc110004f90 tx=0x7fc110009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:38.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128102710 msgr2=0x7fc128112ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128102710 0x7fc128112ee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc118002a50 tx=0x7fc118004600 comp rx=0 tx=0).stop 2024-03-23T08:28:38.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 shutdown_connections 2024-03-23T08:28:38.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc11406cd40 0x7fc11406f1f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc128101d70 0x7fc1281129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 --2- 172.21.15.70:0/402945202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc128102710 0x7fc128112ee0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 >> 172.21.15.70:0/402945202 conn(0x7fc128072bc0 msgr2=0x7fc1280fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:38.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 shutdown_connections 2024-03-23T08:28:38.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.839+0000 7fc12e26b700 1 -- 172.21.15.70:0/402945202 wait complete. 2024-03-23T08:28:38.842 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:28:38.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.951+0000 7ff30241a700 1 -- 172.21.15.70:0/785301426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 msgr2=0x7ff2fc102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.951+0000 7ff30241a700 1 --2- 172.21.15.70:0/785301426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 0x7ff2fc102140 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec004940 tx=0x7ff2ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:38.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.951+0000 7ff30241a700 1 -- 172.21.15.70:0/785301426 shutdown_connections 2024-03-23T08:28:38.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.951+0000 7ff30241a700 1 --2- 172.21.15.70:0/785301426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2fc0fc5b0 0x7ff2fc0fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.951+0000 7ff30241a700 1 --2- 172.21.15.70:0/785301426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 0x7ff2fc102140 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.951+0000 7ff30241a700 1 -- 172.21.15.70:0/785301426 >> 172.21.15.70:0/785301426 conn(0x7ff2fc0714c0 msgr2=0x7ff2fc0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.951+0000 7ff30241a700 1 -- 172.21.15.70:0/785301426 shutdown_connections 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.951+0000 7ff30241a700 1 -- 172.21.15.70:0/785301426 wait complete. 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff30241a700 1 Processor -- start 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff30241a700 1 -- start start 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff30241a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2fc0fc5b0 0x7ff2fc10ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff30241a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 0x7ff2fc10f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff30241a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc10f6a0 con 0x7ff2fc101d70 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff30241a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc10f7e0 con 0x7ff2fc0fc5b0 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 0x7ff2fc10f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 0x7ff2fc10f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54604/0 (socket says 172.21.15.70:54604) 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff2fb7fe700 1 -- 172.21.15.70:0/933026955 learned_addr learned my addr 172.21.15.70:0/933026955 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.952+0000 7ff2fbfff700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2fc0fc5b0 0x7ff2fc10ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.953+0000 7ff2fb7fe700 1 -- 172.21.15.70:0/933026955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2fc0fc5b0 msgr2=0x7ff2fc10ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.953+0000 7ff2fb7fe700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2fc0fc5b0 0x7ff2fc10ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:38.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.953+0000 7ff2fb7fe700 1 -- 172.21.15.70:0/933026955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec005040 con 0x7ff2fc101d70 2024-03-23T08:28:38.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.953+0000 7ff2fb7fe700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 0x7ff2fc10f160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c8c0 tx=0x7ff2f00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:38.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.953+0000 7ff2f8ff9700 1 -- 172.21.15.70:0/933026955 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f000a760 con 0x7ff2fc101d70 2024-03-23T08:28:38.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.953+0000 7ff2f8ff9700 1 -- 172.21.15.70:0/933026955 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2f000ad70 con 0x7ff2fc101d70 2024-03-23T08:28:38.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.953+0000 7ff2f8ff9700 1 -- 172.21.15.70:0/933026955 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f00113a0 con 0x7ff2fc101d70 2024-03-23T08:28:38.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.953+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc117ad0 con 0x7ff2fc101d70 2024-03-23T08:28:38.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.953+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc118010 con 0x7ff2fc101d70 2024-03-23T08:28:38.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.954+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc0fd5c0 con 0x7ff2fc101d70 2024-03-23T08:28:38.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.955+0000 7ff2f8ff9700 1 -- 172.21.15.70:0/933026955 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7ff2f000a8c0 con 0x7ff2fc101d70 2024-03-23T08:28:38.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.955+0000 7ff2f8ff9700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff2e406cd40 0x7ff2e406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:38.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.956+0000 7ff2f8ff9700 1 -- 172.21.15.70:0/933026955 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff2f008c1c0 con 0x7ff2fc101d70 2024-03-23T08:28:38.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.956+0000 7ff2fbfff700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff2e406cd40 0x7ff2e406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:38.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.956+0000 7ff2fbfff700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff2e406cd40 0x7ff2e406f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00b3b0 tx=0x7ff2ec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:38.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:38.961+0000 7ff2f8ff9700 1 -- 172.21.15.70:0/933026955 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff2f0056190 con 0x7ff2fc101d70 2024-03-23T08:28:39.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.184+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2fc108cb0 con 0x7ff2e406cd40 2024-03-23T08:28:39.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.185+0000 7ff2f8ff9700 1 -- 172.21.15.70:0/933026955 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7ff2fc108cb0 con 0x7ff2e406cd40 2024-03-23T08:28:39.185 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:28:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:28:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:28:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:28:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:28:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:28:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr" 2024-03-23T08:28:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:28:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "4/21 daemons upgraded", 2024-03-23T08:28:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading crash daemons", 2024-03-23T08:28:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:28:39.186 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:28:39.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff2e406cd40 msgr2=0x7ff2e406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff2e406cd40 0x7ff2e406f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00b3b0 tx=0x7ff2ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 msgr2=0x7ff2fc10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 0x7ff2fc10f160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c8c0 tx=0x7ff2f00044c0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 shutdown_connections 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff2e406cd40 0x7ff2e406f1f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2fc0fc5b0 0x7ff2fc10ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 --2- 172.21.15.70:0/933026955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2fc101d70 0x7ff2fc10f160 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 >> 172.21.15.70:0/933026955 conn(0x7ff2fc0714c0 msgr2=0x7ff2fc104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 shutdown_connections 2024-03-23T08:28:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.187+0000 7ff30241a700 1 -- 172.21.15.70:0/933026955 wait complete. 2024-03-23T08:28:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.291+0000 7ff238c83700 1 -- 172.21.15.70:0/423582904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 msgr2=0x7ff234106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.291+0000 7ff238c83700 1 --2- 172.21.15.70:0/423582904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 0x7ff234106f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff224004940 tx=0x7ff224009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.291+0000 7ff238c83700 1 -- 172.21.15.70:0/423582904 shutdown_connections 2024-03-23T08:28:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.291+0000 7ff238c83700 1 --2- 172.21.15.70:0/423582904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 0x7ff234106f00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.291+0000 7ff238c83700 1 --2- 172.21.15.70:0/423582904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2340fde90 0x7ff2340fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.291+0000 7ff238c83700 1 -- 172.21.15.70:0/423582904 >> 172.21.15.70:0/423582904 conn(0x7ff234072bc0 msgr2=0x7ff234072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff238c83700 1 -- 172.21.15.70:0/423582904 shutdown_connections 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff238c83700 1 -- 172.21.15.70:0/423582904 wait complete. 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff238c83700 1 Processor -- start 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff238c83700 1 -- start start 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff238c83700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2340fde90 0x7ff234113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff238c83700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 0x7ff234114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff238c83700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234114670 con 0x7ff2340fe830 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff238c83700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23410da60 con 0x7ff2340fde90 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff231d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 0x7ff234114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff231d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 0x7ff234114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54622/0 (socket says 172.21.15.70:54622) 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff231d9b700 1 -- 172.21.15.70:0/2246991281 learned_addr learned my addr 172.21.15.70:0/2246991281 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.292+0000 7ff23259c700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2340fde90 0x7ff234113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.293+0000 7ff231d9b700 1 -- 172.21.15.70:0/2246991281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2340fde90 msgr2=0x7ff234113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.293+0000 7ff231d9b700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2340fde90 0x7ff234113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.293+0000 7ff231d9b700 1 -- 172.21.15.70:0/2246991281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224005040 con 0x7ff2340fe830 2024-03-23T08:28:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.293+0000 7ff231d9b700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 0x7ff234114020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff224009390 tx=0x7ff224009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.293+0000 7ff222ffd700 1 -- 172.21.15.70:0/2246991281 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff22400e040 con 0x7ff2340fe830 2024-03-23T08:28:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.293+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23410dce0 con 0x7ff2340fe830 2024-03-23T08:28:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.293+0000 7ff222ffd700 1 -- 172.21.15.70:0/2246991281 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff224004290 con 0x7ff2340fe830 2024-03-23T08:28:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.293+0000 7ff222ffd700 1 -- 172.21.15.70:0/2246991281 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff22401c3f0 con 0x7ff2340fe830 2024-03-23T08:28:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.293+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23410e220 con 0x7ff2340fe830 2024-03-23T08:28:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.294+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff234065e30 con 0x7ff2340fe830 2024-03-23T08:28:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.294+0000 7ff222ffd700 1 -- 172.21.15.70:0/2246991281 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7ff2240136b0 con 0x7ff2340fe830 2024-03-23T08:28:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.295+0000 7ff222ffd700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff21c06cd40 0x7ff21c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:28:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.295+0000 7ff222ffd700 1 -- 172.21.15.70:0/2246991281 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff22408d3c0 con 0x7ff2340fe830 2024-03-23T08:28:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.295+0000 7ff23259c700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff21c06cd40 0x7ff21c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:28:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.295+0000 7ff23259c700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff21c06cd40 0x7ff21c06f1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff228004f40 tx=0x7ff22800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:28:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.299+0000 7ff222ffd700 1 -- 172.21.15.70:0/2246991281 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff22405ad10 con 0x7ff2340fe830 2024-03-23T08:28:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:39 smithi193 ceph-mon[108533]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:39 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2178601408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:39 smithi193 ceph-mon[108533]: pgmap v29: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:39 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/402945202' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:28:39.435 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:39 smithi070 ceph-mon[145869]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:39.435 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:39 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2178601408' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:39.435 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:39 smithi070 ceph-mon[145869]: pgmap v29: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:39.435 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:39 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/402945202' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:28:39.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.585+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff23405f5f0 con 0x7ff2340fe830 2024-03-23T08:28:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.585+0000 7ff222ffd700 1 -- 172.21.15.70:0/2246991281 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff22405a8a0 con 0x7ff2340fe830 2024-03-23T08:28:39.586 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-03-23T08:28:39.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff21c06cd40 msgr2=0x7ff21c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:39.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff21c06cd40 0x7ff21c06f1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff228004f40 tx=0x7ff22800b040 comp rx=0 tx=0).stop 2024-03-23T08:28:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 msgr2=0x7ff234114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:28:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 0x7ff234114020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff224009390 tx=0x7ff224009e30 comp rx=0 tx=0).stop 2024-03-23T08:28:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 shutdown_connections 2024-03-23T08:28:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff21c06cd40 0x7ff21c06f1f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2340fde90 0x7ff234113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 --2- 172.21.15.70:0/2246991281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2340fe830 0x7ff234114020 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:28:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 >> 172.21.15.70:0/2246991281 conn(0x7ff234072bc0 msgr2=0x7ff234103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:28:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 shutdown_connections 2024-03-23T08:28:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:28:39.587+0000 7ff238c83700 1 -- 172.21.15.70:0/2246991281 wait complete. 2024-03-23T08:28:40.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:40 smithi193 ceph-mon[108533]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:40.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:40 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:40.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:40 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:40.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:40 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2246991281' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:28:40.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:40 smithi070 ceph-mon[145869]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:28:40.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:40 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:40.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:40 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:40.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:40 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2246991281' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:28:41.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:41 smithi193 ceph-mon[108533]: pgmap v30: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:28:41.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:41.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:28:41.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:41 smithi070 ceph-mon[145869]: pgmap v30: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:28:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-23T08:28:41.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:43.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:42 smithi193 ceph-mon[108533]: Upgrade: Updating crash.smithi193 (2/2) 2024-03-23T08:28:43.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:42 smithi193 ceph-mon[108533]: Deploying daemon crash.smithi193 on smithi193 2024-03-23T08:28:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:42 smithi070 ceph-mon[145869]: Upgrade: Updating crash.smithi193 (2/2) 2024-03-23T08:28:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:42 smithi070 ceph-mon[145869]: Deploying daemon crash.smithi193 on smithi193 2024-03-23T08:28:43.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:43 smithi193 ceph-mon[108533]: pgmap v31: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:43 smithi070 ceph-mon[145869]: pgmap v31: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:45.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:45.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:45.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:28:45.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:45 smithi070 ceph-mon[145869]: pgmap v32: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:45.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:45.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:45.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:28:45.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:45 smithi193 ceph-mon[108533]: pgmap v32: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:46.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:28:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:28:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:47 smithi070 ceph-mon[145869]: pgmap v33: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:48.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:47 smithi193 ceph-mon[108533]: pgmap v33: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:48.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:47 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:48.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:47 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:48 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:48 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:48 smithi070 ceph-mon[145869]: pgmap v34: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:49.256 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:48 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:49.256 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:48 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:49.256 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:48 smithi193 ceph-mon[108533]: pgmap v34: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:50.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:50.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:50.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:50.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:50 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:50.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:50 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:50.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:50 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:50.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:50 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:51.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:51 smithi070 ceph-mon[145869]: pgmap v35: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:51.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:51 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:51.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:51 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:51 smithi193 ceph-mon[108533]: pgmap v35: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:51 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:51 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:53.981 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:53 smithi070 ceph-mon[145869]: pgmap v36: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:53 smithi193 ceph-mon[108533]: pgmap v36: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:28:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all crash 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]: dispatch 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]': finished 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]: dispatch 2024-03-23T08:28:55.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]': finished 2024-03-23T08:28:55.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-23T08:28:55.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-23T08:28:55.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: Upgrade: osd.0 is safe to restart 2024-03-23T08:28:55.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:55 smithi070 ceph-mon[145869]: pgmap v37: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:55.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:55.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:55.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all crash 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]: dispatch 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]': finished 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]: dispatch 2024-03-23T08:28:55.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]': finished 2024-03-23T08:28:55.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-23T08:28:55.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-23T08:28:55.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: Upgrade: osd.0 is safe to restart 2024-03-23T08:28:55.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:55 smithi193 ceph-mon[108533]: pgmap v37: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:56.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:56 smithi070 ceph-mon[145869]: Upgrade: Updating osd.0 2024-03-23T08:28:56.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:56 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:56.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:56 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T08:28:56.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:56 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:56.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:56 smithi070 ceph-mon[145869]: Deploying daemon osd.0 on smithi070 2024-03-23T08:28:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:56 smithi193 ceph-mon[108533]: Upgrade: Updating osd.0 2024-03-23T08:28:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:56 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:28:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:56 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-23T08:28:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:56 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:28:57.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:56 smithi193 ceph-mon[108533]: Deploying daemon osd.0 on smithi070 2024-03-23T08:28:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:57 smithi070 ceph-mon[145869]: pgmap v38: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:57 smithi193 ceph-mon[108533]: pgmap v38: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:28:59.233 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:28:58 smithi070 systemd[1]: Stopping Ceph osd.0 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:28:59.233 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:28:59 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[63280]: 2024-03-23T08:28:59.186+0000 7f950a1a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:28:59.233 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:28:59 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[63280]: 2024-03-23T08:28:59.186+0000 7f950a1a3700 -1 osd.0 51 *** Got signal Terminated *** 2024-03-23T08:28:59.233 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:28:59 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[63280]: 2024-03-23T08:28:59.186+0000 7f950a1a3700 -1 osd.0 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:29:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:59 smithi193 ceph-mon[108533]: pgmap v39: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:29:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:28:59 smithi193 ceph-mon[108533]: osd.0 marked itself down and dead 2024-03-23T08:29:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:59 smithi070 ceph-mon[145869]: pgmap v39: 65 pgs: 65 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:29:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:28:59 smithi070 ceph-mon[145869]: osd.0 marked itself down and dead 2024-03-23T08:29:00.765 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:00 smithi070 bash[159153]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0 2024-03-23T08:29:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:00 smithi193 ceph-mon[108533]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:29:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:00 smithi193 ceph-mon[108533]: osdmap e52: 6 total, 5 up, 6 in 2024-03-23T08:29:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:00 smithi193 ceph-mon[108533]: osdmap e53: 6 total, 5 up, 6 in 2024-03-23T08:29:01.218 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:00 smithi070 ceph-mon[145869]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:29:01.218 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:00 smithi070 ceph-mon[145869]: osdmap e52: 6 total, 5 up, 6 in 2024-03-23T08:29:01.218 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:00 smithi070 ceph-mon[145869]: osdmap e53: 6 total, 5 up, 6 in 2024-03-23T08:29:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:01 smithi070 ceph-mon[145869]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-23T08:29:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:29:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:01 smithi193 ceph-mon[108533]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-23T08:29:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:29:02.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:02 smithi070 systemd[1]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.0.service: Succeeded. 2024-03-23T08:29:02.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:02 smithi070 systemd[1]: Stopped Ceph osd.0 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:29:02.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:02 smithi070 systemd[1]: Starting Ceph osd.0 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:29:03.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:02 smithi193 ceph-mon[108533]: Health check failed: Degraded data redundancy: 39/222 objects degraded (17.568%), 18 pgs degraded (PG_DEGRADED) 2024-03-23T08:29:03.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:02 smithi070 ceph-mon[145869]: Health check failed: Degraded data redundancy: 39/222 objects degraded (17.568%), 18 pgs degraded (PG_DEGRADED) 2024-03-23T08:29:03.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:03 smithi070 bash[159426]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-23T08:29:03.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:03 smithi070 bash[159426]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-23T08:29:03.734 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:03 smithi070 bash[159426]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-23T08:29:03.734 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:03 smithi070 bash[159426]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-23T08:29:03.734 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:03 smithi070 bash[159426]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-23T08:29:03.734 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:03 smithi070 bash[159426]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-23T08:29:03.734 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:03 smithi070 bash[159426]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-23T08:29:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:03 smithi193 ceph-mon[108533]: pgmap v43: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:04.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:03 smithi070 ceph-mon[145869]: pgmap v43: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:04.983 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:04 smithi070 ceph-osd[159723]: -- 172.21.15.70:0/2041284600 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559f25d311e0 con 0x559f25d00c00 2024-03-23T08:29:04.983 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:04 smithi070 bash[159685]: 9396a3debb4044c6f992f400c6f5476067b53beed264517180f6781f07b85002 2024-03-23T08:29:04.983 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:04 smithi070 systemd[1]: Started Ceph osd.0 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:29:05.483 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:05 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[159700]: 2024-03-23T08:29:05.231+0000 7fc1c6df1640 -1 Falling back to public interface 2024-03-23T08:29:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:05 smithi193 ceph-mon[108533]: pgmap v44: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:06.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:29:06.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:05 smithi070 ceph-mon[145869]: pgmap v44: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:06.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:06.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:06.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:29:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:07 smithi193 ceph-mon[108533]: pgmap v45: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:07 smithi070 ceph-mon[145869]: pgmap v45: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 -- 172.21.15.70:0/2585345985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fde80 msgr2=0x7f264c0fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 --2- 172.21.15.70:0/2585345985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fde80 0x7f264c0fe250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f263c004940 tx=0x7f263c009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 -- 172.21.15.70:0/2585345985 shutdown_connections 2024-03-23T08:29:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 --2- 172.21.15.70:0/2585345985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f264c0fe820 0x7f264c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 --2- 172.21.15.70:0/2585345985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fde80 0x7f264c0fe250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 -- 172.21.15.70:0/2585345985 >> 172.21.15.70:0/2585345985 conn(0x7f264c0f78d0 msgr2=0x7f264c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 -- 172.21.15.70:0/2585345985 shutdown_connections 2024-03-23T08:29:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 -- 172.21.15.70:0/2585345985 wait complete. 2024-03-23T08:29:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 Processor -- start 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 -- start start 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.695+0000 7f26508e8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f264c0fde80 0x7f264c0fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f26508e8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fe820 0x7f264c0fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f26508e8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c0fc780 con 0x7f264c0fe820 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f26508e8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c0fc8c0 con 0x7f264c0fde80 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f2649d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fe820 0x7f264c0fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f2649d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fe820 0x7f264c0fc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48490/0 (socket says 172.21.15.70:48490) 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f2649d9b700 1 -- 172.21.15.70:0/3013104308 learned_addr learned my addr 172.21.15.70:0/3013104308 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f264a59c700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f264c0fde80 0x7f264c0fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f2649d9b700 1 -- 172.21.15.70:0/3013104308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f264c0fde80 msgr2=0x7f264c0fbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f2649d9b700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f264c0fde80 0x7f264c0fbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f2649d9b700 1 -- 172.21.15.70:0/3013104308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c005040 con 0x7f264c0fe820 2024-03-23T08:29:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f2649d9b700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fe820 0x7f264c0fc1f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2640002a50 tx=0x7f26400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f263affd700 1 -- 172.21.15.70:0/3013104308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2640007cc0 con 0x7f264c0fe820 2024-03-23T08:29:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c119d30 con 0x7f264c0fe820 2024-03-23T08:29:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f263affd700 1 -- 172.21.15.70:0/3013104308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f264000f040 con 0x7f264c0fe820 2024-03-23T08:29:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.696+0000 7f263affd700 1 -- 172.21.15.70:0/3013104308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f264000d9c0 con 0x7f264c0fe820 2024-03-23T08:29:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.697+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c11a130 con 0x7f264c0fe820 2024-03-23T08:29:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.697+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c066710 con 0x7f264c0fe820 2024-03-23T08:29:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.698+0000 7f263affd700 1 -- 172.21.15.70:0/3013104308 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f264000db20 con 0x7f264c0fe820 2024-03-23T08:29:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.699+0000 7f263affd700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f263406cd40 0x7f263406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.699+0000 7f263affd700 1 -- 172.21.15.70:0/3013104308 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f264008c5d0 con 0x7f264c0fe820 2024-03-23T08:29:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.699+0000 7f264a59c700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f263406cd40 0x7f263406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.700+0000 7f264a59c700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f263406cd40 0x7f263406f1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f263c004f90 tx=0x7f263c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:09.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.703+0000 7f263affd700 1 -- 172.21.15.70:0/3013104308 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2640056430 con 0x7f264c0fe820 2024-03-23T08:29:09.733 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:09 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[159700]: 2024-03-23T08:29:09.455+0000 7fc1c6df1640 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-23T08:29:09.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.932+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f264c04bed0 con 0x7f263406cd40 2024-03-23T08:29:09.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.933+0000 7f263affd700 1 -- 172.21.15.70:0/3013104308 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f264c04bed0 con 0x7f263406cd40 2024-03-23T08:29:09.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f263406cd40 msgr2=0x7f263406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:09.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f263406cd40 0x7f263406f1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f263c004f90 tx=0x7f263c009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:09.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fe820 msgr2=0x7f264c0fc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:09.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fe820 0x7f264c0fc1f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2640002a50 tx=0x7f26400046b0 comp rx=0 tx=0).stop 2024-03-23T08:29:09.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 shutdown_connections 2024-03-23T08:29:09.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f263406cd40 0x7f263406f1f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:09.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f264c0fde80 0x7f264c0fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:09.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 --2- 172.21.15.70:0/3013104308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264c0fe820 0x7f264c0fc1f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:09.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 >> 172.21.15.70:0/3013104308 conn(0x7f264c0f78d0 msgr2=0x7f264c0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:09.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 shutdown_connections 2024-03-23T08:29:09.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:09.936+0000 7f26508e8700 1 -- 172.21.15.70:0/3013104308 wait complete. 2024-03-23T08:29:09.947 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:29:10.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 -- 172.21.15.70:0/1085446770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9010101d10 msgr2=0x7f90101020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 --2- 172.21.15.70:0/1085446770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9010101d10 0x7f90101020e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9000004940 tx=0x7f9000009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 -- 172.21.15.70:0/1085446770 shutdown_connections 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 --2- 172.21.15.70:0/1085446770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90101026b0 0x7f90100fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 --2- 172.21.15.70:0/1085446770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9010101d10 0x7f90101020e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 -- 172.21.15.70:0/1085446770 >> 172.21.15.70:0/1085446770 conn(0x7f90100f7880 msgr2=0x7f90100f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 -- 172.21.15.70:0/1085446770 shutdown_connections 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 -- 172.21.15.70:0/1085446770 wait complete. 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 Processor -- start 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 -- start start 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9010101d10 0x7f9010113dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.056+0000 7f9015cc3700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90101026b0 0x7f9010114300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f9015cc3700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9010114950 con 0x7f90101026b0 2024-03-23T08:29:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f9015cc3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901010dd20 con 0x7f9010101d10 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f900effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90101026b0 0x7f9010114300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f900effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90101026b0 0x7f9010114300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48502/0 (socket says 172.21.15.70:48502) 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f900effd700 1 -- 172.21.15.70:0/3850295623 learned_addr learned my addr 172.21.15.70:0/3850295623 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f900f7fe700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9010101d10 0x7f9010113dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f900effd700 1 -- 172.21.15.70:0/3850295623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9010101d10 msgr2=0x7f9010113dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f900effd700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9010101d10 0x7f9010113dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f900effd700 1 -- 172.21.15.70:0/3850295623 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9000005040 con 0x7f90101026b0 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f900f7fe700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9010101d10 0x7f9010113dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f900effd700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90101026b0 0x7f9010114300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c890 tx=0x7f8ff80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f9007fff700 1 -- 172.21.15.70:0/3850295623 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff800a7d0 con 0x7f90101026b0 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901010e000 con 0x7f90101026b0 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f9007fff700 1 -- 172.21.15.70:0/3850295623 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8ff800ade0 con 0x7f90101026b0 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f9007fff700 1 -- 172.21.15.70:0/3850295623 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff80113a0 con 0x7f90101026b0 2024-03-23T08:29:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.057+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901010e5c0 con 0x7f90101026b0 2024-03-23T08:29:10.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.058+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90100fd460 con 0x7f90101026b0 2024-03-23T08:29:10.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.059+0000 7f9007fff700 1 -- 172.21.15.70:0/3850295623 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f8ff8012400 con 0x7f90101026b0 2024-03-23T08:29:10.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.059+0000 7f9007fff700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8ffc06cd40 0x7f8ffc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:10.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.059+0000 7f9007fff700 1 -- 172.21.15.70:0/3850295623 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8ff808c1d0 con 0x7f90101026b0 2024-03-23T08:29:10.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.059+0000 7f900f7fe700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8ffc06cd40 0x7f8ffc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:10.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.059+0000 7f900f7fe700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8ffc06cd40 0x7f8ffc06f1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f900000b3b0 tx=0x7f9000009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:10.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.063+0000 7f9007fff700 1 -- 172.21.15.70:0/3850295623 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8ff8059c30 con 0x7f90101026b0 2024-03-23T08:29:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:09 smithi070 ceph-mon[145869]: pgmap v46: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:10.233 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:09 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[159700]: 2024-03-23T08:29:09.904+0000 7fc1c6df1640 -1 osd.0 51 log_to_monitors true 2024-03-23T08:29:10.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.299+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9010108d40 con 0x7f8ffc06cd40 2024-03-23T08:29:10.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.300+0000 7f9007fff700 1 -- 172.21.15.70:0/3850295623 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9010108d40 con 0x7f8ffc06cd40 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.302+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8ffc06cd40 msgr2=0x7f8ffc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.302+0000 7f9015cc3700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8ffc06cd40 0x7f8ffc06f1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f900000b3b0 tx=0x7f9000009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.303+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90101026b0 msgr2=0x7f9010114300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.303+0000 7f9015cc3700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90101026b0 0x7f9010114300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c890 tx=0x7f8ff80044c0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.303+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 shutdown_connections 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.303+0000 7f9015cc3700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8ffc06cd40 0x7f8ffc06f1f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.303+0000 7f9015cc3700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9010101d10 0x7f9010113dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.303+0000 7f9015cc3700 1 --2- 172.21.15.70:0/3850295623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90101026b0 0x7f9010114300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.303+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 >> 172.21.15.70:0/3850295623 conn(0x7f90100f7880 msgr2=0x7f90100f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.303+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 shutdown_connections 2024-03-23T08:29:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.303+0000 7f9015cc3700 1 -- 172.21.15.70:0/3850295623 wait complete. 2024-03-23T08:29:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:09 smithi193 ceph-mon[108533]: pgmap v46: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:10.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 -- 172.21.15.70:0/4260946404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070300 msgr2=0x7f95780706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 --2- 172.21.15.70:0/4260946404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070300 0x7f95780706d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9568004970 tx=0x7f9568009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:10.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 -- 172.21.15.70:0/4260946404 shutdown_connections 2024-03-23T08:29:10.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 --2- 172.21.15.70:0/4260946404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9578070c10 0x7f9578109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 --2- 172.21.15.70:0/4260946404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070300 0x7f95780706d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 -- 172.21.15.70:0/4260946404 >> 172.21.15.70:0/4260946404 conn(0x7f95780f89b0 msgr2=0x7f95780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:10.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 -- 172.21.15.70:0/4260946404 shutdown_connections 2024-03-23T08:29:10.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 -- 172.21.15.70:0/4260946404 wait complete. 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 Processor -- start 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.408+0000 7f957d8b9700 1 -- start start 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f957d8b9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9578070300 0x7f9578115a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f957d8b9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070c10 0x7f9578115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f957d8b9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9578116600 con 0x7f9578070c10 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f957d8b9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957810fbe0 con 0x7f9578070300 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f95767fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070c10 0x7f9578115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f95767fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070c10 0x7f9578115fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48524/0 (socket says 172.21.15.70:48524) 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f95767fc700 1 -- 172.21.15.70:0/249340856 learned_addr learned my addr 172.21.15.70:0/249340856 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f9576ffd700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9578070300 0x7f9578115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f95767fc700 1 -- 172.21.15.70:0/249340856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9578070300 msgr2=0x7f9578115a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.409 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f95767fc700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9578070300 0x7f9578115a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f95767fc700 1 -- 172.21.15.70:0/249340856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9568005040 con 0x7f9578070c10 2024-03-23T08:29:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f95767fc700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070c10 0x7f9578115fb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f956c002a80 tx=0x7f956c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f955f7fe700 1 -- 172.21.15.70:0/249340856 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f956c007cc0 con 0x7f9578070c10 2024-03-23T08:29:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957810fec0 con 0x7f9578070c10 2024-03-23T08:29:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f955f7fe700 1 -- 172.21.15.70:0/249340856 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f956c00f040 con 0x7f9578070c10 2024-03-23T08:29:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.409+0000 7f955f7fe700 1 -- 172.21.15.70:0/249340856 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f956c009640 con 0x7f9578070c10 2024-03-23T08:29:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.410+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9578110400 con 0x7f9578070c10 2024-03-23T08:29:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.410+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9578103bf0 con 0x7f9578070c10 2024-03-23T08:29:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.411+0000 7f955f7fe700 1 -- 172.21.15.70:0/249340856 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f956c0097a0 con 0x7f9578070c10 2024-03-23T08:29:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.412+0000 7f955f7fe700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f956006cd50 0x7f956006f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.412+0000 7f955f7fe700 1 -- 172.21.15.70:0/249340856 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f956c08b9f0 con 0x7f9578070c10 2024-03-23T08:29:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.412+0000 7f9576ffd700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f956006cd50 0x7f956006f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.412+0000 7f9576ffd700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f956006cd50 0x7f956006f200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9568004f50 tx=0x7f9568009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:10.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.417+0000 7f955f7fe700 1 -- 172.21.15.70:0/249340856 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f956c08de60 con 0x7f9578070c10 2024-03-23T08:29:10.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.646+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f957810bfb0 con 0x7f956006cd50 2024-03-23T08:29:10.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.656+0000 7f955f7fe700 1 -- 172.21.15.70:0/249340856 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f957810bfb0 con 0x7f956006cd50 2024-03-23T08:29:10.656 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (2m) 0s ago 13m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (31s) 0s ago 14m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (26s) 23s ago 13m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (2m) 0s ago 13m 79.8M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (8m) 0s ago 8m 147M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (8m) 0s ago 8m 14.1M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (8m) 23s ago 8m 97.4M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (8m) 23s ago 8m 15.3M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (4m) 0s ago 15m 494M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (3m) 23s ago 13m 447M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (94s) 0s ago 15m 44.3M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (68s) 23s ago 12m 34.9M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (3m) 0s ago 14m 18.5M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (3m) 23s ago 13m 16.2M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (6s) 0s ago 12m 10.3M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:29:10.657 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (11m) 0s ago 11m 233M 1568M 17.2.7-726-g9c4e0c34 837124da804f 80ec0d92182a 2024-03-23T08:29:10.658 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (11m) 0s ago 11m 203M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:29:10.658 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (10m) 23s ago 10m 279M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:29:10.658 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (10m) 23s ago 10m 242M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:29:10.658 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (10m) 23s ago 9m 225M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:29:10.658 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (3m) 0s ago 13m 60.0M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.658+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f956006cd50 msgr2=0x7f956006f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.658+0000 7f957d8b9700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f956006cd50 0x7f956006f200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9568004f50 tx=0x7f9568009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.658+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070c10 msgr2=0x7f9578115fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.658+0000 7f957d8b9700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070c10 0x7f9578115fb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f956c002a80 tx=0x7f956c0046b0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.658+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 shutdown_connections 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.658+0000 7f957d8b9700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f956006cd50 0x7f956006f200 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.658+0000 7f957d8b9700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9578070300 0x7f9578115a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.658+0000 7f957d8b9700 1 --2- 172.21.15.70:0/249340856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9578070c10 0x7f9578115fb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.658+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 >> 172.21.15.70:0/249340856 conn(0x7f95780f89b0 msgr2=0x7f9578101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.659+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 shutdown_connections 2024-03-23T08:29:10.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.659+0000 7f957d8b9700 1 -- 172.21.15.70:0/249340856 wait complete. 2024-03-23T08:29:10.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.766+0000 7f49227bd700 1 -- 172.21.15.70:0/4152298869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 msgr2=0x7f491c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.766+0000 7f49227bd700 1 --2- 172.21.15.70:0/4152298869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 0x7f491c106f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4910004940 tx=0x7f4910009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:10.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.766+0000 7f49227bd700 1 -- 172.21.15.70:0/4152298869 shutdown_connections 2024-03-23T08:29:10.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.766+0000 7f49227bd700 1 --2- 172.21.15.70:0/4152298869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 0x7f491c106f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.766+0000 7f49227bd700 1 --2- 172.21.15.70:0/4152298869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491c0fde90 0x7f491c0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.766+0000 7f49227bd700 1 -- 172.21.15.70:0/4152298869 >> 172.21.15.70:0/4152298869 conn(0x7f491c072bc0 msgr2=0x7f491c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:10.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.766+0000 7f49227bd700 1 -- 172.21.15.70:0/4152298869 shutdown_connections 2024-03-23T08:29:10.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.766+0000 7f49227bd700 1 -- 172.21.15.70:0/4152298869 wait complete. 2024-03-23T08:29:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.767+0000 7f49227bd700 1 Processor -- start 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.767+0000 7f49227bd700 1 -- start start 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.767+0000 7f49227bd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491c0fde90 0x7f491c113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.767+0000 7f49227bd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 0x7f491c113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.767+0000 7f49227bd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c114610 con 0x7f491c0fe830 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.767+0000 7f49227bd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c10da00 con 0x7f491c0fde90 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.767+0000 7f491b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 0x7f491c113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.767+0000 7f491b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 0x7f491c113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48538/0 (socket says 172.21.15.70:48538) 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f491b7fe700 1 -- 172.21.15.70:0/1863506079 learned_addr learned my addr 172.21.15.70:0/1863506079 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f491bfff700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491c0fde90 0x7f491c113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f491b7fe700 1 -- 172.21.15.70:0/1863506079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491c0fde90 msgr2=0x7f491c113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f491b7fe700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491c0fde90 0x7f491c113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f491b7fe700 1 -- 172.21.15.70:0/1863506079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910005040 con 0x7f491c0fe830 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f491b7fe700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 0x7f491c113fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4910000c00 tx=0x7f4910009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f4918ff9700 1 -- 172.21.15.70:0/1863506079 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4910021070 con 0x7f491c0fe830 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c10dc80 con 0x7f491c0fe830 2024-03-23T08:29:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f4918ff9700 1 -- 172.21.15.70:0/1863506079 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4910013e00 con 0x7f491c0fe830 2024-03-23T08:29:10.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f4918ff9700 1 -- 172.21.15.70:0/1863506079 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491000e070 con 0x7f491c0fe830 2024-03-23T08:29:10.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.768+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c10e1c0 con 0x7f491c0fe830 2024-03-23T08:29:10.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.769+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c101980 con 0x7f491c0fe830 2024-03-23T08:29:10.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.770+0000 7f4918ff9700 1 -- 172.21.15.70:0/1863506079 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f491001c590 con 0x7f491c0fe830 2024-03-23T08:29:10.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.771+0000 7f4918ff9700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f490406cd40 0x7f490406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:10.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.771+0000 7f4918ff9700 1 -- 172.21.15.70:0/1863506079 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f491008db90 con 0x7f491c0fe830 2024-03-23T08:29:10.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.771+0000 7f491bfff700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f490406cd40 0x7f490406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:10.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.771+0000 7f491bfff700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f490406cd40 0x7f490406f1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f490c009b80 tx=0x7f490c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:10.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:10.775+0000 7f4918ff9700 1 -- 172.21.15.70:0/1863506079 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f491005b580 con 0x7f491c0fe830 2024-03-23T08:29:11.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.082+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f491c065e30 con 0x7f491c0fe830 2024-03-23T08:29:11.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.083+0000 7f4918ff9700 1 -- 172.21.15.70:0/1863506079 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+738 (secure 0 0 0) 0x7f491005b110 con 0x7f491c0fe830 2024-03-23T08:29:11.083 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:29:11.083 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:29:11.083 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:29:11.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:29:11.083 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 9, 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 4 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:29:11.084 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:29:11.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f490406cd40 msgr2=0x7f490406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f490406cd40 0x7f490406f1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f490c009b80 tx=0x7f490c00b040 comp rx=0 tx=0).stop 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 msgr2=0x7f491c113fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 0x7f491c113fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4910000c00 tx=0x7f4910009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 shutdown_connections 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f490406cd40 0x7f490406f1f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491c0fde90 0x7f491c113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 --2- 172.21.15.70:0/1863506079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f491c0fe830 0x7f491c113fc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 >> 172.21.15.70:0/1863506079 conn(0x7f491c072bc0 msgr2=0x7f491c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 shutdown_connections 2024-03-23T08:29:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.085+0000 7f49227bd700 1 -- 172.21.15.70:0/1863506079 wait complete. 2024-03-23T08:29:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.187+0000 7f4048e8e700 1 -- 172.21.15.70:0/833269983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 msgr2=0x7f40441031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.187+0000 7f4048e8e700 1 --2- 172.21.15.70:0/833269983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 0x7f40441031d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f403c009ce0 tx=0x7f403c004d10 comp rx=0 tx=0).stop 2024-03-23T08:29:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.187+0000 7f4048e8e700 1 -- 172.21.15.70:0/833269983 shutdown_connections 2024-03-23T08:29:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.187+0000 7f4048e8e700 1 --2- 172.21.15.70:0/833269983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40441037a0 0x7f40440fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.187+0000 7f4048e8e700 1 --2- 172.21.15.70:0/833269983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 0x7f40441031d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.187+0000 7f4048e8e700 1 -- 172.21.15.70:0/833269983 >> 172.21.15.70:0/833269983 conn(0x7f40440f89b0 msgr2=0x7f40440fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:11.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.187+0000 7f4048e8e700 1 -- 172.21.15.70:0/833269983 shutdown_connections 2024-03-23T08:29:11.188 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.187+0000 7f4048e8e700 1 -- 172.21.15.70:0/833269983 wait complete. 2024-03-23T08:29:11.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f4048e8e700 1 Processor -- start 2024-03-23T08:29:11.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f4048e8e700 1 -- start start 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f4048e8e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 0x7f4044113c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f4048e8e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40441037a0 0x7f4044114150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f4048e8e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4044114830 con 0x7f4044102e00 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f4048e8e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404410db90 con 0x7f40441037a0 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f404259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 0x7f4044113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f404259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 0x7f4044113c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48558/0 (socket says 172.21.15.70:48558) 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f404259c700 1 -- 172.21.15.70:0/3631239391 learned_addr learned my addr 172.21.15.70:0/3631239391 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f4041d9b700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40441037a0 0x7f4044114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f404259c700 1 -- 172.21.15.70:0/3631239391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40441037a0 msgr2=0x7f4044114150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f404259c700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40441037a0 0x7f4044114150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f404259c700 1 -- 172.21.15.70:0/3631239391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c009940 con 0x7f4044102e00 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.188+0000 7f4041d9b700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40441037a0 0x7f4044114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.189+0000 7f404259c700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 0x7f4044113c10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f403c009910 tx=0x7f403c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.189+0000 7f4032ffd700 1 -- 172.21.15.70:0/3631239391 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f403c00e040 con 0x7f4044102e00 2024-03-23T08:29:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.189+0000 7f4032ffd700 1 -- 172.21.15.70:0/3631239391 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f403c004610 con 0x7f4044102e00 2024-03-23T08:29:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.189+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404410de10 con 0x7f4044102e00 2024-03-23T08:29:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.189+0000 7f4032ffd700 1 -- 172.21.15.70:0/3631239391 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f403c01c640 con 0x7f4044102e00 2024-03-23T08:29:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.189+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404410e3d0 con 0x7f4044102e00 2024-03-23T08:29:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.190+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40440fe550 con 0x7f4044102e00 2024-03-23T08:29:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.190+0000 7f4032ffd700 1 -- 172.21.15.70:0/3631239391 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f403c012ac0 con 0x7f4044102e00 2024-03-23T08:29:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.191+0000 7f4032ffd700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f402c06cd40 0x7f402c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.191+0000 7f4032ffd700 1 -- 172.21.15.70:0/3631239391 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f403c08ddb0 con 0x7f4044102e00 2024-03-23T08:29:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.191+0000 7f4041d9b700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f402c06cd40 0x7f402c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.191+0000 7f4041d9b700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f402c06cd40 0x7f402c06f1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4038006fd0 tx=0x7f4038008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:11.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.196+0000 7f4032ffd700 1 -- 172.21.15.70:0/3631239391 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f403c05b680 con 0x7f4044102e00 2024-03-23T08:29:11.232 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:29:10 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[159700]: 2024-03-23T08:29:10.848+0000 7fc1b824a700 -1 osd.0 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:29:11.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:10 smithi070 ceph-mon[145869]: from='osd.0 [v2:172.21.15.70:6802/2783750869,v1:172.21.15.70:6803/2783750869]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T08:29:11.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:10 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:11.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:10 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:11.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:10 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:11.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:10 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:10 smithi193 ceph-mon[108533]: from='osd.0 [v2:172.21.15.70:6802/2783750869,v1:172.21.15.70:6803/2783750869]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-23T08:29:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:10 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:10 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:10 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:10 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:11.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.441+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f404404bed0 con 0x7f4044102e00 2024-03-23T08:29:11.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.442+0000 7f4032ffd700 1 -- 172.21.15.70:0/3631239391 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f403c004e10 con 0x7f4044102e00 2024-03-23T08:29:11.442 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:29:11.443 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:29:11.444 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:29:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:29:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:29:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:29:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:29:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:29:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:29:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:29:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:29:11.445 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f402c06cd40 msgr2=0x7f402c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f402c06cd40 0x7f402c06f1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4038006fd0 tx=0x7f4038008040 comp rx=0 tx=0).stop 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 msgr2=0x7f4044113c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 0x7f4044113c10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f403c009910 tx=0x7f403c004d10 comp rx=0 tx=0).stop 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 shutdown_connections 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f402c06cd40 0x7f402c06f1f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4044102e00 0x7f4044113c10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 --2- 172.21.15.70:0/3631239391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40441037a0 0x7f4044114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 >> 172.21.15.70:0/3631239391 conn(0x7f40440f89b0 msgr2=0x7f40440f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 shutdown_connections 2024-03-23T08:29:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.444+0000 7f4048e8e700 1 -- 172.21.15.70:0/3631239391 wait complete. 2024-03-23T08:29:11.447 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:29:11.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.547+0000 7f531bfd5700 1 -- 172.21.15.70:0/2372611110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5314102e70 msgr2=0x7f5314103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.547+0000 7f531bfd5700 1 --2- 172.21.15.70:0/2372611110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5314102e70 0x7f5314103240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5310004940 tx=0x7f5310009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.547+0000 7f531bfd5700 1 -- 172.21.15.70:0/2372611110 shutdown_connections 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.547+0000 7f531bfd5700 1 --2- 172.21.15.70:0/2372611110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53140fb4a0 0x7f53140fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.547+0000 7f531bfd5700 1 --2- 172.21.15.70:0/2372611110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5314102e70 0x7f5314103240 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.547+0000 7f531bfd5700 1 -- 172.21.15.70:0/2372611110 >> 172.21.15.70:0/2372611110 conn(0x7f53140714c0 msgr2=0x7f53140718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.547+0000 7f531bfd5700 1 -- 172.21.15.70:0/2372611110 shutdown_connections 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.547+0000 7f531bfd5700 1 -- 172.21.15.70:0/2372611110 wait complete. 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f531bfd5700 1 Processor -- start 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f531bfd5700 1 -- start start 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f531bfd5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53140fb4a0 0x7f5314113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f531bfd5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5314102e70 0x7f5314114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f531bfd5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53141146e0 con 0x7f53140fb4a0 2024-03-23T08:29:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f531bfd5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531410dad0 con 0x7f5314102e70 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f5319d71700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53140fb4a0 0x7f5314113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f5319d71700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53140fb4a0 0x7f5314113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48586/0 (socket says 172.21.15.70:48586) 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f5319d71700 1 -- 172.21.15.70:0/4087535146 learned_addr learned my addr 172.21.15.70:0/4087535146 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f5319d71700 1 -- 172.21.15.70:0/4087535146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5314102e70 msgr2=0x7f5314114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f5319d71700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5314102e70 0x7f5314114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f5319d71700 1 -- 172.21.15.70:0/4087535146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5310005040 con 0x7f53140fb4a0 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f5319d71700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53140fb4a0 0x7f5314113b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5310009180 tx=0x7f5310009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.548+0000 7f530a7fc700 1 -- 172.21.15.70:0/4087535146 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f531000e040 con 0x7f53140fb4a0 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.549+0000 7f530a7fc700 1 -- 172.21.15.70:0/4087535146 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5310004290 con 0x7f53140fb4a0 2024-03-23T08:29:11.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.549+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531410dd50 con 0x7f53140fb4a0 2024-03-23T08:29:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.549+0000 7f530a7fc700 1 -- 172.21.15.70:0/4087535146 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f531001c3f0 con 0x7f53140fb4a0 2024-03-23T08:29:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.549+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531410e290 con 0x7f53140fb4a0 2024-03-23T08:29:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.549+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5314105320 con 0x7f53140fb4a0 2024-03-23T08:29:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.550+0000 7f530a7fc700 1 -- 172.21.15.70:0/4087535146 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f5310013790 con 0x7f53140fb4a0 2024-03-23T08:29:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.550+0000 7f530a7fc700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f530006cd40 0x7f530006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.551+0000 7f530a7fc700 1 -- 172.21.15.70:0/4087535146 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f531008d4a0 con 0x7f53140fb4a0 2024-03-23T08:29:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.551+0000 7f5319570700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f530006cd40 0x7f530006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.551+0000 7f5319570700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f530006cd40 0x7f530006f1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f531410ec90 tx=0x7f5304007d40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:11.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.556+0000 7f530a7fc700 1 -- 172.21.15.70:0/4087535146 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f531005adf0 con 0x7f53140fb4a0 2024-03-23T08:29:11.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.774+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f531410f210 con 0x7f530006cd40 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.775+0000 7f530a7fc700 1 -- 172.21.15.70:0/4087535146 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f531410f210 con 0x7f530006cd40 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-03-23T08:29:11.779 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:29:11.780 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "7/21 daemons upgraded", 2024-03-23T08:29:11.780 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-03-23T08:29:11.780 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:29:11.780 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:29:11.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.777+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f530006cd40 msgr2=0x7f530006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.777+0000 7f531bfd5700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f530006cd40 0x7f530006f1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f531410ec90 tx=0x7f5304007d40 comp rx=0 tx=0).stop 2024-03-23T08:29:11.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.777+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53140fb4a0 msgr2=0x7f5314113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.777+0000 7f531bfd5700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53140fb4a0 0x7f5314113b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5310009180 tx=0x7f5310009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:11.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.778+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 shutdown_connections 2024-03-23T08:29:11.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.778+0000 7f531bfd5700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f530006cd40 0x7f530006f1f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.778+0000 7f531bfd5700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53140fb4a0 0x7f5314113b50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.778+0000 7f531bfd5700 1 --2- 172.21.15.70:0/4087535146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5314102e70 0x7f5314114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.778+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 >> 172.21.15.70:0/4087535146 conn(0x7f53140714c0 msgr2=0x7f53140fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:11.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.778+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 shutdown_connections 2024-03-23T08:29:11.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.778+0000 7f531bfd5700 1 -- 172.21.15.70:0/4087535146 wait complete. 2024-03-23T08:29:11.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.893+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/233620439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 msgr2=0x7f9bd80fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.893+0000 7f9bdfe7e700 1 --2- 172.21.15.70:0/233620439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 0x7f9bd80fe260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8004940 tx=0x7f9bc8009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:11.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.893+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/233620439 shutdown_connections 2024-03-23T08:29:11.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.893+0000 7f9bdfe7e700 1 --2- 172.21.15.70:0/233620439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd80fe830 0x7f9bd8106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.893+0000 7f9bdfe7e700 1 --2- 172.21.15.70:0/233620439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 0x7f9bd80fe260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.893+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/233620439 >> 172.21.15.70:0/233620439 conn(0x7f9bd8072bc0 msgr2=0x7f9bd8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.893+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/233620439 shutdown_connections 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.893+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/233620439 wait complete. 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bdfe7e700 1 Processor -- start 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bdfe7e700 1 -- start start 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bdfe7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 0x7f9bd80fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bdfe7e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd80fe830 0x7f9bd80fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bdfe7e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80fc820 con 0x7f9bd80fde90 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bdfe7e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80fc960 con 0x7f9bd80fe830 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bddc1a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 0x7f9bd80fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bddc1a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 0x7f9bd80fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48598/0 (socket says 172.21.15.70:48598) 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bddc1a700 1 -- 172.21.15.70:0/3373115847 learned_addr learned my addr 172.21.15.70:0/3373115847 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bdd419700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd80fe830 0x7f9bd80fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bddc1a700 1 -- 172.21.15.70:0/3373115847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd80fe830 msgr2=0x7f9bd80fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bddc1a700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd80fe830 0x7f9bd80fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bddc1a700 1 -- 172.21.15.70:0/3373115847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8005040 con 0x7f9bd80fde90 2024-03-23T08:29:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.894+0000 7f9bdd419700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd80fe830 0x7f9bd80fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:29:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.895+0000 7f9bddc1a700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 0x7f9bd80fbd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800b3b0 tx=0x7f9bc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.895+0000 7f9bce7fc700 1 -- 172.21.15.70:0/3373115847 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc800e040 con 0x7f9bd80fde90 2024-03-23T08:29:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.895+0000 7f9bce7fc700 1 -- 172.21.15.70:0/3373115847 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9bc8013420 con 0x7f9bd80fde90 2024-03-23T08:29:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.895+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd80fcbb0 con 0x7f9bd80fde90 2024-03-23T08:29:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.895+0000 7f9bce7fc700 1 -- 172.21.15.70:0/3373115847 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc801c5d0 con 0x7f9bd80fde90 2024-03-23T08:29:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.895+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd811a1a0 con 0x7f9bd80fde90 2024-03-23T08:29:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.896+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd8065e30 con 0x7f9bd80fde90 2024-03-23T08:29:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.896+0000 7f9bce7fc700 1 -- 172.21.15.70:0/3373115847 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 39) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f9bc8013590 con 0x7f9bd80fde90 2024-03-23T08:29:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.896+0000 7f9bce7fc700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f9bc406cd40 0x7f9bc406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.897+0000 7f9bce7fc700 1 -- 172.21.15.70:0/3373115847 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9bc808d440 con 0x7f9bd80fde90 2024-03-23T08:29:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.897+0000 7f9bdd419700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f9bc406cd40 0x7f9bc406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.897+0000 7f9bdd419700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f9bc406cd40 0x7f9bc406f1f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009fd0 tx=0x7f9bd400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:11.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:11.901+0000 7f9bce7fc700 1 -- 172.21.15.70:0/3373115847 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9bc8057280 con 0x7f9bd80fde90 2024-03-23T08:29:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.174+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9bd8109dc0 con 0x7f9bd80fde90 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.174+0000 7f9bce7fc700 1 -- 172.21.15.70:0/3373115847 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1182 (secure 0 0 0) 0x7f9bc805a8a0 con 0x7f9bd80fde90 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN Degraded data redundancy: 39/222 objects degraded (17.568%), 18 pgs degraded 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 39/222 objects degraded (17.568%), 18 pgs degraded 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-03-23T08:29:12.175 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-03-23T08:29:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-03-23T08:29:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-03-23T08:29:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-03-23T08:29:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-03-23T08:29:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-03-23T08:29:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-03-23T08:29:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-03-23T08:29:12.176 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.176+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f9bc406cd40 msgr2=0x7f9bc406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.176+0000 7f9bdfe7e700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f9bc406cd40 0x7f9bc406f1f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009fd0 tx=0x7f9bd400b040 comp rx=0 tx=0).stop 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.176+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 msgr2=0x7f9bd80fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.176+0000 7f9bdfe7e700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 0x7f9bd80fbd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800b3b0 tx=0x7f9bc8009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.176+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 shutdown_connections 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.176+0000 7f9bdfe7e700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f9bc406cd40 0x7f9bc406f1f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.176+0000 7f9bdfe7e700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bd80fde90 0x7f9bd80fbd50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.176+0000 7f9bdfe7e700 1 --2- 172.21.15.70:0/3373115847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd80fe830 0x7f9bd80fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.176+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 >> 172.21.15.70:0/3373115847 conn(0x7f9bd8072bc0 msgr2=0x7f9bd80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.177+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 shutdown_connections 2024-03-23T08:29:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:12.177+0000 7f9bdfe7e700 1 -- 172.21.15.70:0/3373115847 wait complete. 2024-03-23T08:29:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: from='osd.0 [v2:172.21.15.70:6802/2783750869,v1:172.21.15.70:6803/2783750869]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T08:29:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: osdmap e54: 6 total, 5 up, 6 in 2024-03-23T08:29:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: from='osd.0 [v2:172.21.15.70:6802/2783750869,v1:172.21.15.70:6803/2783750869]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:29:12.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1863506079' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:12.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3631239391' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:29:12.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:11 smithi070 ceph-mon[145869]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:12.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:12.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 201 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:29:12.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:12.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: from='osd.0 [v2:172.21.15.70:6802/2783750869,v1:172.21.15.70:6803/2783750869]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-23T08:29:12.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: osdmap e54: 6 total, 5 up, 6 in 2024-03-23T08:29:12.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: from='osd.0 [v2:172.21.15.70:6802/2783750869,v1:172.21.15.70:6803/2783750869]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:29:12.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1863506079' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:12.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3631239391' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:29:12.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:11 smithi193 ceph-mon[108533]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:12.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:12 smithi070 ceph-mon[145869]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:29:12.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:12 smithi070 ceph-mon[145869]: osd.0 [v2:172.21.15.70:6802/2783750869,v1:172.21.15.70:6803/2783750869] boot 2024-03-23T08:29:12.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:12 smithi070 ceph-mon[145869]: osdmap e55: 6 total, 6 up, 6 in 2024-03-23T08:29:12.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:12 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:29:12.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:12 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3373115847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:29:12.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:12 smithi070 ceph-mon[145869]: pgmap v50: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 6/222 objects degraded (2.703%) 2024-03-23T08:29:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:12 smithi193 ceph-mon[108533]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:29:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:12 smithi193 ceph-mon[108533]: osd.0 [v2:172.21.15.70:6802/2783750869,v1:172.21.15.70:6803/2783750869] boot 2024-03-23T08:29:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:12 smithi193 ceph-mon[108533]: osdmap e55: 6 total, 6 up, 6 in 2024-03-23T08:29:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:12 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-23T08:29:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:12 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3373115847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:29:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:12 smithi193 ceph-mon[108533]: pgmap v50: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 6/222 objects degraded (2.703%) 2024-03-23T08:29:13.948 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:13 smithi070 ceph-mon[145869]: Health check update: Degraded data redundancy: 6/222 objects degraded (2.703%), 3 pgs degraded (PG_DEGRADED) 2024-03-23T08:29:13.948 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:13 smithi070 ceph-mon[145869]: osdmap e56: 6 total, 6 up, 6 in 2024-03-23T08:29:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:13 smithi193 ceph-mon[108533]: Health check update: Degraded data redundancy: 6/222 objects degraded (2.703%), 3 pgs degraded (PG_DEGRADED) 2024-03-23T08:29:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:13 smithi193 ceph-mon[108533]: osdmap e56: 6 total, 6 up, 6 in 2024-03-23T08:29:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:29:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:29:15.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:15.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:29:15.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:15.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:15.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:15.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-23T08:29:15.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-23T08:29:15.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-23T08:29:15.734 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:15 smithi070 ceph-mon[145869]: pgmap v52: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 6/222 objects degraded (2.703%) 2024-03-23T08:29:15.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:15.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-23T08:29:15.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:15 smithi193 ceph-mon[108533]: pgmap v52: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 6/222 objects degraded (2.703%) 2024-03-23T08:29:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:17 smithi070 ceph-mon[145869]: pgmap v53: 65 pgs: 25 peering, 40 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:29:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:29:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:17 smithi070 ceph-mon[145869]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/222 objects degraded (2.703%), 3 pgs degraded) 2024-03-23T08:29:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:17 smithi070 ceph-mon[145869]: Cluster is now healthy 2024-03-23T08:29:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:17 smithi193 ceph-mon[108533]: pgmap v53: 65 pgs: 25 peering, 40 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:29:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:29:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:17 smithi193 ceph-mon[108533]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/222 objects degraded (2.703%), 3 pgs degraded) 2024-03-23T08:29:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:17 smithi193 ceph-mon[108533]: Cluster is now healthy 2024-03-23T08:29:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:18 smithi193 ceph-mon[108533]: mgrmap e40: smithi070.jhsbch(active, since 91s), standbys: smithi193.zgpdbu 2024-03-23T08:29:19.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:18 smithi070 ceph-mon[145869]: mgrmap e40: smithi070.jhsbch(active, since 91s), standbys: smithi193.zgpdbu 2024-03-23T08:29:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:19 smithi193 ceph-mon[108533]: pgmap v54: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 396 B/s wr, 2 op/s 2024-03-23T08:29:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:19 smithi070 ceph-mon[145869]: pgmap v54: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 396 B/s wr, 2 op/s 2024-03-23T08:29:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:21 smithi193 ceph-mon[108533]: pgmap v55: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 351 B/s wr, 2 op/s 2024-03-23T08:29:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:21 smithi070 ceph-mon[145869]: pgmap v55: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 351 B/s wr, 2 op/s 2024-03-23T08:29:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:23 smithi193 ceph-mon[108533]: pgmap v56: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-03-23T08:29:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:23 smithi070 ceph-mon[145869]: pgmap v56: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-03-23T08:29:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:25 smithi193 ceph-mon[108533]: pgmap v57: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 959 B/s rd, 261 B/s wr, 2 op/s 2024-03-23T08:29:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:25 smithi070 ceph-mon[145869]: pgmap v57: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 959 B/s rd, 261 B/s wr, 2 op/s 2024-03-23T08:29:27.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:27 smithi070 ceph-mon[145869]: pgmap v58: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-23T08:29:27.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:27 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:27 smithi193 ceph-mon[108533]: pgmap v58: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-23T08:29:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:27 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:29 smithi070 ceph-mon[145869]: pgmap v59: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-23T08:29:29.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:29 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-23T08:29:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:29 smithi193 ceph-mon[108533]: pgmap v59: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-23T08:29:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:29 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-23T08:29:30.728 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:30 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-23T08:29:30.728 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:30 smithi070 ceph-mon[145869]: Upgrade: osd.1 is safe to restart 2024-03-23T08:29:31.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:30 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-23T08:29:31.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:30 smithi193 ceph-mon[108533]: Upgrade: osd.1 is safe to restart 2024-03-23T08:29:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:31 smithi070 ceph-mon[145869]: pgmap v60: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:29:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T08:29:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:29:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:31.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:29:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:31 smithi193 ceph-mon[108533]: pgmap v60: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:29:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-23T08:29:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:29:32.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:32.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:29:32.936 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:32 smithi070 ceph-mon[145869]: Upgrade: Updating osd.1 2024-03-23T08:29:32.936 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:32 smithi070 ceph-mon[145869]: Deploying daemon osd.1 on smithi070 2024-03-23T08:29:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:32 smithi193 ceph-mon[108533]: Upgrade: Updating osd.1 2024-03-23T08:29:33.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:32 smithi193 ceph-mon[108533]: Deploying daemon osd.1 on smithi070 2024-03-23T08:29:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:33 smithi070 ceph-mon[145869]: pgmap v61: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:29:34.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:33 smithi193 ceph-mon[108533]: pgmap v61: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:29:34.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:34 smithi070 systemd[1]: Stopping Ceph osd.1 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:29:34.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:34 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[72673]: 2024-03-23T08:29:34.606+0000 7fe9ab3ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:29:34.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:34 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[72673]: 2024-03-23T08:29:34.606+0000 7fe9ab3ab700 -1 osd.1 56 *** Got signal Terminated *** 2024-03-23T08:29:34.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:34 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[72673]: 2024-03-23T08:29:34.606+0000 7fe9ab3ab700 -1 osd.1 56 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:29:35.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:34 smithi193 ceph-mon[108533]: osd.1 marked itself down and dead 2024-03-23T08:29:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:34 smithi070 ceph-mon[145869]: osd.1 marked itself down and dead 2024-03-23T08:29:35.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:35 smithi070 bash[165961]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1 2024-03-23T08:29:36.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:35 smithi193 ceph-mon[108533]: pgmap v62: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:29:36.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:35 smithi193 ceph-mon[108533]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:29:36.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:35 smithi193 ceph-mon[108533]: osdmap e57: 6 total, 5 up, 6 in 2024-03-23T08:29:36.227 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:35 smithi070 ceph-mon[145869]: pgmap v62: 65 pgs: 65 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:29:36.227 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:35 smithi070 ceph-mon[145869]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:29:36.227 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:35 smithi070 ceph-mon[145869]: osdmap e57: 6 total, 5 up, 6 in 2024-03-23T08:29:36.978 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:36 smithi070 ceph-mon[145869]: osdmap e58: 6 total, 5 up, 6 in 2024-03-23T08:29:37.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:36 smithi193 ceph-mon[108533]: osdmap e58: 6 total, 5 up, 6 in 2024-03-23T08:29:37.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:37 smithi070 systemd[1]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.1.service: Succeeded. 2024-03-23T08:29:37.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:37 smithi070 systemd[1]: Stopped Ceph osd.1 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:29:37.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:37 smithi070 systemd[1]: Starting Ceph osd.1 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:29:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:37 smithi193 ceph-mon[108533]: pgmap v65: 65 pgs: 1 active+undersized+degraded, 9 stale+active+clean, 4 active+undersized, 51 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 1/222 objects degraded (0.450%) 2024-03-23T08:29:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:37 smithi193 ceph-mon[108533]: Health check failed: Degraded data redundancy: 1/222 objects degraded (0.450%), 1 pg degraded (PG_DEGRADED) 2024-03-23T08:29:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:37 smithi070 ceph-mon[145869]: pgmap v65: 65 pgs: 1 active+undersized+degraded, 9 stale+active+clean, 4 active+undersized, 51 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 1/222 objects degraded (0.450%) 2024-03-23T08:29:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:37 smithi070 ceph-mon[145869]: Health check failed: Degraded data redundancy: 1/222 objects degraded (0.450%), 1 pg degraded (PG_DEGRADED) 2024-03-23T08:29:38.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:38 smithi070 bash[166240]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-23T08:29:38.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:38 smithi070 bash[166240]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-23T08:29:38.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:38 smithi070 bash[166240]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-23T08:29:38.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:38 smithi070 bash[166240]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-23T08:29:38.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:38 smithi070 bash[166240]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-23T08:29:38.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:38 smithi070 bash[166240]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-23T08:29:38.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:38 smithi070 bash[166240]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-23T08:29:39.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:39 smithi070 ceph-osd[166537]: -- 172.21.15.70:0/3976887793 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5626580d51e0 con 0x5626580a4c00 2024-03-23T08:29:39.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:39 smithi070 bash[166500]: 13fce63c86eb677d140926517ab4b71172ee883c0cfd8552806e90ea39394c29 2024-03-23T08:29:39.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:39 smithi070 systemd[1]: Started Ceph osd.1 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:29:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:39 smithi193 ceph-mon[108533]: pgmap v66: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:39 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:39 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:39 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:29:40.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:39 smithi070 ceph-mon[145869]: pgmap v66: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:40.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:39 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:40.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:39 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:40.089 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:39 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:29:40.426 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:40 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[166514]: 2024-03-23T08:29:40.088+0000 7f77e7523640 -1 Falling back to public interface 2024-03-23T08:29:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:41 smithi193 ceph-mon[108533]: pgmap v67: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:41 smithi070 ceph-mon[145869]: pgmap v67: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:42.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.285+0000 7fcc824c7700 1 -- 172.21.15.70:0/3942404247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 msgr2=0x7fcc7c109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.285+0000 7fcc824c7700 1 --2- 172.21.15.70:0/3942404247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 0x7fcc7c109250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc70004940 tx=0x7fcc70009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:42.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.285+0000 7fcc824c7700 1 -- 172.21.15.70:0/3942404247 shutdown_connections 2024-03-23T08:29:42.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.285+0000 7fcc824c7700 1 --2- 172.21.15.70:0/3942404247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 0x7fcc7c109250 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.285+0000 7fcc824c7700 1 --2- 172.21.15.70:0/3942404247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc7c0fbbe0 0x7fcc7c0fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.285+0000 7fcc824c7700 1 -- 172.21.15.70:0/3942404247 >> 172.21.15.70:0/3942404247 conn(0x7fcc7c0f78d0 msgr2=0x7fcc7c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:42.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.285+0000 7fcc824c7700 1 -- 172.21.15.70:0/3942404247 shutdown_connections 2024-03-23T08:29:42.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.285+0000 7fcc824c7700 1 -- 172.21.15.70:0/3942404247 wait complete. 2024-03-23T08:29:42.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.285+0000 7fcc824c7700 1 Processor -- start 2024-03-23T08:29:42.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc824c7700 1 -- start start 2024-03-23T08:29:42.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc824c7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc7c0fbbe0 0x7fcc7c115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:42.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc824c7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 0x7fcc7c1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:42.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc824c7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c116740 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc824c7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c10fb30 con 0x7fcc7c0fbbe0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc7b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 0x7fcc7c1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc7b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 0x7fcc7c1160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40830/0 (socket says 172.21.15.70:40830) 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc7b7fe700 1 -- 172.21.15.70:0/2647116733 learned_addr learned my addr 172.21.15.70:0/2647116733 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc7bfff700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc7c0fbbe0 0x7fcc7c115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc7b7fe700 1 -- 172.21.15.70:0/2647116733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc7c0fbbe0 msgr2=0x7fcc7c115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc7b7fe700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc7c0fbbe0 0x7fcc7c115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.286+0000 7fcc7b7fe700 1 -- 172.21.15.70:0/2647116733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc70005040 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.287+0000 7fcc7b7fe700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 0x7fcc7c1160f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc70000c00 tx=0x7fcc70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.287+0000 7fcc78ff9700 1 -- 172.21.15.70:0/2647116733 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc70021070 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.287+0000 7fcc78ff9700 1 -- 172.21.15.70:0/2647116733 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc70013e00 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.287+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c10fdb0 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.287+0000 7fcc78ff9700 1 -- 172.21.15.70:0/2647116733 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7000e070 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.287+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c110370 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.288+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c066710 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.288+0000 7fcc78ff9700 1 -- 172.21.15.70:0/2647116733 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcc7001c590 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.289+0000 7fcc78ff9700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcc6406cc20 0x7fcc6406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:42.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.289+0000 7fcc78ff9700 1 -- 172.21.15.70:0/2647116733 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcc7008cfd0 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.289+0000 7fcc7bfff700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcc6406cc20 0x7fcc6406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:42.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.289+0000 7fcc7bfff700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcc6406cc20 0x7fcc6406f0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009b80 tx=0x7fcc6c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:42.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.294+0000 7fcc78ff9700 1 -- 172.21.15.70:0/2647116733 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcc7005abe0 con 0x7fcc7c0fc4f0 2024-03-23T08:29:42.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.514+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc7c10bef0 con 0x7fcc6406cc20 2024-03-23T08:29:42.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.515+0000 7fcc78ff9700 1 -- 172.21.15.70:0/2647116733 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fcc7c10bef0 con 0x7fcc6406cc20 2024-03-23T08:29:42.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcc6406cc20 msgr2=0x7fcc6406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcc6406cc20 0x7fcc6406f0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009b80 tx=0x7fcc6c00b040 comp rx=0 tx=0).stop 2024-03-23T08:29:42.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 msgr2=0x7fcc7c1160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 0x7fcc7c1160f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc70000c00 tx=0x7fcc70009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:42.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 shutdown_connections 2024-03-23T08:29:42.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcc6406cc20 0x7fcc6406f0d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc7c0fbbe0 0x7fcc7c115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 --2- 172.21.15.70:0/2647116733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7c0fc4f0 0x7fcc7c1160f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 >> 172.21.15.70:0/2647116733 conn(0x7fcc7c0f78d0 msgr2=0x7fcc7c0f7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:42.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 shutdown_connections 2024-03-23T08:29:42.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.518+0000 7fcc824c7700 1 -- 172.21.15.70:0/2647116733 wait complete. 2024-03-23T08:29:42.529 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:29:42.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.643+0000 7ff2193b7700 1 -- 172.21.15.70:0/2770111845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406fd80 msgr2=0x7ff214105020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.643+0000 7ff2193b7700 1 --2- 172.21.15.70:0/2770111845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406fd80 0x7ff214105020 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff20c004940 tx=0x7ff20c009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:42.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.643+0000 7ff2193b7700 1 -- 172.21.15.70:0/2770111845 shutdown_connections 2024-03-23T08:29:42.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.643+0000 7ff2193b7700 1 --2- 172.21.15.70:0/2770111845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406fd80 0x7ff214105020 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.643+0000 7ff2193b7700 1 --2- 172.21.15.70:0/2770111845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21406f3e0 0x7ff21406f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.643+0000 7ff2193b7700 1 -- 172.21.15.70:0/2770111845 >> 172.21.15.70:0/2770111845 conn(0x7ff2140690e0 msgr2=0x7ff2140694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:42.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.643+0000 7ff2193b7700 1 -- 172.21.15.70:0/2770111845 shutdown_connections 2024-03-23T08:29:42.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.643+0000 7ff2193b7700 1 -- 172.21.15.70:0/2770111845 wait complete. 2024-03-23T08:29:42.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.644+0000 7ff2193b7700 1 Processor -- start 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.644+0000 7ff2193b7700 1 -- start start 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.644+0000 7ff2193b7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406f3e0 0x7ff214107c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.644+0000 7ff2193b7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21406fd80 0x7ff2141061a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.644+0000 7ff2193b7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2141066e0 con 0x7ff21406f3e0 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.644+0000 7ff2193b7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff214106850 con 0x7ff21406fd80 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.644+0000 7ff212ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406f3e0 0x7ff214107c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff212ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406f3e0 0x7ff214107c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40858/0 (socket says 172.21.15.70:40858) 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff212ffd700 1 -- 172.21.15.70:0/193786950 learned_addr learned my addr 172.21.15.70:0/193786950 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff2127fc700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21406fd80 0x7ff2141061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff212ffd700 1 -- 172.21.15.70:0/193786950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21406fd80 msgr2=0x7ff2141061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff212ffd700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21406fd80 0x7ff2141061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff212ffd700 1 -- 172.21.15.70:0/193786950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c005040 con 0x7ff21406f3e0 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff212ffd700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406f3e0 0x7ff214107c30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00eb40 tx=0x7ff1fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:42.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff20b7fe700 1 -- 172.21.15.70:0/193786950 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1fc007cc0 con 0x7ff21406f3e0 2024-03-23T08:29:42.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff214106ad0 con 0x7ff21406f3e0 2024-03-23T08:29:42.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff20b7fe700 1 -- 172.21.15.70:0/193786950 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1fc00f040 con 0x7ff21406f3e0 2024-03-23T08:29:42.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff20b7fe700 1 -- 172.21.15.70:0/193786950 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1fc00c990 con 0x7ff21406f3e0 2024-03-23T08:29:42.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.645+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff214106fe0 con 0x7ff21406f3e0 2024-03-23T08:29:42.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.647+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21406ec50 con 0x7ff21406f3e0 2024-03-23T08:29:42.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.648+0000 7ff20b7fe700 1 -- 172.21.15.70:0/193786950 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff1fc00cc00 con 0x7ff21406f3e0 2024-03-23T08:29:42.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.648+0000 7ff20b7fe700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff20006cc20 0x7ff20006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:42.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.648+0000 7ff20b7fe700 1 -- 172.21.15.70:0/193786950 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff1fc08bdb0 con 0x7ff21406f3e0 2024-03-23T08:29:42.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.649+0000 7ff2127fc700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff20006cc20 0x7ff20006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:42.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.649+0000 7ff2127fc700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff20006cc20 0x7ff20006f0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff20c004890 tx=0x7ff20c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.656+0000 7ff20b7fe700 1 -- 172.21.15.70:0/193786950 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff1fc0599d0 con 0x7ff21406f3e0 2024-03-23T08:29:42.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.884+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2141076f0 con 0x7ff20006cc20 2024-03-23T08:29:42.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.885+0000 7ff20b7fe700 1 -- 172.21.15.70:0/193786950 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff2141076f0 con 0x7ff20006cc20 2024-03-23T08:29:42.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff20006cc20 msgr2=0x7ff20006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff20006cc20 0x7ff20006f0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff20c004890 tx=0x7ff20c009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:42.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406f3e0 msgr2=0x7ff214107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406f3e0 0x7ff214107c30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00eb40 tx=0x7ff1fc00b040 comp rx=0 tx=0).stop 2024-03-23T08:29:42.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 shutdown_connections 2024-03-23T08:29:42.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff20006cc20 0x7ff20006f0d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff21406f3e0 0x7ff214107c30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 --2- 172.21.15.70:0/193786950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21406fd80 0x7ff2141061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 >> 172.21.15.70:0/193786950 conn(0x7ff2140690e0 msgr2=0x7ff21410e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:42.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 shutdown_connections 2024-03-23T08:29:42.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.887+0000 7ff2193b7700 1 -- 172.21.15.70:0/193786950 wait complete. 2024-03-23T08:29:42.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.988+0000 7f9037e54700 1 -- 172.21.15.70:0/4180765828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 msgr2=0x7f9030106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.988+0000 7f9037e54700 1 --2- 172.21.15.70:0/4180765828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 0x7f9030106f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9024004940 tx=0x7f9024009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:42.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.988+0000 7f9037e54700 1 -- 172.21.15.70:0/4180765828 shutdown_connections 2024-03-23T08:29:42.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.988+0000 7f9037e54700 1 --2- 172.21.15.70:0/4180765828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 0x7f9030106f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.988+0000 7f9037e54700 1 --2- 172.21.15.70:0/4180765828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90300fde90 0x7f90300fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.988+0000 7f9037e54700 1 -- 172.21.15.70:0/4180765828 >> 172.21.15.70:0/4180765828 conn(0x7f9030072bc0 msgr2=0x7f9030072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:42.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.988+0000 7f9037e54700 1 -- 172.21.15.70:0/4180765828 shutdown_connections 2024-03-23T08:29:42.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.988+0000 7f9037e54700 1 -- 172.21.15.70:0/4180765828 wait complete. 2024-03-23T08:29:42.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f9037e54700 1 Processor -- start 2024-03-23T08:29:42.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f9037e54700 1 -- start start 2024-03-23T08:29:42.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f9037e54700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90300fde90 0x7f9030113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:42.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f9037e54700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 0x7f9030114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:42.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f9037e54700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9030114670 con 0x7f90300fe830 2024-03-23T08:29:42.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f9037e54700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903010da60 con 0x7f90300fde90 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f90353ef700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 0x7f9030114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f90353ef700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 0x7f9030114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40874/0 (socket says 172.21.15.70:40874) 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f90353ef700 1 -- 172.21.15.70:0/2574162051 learned_addr learned my addr 172.21.15.70:0/2574162051 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.989+0000 7f9035bf0700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90300fde90 0x7f9030113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.990+0000 7f90353ef700 1 -- 172.21.15.70:0/2574162051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90300fde90 msgr2=0x7f9030113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.990+0000 7f90353ef700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90300fde90 0x7f9030113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.990+0000 7f90353ef700 1 -- 172.21.15.70:0/2574162051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9024005040 con 0x7f90300fe830 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.990+0000 7f90353ef700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 0x7f9030114020 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9024000c00 tx=0x7f9024009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.990+0000 7f90227fc700 1 -- 172.21.15.70:0/2574162051 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f902400e040 con 0x7f90300fe830 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.990+0000 7f90227fc700 1 -- 172.21.15.70:0/2574162051 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f902400bed0 con 0x7f90300fe830 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.990+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903010dce0 con 0x7f90300fe830 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.990+0000 7f90227fc700 1 -- 172.21.15.70:0/2574162051 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f902401c3f0 con 0x7f90300fe830 2024-03-23T08:29:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.990+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903010e220 con 0x7f90300fe830 2024-03-23T08:29:42.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.991+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9030065e30 con 0x7f90300fe830 2024-03-23T08:29:42.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.991+0000 7f90227fc700 1 -- 172.21.15.70:0/2574162051 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9024013420 con 0x7f90300fe830 2024-03-23T08:29:42.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.992+0000 7f90227fc700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901c06cc30 0x7f901c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:42.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.992+0000 7f90227fc700 1 -- 172.21.15.70:0/2574162051 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f902408d300 con 0x7f90300fe830 2024-03-23T08:29:42.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.992+0000 7f9035bf0700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901c06cc30 0x7f901c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:42.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.992+0000 7f9035bf0700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901c06cc30 0x7f901c06f0e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f902c004f40 tx=0x7f902c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:42.996+0000 7f90227fc700 1 -- 172.21.15.70:0/2574162051 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f902405af10 con 0x7f90300fe830 2024-03-23T08:29:43.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.226+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9030109dd0 con 0x7f901c06cc30 2024-03-23T08:29:43.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.236+0000 7f90227fc700 1 -- 172.21.15.70:0/2574162051 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9030109dd0 con 0x7f901c06cc30 2024-03-23T08:29:43.235 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:29:43.235 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (3m) 33s ago 14m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:29:43.235 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (64s) 33s ago 14m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:29:43.235 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (59s) 55s ago 13m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:29:43.235 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (2m) 33s ago 14m 79.8M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:29:43.235 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (8m) 33s ago 8m 147M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (9m) 33s ago 9m 14.1M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (8m) 55s ago 8m 97.4M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (8m) 55s ago 8m 15.3M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (5m) 33s ago 15m 494M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (4m) 55s ago 13m 447M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (2m) 33s ago 16m 44.3M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (100s) 55s ago 13m 34.9M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (4m) 33s ago 14m 18.5M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (3m) 55s ago 13m 16.2M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (38s) 33s ago 12m 10.3M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 starting - - - 1568M 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (11m) 33s ago 11m 203M 1568M 17.2.7-726-g9c4e0c34 837124da804f 077797cff6ef 2024-03-23T08:29:43.236 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (11m) 55s ago 11m 279M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:29:43.237 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (10m) 55s ago 10m 242M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:29:43.237 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (10m) 55s ago 10m 225M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:29:43.237 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (3m) 33s ago 14m 60.0M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:29:43.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901c06cc30 msgr2=0x7f901c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:43.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901c06cc30 0x7f901c06f0e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f902c004f40 tx=0x7f902c00b040 comp rx=0 tx=0).stop 2024-03-23T08:29:43.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 msgr2=0x7f9030114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:43.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 0x7f9030114020 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9024000c00 tx=0x7f9024009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:43.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 shutdown_connections 2024-03-23T08:29:43.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901c06cc30 0x7f901c06f0e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90300fde90 0x7f9030113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 --2- 172.21.15.70:0/2574162051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90300fe830 0x7f9030114020 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 >> 172.21.15.70:0/2574162051 conn(0x7f9030072bc0 msgr2=0x7f9030103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:43.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 shutdown_connections 2024-03-23T08:29:43.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.238+0000 7f9037e54700 1 -- 172.21.15.70:0/2574162051 wait complete. 2024-03-23T08:29:43.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.345+0000 7fde49949700 1 -- 172.21.15.70:0/1104972671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fde20 msgr2=0x7fde440fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:43.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.345+0000 7fde49949700 1 --2- 172.21.15.70:0/1104972671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fde20 0x7fde440fe1f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fde34004970 tx=0x7fde34009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:43.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.345+0000 7fde49949700 1 -- 172.21.15.70:0/1104972671 shutdown_connections 2024-03-23T08:29:43.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.345+0000 7fde49949700 1 --2- 172.21.15.70:0/1104972671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde440fe7c0 0x7fde44106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.345+0000 7fde49949700 1 --2- 172.21.15.70:0/1104972671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fde20 0x7fde440fe1f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.345+0000 7fde49949700 1 -- 172.21.15.70:0/1104972671 >> 172.21.15.70:0/1104972671 conn(0x7fde440f7880 msgr2=0x7fde440f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:43.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.345+0000 7fde49949700 1 -- 172.21.15.70:0/1104972671 shutdown_connections 2024-03-23T08:29:43.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.345+0000 7fde49949700 1 -- 172.21.15.70:0/1104972671 wait complete. 2024-03-23T08:29:43.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.346+0000 7fde49949700 1 Processor -- start 2024-03-23T08:29:43.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.346+0000 7fde49949700 1 -- start start 2024-03-23T08:29:43.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.346+0000 7fde49949700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde440fde20 0x7fde44113db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:43.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.346+0000 7fde49949700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fe7c0 0x7fde441142f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.346+0000 7fde49949700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde44114940 con 0x7fde440fe7c0 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.346+0000 7fde49949700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4410dd10 con 0x7fde440fde20 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde427fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fe7c0 0x7fde441142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde427fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fe7c0 0x7fde441142f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40880/0 (socket says 172.21.15.70:40880) 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde427fc700 1 -- 172.21.15.70:0/2764270047 learned_addr learned my addr 172.21.15.70:0/2764270047 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde42ffd700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde440fde20 0x7fde44113db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde427fc700 1 -- 172.21.15.70:0/2764270047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde440fde20 msgr2=0x7fde44113db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde427fc700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde440fde20 0x7fde44113db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde427fc700 1 -- 172.21.15.70:0/2764270047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34005040 con 0x7fde440fe7c0 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde427fc700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fe7c0 0x7fde441142f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c8c0 tx=0x7fde380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde2b7fe700 1 -- 172.21.15.70:0/2764270047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde3800a980 con 0x7fde440fe7c0 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4410dff0 con 0x7fde440fe7c0 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde2b7fe700 1 -- 172.21.15.70:0/2764270047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fde3800ce60 con 0x7fde440fe7c0 2024-03-23T08:29:43.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde2b7fe700 1 -- 172.21.15.70:0/2764270047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde380113a0 con 0x7fde440fe7c0 2024-03-23T08:29:43.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.347+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4410e530 con 0x7fde440fe7c0 2024-03-23T08:29:43.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.349+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde440666d0 con 0x7fde440fe7c0 2024-03-23T08:29:43.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.350+0000 7fde2b7fe700 1 -- 172.21.15.70:0/2764270047 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fde38012400 con 0x7fde440fe7c0 2024-03-23T08:29:43.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.350+0000 7fde2b7fe700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fde2c06cc30 0x7fde2c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:43.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.350+0000 7fde2b7fe700 1 -- 172.21.15.70:0/2764270047 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fde3808c0b0 con 0x7fde440fe7c0 2024-03-23T08:29:43.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.350+0000 7fde42ffd700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fde2c06cc30 0x7fde2c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:43.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.351+0000 7fde42ffd700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fde2c06cc30 0x7fde2c06f0e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fde34009fd0 tx=0x7fde34009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:43.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.356+0000 7fde2b7fe700 1 -- 172.21.15.70:0/2764270047 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fde38059c20 con 0x7fde440fe7c0 2024-03-23T08:29:43.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.654+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fde4404bed0 con 0x7fde440fe7c0 2024-03-23T08:29:43.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.655+0000 7fde2b7fe700 1 -- 172.21.15.70:0/2764270047 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7fde380597b0 con 0x7fde440fe7c0 2024-03-23T08:29:43.654 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:29:43.654 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:29:43.654 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:29:43.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:29:43.654 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:29:43.654 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:29:43.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 8, 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 5 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:29:43.655 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:29:43.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fde2c06cc30 msgr2=0x7fde2c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:43.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fde2c06cc30 0x7fde2c06f0e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fde34009fd0 tx=0x7fde34009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:43.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fe7c0 msgr2=0x7fde441142f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:43.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fe7c0 0x7fde441142f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c8c0 tx=0x7fde380044c0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 shutdown_connections 2024-03-23T08:29:43.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fde2c06cc30 0x7fde2c06f0e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde440fde20 0x7fde44113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 --2- 172.21.15.70:0/2764270047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde440fe7c0 0x7fde441142f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 >> 172.21.15.70:0/2764270047 conn(0x7fde440f7880 msgr2=0x7fde440f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 shutdown_connections 2024-03-23T08:29:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.657+0000 7fde49949700 1 -- 172.21.15.70:0/2764270047 wait complete. 2024-03-23T08:29:43.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.757+0000 7f700d22a700 1 -- 172.21.15.70:0/862364146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 msgr2=0x7f7008102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:43.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.757+0000 7f700d22a700 1 --2- 172.21.15.70:0/862364146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 0x7f7008102140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8004940 tx=0x7f6ff8009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:43.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.757+0000 7f700d22a700 1 -- 172.21.15.70:0/862364146 shutdown_connections 2024-03-23T08:29:43.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.757+0000 7f700d22a700 1 --2- 172.21.15.70:0/862364146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7008102710 0x7f70080fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.757+0000 7f700d22a700 1 --2- 172.21.15.70:0/862364146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 0x7f7008102140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.757+0000 7f700d22a700 1 -- 172.21.15.70:0/862364146 >> 172.21.15.70:0/862364146 conn(0x7f7008072bc0 msgr2=0x7f7008072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:43.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.757+0000 7f700d22a700 1 -- 172.21.15.70:0/862364146 shutdown_connections 2024-03-23T08:29:43.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.757+0000 7f700d22a700 1 -- 172.21.15.70:0/862364146 wait complete. 2024-03-23T08:29:43.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f700d22a700 1 Processor -- start 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f700d22a700 1 -- start start 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f700d22a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 0x7f7008113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f700d22a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7008102710 0x7f7008114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f700d22a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008114650 con 0x7f7008101d70 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f700d22a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700810da40 con 0x7f7008102710 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f7006d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 0x7f7008113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f7006d9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 0x7f7008113ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40904/0 (socket says 172.21.15.70:40904) 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f7006d9d700 1 -- 172.21.15.70:0/211607501 learned_addr learned my addr 172.21.15.70:0/211607501 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f700659c700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7008102710 0x7f7008114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f7006d9d700 1 -- 172.21.15.70:0/211607501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7008102710 msgr2=0x7f7008114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f7006d9d700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7008102710 0x7f7008114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f7006d9d700 1 -- 172.21.15.70:0/211607501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff8005040 con 0x7f7008101d70 2024-03-23T08:29:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.758+0000 7f700659c700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7008102710 0x7f7008114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:29:43.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.759+0000 7f7006d9d700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 0x7f7008113ac0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b3b0 tx=0x7f6ff8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:43.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.759+0000 7f6fef7fe700 1 -- 172.21.15.70:0/211607501 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff800e040 con 0x7f7008101d70 2024-03-23T08:29:43.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.759+0000 7f6fef7fe700 1 -- 172.21.15.70:0/211607501 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6ff8013420 con 0x7f7008101d70 2024-03-23T08:29:43.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.759+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700810dcc0 con 0x7f7008101d70 2024-03-23T08:29:43.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.759+0000 7f6fef7fe700 1 -- 172.21.15.70:0/211607501 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff801c630 con 0x7f7008101d70 2024-03-23T08:29:43.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.759+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700810e280 con 0x7f7008101d70 2024-03-23T08:29:43.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.760+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70080fd4c0 con 0x7f7008101d70 2024-03-23T08:29:43.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.760+0000 7f6fef7fe700 1 -- 172.21.15.70:0/211607501 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6ff8013970 con 0x7f7008101d70 2024-03-23T08:29:43.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.761+0000 7f6fef7fe700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6ff006cc20 0x7f6ff006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:43.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.761+0000 7f6fef7fe700 1 -- 172.21.15.70:0/211607501 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6ff808d1e0 con 0x7f7008101d70 2024-03-23T08:29:43.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.761+0000 7f700659c700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6ff006cc20 0x7f6ff006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:43.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.761+0000 7f700659c700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6ff006cc20 0x7f6ff006f0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f700810ec00 tx=0x7f6ffc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:43.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:43.765+0000 7f6fef7fe700 1 -- 172.21.15.70:0/211607501 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6ff805ad50 con 0x7f7008101d70 2024-03-23T08:29:44.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.034+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f700804bed0 con 0x7f7008101d70 2024-03-23T08:29:44.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.035+0000 7f6fef7fe700 1 -- 172.21.15.70:0/211607501 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f6ff805a8e0 con 0x7f7008101d70 2024-03-23T08:29:44.034 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:29:44.034 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:29:44.034 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:29:44.034 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:29:44.034 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:29:44.035 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:29:44.036 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:29:44.037 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:29:44.037 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:29:44.037 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:29:44.037 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:29:44.037 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:29:44.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6ff006cc20 msgr2=0x7f6ff006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6ff006cc20 0x7f6ff006f0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f700810ec00 tx=0x7f6ffc009040 comp rx=0 tx=0).stop 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 msgr2=0x7f7008113ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 0x7f7008113ac0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b3b0 tx=0x7f6ff8009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 shutdown_connections 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6ff006cc20 0x7f6ff006f0d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7008101d70 0x7f7008113ac0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 --2- 172.21.15.70:0/211607501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7008102710 0x7f7008114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 >> 172.21.15.70:0/211607501 conn(0x7f7008072bc0 msgr2=0x7f70080659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 shutdown_connections 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.038+0000 7f700d22a700 1 -- 172.21.15.70:0/211607501 wait complete. 2024-03-23T08:29:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:29:44.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:43 smithi193 ceph-mon[108533]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:44.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:43 smithi193 ceph-mon[108533]: pgmap v68: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:44.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:43 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2764270047' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.148+0000 7f0266bff700 1 -- 172.21.15.70:0/3074011076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 msgr2=0x7f0260103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.148+0000 7f0266bff700 1 --2- 172.21.15.70:0/3074011076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 0x7f0260103340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0254004940 tx=0x7f0254009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.148+0000 7f0266bff700 1 -- 172.21.15.70:0/3074011076 shutdown_connections 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.148+0000 7f0266bff700 1 --2- 172.21.15.70:0/3074011076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02600fb3b0 0x7f02600fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.148+0000 7f0266bff700 1 --2- 172.21.15.70:0/3074011076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 0x7f0260103340 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.148+0000 7f0266bff700 1 -- 172.21.15.70:0/3074011076 >> 172.21.15.70:0/3074011076 conn(0x7f0260072310 msgr2=0x7f0260072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.148+0000 7f0266bff700 1 -- 172.21.15.70:0/3074011076 shutdown_connections 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.148+0000 7f0266bff700 1 -- 172.21.15.70:0/3074011076 wait complete. 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f0266bff700 1 Processor -- start 2024-03-23T08:29:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f0266bff700 1 -- start start 2024-03-23T08:29:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f0266bff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02600fb3b0 0x7f0260113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f0266bff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 0x7f0260113d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f0266bff700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601143b0 con 0x7f0260102f70 2024-03-23T08:29:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f0266bff700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026010d9b0 con 0x7f02600fb3b0 2024-03-23T08:29:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f025ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 0x7f0260113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f025ffff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 0x7f0260113d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40914/0 (socket says 172.21.15.70:40914) 2024-03-23T08:29:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f025ffff700 1 -- 172.21.15.70:0/4248686533 learned_addr learned my addr 172.21.15.70:0/4248686533 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:44.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.149+0000 7f026499b700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02600fb3b0 0x7f0260113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.150+0000 7f025ffff700 1 -- 172.21.15.70:0/4248686533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02600fb3b0 msgr2=0x7f0260113820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.150+0000 7f025ffff700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02600fb3b0 0x7f0260113820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.150+0000 7f025ffff700 1 -- 172.21.15.70:0/4248686533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0254005040 con 0x7f0260102f70 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.150+0000 7f025ffff700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 0x7f0260113d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c8c0 tx=0x7f024c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.150+0000 7f025d7fa700 1 -- 172.21.15.70:0/4248686533 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f024c00a760 con 0x7f0260102f70 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.150+0000 7f025d7fa700 1 -- 172.21.15.70:0/4248686533 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f024c00ad70 con 0x7f0260102f70 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.150+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026010dc90 con 0x7f0260102f70 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.150+0000 7f025d7fa700 1 -- 172.21.15.70:0/4248686533 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f024c0113a0 con 0x7f0260102f70 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.150+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026010e250 con 0x7f0260102f70 2024-03-23T08:29:44.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.151+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02600fe6d0 con 0x7f0260102f70 2024-03-23T08:29:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.152+0000 7f025d7fa700 1 -- 172.21.15.70:0/4248686533 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f024c012400 con 0x7f0260102f70 2024-03-23T08:29:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.152+0000 7f025d7fa700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f025006cc20 0x7f025006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.152+0000 7f025d7fa700 1 -- 172.21.15.70:0/4248686533 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f024c08bfe0 con 0x7f0260102f70 2024-03-23T08:29:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.152+0000 7f026499b700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f025006cc20 0x7f025006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:44.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.152+0000 7f026499b700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f025006cc20 0x7f025006f0d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0254004560 tx=0x7f0254009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:44.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.156+0000 7f025d7fa700 1 -- 172.21.15.70:0/4248686533 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f024c059b50 con 0x7f0260102f70 2024-03-23T08:29:44.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:43 smithi070 ceph-mon[145869]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:43 smithi070 ceph-mon[145869]: pgmap v68: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:43 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2764270047' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:44.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.376+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02601134f0 con 0x7f025006cc20 2024-03-23T08:29:44.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.378+0000 7f025d7fa700 1 -- 172.21.15.70:0/4248686533 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f02601134f0 con 0x7f025006cc20 2024-03-23T08:29:44.377 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:29:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:29:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:29:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:29:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:29:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:29:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-03-23T08:29:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-03-23T08:29:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:29:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "7/21 daemons upgraded", 2024-03-23T08:29:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-03-23T08:29:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:29:44.378 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:29:44.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f025006cc20 msgr2=0x7f025006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f025006cc20 0x7f025006f0d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0254004560 tx=0x7f0254009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:44.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 msgr2=0x7f0260113d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 0x7f0260113d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c8c0 tx=0x7f024c0044c0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 shutdown_connections 2024-03-23T08:29:44.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f025006cc20 0x7f025006f0d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02600fb3b0 0x7f0260113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 --2- 172.21.15.70:0/4248686533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0260102f70 0x7f0260113d60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 >> 172.21.15.70:0/4248686533 conn(0x7f0260072310 msgr2=0x7f02600f9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:44.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 shutdown_connections 2024-03-23T08:29:44.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.380+0000 7f0266bff700 1 -- 172.21.15.70:0/4248686533 wait complete. 2024-03-23T08:29:44.488 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.489+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/2408592738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 msgr2=0x7fd4f4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.488 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.489+0000 7fd4fa8f2700 1 --2- 172.21.15.70:0/2408592738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 0x7fd4f4106f00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8004940 tx=0x7fd4e8009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:44.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.489+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/2408592738 shutdown_connections 2024-03-23T08:29:44.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.489+0000 7fd4fa8f2700 1 --2- 172.21.15.70:0/2408592738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 0x7fd4f4106f00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.489+0000 7fd4fa8f2700 1 --2- 172.21.15.70:0/2408592738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4f40fde90 0x7fd4f40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.489+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/2408592738 >> 172.21.15.70:0/2408592738 conn(0x7fd4f4072bc0 msgr2=0x7fd4f4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:44.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.489+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/2408592738 shutdown_connections 2024-03-23T08:29:44.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.489+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/2408592738 wait complete. 2024-03-23T08:29:44.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.489+0000 7fd4fa8f2700 1 Processor -- start 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4fa8f2700 1 -- start start 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4fa8f2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4f40fde90 0x7fd4f4115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4fa8f2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 0x7fd4f4116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4fa8f2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f41168c0 con 0x7fd4f40fe830 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4fa8f2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f410fc90 con 0x7fd4f40fde90 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4f37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 0x7fd4f4116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4f37fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 0x7fd4f4116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40942/0 (socket says 172.21.15.70:40942) 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4f37fe700 1 -- 172.21.15.70:0/3267286224 learned_addr learned my addr 172.21.15.70:0/3267286224 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4f3fff700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4f40fde90 0x7fd4f4115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4f37fe700 1 -- 172.21.15.70:0/3267286224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4f40fde90 msgr2=0x7fd4f4115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4f37fe700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4f40fde90 0x7fd4f4115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.490+0000 7fd4f37fe700 1 -- 172.21.15.70:0/3267286224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e8005040 con 0x7fd4f40fe830 2024-03-23T08:29:44.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.491+0000 7fd4f37fe700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 0x7fd4f4116270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800b3b0 tx=0x7fd4e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:44.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.491+0000 7fd4f0ff9700 1 -- 172.21.15.70:0/3267286224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e800e040 con 0x7fd4f40fe830 2024-03-23T08:29:44.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.491+0000 7fd4f0ff9700 1 -- 172.21.15.70:0/3267286224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4e8013420 con 0x7fd4f40fe830 2024-03-23T08:29:44.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.491+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f410ff10 con 0x7fd4f40fe830 2024-03-23T08:29:44.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.491+0000 7fd4f0ff9700 1 -- 172.21.15.70:0/3267286224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e801c5d0 con 0x7fd4f40fe830 2024-03-23T08:29:44.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.491+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f4110450 con 0x7fd4f40fe830 2024-03-23T08:29:44.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.492+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f4106a20 con 0x7fd4f40fe830 2024-03-23T08:29:44.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.492+0000 7fd4f0ff9700 1 -- 172.21.15.70:0/3267286224 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd4e8013590 con 0x7fd4f40fe830 2024-03-23T08:29:44.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.493+0000 7fd4f0ff9700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd4dc06cc20 0x7fd4dc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:29:44.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.493+0000 7fd4f0ff9700 1 -- 172.21.15.70:0/3267286224 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd4e808d210 con 0x7fd4f40fe830 2024-03-23T08:29:44.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.493+0000 7fd4f3fff700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd4dc06cc20 0x7fd4dc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:29:44.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.493+0000 7fd4f3fff700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd4dc06cc20 0x7fd4dc06f0d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4004d60 tx=0x7fd4e400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:29:44.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.500+0000 7fd4f0ff9700 1 -- 172.21.15.70:0/3267286224 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd4e805ad00 con 0x7fd4f40fe830 2024-03-23T08:29:44.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:44 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[166514]: 2024-03-23T08:29:44.648+0000 7f77e7523640 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-23T08:29:44.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.839+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd4f41113d0 con 0x7fd4f40fe830 2024-03-23T08:29:44.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.839+0000 7fd4f0ff9700 1 -- 172.21.15.70:0/3267286224 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+936 (secure 0 0 0) 0x7fd4e805a890 con 0x7fd4f40fe830 2024-03-23T08:29:44.838 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 27/222 objects degraded (12.162%), 12 pgs degraded 2024-03-23T08:29:44.838 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: osd.1 (root=default,host=smithi070) is down 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 27/222 objects degraded (12.162%), 12 pgs degraded 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-23T08:29:44.839 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-23T08:29:44.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd4dc06cc20 msgr2=0x7fd4dc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd4dc06cc20 0x7fd4dc06f0d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4004d60 tx=0x7fd4e400b040 comp rx=0 tx=0).stop 2024-03-23T08:29:44.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 msgr2=0x7fd4f4116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:29:44.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 0x7fd4f4116270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800b3b0 tx=0x7fd4e8009e30 comp rx=0 tx=0).stop 2024-03-23T08:29:44.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 shutdown_connections 2024-03-23T08:29:44.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd4dc06cc20 0x7fd4dc06f0d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4f40fde90 0x7fd4f4115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 --2- 172.21.15.70:0/3267286224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f40fe830 0x7fd4f4116270 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:29:44.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 >> 172.21.15.70:0/3267286224 conn(0x7fd4f4072bc0 msgr2=0x7fd4f40fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:29:44.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 shutdown_connections 2024-03-23T08:29:44.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:29:44.841+0000 7fd4fa8f2700 1 -- 172.21.15.70:0/3267286224 wait complete. 2024-03-23T08:29:45.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:44 smithi193 ceph-mon[108533]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:45.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:44 smithi193 ceph-mon[108533]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:45.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:44 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/211607501' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:29:45.199 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:44 smithi070 ceph-mon[145869]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:45.199 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:44 smithi070 ceph-mon[145869]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:45.199 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:44 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/211607501' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:29:45.483 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:45 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[166514]: 2024-03-23T08:29:45.198+0000 7f77e7523640 -1 osd.1 56 log_to_monitors true 2024-03-23T08:29:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:45 smithi193 ceph-mon[108533]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:45 smithi193 ceph-mon[108533]: pgmap v69: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:45 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3267286224' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:29:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:45 smithi193 ceph-mon[108533]: from='osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T08:29:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:46.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:45 smithi070 ceph-mon[145869]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:29:46.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:45 smithi070 ceph-mon[145869]: pgmap v69: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:46.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:45 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3267286224' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:29:46.089 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:46.089 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:46.089 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:45 smithi070 ceph-mon[145869]: from='osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-23T08:29:46.089 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:46.089 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:46.867 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:29:46 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[166514]: 2024-03-23T08:29:46.582+0000 7f77d4974700 -1 osd.1 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:29:47.130 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:46 smithi070 ceph-mon[145869]: from='osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T08:29:47.131 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:46 smithi070 ceph-mon[145869]: osdmap e59: 6 total, 5 up, 6 in 2024-03-23T08:29:47.131 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:46 smithi070 ceph-mon[145869]: from='osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:29:47.131 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:46 smithi070 ceph-mon[145869]: pgmap v71: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:47.131 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:46 smithi070 ceph-mon[145869]: from='osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063]' entity='osd.1' 2024-03-23T08:29:47.131 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:47.131 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:29:47.131 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:46 smithi070 ceph-mon[145869]: Health check update: Degraded data redundancy: 27/222 objects degraded (12.162%), 12 pgs degraded (PG_DEGRADED) 2024-03-23T08:29:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:46 smithi193 ceph-mon[108533]: from='osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-23T08:29:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:46 smithi193 ceph-mon[108533]: osdmap e59: 6 total, 5 up, 6 in 2024-03-23T08:29:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:46 smithi193 ceph-mon[108533]: from='osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:29:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:46 smithi193 ceph-mon[108533]: pgmap v71: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:46 smithi193 ceph-mon[108533]: from='osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063]' entity='osd.1' 2024-03-23T08:29:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:47.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:29:47.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:46 smithi193 ceph-mon[108533]: Health check update: Degraded data redundancy: 27/222 objects degraded (12.162%), 12 pgs degraded (PG_DEGRADED) 2024-03-23T08:29:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:47 smithi070 ceph-mon[145869]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:29:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:47 smithi070 ceph-mon[145869]: osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063] boot 2024-03-23T08:29:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:47 smithi070 ceph-mon[145869]: osdmap e60: 6 total, 6 up, 6 in 2024-03-23T08:29:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:29:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:47 smithi193 ceph-mon[108533]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:29:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:47 smithi193 ceph-mon[108533]: osd.1 [v2:172.21.15.70:6810/1925098063,v1:172.21.15.70:6811/1925098063] boot 2024-03-23T08:29:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:47 smithi193 ceph-mon[108533]: osdmap e60: 6 total, 6 up, 6 in 2024-03-23T08:29:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:47 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-23T08:29:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: pgmap v74: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: osdmap e61: 6 total, 6 up, 6 in 2024-03-23T08:29:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:29:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:29:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:49.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:29:49.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:49.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:49.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:49.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:49 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-23T08:29:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: pgmap v74: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: osdmap e61: 6 total, 6 up, 6 in 2024-03-23T08:29:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:29:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:29:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:29:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:29:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:29:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:49 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-23T08:29:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:50 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-23T08:29:50.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:50 smithi070 ceph-mon[145869]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-23T08:29:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:50 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-23T08:29:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:50 smithi193 ceph-mon[108533]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-23T08:29:51.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:51 smithi070 ceph-mon[145869]: pgmap v75: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:51 smithi193 ceph-mon[108533]: pgmap v75: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 27/222 objects degraded (12.162%) 2024-03-23T08:29:53.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:52 smithi193 ceph-mon[108533]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/222 objects degraded (12.162%), 12 pgs degraded) 2024-03-23T08:29:53.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:52 smithi193 ceph-mon[108533]: Cluster is now healthy 2024-03-23T08:29:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:52 smithi070 ceph-mon[145869]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/222 objects degraded (12.162%), 12 pgs degraded) 2024-03-23T08:29:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:52 smithi070 ceph-mon[145869]: Cluster is now healthy 2024-03-23T08:29:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:53 smithi193 ceph-mon[108533]: pgmap v76: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-23T08:29:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:53 smithi070 ceph-mon[145869]: pgmap v76: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-23T08:29:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:55 smithi193 ceph-mon[108533]: pgmap v77: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-23T08:29:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:55 smithi070 ceph-mon[145869]: pgmap v77: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-23T08:29:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:57 smithi193 ceph-mon[108533]: pgmap v78: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 454 B/s rd, 0 op/s 2024-03-23T08:29:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:57 smithi070 ceph-mon[145869]: pgmap v78: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 454 B/s rd, 0 op/s 2024-03-23T08:30:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:29:59 smithi193 ceph-mon[108533]: pgmap v79: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-23T08:30:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:29:59 smithi070 ceph-mon[145869]: pgmap v79: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-23T08:30:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:00 smithi193 ceph-mon[108533]: overall HEALTH_OK 2024-03-23T08:30:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:00 smithi070 ceph-mon[145869]: overall HEALTH_OK 2024-03-23T08:30:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:01 smithi070 ceph-mon[145869]: pgmap v80: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:30:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:30:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:01 smithi193 ceph-mon[108533]: pgmap v80: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:30:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:30:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:04.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:03 smithi070 ceph-mon[145869]: pgmap v81: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:30:04.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:03 smithi193 ceph-mon[108533]: pgmap v81: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:30:05.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-23T08:30:05.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-23T08:30:06.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:05 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-23T08:30:06.469 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:05 smithi070 ceph-mon[145869]: Upgrade: osd.2 is safe to restart 2024-03-23T08:30:06.469 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:05 smithi070 ceph-mon[145869]: pgmap v82: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:30:06.470 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:05 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-23T08:30:06.470 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:05 smithi193 ceph-mon[108533]: Upgrade: osd.2 is safe to restart 2024-03-23T08:30:06.470 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:05 smithi193 ceph-mon[108533]: pgmap v82: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:30:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:06 smithi070 ceph-mon[145869]: Upgrade: Updating osd.2 2024-03-23T08:30:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:06 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:06 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T08:30:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:06 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:30:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:06 smithi070 ceph-mon[145869]: Deploying daemon osd.2 on smithi070 2024-03-23T08:30:06.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:06 smithi070 ceph-mon[145869]: pgmap v83: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:30:07.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:06 smithi193 ceph-mon[108533]: Upgrade: Updating osd.2 2024-03-23T08:30:07.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:06 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:07.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:06 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-23T08:30:07.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:06 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:30:07.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:06 smithi193 ceph-mon[108533]: Deploying daemon osd.2 on smithi070 2024-03-23T08:30:07.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:06 smithi193 ceph-mon[108533]: pgmap v83: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:30:09.639 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:09 smithi070 systemd[1]: Stopping Ceph osd.2 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:30:09.639 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:09 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[82465]: 2024-03-23T08:30:09.493+0000 7f3066899700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:30:09.639 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:09 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[82465]: 2024-03-23T08:30:09.493+0000 7f3066899700 -1 osd.2 61 *** Got signal Terminated *** 2024-03-23T08:30:09.639 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:09 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[82465]: 2024-03-23T08:30:09.493+0000 7f3066899700 -1 osd.2 61 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:30:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:09 smithi070 ceph-mon[145869]: pgmap v84: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:30:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:09 smithi070 ceph-mon[145869]: osd.2 marked itself down and dead 2024-03-23T08:30:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:09 smithi193 ceph-mon[108533]: pgmap v84: 65 pgs: 65 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:30:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:09 smithi193 ceph-mon[108533]: osd.2 marked itself down and dead 2024-03-23T08:30:10.640 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:10 smithi070 bash[172588]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2 2024-03-23T08:30:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:10 smithi070 ceph-mon[145869]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:30:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:10 smithi070 ceph-mon[145869]: osdmap e62: 6 total, 5 up, 6 in 2024-03-23T08:30:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:10 smithi193 ceph-mon[108533]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:30:11.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:10 smithi193 ceph-mon[108533]: osdmap e62: 6 total, 5 up, 6 in 2024-03-23T08:30:11.889 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:11 smithi070 ceph-mon[145869]: pgmap v86: 65 pgs: 9 stale+active+clean, 56 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-23T08:30:11.889 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:11 smithi070 ceph-mon[145869]: osdmap e63: 6 total, 5 up, 6 in 2024-03-23T08:30:11.976 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:11 smithi193 ceph-mon[108533]: pgmap v86: 65 pgs: 9 stale+active+clean, 56 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-23T08:30:11.976 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:11 smithi193 ceph-mon[108533]: osdmap e63: 6 total, 5 up, 6 in 2024-03-23T08:30:12.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:12 smithi070 systemd[1]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.2.service: Succeeded. 2024-03-23T08:30:12.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:12 smithi070 systemd[1]: Stopped Ceph osd.2 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:30:12.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:12 smithi070 systemd[1]: Starting Ceph osd.2 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:30:13.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:12 smithi193 ceph-mon[108533]: Health check failed: Degraded data redundancy: 29/222 objects degraded (13.063%), 12 pgs degraded (PG_DEGRADED) 2024-03-23T08:30:13.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:12 smithi070 ceph-mon[145869]: Health check failed: Degraded data redundancy: 29/222 objects degraded (13.063%), 12 pgs degraded (PG_DEGRADED) 2024-03-23T08:30:13.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:13 smithi070 bash[172855]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-23T08:30:13.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:13 smithi070 bash[172855]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-23T08:30:13.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:13 smithi070 bash[172855]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-23T08:30:13.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:13 smithi070 bash[172855]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-23T08:30:13.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:13 smithi070 bash[172855]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-23T08:30:13.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:13 smithi070 bash[172855]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-23T08:30:13.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:13 smithi070 bash[172855]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-23T08:30:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:13 smithi193 ceph-mon[108533]: pgmap v88: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:13 smithi070 ceph-mon[145869]: pgmap v88: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.950+0000 7fbd7f25f700 1 -- 172.21.15.70:0/1859154477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd781037a0 msgr2=0x7fbd780fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.950+0000 7fbd7f25f700 1 --2- 172.21.15.70:0/1859154477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd781037a0 0x7fbd780fce10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004940 tx=0x7fbd64009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.951+0000 7fbd7f25f700 1 -- 172.21.15.70:0/1859154477 shutdown_connections 2024-03-23T08:30:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.951+0000 7fbd7f25f700 1 --2- 172.21.15.70:0/1859154477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd781037a0 0x7fbd780fce10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.951+0000 7fbd7f25f700 1 --2- 172.21.15.70:0/1859154477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd78102e00 0x7fbd781031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.951+0000 7fbd7f25f700 1 -- 172.21.15.70:0/1859154477 >> 172.21.15.70:0/1859154477 conn(0x7fbd780f89b0 msgr2=0x7fbd780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.951+0000 7fbd7f25f700 1 -- 172.21.15.70:0/1859154477 shutdown_connections 2024-03-23T08:30:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.951+0000 7fbd7f25f700 1 -- 172.21.15.70:0/1859154477 wait complete. 2024-03-23T08:30:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.951+0000 7fbd7f25f700 1 Processor -- start 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.951+0000 7fbd7f25f700 1 -- start start 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.951+0000 7fbd7f25f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd78102e00 0x7fbd7806f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7f25f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd781037a0 0x7fbd7806b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7f25f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd78070050 con 0x7fbd78102e00 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7f25f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7806be30 con 0x7fbd781037a0 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7cffb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd78102e00 0x7fbd7806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7cffb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd78102e00 0x7fbd7806f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54356/0 (socket says 172.21.15.70:54356) 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7cffb700 1 -- 172.21.15.70:0/2775117857 learned_addr learned my addr 172.21.15.70:0/2775117857 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd77fff700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd781037a0 0x7fbd7806b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7cffb700 1 -- 172.21.15.70:0/2775117857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd781037a0 msgr2=0x7fbd7806b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7cffb700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd781037a0 0x7fbd7806b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7cffb700 1 -- 172.21.15.70:0/2775117857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64005040 con 0x7fbd78102e00 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7cffb700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd78102e00 0x7fbd7806f970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b790 tx=0x7fbd6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd757fa700 1 -- 172.21.15.70:0/2775117857 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd6c00f6f0 con 0x7fbd78102e00 2024-03-23T08:30:14.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd757fa700 1 -- 172.21.15.70:0/2775117857 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd6c00fd00 con 0x7fbd78102e00 2024-03-23T08:30:14.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7806c0b0 con 0x7fbd78102e00 2024-03-23T08:30:14.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd757fa700 1 -- 172.21.15.70:0/2775117857 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd6c0103a0 con 0x7fbd78102e00 2024-03-23T08:30:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.952+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7806c640 con 0x7fbd78102e00 2024-03-23T08:30:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.954+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd780fe550 con 0x7fbd78102e00 2024-03-23T08:30:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.954+0000 7fbd757fa700 1 -- 172.21.15.70:0/2775117857 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbd6c00f850 con 0x7fbd78102e00 2024-03-23T08:30:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.954+0000 7fbd757fa700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbd6806cc20 0x7fbd6806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.954+0000 7fbd757fa700 1 -- 172.21.15.70:0/2775117857 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbd6c08bd20 con 0x7fbd78102e00 2024-03-23T08:30:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.954+0000 7fbd77fff700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbd6806cc20 0x7fbd6806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:14.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.955+0000 7fbd77fff700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbd6806cc20 0x7fbd6806f0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004890 tx=0x7fbd64009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:14.963+0000 7fbd757fa700 1 -- 172.21.15.70:0/2775117857 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbd6c055ea0 con 0x7fbd78102e00 2024-03-23T08:30:14.982 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:14 smithi070 ceph-osd[173154]: -- 172.21.15.70:0/3256287340 <== mon.0 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a4d38dd1e0 con 0x55a4d38acc00 2024-03-23T08:30:14.983 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:14 smithi070 bash[173117]: 73e36e62349150a524b8cc7a4db5f590026a9822764af3fb08127bf0ede7f743 2024-03-23T08:30:14.983 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:14 smithi070 systemd[1]: Started Ceph osd.2 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:30:15.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.207+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd7804bf40 con 0x7fbd6806cc20 2024-03-23T08:30:15.207 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.208+0000 7fbd757fa700 1 -- 172.21.15.70:0/2775117857 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbd7804bf40 con 0x7fbd6806cc20 2024-03-23T08:30:15.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.210+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbd6806cc20 msgr2=0x7fbd6806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.210+0000 7fbd7f25f700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbd6806cc20 0x7fbd6806f0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004890 tx=0x7fbd64009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.210+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd78102e00 msgr2=0x7fbd7806f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.210+0000 7fbd7f25f700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd78102e00 0x7fbd7806f970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b790 tx=0x7fbd6c009250 comp rx=0 tx=0).stop 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.211+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 shutdown_connections 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.211+0000 7fbd7f25f700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbd6806cc20 0x7fbd6806f0d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.211+0000 7fbd7f25f700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd78102e00 0x7fbd7806f970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.211+0000 7fbd7f25f700 1 --2- 172.21.15.70:0/2775117857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd781037a0 0x7fbd7806b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.211+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 >> 172.21.15.70:0/2775117857 conn(0x7fbd780f89b0 msgr2=0x7fbd780f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.211+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 shutdown_connections 2024-03-23T08:30:15.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.211+0000 7fbd7f25f700 1 -- 172.21.15.70:0/2775117857 wait complete. 2024-03-23T08:30:15.220 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.313+0000 7f776d6b1700 1 -- 172.21.15.70:0/2387447231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 msgr2=0x7f7768102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.313+0000 7f776d6b1700 1 --2- 172.21.15.70:0/2387447231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 0x7f7768102140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f775c004940 tx=0x7f775c009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.313+0000 7f776d6b1700 1 -- 172.21.15.70:0/2387447231 shutdown_connections 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.313+0000 7f776d6b1700 1 --2- 172.21.15.70:0/2387447231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7768102710 0x7f77680fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.313+0000 7f776d6b1700 1 --2- 172.21.15.70:0/2387447231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 0x7f7768102140 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.313+0000 7f776d6b1700 1 -- 172.21.15.70:0/2387447231 >> 172.21.15.70:0/2387447231 conn(0x7f7768072bc0 msgr2=0x7f7768072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.313+0000 7f776d6b1700 1 -- 172.21.15.70:0/2387447231 shutdown_connections 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.313+0000 7f776d6b1700 1 -- 172.21.15.70:0/2387447231 wait complete. 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f776d6b1700 1 Processor -- start 2024-03-23T08:30:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f776d6b1700 1 -- start start 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f776d6b1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 0x7f77681129a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f776d6b1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7768102710 0x7f7768112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f776d6b1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7768113530 con 0x7f7768101d70 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f776d6b1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776810c920 con 0x7f7768102710 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f7766ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 0x7f77681129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f7766ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 0x7f77681129a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54366/0 (socket says 172.21.15.70:54366) 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f7766ffd700 1 -- 172.21.15.70:0/1390155721 learned_addr learned my addr 172.21.15.70:0/1390155721 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f77667fc700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7768102710 0x7f7768112ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f7766ffd700 1 -- 172.21.15.70:0/1390155721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7768102710 msgr2=0x7f7768112ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f7766ffd700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7768102710 0x7f7768112ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f7766ffd700 1 -- 172.21.15.70:0/1390155721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c005040 con 0x7f7768101d70 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f7766ffd700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 0x7f77681129a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f775c000c00 tx=0x7f775c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f774f7fe700 1 -- 172.21.15.70:0/1390155721 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775c00e040 con 0x7f7768101d70 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f774f7fe700 1 -- 172.21.15.70:0/1390155721 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f775c013420 con 0x7f7768101d70 2024-03-23T08:30:15.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.314+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776810cba0 con 0x7f7768101d70 2024-03-23T08:30:15.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.315+0000 7f774f7fe700 1 -- 172.21.15.70:0/1390155721 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775c01c630 con 0x7f7768101d70 2024-03-23T08:30:15.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.315+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776810d160 con 0x7f7768101d70 2024-03-23T08:30:15.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.316+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77680fd4c0 con 0x7f7768101d70 2024-03-23T08:30:15.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.316+0000 7f774f7fe700 1 -- 172.21.15.70:0/1390155721 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f775c00bb50 con 0x7f7768101d70 2024-03-23T08:30:15.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.316+0000 7f774f7fe700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f775006cc20 0x7f775006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:15.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.316+0000 7f774f7fe700 1 -- 172.21.15.70:0/1390155721 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f775c08d1e0 con 0x7f7768101d70 2024-03-23T08:30:15.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.316+0000 7f77667fc700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f775006cc20 0x7f775006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:15.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.317+0000 7f77667fc700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f775006cc20 0x7f775006f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f776810dae0 tx=0x7f775800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:15.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.321+0000 7f774f7fe700 1 -- 172.21.15.70:0/1390155721 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f775c05ad50 con 0x7f7768101d70 2024-03-23T08:30:15.483 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:15 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[173131]: 2024-03-23T08:30:15.184+0000 7f5f99db2640 -1 Falling back to public interface 2024-03-23T08:30:15.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.541+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7768108dd0 con 0x7f775006cc20 2024-03-23T08:30:15.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.542+0000 7f774f7fe700 1 -- 172.21.15.70:0/1390155721 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f775800f040 con 0x7f775006cc20 2024-03-23T08:30:15.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f775006cc20 msgr2=0x7f775006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f775006cc20 0x7f775006f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f776810dae0 tx=0x7f775800b040 comp rx=0 tx=0).stop 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 msgr2=0x7f77681129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 0x7f77681129a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f775c000c00 tx=0x7f775c009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 shutdown_connections 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f775006cc20 0x7f775006f0d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7768101d70 0x7f77681129a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 --2- 172.21.15.70:0/1390155721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7768102710 0x7f7768112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 >> 172.21.15.70:0/1390155721 conn(0x7f7768072bc0 msgr2=0x7f77680fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 shutdown_connections 2024-03-23T08:30:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.544+0000 7f776d6b1700 1 -- 172.21.15.70:0/1390155721 wait complete. 2024-03-23T08:30:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.647+0000 7fa325d26700 1 -- 172.21.15.70:0/2458284940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 msgr2=0x7fa320106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.647+0000 7fa325d26700 1 --2- 172.21.15.70:0/2458284940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 0x7fa320106f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa314004940 tx=0x7fa314009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.647+0000 7fa325d26700 1 -- 172.21.15.70:0/2458284940 shutdown_connections 2024-03-23T08:30:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.647+0000 7fa325d26700 1 --2- 172.21.15.70:0/2458284940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 0x7fa320106f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.647+0000 7fa325d26700 1 --2- 172.21.15.70:0/2458284940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3200fde90 0x7fa3200fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.647+0000 7fa325d26700 1 -- 172.21.15.70:0/2458284940 >> 172.21.15.70:0/2458284940 conn(0x7fa320072bc0 msgr2=0x7fa320072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:15.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.647+0000 7fa325d26700 1 -- 172.21.15.70:0/2458284940 shutdown_connections 2024-03-23T08:30:15.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.648+0000 7fa325d26700 1 -- 172.21.15.70:0/2458284940 wait complete. 2024-03-23T08:30:15.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.648+0000 7fa325d26700 1 Processor -- start 2024-03-23T08:30:15.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.648+0000 7fa325d26700 1 -- start start 2024-03-23T08:30:15.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.648+0000 7fa325d26700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3200fde90 0x7fa320113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:15.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.648+0000 7fa325d26700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 0x7fa320113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:15.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.648+0000 7fa325d26700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa320114630 con 0x7fa3200fe830 2024-03-23T08:30:15.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.648+0000 7fa325d26700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32010da20 con 0x7fa3200fde90 2024-03-23T08:30:15.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa31effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 0x7fa320113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa31effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 0x7fa320113fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54388/0 (socket says 172.21.15.70:54388) 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa31effd700 1 -- 172.21.15.70:0/3869374902 learned_addr learned my addr 172.21.15.70:0/3869374902 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa31f7fe700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3200fde90 0x7fa320113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa31effd700 1 -- 172.21.15.70:0/3869374902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3200fde90 msgr2=0x7fa320113aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa31effd700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3200fde90 0x7fa320113aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa31effd700 1 -- 172.21.15.70:0/3869374902 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa314005040 con 0x7fa3200fe830 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa31effd700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 0x7fa320113fe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3140134a0 tx=0x7fa314009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa307fff700 1 -- 172.21.15.70:0/3869374902 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31400e070 con 0x7fa3200fe830 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa307fff700 1 -- 172.21.15.70:0/3869374902 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa314009af0 con 0x7fa3200fe830 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32010dca0 con 0x7fa3200fe830 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa307fff700 1 -- 172.21.15.70:0/3869374902 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31401ccd0 con 0x7fa3200fe830 2024-03-23T08:30:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.649+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32010e1e0 con 0x7fa3200fe830 2024-03-23T08:30:15.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.650+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa320101980 con 0x7fa3200fe830 2024-03-23T08:30:15.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.651+0000 7fa307fff700 1 -- 172.21.15.70:0/3869374902 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa314013950 con 0x7fa3200fe830 2024-03-23T08:30:15.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.651+0000 7fa307fff700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa30806cc20 0x7fa30806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:15.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.651+0000 7fa307fff700 1 -- 172.21.15.70:0/3869374902 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa31408e0b0 con 0x7fa3200fe830 2024-03-23T08:30:15.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.652+0000 7fa31f7fe700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa30806cc20 0x7fa30806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:15.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.652+0000 7fa31f7fe700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa30806cc20 0x7fa30806f0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa310009b80 tx=0x7fa31000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:15.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.655+0000 7fa307fff700 1 -- 172.21.15.70:0/3869374902 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa31405bcc0 con 0x7fa3200fe830 2024-03-23T08:30:15.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.864+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa320113770 con 0x7fa30806cc20 2024-03-23T08:30:15.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.874+0000 7fa307fff700 1 -- 172.21.15.70:0/3869374902 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7fa320113770 con 0x7fa30806cc20 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (3m) 31s ago 14m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (96s) 31s ago 15m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (92s) 88s ago 14m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (3m) 31s ago 14m 81.9M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (9m) 31s ago 9m 146M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (9m) 31s ago 9m 14.2M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (9m) 88s ago 9m 97.4M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (9m) 88s ago 9m 15.3M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (5m) 31s ago 16m 496M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (4m) 88s ago 14m 447M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (2m) 31s ago 16m 47.5M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (2m) 88s ago 14m 34.9M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:30:15.874 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (4m) 31s ago 15m 18.7M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:30:15.875 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (4m) 88s ago 14m 16.2M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:30:15.875 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (71s) 31s ago 13m 84.9M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:30:15.875 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (36s) 31s ago 12m 10.7M 1568M 18.2.2-954-g82316b04 b24e1baba637 13fce63c86eb 2024-03-23T08:30:15.875 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 starting - - - 1568M 2024-03-23T08:30:15.875 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (11m) 88s ago 11m 279M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:30:15.875 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (11m) 88s ago 11m 242M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:30:15.875 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (11m) 88s ago 11m 225M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:30:15.875 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (4m) 31s ago 14m 60.0M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.876+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa30806cc20 msgr2=0x7fa30806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.876+0000 7fa325d26700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa30806cc20 0x7fa30806f0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa310009b80 tx=0x7fa31000b040 comp rx=0 tx=0).stop 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.876+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 msgr2=0x7fa320113fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.876+0000 7fa325d26700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 0x7fa320113fe0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3140134a0 tx=0x7fa314009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.877+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 shutdown_connections 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.877+0000 7fa325d26700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa30806cc20 0x7fa30806f0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.877+0000 7fa325d26700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3200fde90 0x7fa320113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.877+0000 7fa325d26700 1 --2- 172.21.15.70:0/3869374902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3200fe830 0x7fa320113fe0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.877+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 >> 172.21.15.70:0/3869374902 conn(0x7fa320072bc0 msgr2=0x7fa3200ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:15.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.877+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 shutdown_connections 2024-03-23T08:30:15.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.877+0000 7fa325d26700 1 -- 172.21.15.70:0/3869374902 wait complete. 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.995+0000 7fcefacfd700 1 -- 172.21.15.70:0/1266725964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 msgr2=0x7fcef4109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.995+0000 7fcefacfd700 1 --2- 172.21.15.70:0/1266725964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 0x7fcef4109150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcee00048b0 tx=0x7fcee0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.995+0000 7fcefacfd700 1 -- 172.21.15.70:0/1266725964 shutdown_connections 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.995+0000 7fcefacfd700 1 --2- 172.21.15.70:0/1266725964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 0x7fcef4109150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.995+0000 7fcefacfd700 1 --2- 172.21.15.70:0/1266725964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcef4070300 0x7fcef40706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.995+0000 7fcefacfd700 1 -- 172.21.15.70:0/1266725964 >> 172.21.15.70:0/1266725964 conn(0x7fcef40f8950 msgr2=0x7fcef40fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.995+0000 7fcefacfd700 1 -- 172.21.15.70:0/1266725964 shutdown_connections 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.995+0000 7fcefacfd700 1 -- 172.21.15.70:0/1266725964 wait complete. 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcefacfd700 1 Processor -- start 2024-03-23T08:30:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcefacfd700 1 -- start start 2024-03-23T08:30:15.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcefacfd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcef4070300 0x7fcef4115a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:15.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcefacfd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 0x7fcef4115f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:15.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcefacfd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef41165d0 con 0x7fcef4070c10 2024-03-23T08:30:15.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcefacfd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef410fbb0 con 0x7fcef4070300 2024-03-23T08:30:15.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcef3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 0x7fcef4115f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:15.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcef3fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 0x7fcef4115f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54410/0 (socket says 172.21.15.70:54410) 2024-03-23T08:30:15.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcef3fff700 1 -- 172.21.15.70:0/3398520760 learned_addr learned my addr 172.21.15.70:0/3398520760 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:15.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcef8a99700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcef4070300 0x7fcef4115a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:15.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcef3fff700 1 -- 172.21.15.70:0/3398520760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcef4070300 msgr2=0x7fcef4115a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcef3fff700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcef4070300 0x7fcef4115a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.996+0000 7fcef3fff700 1 -- 172.21.15.70:0/3398520760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee0005040 con 0x7fcef4070c10 2024-03-23T08:30:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.997+0000 7fcef3fff700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 0x7fcef4115f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009360 tx=0x7fcee0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.997+0000 7fcef17fa700 1 -- 172.21.15.70:0/3398520760 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcee000e040 con 0x7fcef4070c10 2024-03-23T08:30:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.997+0000 7fcef17fa700 1 -- 172.21.15.70:0/3398520760 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcee001c430 con 0x7fcef4070c10 2024-03-23T08:30:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.997+0000 7fcef17fa700 1 -- 172.21.15.70:0/3398520760 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcee0012600 con 0x7fcef4070c10 2024-03-23T08:30:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.997+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef410fe30 con 0x7fcef4070c10 2024-03-23T08:30:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.997+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef41102c0 con 0x7fcef4070c10 2024-03-23T08:30:15.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.998+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef4071a40 con 0x7fcef4070c10 2024-03-23T08:30:15.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.999+0000 7fcef17fa700 1 -- 172.21.15.70:0/3398520760 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcee001c5a0 con 0x7fcef4070c10 2024-03-23T08:30:15.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.999+0000 7fcef17fa700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcee406cc20 0x7fcee406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:15.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.999+0000 7fcef17fa700 1 -- 172.21.15.70:0/3398520760 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcee008c990 con 0x7fcef4070c10 2024-03-23T08:30:15.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:15.999+0000 7fcef8a99700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcee406cc20 0x7fcee406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:15.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.000+0000 7fcef8a99700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcee406cc20 0x7fcee406f0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcee80046c0 tx=0x7fcee800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:16.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.007+0000 7fcef17fa700 1 -- 172.21.15.70:0/3398520760 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcee005a3f0 con 0x7fcef4070c10 2024-03-23T08:30:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:15 smithi193 ceph-mon[108533]: pgmap v89: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:15 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:30:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:15 smithi070 ceph-mon[145869]: pgmap v89: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:15 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:30:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.291+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcef410bfa0 con 0x7fcef4070c10 2024-03-23T08:30:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.292+0000 7fcef17fa700 1 -- 172.21.15.70:0/3398520760 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7fcee0059f80 con 0x7fcef4070c10 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:30:16.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:30:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:30:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:30:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:30:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:30:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 7, 2024-03-23T08:30:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 6 2024-03-23T08:30:16.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:30:16.292 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcee406cc20 msgr2=0x7fcee406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcee406cc20 0x7fcee406f0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcee80046c0 tx=0x7fcee800b040 comp rx=0 tx=0).stop 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 msgr2=0x7fcef4115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 0x7fcef4115f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009360 tx=0x7fcee0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 shutdown_connections 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcee406cc20 0x7fcee406f0d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcef4070300 0x7fcef4115a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 --2- 172.21.15.70:0/3398520760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef4070c10 0x7fcef4115f80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 >> 172.21.15.70:0/3398520760 conn(0x7fcef40f8950 msgr2=0x7fcef4105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 shutdown_connections 2024-03-23T08:30:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.294+0000 7fcefacfd700 1 -- 172.21.15.70:0/3398520760 wait complete. 2024-03-23T08:30:16.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.395+0000 7f26f7975700 1 -- 172.21.15.70:0/4231119680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070c10 msgr2=0x7f26f0109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.395+0000 7f26f7975700 1 --2- 172.21.15.70:0/4231119680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070c10 0x7f26f0109180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26ec004970 tx=0x7f26ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:16.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.395+0000 7f26f7975700 1 -- 172.21.15.70:0/4231119680 shutdown_connections 2024-03-23T08:30:16.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.395+0000 7f26f7975700 1 --2- 172.21.15.70:0/4231119680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070c10 0x7f26f0109180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.395+0000 7f26f7975700 1 --2- 172.21.15.70:0/4231119680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f0070300 0x7f26f00706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.395+0000 7f26f7975700 1 -- 172.21.15.70:0/4231119680 >> 172.21.15.70:0/4231119680 conn(0x7f26f00f89a0 msgr2=0x7f26f00fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:16.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.395+0000 7f26f7975700 1 -- 172.21.15.70:0/4231119680 shutdown_connections 2024-03-23T08:30:16.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.395+0000 7f26f7975700 1 -- 172.21.15.70:0/4231119680 wait complete. 2024-03-23T08:30:16.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f7975700 1 Processor -- start 2024-03-23T08:30:16.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f7975700 1 -- start start 2024-03-23T08:30:16.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f7975700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070300 0x7f26f0115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:16.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f7975700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f0070c10 0x7f26f01161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:16.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f7975700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f01167f0 con 0x7f26f0070300 2024-03-23T08:30:16.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f7975700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f010fbc0 con 0x7f26f0070c10 2024-03-23T08:30:16.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f5711700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070300 0x7f26f0115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:16.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f5711700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070300 0x7f26f0115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54430/0 (socket says 172.21.15.70:54430) 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f5711700 1 -- 172.21.15.70:0/3833966810 learned_addr learned my addr 172.21.15.70:0/3833966810 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f4f10700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f0070c10 0x7f26f01161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f5711700 1 -- 172.21.15.70:0/3833966810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f0070c10 msgr2=0x7f26f01161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f5711700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f0070c10 0x7f26f01161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.396+0000 7f26f5711700 1 -- 172.21.15.70:0/3833966810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec005040 con 0x7f26f0070300 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.397+0000 7f26f5711700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070300 0x7f26f0115c60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f26e0002a50 tx=0x7f26e00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.397+0000 7f26e5ffb700 1 -- 172.21.15.70:0/3833966810 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e0007cc0 con 0x7f26f0070300 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.397+0000 7f26e5ffb700 1 -- 172.21.15.70:0/3833966810 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26e000f040 con 0x7f26f0070300 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.397+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f010fea0 con 0x7f26f0070300 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.397+0000 7f26e5ffb700 1 -- 172.21.15.70:0/3833966810 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e000d9c0 con 0x7f26f0070300 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.397+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f01103b0 con 0x7f26f0070300 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.398+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f0071a40 con 0x7f26f0070300 2024-03-23T08:30:16.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.398+0000 7f26e5ffb700 1 -- 172.21.15.70:0/3833966810 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f26e000db60 con 0x7f26f0070300 2024-03-23T08:30:16.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.399+0000 7f26e5ffb700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f26dc06cc20 0x7f26dc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:16.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.399+0000 7f26e5ffb700 1 -- 172.21.15.70:0/3833966810 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f26e008b560 con 0x7f26f0070300 2024-03-23T08:30:16.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.399+0000 7f26f4f10700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f26dc06cc20 0x7f26dc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:16.399 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.399+0000 7f26f4f10700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f26dc06cc20 0x7f26dc06f0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f26ec004f50 tx=0x7f26ec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:16.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.403+0000 7f26e5ffb700 1 -- 172.21.15.70:0/3833966810 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f26e00590d0 con 0x7f26f0070300 2024-03-23T08:30:16.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.644+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f26f004bf20 con 0x7f26f0070300 2024-03-23T08:30:16.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.644+0000 7f26e5ffb700 1 -- 172.21.15.70:0/3833966810 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f26e0058c60 con 0x7f26f0070300 2024-03-23T08:30:16.643 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:30:16.644 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:30:16.645 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:30:16.646 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:30:16.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f26dc06cc20 msgr2=0x7f26dc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f26dc06cc20 0x7f26dc06f0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f26ec004f50 tx=0x7f26ec009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:16.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070300 msgr2=0x7f26f0115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070300 0x7f26f0115c60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f26e0002a50 tx=0x7f26e00046b0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 shutdown_connections 2024-03-23T08:30:16.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f26dc06cc20 0x7f26dc06f0d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0070300 0x7f26f0115c60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 --2- 172.21.15.70:0/3833966810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f0070c10 0x7f26f01161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 >> 172.21.15.70:0/3833966810 conn(0x7f26f00f89a0 msgr2=0x7f26f00f9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:16.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 shutdown_connections 2024-03-23T08:30:16.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.647+0000 7f26f7975700 1 -- 172.21.15.70:0/3833966810 wait complete. 2024-03-23T08:30:16.648 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:30:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.758+0000 7f68d122f700 1 -- 172.21.15.70:0/1872384032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 msgr2=0x7f68cc10d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.758+0000 7f68d122f700 1 --2- 172.21.15.70:0/1872384032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 0x7f68cc10d420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f68c0004970 tx=0x7f68c0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.758+0000 7f68d122f700 1 -- 172.21.15.70:0/1872384032 shutdown_connections 2024-03-23T08:30:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.758+0000 7f68d122f700 1 --2- 172.21.15.70:0/1872384032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 0x7f68cc10d420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.758+0000 7f68d122f700 1 --2- 172.21.15.70:0/1872384032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68cc0fb720 0x7f68cc0fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.758+0000 7f68d122f700 1 -- 172.21.15.70:0/1872384032 >> 172.21.15.70:0/1872384032 conn(0x7f68cc0714c0 msgr2=0x7f68cc0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.758+0000 7f68d122f700 1 -- 172.21.15.70:0/1872384032 shutdown_connections 2024-03-23T08:30:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.758+0000 7f68d122f700 1 -- 172.21.15.70:0/1872384032 wait complete. 2024-03-23T08:30:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68d122f700 1 Processor -- start 2024-03-23T08:30:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68d122f700 1 -- start start 2024-03-23T08:30:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68d122f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68cc0fb720 0x7f68cc119f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68d122f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 0x7f68cc11a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68d122f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc11ab30 con 0x7f68cc0fc030 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68d122f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc114080 con 0x7f68cc0fb720 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68ca59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 0x7f68cc11a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68ca59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 0x7f68cc11a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54452/0 (socket says 172.21.15.70:54452) 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68ca59c700 1 -- 172.21.15.70:0/3184628385 learned_addr learned my addr 172.21.15.70:0/3184628385 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68cad9d700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68cc0fb720 0x7f68cc119f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68ca59c700 1 -- 172.21.15.70:0/3184628385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68cc0fb720 msgr2=0x7f68cc119f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68ca59c700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68cc0fb720 0x7f68cc119f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68ca59c700 1 -- 172.21.15.70:0/3184628385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0005040 con 0x7f68cc0fc030 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.759+0000 7f68cad9d700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68cc0fb720 0x7f68cc119f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.760+0000 7f68ca59c700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 0x7f68cc11a450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009150 tx=0x7f68c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.760+0000 7f68b37fe700 1 -- 172.21.15.70:0/3184628385 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68c000e040 con 0x7f68cc0fc030 2024-03-23T08:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.760+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68cc1142b0 con 0x7f68cc0fc030 2024-03-23T08:30:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.760+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc114790 con 0x7f68cc0fc030 2024-03-23T08:30:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.760+0000 7f68b37fe700 1 -- 172.21.15.70:0/3184628385 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f68c0013420 con 0x7f68cc0fc030 2024-03-23T08:30:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.760+0000 7f68b37fe700 1 -- 172.21.15.70:0/3184628385 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68c0025640 con 0x7f68cc0fc030 2024-03-23T08:30:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.762+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc0fe0a0 con 0x7f68cc0fc030 2024-03-23T08:30:16.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.763+0000 7f68b37fe700 1 -- 172.21.15.70:0/3184628385 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f68c00139c0 con 0x7f68cc0fc030 2024-03-23T08:30:16.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.763+0000 7f68b37fe700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f68b406cc20 0x7f68b406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:16.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.763+0000 7f68b37fe700 1 -- 172.21.15.70:0/3184628385 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f68c008d230 con 0x7f68cc0fc030 2024-03-23T08:30:16.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.769+0000 7f68cad9d700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f68b406cc20 0x7f68b406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:16.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.770+0000 7f68cad9d700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f68b406cc20 0x7f68b406f0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009b10 tx=0x7f68bc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:16.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.772+0000 7f68b37fe700 1 -- 172.21.15.70:0/3184628385 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f68c005adb0 con 0x7f68cc0fc030 2024-03-23T08:30:16.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.993+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68cc115600 con 0x7f68b406cc20 2024-03-23T08:30:16.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.994+0000 7f68b37fe700 1 -- 172.21.15.70:0/3184628385 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f68cc115600 con 0x7f68b406cc20 2024-03-23T08:30:16.993 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:30:16.993 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "8/21 daemons upgraded", 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:30:16.994 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:30:16.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f68b406cc20 msgr2=0x7f68b406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f68b406cc20 0x7f68b406f0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009b10 tx=0x7f68bc008040 comp rx=0 tx=0).stop 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 msgr2=0x7f68cc11a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 0x7f68cc11a450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009150 tx=0x7f68c0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 shutdown_connections 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f68b406cc20 0x7f68b406f0d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68cc0fb720 0x7f68cc119f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 --2- 172.21.15.70:0/3184628385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68cc0fc030 0x7f68cc11a450 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 >> 172.21.15.70:0/3184628385 conn(0x7f68cc0714c0 msgr2=0x7f68cc101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.996+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 shutdown_connections 2024-03-23T08:30:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:16.997+0000 7f68d122f700 1 -- 172.21.15.70:0/3184628385 wait complete. 2024-03-23T08:30:17.060 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:16 smithi070 ceph-mon[145869]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:17.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:16 smithi070 ceph-mon[145869]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:17.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:16 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3398520760' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:17.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:16 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:17.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:16 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:30:17.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:16 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3833966810' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:30:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:16 smithi193 ceph-mon[108533]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:16 smithi193 ceph-mon[108533]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:16 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3398520760' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:16 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:16 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:30:17.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:16 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3833966810' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:30:17.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.108+0000 7f6f59834700 1 -- 172.21.15.70:0/1893322017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 msgr2=0x7f6f540fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.108+0000 7f6f59834700 1 --2- 172.21.15.70:0/1893322017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 0x7f6f540fe260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f44004940 tx=0x7f6f44009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.108+0000 7f6f59834700 1 -- 172.21.15.70:0/1893322017 shutdown_connections 2024-03-23T08:30:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.108+0000 7f6f59834700 1 --2- 172.21.15.70:0/1893322017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f540fe830 0x7f6f54106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.108+0000 7f6f59834700 1 --2- 172.21.15.70:0/1893322017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 0x7f6f540fe260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.108+0000 7f6f59834700 1 -- 172.21.15.70:0/1893322017 >> 172.21.15.70:0/1893322017 conn(0x7f6f54072bc0 msgr2=0x7f6f54072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.108+0000 7f6f59834700 1 -- 172.21.15.70:0/1893322017 shutdown_connections 2024-03-23T08:30:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.108+0000 7f6f59834700 1 -- 172.21.15.70:0/1893322017 wait complete. 2024-03-23T08:30:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.109+0000 7f6f59834700 1 Processor -- start 2024-03-23T08:30:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.109+0000 7f6f59834700 1 -- start start 2024-03-23T08:30:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.109+0000 7f6f59834700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 0x7f6f54113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.109+0000 7f6f59834700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f540fe830 0x7f6f54114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.109+0000 7f6f59834700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f54114690 con 0x7f6f540fde90 2024-03-23T08:30:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.109+0000 7f6f59834700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5410da80 con 0x7f6f540fe830 2024-03-23T08:30:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f52ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 0x7f6f54113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f52ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 0x7f6f54113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54464/0 (socket says 172.21.15.70:54464) 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f52ffd700 1 -- 172.21.15.70:0/2554980738 learned_addr learned my addr 172.21.15.70:0/2554980738 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f527fc700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f540fe830 0x7f6f54114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f52ffd700 1 -- 172.21.15.70:0/2554980738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f540fe830 msgr2=0x7f6f54114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f52ffd700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f540fe830 0x7f6f54114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f52ffd700 1 -- 172.21.15.70:0/2554980738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f44005040 con 0x7f6f540fde90 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f52ffd700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 0x7f6f54113b00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009360 tx=0x7f6f44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f3b7fe700 1 -- 172.21.15.70:0/2554980738 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f4400e040 con 0x7f6f540fde90 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f3b7fe700 1 -- 172.21.15.70:0/2554980738 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f44013420 con 0x7f6f540fde90 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5410dd00 con 0x7f6f540fde90 2024-03-23T08:30:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f3b7fe700 1 -- 172.21.15.70:0/2554980738 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f4401c630 con 0x7f6f540fde90 2024-03-23T08:30:17.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.110+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5410e240 con 0x7f6f540fde90 2024-03-23T08:30:17.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.111+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f54065e30 con 0x7f6f540fde90 2024-03-23T08:30:17.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.112+0000 7f6f3b7fe700 1 -- 172.21.15.70:0/2554980738 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6f440139e0 con 0x7f6f540fde90 2024-03-23T08:30:17.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.112+0000 7f6f3b7fe700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6f3c06cc20 0x7f6f3c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:17.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.112+0000 7f6f3b7fe700 1 -- 172.21.15.70:0/2554980738 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6f4408d210 con 0x7f6f540fde90 2024-03-23T08:30:17.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.112+0000 7f6f527fc700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6f3c06cc20 0x7f6f3c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:17.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.113+0000 7f6f527fc700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6f3c06cc20 0x7f6f3c06f0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f5410ec40 tx=0x7f6f48009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:17.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.117+0000 7f6f3b7fe700 1 -- 172.21.15.70:0/2554980738 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6f4405ad80 con 0x7f6f540fde90 2024-03-23T08:30:17.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.398+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6f54109e20 con 0x7f6f540fde90 2024-03-23T08:30:17.398 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.399+0000 7f6f3b7fe700 1 -- 172.21.15.70:0/2554980738 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+940 (secure 0 0 0) 0x7f6f4405a910 con 0x7f6f540fde90 2024-03-23T08:30:17.398 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 29/222 objects degraded (13.063%), 12 pgs degraded 2024-03-23T08:30:17.398 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-23T08:30:17.398 INFO:teuthology.orchestra.run.smithi070.stdout: osd.2 (root=default,host=smithi070) is down 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 29/222 objects degraded (13.063%), 12 pgs degraded 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-03-23T08:30:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-03-23T08:30:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6f3c06cc20 msgr2=0x7f6f3c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6f3c06cc20 0x7f6f3c06f0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f5410ec40 tx=0x7f6f48009040 comp rx=0 tx=0).stop 2024-03-23T08:30:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 msgr2=0x7f6f54113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 0x7f6f54113b00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009360 tx=0x7f6f44009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 shutdown_connections 2024-03-23T08:30:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6f3c06cc20 0x7f6f3c06f0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f540fde90 0x7f6f54113b00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 --2- 172.21.15.70:0/2554980738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f540fe830 0x7f6f54114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 >> 172.21.15.70:0/2554980738 conn(0x7f6f54072bc0 msgr2=0x7f6f54103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:17.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 shutdown_connections 2024-03-23T08:30:17.401 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:17.401+0000 7f6f59834700 1 -- 172.21.15.70:0/2554980738 wait complete. 2024-03-23T08:30:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:17 smithi193 ceph-mon[108533]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:17 smithi193 ceph-mon[108533]: pgmap v90: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:17 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2554980738' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:30:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:17 smithi070 ceph-mon[145869]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:17 smithi070 ceph-mon[145869]: pgmap v90: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:17 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2554980738' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:30:18.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:18 smithi070 ceph-mon[145869]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:18 smithi193 ceph-mon[108533]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:19.483 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:19 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[173131]: 2024-03-23T08:30:19.135+0000 7f5f99db2640 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-23T08:30:19.803 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:19 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[173131]: 2024-03-23T08:30:19.543+0000 7f5f99db2640 -1 osd.2 61 log_to_monitors true 2024-03-23T08:30:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:19 smithi193 ceph-mon[108533]: pgmap v91: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:19 smithi193 ceph-mon[108533]: from='osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T08:30:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:19 smithi070 ceph-mon[145869]: pgmap v91: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:19 smithi070 ceph-mon[145869]: from='osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-23T08:30:20.804 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:30:20 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[173131]: 2024-03-23T08:30:20.541+0000 7f5f87203700 -1 osd.2 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:30:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:20 smithi193 ceph-mon[108533]: from='osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T08:30:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:20 smithi193 ceph-mon[108533]: osdmap e64: 6 total, 5 up, 6 in 2024-03-23T08:30:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:20 smithi193 ceph-mon[108533]: from='osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:30:21.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:20 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:21.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:20 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:21.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:20 smithi193 ceph-mon[108533]: from='osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512]' entity='osd.2' 2024-03-23T08:30:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:20 smithi070 ceph-mon[145869]: from='osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-23T08:30:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:20 smithi070 ceph-mon[145869]: osdmap e64: 6 total, 5 up, 6 in 2024-03-23T08:30:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:20 smithi070 ceph-mon[145869]: from='osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-03-23T08:30:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:20 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:20 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:20 smithi070 ceph-mon[145869]: from='osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512]' entity='osd.2' 2024-03-23T08:30:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:21 smithi070 ceph-mon[145869]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:21 smithi070 ceph-mon[145869]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:30:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:21 smithi070 ceph-mon[145869]: osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512] boot 2024-03-23T08:30:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:21 smithi070 ceph-mon[145869]: osdmap e65: 6 total, 6 up, 6 in 2024-03-23T08:30:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:30:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:21 smithi193 ceph-mon[108533]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29/222 objects degraded (13.063%) 2024-03-23T08:30:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:22.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:21 smithi193 ceph-mon[108533]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:30:22.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:21 smithi193 ceph-mon[108533]: osd.2 [v2:172.21.15.70:6818/357933512,v1:172.21.15.70:6819/357933512] boot 2024-03-23T08:30:22.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:21 smithi193 ceph-mon[108533]: osdmap e65: 6 total, 6 up, 6 in 2024-03-23T08:30:22.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-23T08:30:23.666 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:23 smithi070 ceph-mon[145869]: osdmap e66: 6 total, 6 up, 6 in 2024-03-23T08:30:23.666 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:23 smithi070 ceph-mon[145869]: pgmap v96: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 20/222 objects degraded (9.009%) 2024-03-23T08:30:23.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:23 smithi193 ceph-mon[108533]: osdmap e66: 6 total, 6 up, 6 in 2024-03-23T08:30:23.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:23 smithi193 ceph-mon[108533]: pgmap v96: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 20/222 objects degraded (9.009%) 2024-03-23T08:30:24.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: Health check update: Degraded data redundancy: 20/222 objects degraded (9.009%), 8 pgs degraded (PG_DEGRADED) 2024-03-23T08:30:24.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:24.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:24.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:30:24.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:30:24.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:24.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:30:24.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:24.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:24.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:24.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:24 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: Health check update: Degraded data redundancy: 20/222 objects degraded (9.009%), 8 pgs degraded (PG_DEGRADED) 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:24.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:24 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-23T08:30:25.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:25 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-23T08:30:25.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:25 smithi193 ceph-mon[108533]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-23T08:30:25.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:25 smithi193 ceph-mon[108533]: pgmap v97: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 20/222 objects degraded (9.009%) 2024-03-23T08:30:25.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:25 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-23T08:30:25.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:25 smithi070 ceph-mon[145869]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-23T08:30:25.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:25 smithi070 ceph-mon[145869]: pgmap v97: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 20/222 objects degraded (9.009%) 2024-03-23T08:30:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:26 smithi070 ceph-mon[145869]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/222 objects degraded (9.009%), 8 pgs degraded) 2024-03-23T08:30:26.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:26 smithi070 ceph-mon[145869]: Cluster is now healthy 2024-03-23T08:30:27.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:26 smithi193 ceph-mon[108533]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/222 objects degraded (9.009%), 8 pgs degraded) 2024-03-23T08:30:27.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:26 smithi193 ceph-mon[108533]: Cluster is now healthy 2024-03-23T08:30:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:27 smithi193 ceph-mon[108533]: pgmap v98: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 754 B/s rd, 0 op/s 2024-03-23T08:30:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:27 smithi070 ceph-mon[145869]: pgmap v98: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 754 B/s rd, 0 op/s 2024-03-23T08:30:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:29 smithi193 ceph-mon[108533]: pgmap v99: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-23T08:30:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:29 smithi070 ceph-mon[145869]: pgmap v99: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-23T08:30:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:31 smithi193 ceph-mon[108533]: pgmap v100: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-03-23T08:30:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:30:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:31 smithi070 ceph-mon[145869]: pgmap v100: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-03-23T08:30:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:30:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:34.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:33 smithi193 ceph-mon[108533]: pgmap v101: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-23T08:30:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:33 smithi070 ceph-mon[145869]: pgmap v101: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-23T08:30:36.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:35 smithi193 ceph-mon[108533]: pgmap v102: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-03-23T08:30:36.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:35 smithi070 ceph-mon[145869]: pgmap v102: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-03-23T08:30:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:37 smithi193 ceph-mon[108533]: pgmap v103: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-03-23T08:30:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:37 smithi070 ceph-mon[145869]: pgmap v103: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-03-23T08:30:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:39 smithi193 ceph-mon[108533]: pgmap v104: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:30:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:39 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-23T08:30:40.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:39 smithi070 ceph-mon[145869]: pgmap v104: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:30:40.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:39 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-23T08:30:41.010 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:40 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-23T08:30:41.010 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:40 smithi193 ceph-mon[108533]: Upgrade: osd.3 is safe to restart 2024-03-23T08:30:41.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:40 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-23T08:30:41.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:40 smithi070 ceph-mon[145869]: Upgrade: osd.3 is safe to restart 2024-03-23T08:30:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:41 smithi193 ceph-mon[108533]: pgmap v105: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:30:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T08:30:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:30:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:41 smithi070 ceph-mon[145869]: pgmap v105: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:30:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-23T08:30:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:30:43.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:42 smithi193 ceph-mon[108533]: Upgrade: Updating osd.3 2024-03-23T08:30:43.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:42 smithi193 ceph-mon[108533]: Deploying daemon osd.3 on smithi193 2024-03-23T08:30:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:42 smithi070 ceph-mon[145869]: Upgrade: Updating osd.3 2024-03-23T08:30:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:42 smithi070 ceph-mon[145869]: Deploying daemon osd.3 on smithi193 2024-03-23T08:30:44.051 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:43 smithi193 ceph-mon[108533]: pgmap v106: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:30:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:43 smithi070 ceph-mon[145869]: pgmap v106: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:30:44.305 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:44 smithi193 systemd[1]: Stopping Ceph osd.3 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:30:44.575 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:44 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[54804]: 2024-03-23T08:30:44.304+0000 7f153f687700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:30:44.575 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:44 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[54804]: 2024-03-23T08:30:44.304+0000 7f153f687700 -1 osd.3 66 *** Got signal Terminated *** 2024-03-23T08:30:44.575 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:44 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[54804]: 2024-03-23T08:30:44.304+0000 7f153f687700 -1 osd.3 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:30:45.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:44 smithi193 ceph-mon[108533]: osd.3 marked itself down and dead 2024-03-23T08:30:45.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:44 smithi070 ceph-mon[145869]: osd.3 marked itself down and dead 2024-03-23T08:30:46.075 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:45 smithi193 bash[120845]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3 2024-03-23T08:30:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:45 smithi193 ceph-mon[108533]: pgmap v107: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:30:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:45 smithi193 ceph-mon[108533]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:30:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:45 smithi193 ceph-mon[108533]: osdmap e67: 6 total, 5 up, 6 in 2024-03-23T08:30:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:45 smithi070 ceph-mon[145869]: pgmap v107: 65 pgs: 65 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:30:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:45 smithi070 ceph-mon[145869]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:30:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:45 smithi070 ceph-mon[145869]: osdmap e67: 6 total, 5 up, 6 in 2024-03-23T08:30:47.064 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:46 smithi193 ceph-mon[108533]: osdmap e68: 6 total, 5 up, 6 in 2024-03-23T08:30:47.065 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:47.065 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:30:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:46 smithi070 ceph-mon[145869]: osdmap e68: 6 total, 5 up, 6 in 2024-03-23T08:30:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.516+0000 7f52794d7700 1 -- 172.21.15.70:0/1547084386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5274101d20 msgr2=0x7f52741020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.516+0000 7f52794d7700 1 --2- 172.21.15.70:0/1547084386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5274101d20 0x7f52741020f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5264004940 tx=0x7f5264009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.516+0000 7f52794d7700 1 -- 172.21.15.70:0/1547084386 shutdown_connections 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.517+0000 7f52794d7700 1 --2- 172.21.15.70:0/1547084386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52741026c0 0x7f52740fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.517+0000 7f52794d7700 1 --2- 172.21.15.70:0/1547084386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5274101d20 0x7f52741020f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.517+0000 7f52794d7700 1 -- 172.21.15.70:0/1547084386 >> 172.21.15.70:0/1547084386 conn(0x7f52740f78d0 msgr2=0x7f52740f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.517+0000 7f52794d7700 1 -- 172.21.15.70:0/1547084386 shutdown_connections 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.517+0000 7f52794d7700 1 -- 172.21.15.70:0/1547084386 wait complete. 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.517+0000 7f52794d7700 1 Processor -- start 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.517+0000 7f52794d7700 1 -- start start 2024-03-23T08:30:47.518 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.517+0000 7f52794d7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5274101d20 0x7f527410e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52794d7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52741026c0 0x7f527410ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52794d7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527410f580 con 0x7f52741026c0 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52794d7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527410f6c0 con 0x7f5274101d20 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52727fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52741026c0 0x7f527410ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f5272ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5274101d20 0x7f527410e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52727fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52741026c0 0x7f527410ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40336/0 (socket says 172.21.15.70:40336) 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52727fc700 1 -- 172.21.15.70:0/3558805395 learned_addr learned my addr 172.21.15.70:0/3558805395 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52727fc700 1 -- 172.21.15.70:0/3558805395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5274101d20 msgr2=0x7f527410e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52727fc700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5274101d20 0x7f527410e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52727fc700 1 -- 172.21.15.70:0/3558805395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5264005040 con 0x7f52741026c0 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f5272ffd700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5274101d20 0x7f527410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52727fc700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52741026c0 0x7f527410ef30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f525c00eb40 tx=0x7f525c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f526b7fe700 1 -- 172.21.15.70:0/3558805395 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f525c007cc0 con 0x7f52741026c0 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5274117a60 con 0x7f52741026c0 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f526b7fe700 1 -- 172.21.15.70:0/3558805395 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f525c00f040 con 0x7f52741026c0 2024-03-23T08:30:47.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f526b7fe700 1 -- 172.21.15.70:0/3558805395 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f525c00c8f0 con 0x7f52741026c0 2024-03-23T08:30:47.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.518+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5274117f70 con 0x7f52741026c0 2024-03-23T08:30:47.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.519+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52740fd470 con 0x7f52741026c0 2024-03-23T08:30:47.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.520+0000 7f526b7fe700 1 -- 172.21.15.70:0/3558805395 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f525c00cb60 con 0x7f52741026c0 2024-03-23T08:30:47.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.520+0000 7f526b7fe700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f526006cc20 0x7f526006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:47.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.520+0000 7f526b7fe700 1 -- 172.21.15.70:0/3558805395 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f525c08b9e0 con 0x7f52741026c0 2024-03-23T08:30:47.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.520+0000 7f5272ffd700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f526006cc20 0x7f526006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:47.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.521+0000 7f5272ffd700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f526006cc20 0x7f526006f0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5264009fd0 tx=0x7f5264009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:47.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.525+0000 7f526b7fe700 1 -- 172.21.15.70:0/3558805395 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f525c055af0 con 0x7f52741026c0 2024-03-23T08:30:47.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.737+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f527404bed0 con 0x7f526006cc20 2024-03-23T08:30:47.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.738+0000 7f526b7fe700 1 -- 172.21.15.70:0/3558805395 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f527404bed0 con 0x7f526006cc20 2024-03-23T08:30:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f526006cc20 msgr2=0x7f526006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f526006cc20 0x7f526006f0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5264009fd0 tx=0x7f5264009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52741026c0 msgr2=0x7f527410ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52741026c0 0x7f527410ef30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f525c00eb40 tx=0x7f525c00b040 comp rx=0 tx=0).stop 2024-03-23T08:30:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 shutdown_connections 2024-03-23T08:30:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f526006cc20 0x7f526006f0d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5274101d20 0x7f527410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:47.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 --2- 172.21.15.70:0/3558805395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52741026c0 0x7f527410ef30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:47.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 >> 172.21.15.70:0/3558805395 conn(0x7f52740f78d0 msgr2=0x7f52740f8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:47.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 shutdown_connections 2024-03-23T08:30:47.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.740+0000 7f52794d7700 1 -- 172.21.15.70:0/3558805395 wait complete. 2024-03-23T08:30:47.749 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:30:47.812 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:47 smithi193 systemd[1]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.3.service: Succeeded. 2024-03-23T08:30:47.813 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:47 smithi193 systemd[1]: Stopped Ceph osd.3 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:30:47.813 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:47 smithi193 systemd[1]: Starting Ceph osd.3 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:30:47.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 -- 172.21.15.70:0/3727663058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff01037b0 msgr2=0x7f7ff00fb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:47.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 --2- 172.21.15.70:0/3727663058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff01037b0 0x7f7ff00fb3b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0004940 tx=0x7f7fe0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:47.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 -- 172.21.15.70:0/3727663058 shutdown_connections 2024-03-23T08:30:47.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 --2- 172.21.15.70:0/3727663058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff01037b0 0x7f7ff00fb3b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:47.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 --2- 172.21.15.70:0/3727663058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff0102e10 0x7f7ff01031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:47.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 -- 172.21.15.70:0/3727663058 >> 172.21.15.70:0/3727663058 conn(0x7f7ff0072310 msgr2=0x7f7ff0072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:47.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 -- 172.21.15.70:0/3727663058 shutdown_connections 2024-03-23T08:30:47.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 -- 172.21.15.70:0/3727663058 wait complete. 2024-03-23T08:30:47.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 Processor -- start 2024-03-23T08:30:47.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.844+0000 7f7ff83f8700 1 -- start start 2024-03-23T08:30:47.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff83f8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff0102e10 0x7f7ff0113890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:47.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff83f8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff01037b0 0x7f7ff0113dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:47.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff83f8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff0114420 con 0x7f7ff0102e10 2024-03-23T08:30:47.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff83f8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff010da20 con 0x7f7ff01037b0 2024-03-23T08:30:47.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff6194700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff0102e10 0x7f7ff0113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:47.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff6194700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff0102e10 0x7f7ff0113890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40352/0 (socket says 172.21.15.70:40352) 2024-03-23T08:30:47.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff6194700 1 -- 172.21.15.70:0/748450126 learned_addr learned my addr 172.21.15.70:0/748450126 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff5993700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff01037b0 0x7f7ff0113dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff6194700 1 -- 172.21.15.70:0/748450126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff01037b0 msgr2=0x7f7ff0113dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff6194700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff01037b0 0x7f7ff0113dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff6194700 1 -- 172.21.15.70:0/748450126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe0005040 con 0x7f7ff0102e10 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff6194700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff0102e10 0x7f7ff0113890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00eb70 tx=0x7f7fec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7fe6ffd700 1 -- 172.21.15.70:0/748450126 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fec007cc0 con 0x7f7ff0102e10 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff010dca0 con 0x7f7ff0102e10 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7fe6ffd700 1 -- 172.21.15.70:0/748450126 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7fec00f040 con 0x7f7ff0102e10 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7fe6ffd700 1 -- 172.21.15.70:0/748450126 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fec00c9d0 con 0x7f7ff0102e10 2024-03-23T08:30:47.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.845+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff010e1b0 con 0x7f7ff0102e10 2024-03-23T08:30:47.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.847+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff00fe6d0 con 0x7f7ff0102e10 2024-03-23T08:30:47.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.847+0000 7f7fe6ffd700 1 -- 172.21.15.70:0/748450126 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7fec00cc40 con 0x7f7ff0102e10 2024-03-23T08:30:47.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.847+0000 7f7fe6ffd700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7fdc06cc20 0x7f7fdc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:47.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.847+0000 7f7fe6ffd700 1 -- 172.21.15.70:0/748450126 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7fec08ba50 con 0x7f7ff0102e10 2024-03-23T08:30:47.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.847+0000 7f7ff5993700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7fdc06cc20 0x7f7fdc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:47.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.848+0000 7f7ff5993700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7fdc06cc20 0x7f7fdc06f0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000b0d0 tx=0x7f7fe0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:47.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:47.852+0000 7f7fe6ffd700 1 -- 172.21.15.70:0/748450126 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7fec0595f0 con 0x7f7ff0102e10 2024-03-23T08:30:48.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.063+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ff0109e40 con 0x7f7fdc06cc20 2024-03-23T08:30:48.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.064+0000 7f7fe6ffd700 1 -- 172.21.15.70:0/748450126 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7ff0109e40 con 0x7f7fdc06cc20 2024-03-23T08:30:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7fdc06cc20 msgr2=0x7f7fdc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7fdc06cc20 0x7f7fdc06f0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000b0d0 tx=0x7f7fe0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff0102e10 msgr2=0x7f7ff0113890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff0102e10 0x7f7ff0113890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00eb70 tx=0x7f7fec00b040 comp rx=0 tx=0).stop 2024-03-23T08:30:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 shutdown_connections 2024-03-23T08:30:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7fdc06cc20 0x7f7fdc06f0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ff0102e10 0x7f7ff0113890 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 --2- 172.21.15.70:0/748450126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff01037b0 0x7f7ff0113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 >> 172.21.15.70:0/748450126 conn(0x7f7ff0072310 msgr2=0x7f7ff00f9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:48.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 shutdown_connections 2024-03-23T08:30:48.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.066+0000 7f7ff83f8700 1 -- 172.21.15.70:0/748450126 wait complete. 2024-03-23T08:30:48.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:47 smithi193 ceph-mon[108533]: pgmap v110: 65 pgs: 8 active+undersized+degraded, 4 peering, 10 stale+active+clean, 8 active+undersized, 35 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 20/222 objects degraded (9.009%) 2024-03-23T08:30:48.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:47 smithi193 ceph-mon[108533]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-23T08:30:48.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:47 smithi193 ceph-mon[108533]: Health check failed: Degraded data redundancy: 20/222 objects degraded (9.009%), 8 pgs degraded (PG_DEGRADED) 2024-03-23T08:30:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 -- 172.21.15.70:0/4102901306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 msgr2=0x7fdde4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 --2- 172.21.15.70:0/4102901306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 0x7fdde4106f00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdde0004940 tx=0x7fdde0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 -- 172.21.15.70:0/4102901306 shutdown_connections 2024-03-23T08:30:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 --2- 172.21.15.70:0/4102901306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 0x7fdde4106f00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 --2- 172.21.15.70:0/4102901306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde40fde90 0x7fdde40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 -- 172.21.15.70:0/4102901306 >> 172.21.15.70:0/4102901306 conn(0x7fdde4072bc0 msgr2=0x7fdde4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 -- 172.21.15.70:0/4102901306 shutdown_connections 2024-03-23T08:30:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 -- 172.21.15.70:0/4102901306 wait complete. 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 Processor -- start 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.165+0000 7fddec6cc700 1 -- start start 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fddec6cc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde40fde90 0x7fdde4115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fddec6cc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 0x7fdde4116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fddec6cc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde41168c0 con 0x7fdde40fe830 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fddec6cc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde410fc90 con 0x7fdde40fde90 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdde9c67700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 0x7fdde4116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdde9c67700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 0x7fdde4116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40370/0 (socket says 172.21.15.70:40370) 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdde9c67700 1 -- 172.21.15.70:0/2597832525 learned_addr learned my addr 172.21.15.70:0/2597832525 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fddea468700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde40fde90 0x7fdde4115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdde9c67700 1 -- 172.21.15.70:0/2597832525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde40fde90 msgr2=0x7fdde4115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdde9c67700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde40fde90 0x7fdde4115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdde9c67700 1 -- 172.21.15.70:0/2597832525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0005040 con 0x7fdde40fe830 2024-03-23T08:30:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdde9c67700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 0x7fdde4116270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009360 tx=0x7fdde0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdddaffd700 1 -- 172.21.15.70:0/2597832525 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde000e040 con 0x7fdde40fe830 2024-03-23T08:30:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdddaffd700 1 -- 172.21.15.70:0/2597832525 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdde000bed0 con 0x7fdde40fe830 2024-03-23T08:30:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde410ff10 con 0x7fdde40fe830 2024-03-23T08:30:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fdddaffd700 1 -- 172.21.15.70:0/2597832525 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde001c3f0 con 0x7fdde40fe830 2024-03-23T08:30:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.166+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde4110450 con 0x7fdde40fe830 2024-03-23T08:30:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.167+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde4106a20 con 0x7fdde40fe830 2024-03-23T08:30:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.168+0000 7fdddaffd700 1 -- 172.21.15.70:0/2597832525 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fdde0013790 con 0x7fdde40fe830 2024-03-23T08:30:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.168+0000 7fdddaffd700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fddd006cc20 0x7fddd006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.168+0000 7fdddaffd700 1 -- 172.21.15.70:0/2597832525 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdde008d360 con 0x7fdde40fe830 2024-03-23T08:30:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.168+0000 7fddea468700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fddd006cc20 0x7fddd006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.169+0000 7fddea468700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fddd006cc20 0x7fddd006f0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fddd4004f40 tx=0x7fddd400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:48.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.173+0000 7fdddaffd700 1 -- 172.21.15.70:0/2597832525 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdde005ae50 con 0x7fdde40fe830 2024-03-23T08:30:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:47 smithi070 ceph-mon[145869]: pgmap v110: 65 pgs: 8 active+undersized+degraded, 4 peering, 10 stale+active+clean, 8 active+undersized, 35 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 20/222 objects degraded (9.009%) 2024-03-23T08:30:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:47 smithi070 ceph-mon[145869]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-23T08:30:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:47 smithi070 ceph-mon[145869]: Health check failed: Degraded data redundancy: 20/222 objects degraded (9.009%), 8 pgs degraded (PG_DEGRADED) 2024-03-23T08:30:48.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.374+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdde4111380 con 0x7fddd006cc20 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.384+0000 7fdddaffd700 1 -- 172.21.15.70:0/2597832525 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdde4111380 con 0x7fddd006cc20 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (4m) 28s ago 15m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (2m) 28s ago 15m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (2m) 2m ago 14m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (4m) 28s ago 15m 83.9M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (10m) 28s ago 10m 146M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (10m) 28s ago 10m 14.4M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (9m) 2m ago 9m 97.4M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (10m) 2m ago 10m 15.3M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (6m) 28s ago 17m 501M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:30:48.383 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (5m) 2m ago 14m 447M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (3m) 28s ago 17m 48.9M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (2m) 2m ago 14m 34.9M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (5m) 28s ago 15m 18.6M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (5m) 2m ago 14m 16.2M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (104s) 28s ago 13m 85.0M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (69s) 28s ago 13m 96.3M 1568M 18.2.2-954-g82316b04 b24e1baba637 13fce63c86eb 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (34s) 28s ago 12m 10.7M 1568M 18.2.2-954-g82316b04 b24e1baba637 73e36e623491 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (12m) 2m ago 12m 279M 2592M 17.2.7-726-g9c4e0c34 837124da804f 0d32e6f79a85 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (12m) 2m ago 12m 242M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (11m) 2m ago 11m 225M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:30:48.384 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (4m) 28s ago 15m 60.3M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fddd006cc20 msgr2=0x7fddd006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fddd006cc20 0x7fddd006f0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fddd4004f40 tx=0x7fddd400b040 comp rx=0 tx=0).stop 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 msgr2=0x7fdde4116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 0x7fdde4116270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009360 tx=0x7fdde0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 shutdown_connections 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fddd006cc20 0x7fddd006f0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdde40fde90 0x7fdde4115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 --2- 172.21.15.70:0/2597832525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdde40fe830 0x7fdde4116270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 >> 172.21.15.70:0/2597832525 conn(0x7fdde4072bc0 msgr2=0x7fdde40fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 shutdown_connections 2024-03-23T08:30:48.386 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.386+0000 7fddec6cc700 1 -- 172.21.15.70:0/2597832525 wait complete. 2024-03-23T08:30:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 -- 172.21.15.70:0/675949424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff81026b0 msgr2=0x7efff80fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 --2- 172.21.15.70:0/675949424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff81026b0 0x7efff80fbd20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7effe4004940 tx=0x7effe4009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 -- 172.21.15.70:0/675949424 shutdown_connections 2024-03-23T08:30:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 --2- 172.21.15.70:0/675949424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff81026b0 0x7efff80fbd20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 --2- 172.21.15.70:0/675949424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efff8101d10 0x7efff81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 -- 172.21.15.70:0/675949424 >> 172.21.15.70:0/675949424 conn(0x7efff80f7880 msgr2=0x7efff80f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 -- 172.21.15.70:0/675949424 shutdown_connections 2024-03-23T08:30:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 -- 172.21.15.70:0/675949424 wait complete. 2024-03-23T08:30:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 Processor -- start 2024-03-23T08:30:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.484+0000 7effff097700 1 -- start start 2024-03-23T08:30:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7effff097700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff8101d10 0x7efff810ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7effff097700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efff81026b0 0x7efff810f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7effff097700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff810f9c0 con 0x7efff8101d10 2024-03-23T08:30:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7effff097700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff810fb00 con 0x7efff81026b0 2024-03-23T08:30:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7efffce33700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff8101d10 0x7efff810ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7efffce33700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff8101d10 0x7efff810ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55434/0 (socket says 172.21.15.70:55434) 2024-03-23T08:30:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7efffce33700 1 -- 172.21.15.70:0/125034255 learned_addr learned my addr 172.21.15.70:0/125034255 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7efff7fff700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efff81026b0 0x7efff810f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7efffce33700 1 -- 172.21.15.70:0/125034255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efff81026b0 msgr2=0x7efff810f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7efffce33700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efff81026b0 0x7efff810f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7efffce33700 1 -- 172.21.15.70:0/125034255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe4005040 con 0x7efff8101d10 2024-03-23T08:30:48.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.485+0000 7efffce33700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff8101d10 0x7efff810ef40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7effec002a50 tx=0x7effec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:48.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.486+0000 7efff57fa700 1 -- 172.21.15.70:0/125034255 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effec007cc0 con 0x7efff8101d10 2024-03-23T08:30:48.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.486+0000 7efff57fa700 1 -- 172.21.15.70:0/125034255 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7effec00f040 con 0x7efff8101d10 2024-03-23T08:30:48.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.486+0000 7effff097700 1 -- 172.21.15.70:0/125034255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff8117df0 con 0x7efff8101d10 2024-03-23T08:30:48.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.486+0000 7efff57fa700 1 -- 172.21.15.70:0/125034255 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effec00d9c0 con 0x7efff8101d10 2024-03-23T08:30:48.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.486+0000 7effff097700 1 -- 172.21.15.70:0/125034255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff81182d0 con 0x7efff8101d10 2024-03-23T08:30:48.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.487+0000 7effff097700 1 -- 172.21.15.70:0/125034255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff80fd460 con 0x7efff8101d10 2024-03-23T08:30:48.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.487+0000 7efff57fa700 1 -- 172.21.15.70:0/125034255 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7effec00dc40 con 0x7efff8101d10 2024-03-23T08:30:48.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.487+0000 7efff57fa700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7effe806cc20 0x7effe806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:48.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.488+0000 7efff57fa700 1 -- 172.21.15.70:0/125034255 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6324+0+0 (secure 0 0 0) 0x7effec08bce0 con 0x7efff8101d10 2024-03-23T08:30:48.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.488+0000 7efff7fff700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7effe806cc20 0x7effe806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:48.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.488+0000 7efff7fff700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7effe806cc20 0x7effe806f0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7effe4004890 tx=0x7effe4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:48.491 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.492+0000 7efff57fa700 1 -- 172.21.15.70:0/125034255 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7effec055d40 con 0x7efff8101d10 2024-03-23T08:30:48.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.764+0000 7effff097700 1 -- 172.21.15.70:0/125034255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efff8066660 con 0x7efff8101d10 2024-03-23T08:30:48.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.765+0000 7efff57fa700 1 -- 172.21.15.70:0/125034255 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7effec059360 con 0x7efff8101d10 2024-03-23T08:30:48.764 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:30:48.764 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:30:48.764 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:30:48.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:30:48.764 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:30:48.764 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:30:48.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2, 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 3 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6, 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 7 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:30:48.765 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 -- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7effe806cc20 msgr2=0x7effe806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7effe806cc20 0x7effe806f0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7effe4004890 tx=0x7effe4009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 -- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff8101d10 msgr2=0x7efff810ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff8101d10 0x7efff810ef40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7effec002a50 tx=0x7effec0046b0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 -- 172.21.15.70:0/125034255 shutdown_connections 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7effe806cc20 0x7effe806f0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efff8101d10 0x7efff810ef40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 --2- 172.21.15.70:0/125034255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efff81026b0 0x7efff810f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 -- 172.21.15.70:0/125034255 >> 172.21.15.70:0/125034255 conn(0x7efff80f7880 msgr2=0x7efff80f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 -- 172.21.15.70:0/125034255 shutdown_connections 2024-03-23T08:30:48.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.767+0000 7effff097700 1 -- 172.21.15.70:0/125034255 wait complete. 2024-03-23T08:30:48.816 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:48 smithi193 bash[121117]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-23T08:30:48.816 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:48 smithi193 bash[121117]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-23T08:30:48.816 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:48 smithi193 bash[121117]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-23T08:30:48.816 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:48 smithi193 bash[121117]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-23T08:30:48.816 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:48 smithi193 bash[121117]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-23T08:30:48.816 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:48 smithi193 bash[121117]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-23T08:30:48.817 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:48 smithi193 bash[121117]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-23T08:30:48.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 -- 172.21.15.70:0/3400647195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 msgr2=0x7f2300102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 --2- 172.21.15.70:0/3400647195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 0x7f2300102140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f22f0004940 tx=0x7f22f0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:48.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 -- 172.21.15.70:0/3400647195 shutdown_connections 2024-03-23T08:30:48.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 --2- 172.21.15.70:0/3400647195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2300102710 0x7f23000fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 --2- 172.21.15.70:0/3400647195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 0x7f2300102140 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 -- 172.21.15.70:0/3400647195 >> 172.21.15.70:0/3400647195 conn(0x7f2300072bc0 msgr2=0x7f2300072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:48.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 -- 172.21.15.70:0/3400647195 shutdown_connections 2024-03-23T08:30:48.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 -- 172.21.15.70:0/3400647195 wait complete. 2024-03-23T08:30:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 Processor -- start 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 -- start start 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 0x7f2300115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2300102710 0x7f2300116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300116910 con 0x7f2300101d70 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.869+0000 7f230611e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230010fc90 con 0x7f2300102710 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22ff7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 0x7f2300115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22ff7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 0x7f2300115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55454/0 (socket says 172.21.15.70:55454) 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22ff7fe700 1 -- 172.21.15.70:0/3064866542 learned_addr learned my addr 172.21.15.70:0/3064866542 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22f7fff700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2300102710 0x7f2300116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22ff7fe700 1 -- 172.21.15.70:0/3064866542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2300102710 msgr2=0x7f2300116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22ff7fe700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2300102710 0x7f2300116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22ff7fe700 1 -- 172.21.15.70:0/3064866542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f0005040 con 0x7f2300101d70 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22ff7fe700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 0x7f2300115cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f22f000b3b0 tx=0x7f22f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22fcff9700 1 -- 172.21.15.70:0/3064866542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f000e030 con 0x7f2300101d70 2024-03-23T08:30:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230010ff10 con 0x7f2300101d70 2024-03-23T08:30:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22fcff9700 1 -- 172.21.15.70:0/3064866542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22f0004290 con 0x7f2300101d70 2024-03-23T08:30:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f22fcff9700 1 -- 172.21.15.70:0/3064866542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f0013ca0 con 0x7f2300101d70 2024-03-23T08:30:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.870+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23001104d0 con 0x7f2300101d70 2024-03-23T08:30:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.871+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23000fd4c0 con 0x7f2300101d70 2024-03-23T08:30:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.872+0000 7f22fcff9700 1 -- 172.21.15.70:0/3064866542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f22f00134b0 con 0x7f2300101d70 2024-03-23T08:30:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.872+0000 7f22fcff9700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22ec06cc20 0x7f22ec06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.872+0000 7f22fcff9700 1 -- 172.21.15.70:0/3064866542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f22f008d960 con 0x7f2300101d70 2024-03-23T08:30:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.872+0000 7f22f7fff700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22ec06cc20 0x7f22ec06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.872+0000 7f22f7fff700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22ec06cc20 0x7f22ec06f0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f22e8006fd0 tx=0x7f22e8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:48.876+0000 7f22fcff9700 1 -- 172.21.15.70:0/3064866542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f22f00579c0 con 0x7f2300101d70 2024-03-23T08:30:49.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:48 smithi193 ceph-mon[108533]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:49.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:48 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/125034255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:49.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.112+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2300111380 con 0x7f2300101d70 2024-03-23T08:30:49.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.113+0000 7f22fcff9700 1 -- 172.21.15.70:0/3064866542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f22f005afe0 con 0x7f2300101d70 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:30:49.113 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:30:49.114 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:30:49.115 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22ec06cc20 msgr2=0x7f22ec06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22ec06cc20 0x7f22ec06f0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f22e8006fd0 tx=0x7f22e8008040 comp rx=0 tx=0).stop 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 msgr2=0x7f2300115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 0x7f2300115cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f22f000b3b0 tx=0x7f22f0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 shutdown_connections 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22ec06cc20 0x7f22ec06f0d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2300101d70 0x7f2300115cf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 --2- 172.21.15.70:0/3064866542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2300102710 0x7f2300116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 >> 172.21.15.70:0/3064866542 conn(0x7f2300072bc0 msgr2=0x7f23000659e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 shutdown_connections 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.115+0000 7f230611e700 1 -- 172.21.15.70:0/3064866542 wait complete. 2024-03-23T08:30:49.116 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:30:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.216+0000 7f0af239e700 1 -- 172.21.15.70:0/2951827314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec102710 msgr2=0x7f0aec0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.216+0000 7f0af239e700 1 --2- 172.21.15.70:0/2951827314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec102710 0x7f0aec0fbd80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0004940 tx=0x7f0ae0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.216+0000 7f0af239e700 1 -- 172.21.15.70:0/2951827314 shutdown_connections 2024-03-23T08:30:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.216+0000 7f0af239e700 1 --2- 172.21.15.70:0/2951827314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec102710 0x7f0aec0fbd80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.216+0000 7f0af239e700 1 --2- 172.21.15.70:0/2951827314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aec101d70 0x7f0aec102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.216+0000 7f0af239e700 1 -- 172.21.15.70:0/2951827314 >> 172.21.15.70:0/2951827314 conn(0x7f0aec072bc0 msgr2=0x7f0aec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.216+0000 7f0af239e700 1 -- 172.21.15.70:0/2951827314 shutdown_connections 2024-03-23T08:30:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.216+0000 7f0af239e700 1 -- 172.21.15.70:0/2951827314 wait complete. 2024-03-23T08:30:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0af239e700 1 Processor -- start 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0af239e700 1 -- start start 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0af239e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec101d70 0x7f0aec1139e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0af239e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aec102710 0x7f0aec113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0af239e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec114570 con 0x7f0aec101d70 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0af239e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec10d960 con 0x7f0aec102710 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0aebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec101d70 0x7f0aec1139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0aebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec101d70 0x7f0aec1139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55480/0 (socket says 172.21.15.70:55480) 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0aebfff700 1 -- 172.21.15.70:0/746059389 learned_addr learned my addr 172.21.15.70:0/746059389 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0aeb7fe700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aec102710 0x7f0aec113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.217+0000 7f0aebfff700 1 -- 172.21.15.70:0/746059389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aec102710 msgr2=0x7f0aec113f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.218+0000 7f0aebfff700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aec102710 0x7f0aec113f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.218+0000 7f0aebfff700 1 -- 172.21.15.70:0/746059389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0005040 con 0x7f0aec101d70 2024-03-23T08:30:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.218+0000 7f0aeb7fe700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aec102710 0x7f0aec113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:30:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.218+0000 7f0aebfff700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec101d70 0x7f0aec1139e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00eb40 tx=0x7f0adc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.218+0000 7f0ae8ff9700 1 -- 172.21.15.70:0/746059389 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0adc007cc0 con 0x7f0aec101d70 2024-03-23T08:30:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.218+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aec10dbe0 con 0x7f0aec101d70 2024-03-23T08:30:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.218+0000 7f0ae8ff9700 1 -- 172.21.15.70:0/746059389 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0adc00f040 con 0x7f0aec101d70 2024-03-23T08:30:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.218+0000 7f0ae8ff9700 1 -- 172.21.15.70:0/746059389 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0adc009570 con 0x7f0aec101d70 2024-03-23T08:30:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.218+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aec10e1a0 con 0x7f0aec101d70 2024-03-23T08:30:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.219+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aec0fd4c0 con 0x7f0aec101d70 2024-03-23T08:30:49.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.219+0000 7f0ae8ff9700 1 -- 172.21.15.70:0/746059389 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0adc0096d0 con 0x7f0aec101d70 2024-03-23T08:30:49.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.220+0000 7f0ae8ff9700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0ad406cc20 0x7f0ad406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:49.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.220+0000 7f0ae8ff9700 1 -- 172.21.15.70:0/746059389 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0adc08bc50 con 0x7f0aec101d70 2024-03-23T08:30:49.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.220+0000 7f0aeb7fe700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0ad406cc20 0x7f0ad406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:49.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.220+0000 7f0aeb7fe700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0ad406cc20 0x7f0ad406f0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0aec10eb20 tx=0x7f0ae0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:49.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.224+0000 7f0ae8ff9700 1 -- 172.21.15.70:0/746059389 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0adc0597f0 con 0x7f0aec101d70 2024-03-23T08:30:49.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:48 smithi070 ceph-mon[145869]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:49.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:48 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/125034255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.434+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0aec108d10 con 0x7f0ad406cc20 2024-03-23T08:30:49.434 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.435+0000 7f0ae8ff9700 1 -- 172.21.15.70:0/746059389 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0aec108d10 con 0x7f0ad406cc20 2024-03-23T08:30:49.434 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:30:49.434 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:30:49.434 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:30:49.434 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:30:49.435 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:30:49.435 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:30:49.435 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-03-23T08:30:49.435 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-03-23T08:30:49.435 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:30:49.435 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "9/21 daemons upgraded", 2024-03-23T08:30:49.435 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-03-23T08:30:49.435 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:30:49.435 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:30:49.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0ad406cc20 msgr2=0x7f0ad406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0ad406cc20 0x7f0ad406f0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0aec10eb20 tx=0x7f0ae0009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:49.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec101d70 msgr2=0x7f0aec1139e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec101d70 0x7f0aec1139e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00eb40 tx=0x7f0adc00b040 comp rx=0 tx=0).stop 2024-03-23T08:30:49.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 shutdown_connections 2024-03-23T08:30:49.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0ad406cc20 0x7f0ad406f0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aec101d70 0x7f0aec1139e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.436 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 --2- 172.21.15.70:0/746059389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aec102710 0x7f0aec113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 >> 172.21.15.70:0/746059389 conn(0x7f0aec072bc0 msgr2=0x7f0aec0658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:49.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 shutdown_connections 2024-03-23T08:30:49.437 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.437+0000 7f0af239e700 1 -- 172.21.15.70:0/746059389 wait complete. 2024-03-23T08:30:49.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.536+0000 7fc8632ae700 1 -- 172.21.15.70:0/2065611769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 msgr2=0x7fc85c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.536+0000 7fc8632ae700 1 --2- 172.21.15.70:0/2065611769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 0x7fc85c0fbd80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc858004940 tx=0x7fc858009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:49.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.536+0000 7fc8632ae700 1 -- 172.21.15.70:0/2065611769 shutdown_connections 2024-03-23T08:30:49.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.536+0000 7fc8632ae700 1 --2- 172.21.15.70:0/2065611769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 0x7fc85c0fbd80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.536+0000 7fc8632ae700 1 --2- 172.21.15.70:0/2065611769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c101d70 0x7fc85c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.536+0000 7fc8632ae700 1 -- 172.21.15.70:0/2065611769 >> 172.21.15.70:0/2065611769 conn(0x7fc85c072bc0 msgr2=0x7fc85c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:49.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.536+0000 7fc8632ae700 1 -- 172.21.15.70:0/2065611769 shutdown_connections 2024-03-23T08:30:49.537 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.536+0000 7fc8632ae700 1 -- 172.21.15.70:0/2065611769 wait complete. 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc8632ae700 1 Processor -- start 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc8632ae700 1 -- start start 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc8632ae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c101d70 0x7fc85c1129f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc8632ae700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 0x7fc85c112f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc8632ae700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c113610 con 0x7fc85c102710 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc8632ae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c10c970 con 0x7fc85c101d70 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc860849700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 0x7fc85c112f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc860849700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 0x7fc85c112f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55494/0 (socket says 172.21.15.70:55494) 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc860849700 1 -- 172.21.15.70:0/3579045728 learned_addr learned my addr 172.21.15.70:0/3579045728 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.537+0000 7fc86104a700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c101d70 0x7fc85c1129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.538+0000 7fc860849700 1 -- 172.21.15.70:0/3579045728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c101d70 msgr2=0x7fc85c1129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.538+0000 7fc860849700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c101d70 0x7fc85c1129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.538+0000 7fc860849700 1 -- 172.21.15.70:0/3579045728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858005040 con 0x7fc85c102710 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.538+0000 7fc860849700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 0x7fc85c112f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc8580093b0 tx=0x7fc858009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.538+0000 7fc851ffb700 1 -- 172.21.15.70:0/3579045728 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc85800e040 con 0x7fc85c102710 2024-03-23T08:30:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.538+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c10cbf0 con 0x7fc85c102710 2024-03-23T08:30:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.538+0000 7fc851ffb700 1 -- 172.21.15.70:0/3579045728 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc858013420 con 0x7fc85c102710 2024-03-23T08:30:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.538+0000 7fc851ffb700 1 -- 172.21.15.70:0/3579045728 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc85801c630 con 0x7fc85c102710 2024-03-23T08:30:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.538+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c10d1b0 con 0x7fc85c102710 2024-03-23T08:30:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.539+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c0ff080 con 0x7fc85c102710 2024-03-23T08:30:49.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.539+0000 7fc851ffb700 1 -- 172.21.15.70:0/3579045728 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc8580139e0 con 0x7fc85c102710 2024-03-23T08:30:49.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.540+0000 7fc851ffb700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc84806cc20 0x7fc84806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:30:49.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.540+0000 7fc851ffb700 1 -- 172.21.15.70:0/3579045728 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc85808d140 con 0x7fc85c102710 2024-03-23T08:30:49.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.540+0000 7fc86104a700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc84806cc20 0x7fc84806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:30:49.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.540+0000 7fc86104a700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc84806cc20 0x7fc84806f0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009bf0 tx=0x7fc84c0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:30:49.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.544+0000 7fc851ffb700 1 -- 172.21.15.70:0/3579045728 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc85805ad00 con 0x7fc85c102710 2024-03-23T08:30:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.823+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc85c0fd4c0 con 0x7fc85c102710 2024-03-23T08:30:49.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.823+0000 7fc851ffb700 1 -- 172.21.15.70:0/3579045728 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1342 (secure 0 0 0) 0x7fc85805a890 con 0x7fc85c102710 2024-03-23T08:30:49.822 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 1 pg peering; Degraded data redundancy: 49/222 objects degraded (22.072%), 16 pgs degraded 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: osd.3 (root=default,host=smithi193) is down 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 1 pg peering 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 3.d is stuck peering for 5m, current state peering, last acting [2,4] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 49/222 objects degraded (22.072%), 16 pgs degraded 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-03-23T08:30:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-03-23T08:30:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-03-23T08:30:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-03-23T08:30:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-03-23T08:30:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-03-23T08:30:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.825+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc84806cc20 msgr2=0x7fc84806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.825+0000 7fc8632ae700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc84806cc20 0x7fc84806f0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009bf0 tx=0x7fc84c0091a0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.825+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 msgr2=0x7fc85c112f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.825+0000 7fc8632ae700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 0x7fc85c112f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc8580093b0 tx=0x7fc858009e30 comp rx=0 tx=0).stop 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.825+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 shutdown_connections 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.825+0000 7fc8632ae700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc84806cc20 0x7fc84806f0d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.825+0000 7fc8632ae700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c101d70 0x7fc85c1129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.825+0000 7fc8632ae700 1 --2- 172.21.15.70:0/3579045728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85c102710 0x7fc85c112f30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.825+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 >> 172.21.15.70:0/3579045728 conn(0x7fc85c072bc0 msgr2=0x7fc85c0fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.826+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 shutdown_connections 2024-03-23T08:30:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:30:49.826+0000 7fc8632ae700 1 -- 172.21.15.70:0/3579045728 wait complete. 2024-03-23T08:30:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:49 smithi193 ceph-mon[108533]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:49 smithi193 ceph-mon[108533]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:49 smithi193 ceph-mon[108533]: pgmap v111: 65 pgs: 16 active+undersized+degraded, 4 peering, 18 active+undersized, 27 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 49/222 objects degraded (22.072%) 2024-03-23T08:30:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:49 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3064866542' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:30:50.075 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:49 smithi193 ceph-osd[121412]: -- 172.21.15.193:0/794800427 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1420a31e0 con 0x55f142073000 2024-03-23T08:30:50.076 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:49 smithi193 bash[121374]: 8a7617f5541a8fffb3f09eb568fbed5017fc5feb43b26802ac795e2392482016 2024-03-23T08:30:50.076 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:49 smithi193 systemd[1]: Started Ceph osd.3 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:30:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:49 smithi070 ceph-mon[145869]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:49 smithi070 ceph-mon[145869]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:49 smithi070 ceph-mon[145869]: pgmap v111: 65 pgs: 16 active+undersized+degraded, 4 peering, 18 active+undersized, 27 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 49/222 objects degraded (22.072%) 2024-03-23T08:30:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:49 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3064866542' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:30:50.575 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:50 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[121389]: 2024-03-23T08:30:50.269+0000 7f37c8ba6640 -1 Falling back to public interface 2024-03-23T08:30:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:50 smithi193 ceph-mon[108533]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:50 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3579045728' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:30:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:50 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:50 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:50 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:30:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:50 smithi070 ceph-mon[145869]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:30:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:50 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3579045728' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:30:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:50 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:30:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:51 smithi193 ceph-mon[108533]: pgmap v112: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 53/222 objects degraded (23.874%) 2024-03-23T08:30:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:51 smithi193 ceph-mon[108533]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-23T08:30:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:51 smithi070 ceph-mon[145869]: pgmap v112: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 53/222 objects degraded (23.874%) 2024-03-23T08:30:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:51 smithi070 ceph-mon[145869]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-23T08:30:53.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:53 smithi070 ceph-mon[145869]: pgmap v113: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 53/222 objects degraded (23.874%) 2024-03-23T08:30:53.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:53 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:53.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:53 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:53.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:53 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:53.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:53 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:53 smithi193 ceph-mon[108533]: pgmap v113: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 53/222 objects degraded (23.874%) 2024-03-23T08:30:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:53 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:54.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:53 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:54.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:53 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:54.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:53 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:55.325 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:54 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[121389]: 2024-03-23T08:30:54.860+0000 7f37c8ba6640 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-23T08:30:55.674 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:55 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[121389]: 2024-03-23T08:30:55.417+0000 7f37c8ba6640 -1 osd.3 66 log_to_monitors true 2024-03-23T08:30:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:55 smithi070 ceph-mon[145869]: pgmap v114: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 53/222 objects degraded (23.874%) 2024-03-23T08:30:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:55 smithi070 ceph-mon[145869]: from='osd.3 [v2:172.21.15.193:6800/457144182,v1:172.21.15.193:6801/457144182]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T08:30:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:55 smithi070 ceph-mon[145869]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T08:30:56.075 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:30:55 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[121389]: 2024-03-23T08:30:55.695+0000 7f37b9fff700 -1 osd.3 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:30:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:55 smithi193 ceph-mon[108533]: pgmap v114: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 201 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 53/222 objects degraded (23.874%) 2024-03-23T08:30:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:55 smithi193 ceph-mon[108533]: from='osd.3 [v2:172.21.15.193:6800/457144182,v1:172.21.15.193:6801/457144182]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T08:30:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:55 smithi193 ceph-mon[108533]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-23T08:30:56.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:56 smithi193 ceph-mon[108533]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T08:30:56.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:56 smithi193 ceph-mon[108533]: osdmap e69: 6 total, 5 up, 6 in 2024-03-23T08:30:56.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:56 smithi193 ceph-mon[108533]: from='osd.3 [v2:172.21.15.193:6800/457144182,v1:172.21.15.193:6801/457144182]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:30:56.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:56 smithi193 ceph-mon[108533]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:30:56.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:56 smithi070 ceph-mon[145869]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-23T08:30:56.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:56 smithi070 ceph-mon[145869]: osdmap e69: 6 total, 5 up, 6 in 2024-03-23T08:30:56.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:56 smithi070 ceph-mon[145869]: from='osd.3 [v2:172.21.15.193:6800/457144182,v1:172.21.15.193:6801/457144182]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:30:56.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:56 smithi070 ceph-mon[145869]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 53/222 objects degraded (23.874%) 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: osd.3 [v2:172.21.15.193:6800/457144182,v1:172.21.15.193:6801/457144182] boot 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: osdmap e70: 6 total, 6 up, 6 in 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: Health check update: Degraded data redundancy: 53/222 objects degraded (23.874%), 18 pgs degraded (PG_DEGRADED) 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:30:57.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:30:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-23T08:30:57.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:57 smithi070 ceph-mon[145869]: osdmap e71: 6 total, 6 up, 6 in 2024-03-23T08:30:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 53/222 objects degraded (23.874%) 2024-03-23T08:30:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:30:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: osd.3 [v2:172.21.15.193:6800/457144182,v1:172.21.15.193:6801/457144182] boot 2024-03-23T08:30:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: osdmap e70: 6 total, 6 up, 6 in 2024-03-23T08:30:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-23T08:30:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: Health check update: Degraded data redundancy: 53/222 objects degraded (23.874%), 18 pgs degraded (PG_DEGRADED) 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-23T08:30:58.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:57 smithi193 ceph-mon[108533]: osdmap e71: 6 total, 6 up, 6 in 2024-03-23T08:30:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:58 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-23T08:30:58.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:58 smithi070 ceph-mon[145869]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-23T08:30:59.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:58 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-23T08:30:59.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:58 smithi193 ceph-mon[108533]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-23T08:30:59.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:30:59 smithi070 ceph-mon[145869]: pgmap v119: 65 pgs: 8 peering, 16 active+undersized+degraded, 14 active+undersized, 27 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 44/222 objects degraded (19.820%) 2024-03-23T08:31:00.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:30:59 smithi193 ceph-mon[108533]: pgmap v119: 65 pgs: 8 peering, 16 active+undersized+degraded, 14 active+undersized, 27 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 44/222 objects degraded (19.820%) 2024-03-23T08:31:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:01 smithi070 ceph-mon[145869]: pgmap v120: 65 pgs: 8 peering, 14 active+undersized+degraded, 13 active+undersized, 30 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 40/222 objects degraded (18.018%) 2024-03-23T08:31:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:31:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:01 smithi193 ceph-mon[108533]: pgmap v120: 65 pgs: 8 peering, 14 active+undersized+degraded, 13 active+undersized, 30 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 40/222 objects degraded (18.018%) 2024-03-23T08:31:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:31:02.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:02 smithi070 ceph-mon[145869]: Health check update: Degraded data redundancy: 40/222 objects degraded (18.018%), 14 pgs degraded (PG_DEGRADED) 2024-03-23T08:31:03.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:02 smithi193 ceph-mon[108533]: Health check update: Degraded data redundancy: 40/222 objects degraded (18.018%), 14 pgs degraded (PG_DEGRADED) 2024-03-23T08:31:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:03 smithi070 ceph-mon[145869]: pgmap v121: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 886 B/s rd, 1 op/s 2024-03-23T08:31:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:03 smithi070 ceph-mon[145869]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 40/222 objects degraded (18.018%), 14 pgs degraded) 2024-03-23T08:31:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:03 smithi070 ceph-mon[145869]: Cluster is now healthy 2024-03-23T08:31:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:03 smithi193 ceph-mon[108533]: pgmap v121: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 886 B/s rd, 1 op/s 2024-03-23T08:31:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:03 smithi193 ceph-mon[108533]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 40/222 objects degraded (18.018%), 14 pgs degraded) 2024-03-23T08:31:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:03 smithi193 ceph-mon[108533]: Cluster is now healthy 2024-03-23T08:31:05.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:05 smithi070 ceph-mon[145869]: pgmap v122: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:31:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:05 smithi193 ceph-mon[108533]: pgmap v122: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:31:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:07 smithi193 ceph-mon[108533]: pgmap v123: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-23T08:31:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:07 smithi070 ceph-mon[145869]: pgmap v123: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-23T08:31:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:09 smithi193 ceph-mon[108533]: pgmap v124: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-23T08:31:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:09 smithi070 ceph-mon[145869]: pgmap v124: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-23T08:31:12.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:11 smithi193 ceph-mon[108533]: pgmap v125: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 2 op/s 2024-03-23T08:31:12.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:11 smithi070 ceph-mon[145869]: pgmap v125: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 2 op/s 2024-03-23T08:31:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:13.022 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:12 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-23T08:31:13.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:12 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-23T08:31:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:13 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-23T08:31:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:13 smithi193 ceph-mon[108533]: Upgrade: osd.4 is safe to restart 2024-03-23T08:31:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:13 smithi193 ceph-mon[108533]: pgmap v126: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-23T08:31:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:13 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:13 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T08:31:14.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:13 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:31:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:13 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-23T08:31:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:13 smithi070 ceph-mon[145869]: Upgrade: osd.4 is safe to restart 2024-03-23T08:31:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:13 smithi070 ceph-mon[145869]: pgmap v126: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-23T08:31:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:13 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:13 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-23T08:31:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:13 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:31:15.028 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:14 smithi193 ceph-mon[108533]: Upgrade: Updating osd.4 2024-03-23T08:31:15.028 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:14 smithi193 ceph-mon[108533]: Deploying daemon osd.4 on smithi193 2024-03-23T08:31:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:14 smithi070 ceph-mon[145869]: Upgrade: Updating osd.4 2024-03-23T08:31:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:14 smithi070 ceph-mon[145869]: Deploying daemon osd.4 on smithi193 2024-03-23T08:31:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:15 smithi193 ceph-mon[108533]: pgmap v127: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:31:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:15 smithi070 ceph-mon[145869]: pgmap v127: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:31:17.075 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:16 smithi193 systemd[1]: Stopping Ceph osd.4 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:31:17.075 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:16 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[63545]: 2024-03-23T08:31:16.920+0000 7f9ceea7c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:31:17.075 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:16 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[63545]: 2024-03-23T08:31:16.920+0000 7f9ceea7c700 -1 osd.4 71 *** Got signal Terminated *** 2024-03-23T08:31:17.075 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:16 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[63545]: 2024-03-23T08:31:16.920+0000 7f9ceea7c700 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:31:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:17 smithi070 ceph-mon[145869]: pgmap v128: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:31:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:31:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:17 smithi070 ceph-mon[145869]: osd.4 marked itself down and dead 2024-03-23T08:31:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:17 smithi193 ceph-mon[108533]: pgmap v128: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:31:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:31:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:17 smithi193 ceph-mon[108533]: osd.4 marked itself down and dead 2024-03-23T08:31:18.825 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:18 smithi193 bash[126932]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4 2024-03-23T08:31:18.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:18 smithi193 ceph-mon[108533]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:31:18.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:18 smithi193 ceph-mon[108533]: osdmap e72: 6 total, 5 up, 6 in 2024-03-23T08:31:18.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:18 smithi070 ceph-mon[145869]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:31:18.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:18 smithi070 ceph-mon[145869]: osdmap e72: 6 total, 5 up, 6 in 2024-03-23T08:31:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.937+0000 7f6329859700 1 -- 172.21.15.70:0/2211751417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 msgr2=0x7f63240fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.937+0000 7f6329859700 1 --2- 172.21.15.70:0/2211751417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 0x7f63240fce10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6318004940 tx=0x7f6318009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.938+0000 7f6329859700 1 -- 172.21.15.70:0/2211751417 shutdown_connections 2024-03-23T08:31:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.938+0000 7f6329859700 1 --2- 172.21.15.70:0/2211751417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 0x7f63240fce10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.938+0000 7f6329859700 1 --2- 172.21.15.70:0/2211751417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6324102e00 0x7f63241031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.938+0000 7f6329859700 1 -- 172.21.15.70:0/2211751417 >> 172.21.15.70:0/2211751417 conn(0x7f63240f89b0 msgr2=0x7f63240fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.938+0000 7f6329859700 1 -- 172.21.15.70:0/2211751417 shutdown_connections 2024-03-23T08:31:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.938+0000 7f6329859700 1 -- 172.21.15.70:0/2211751417 wait complete. 2024-03-23T08:31:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.938+0000 7f6329859700 1 Processor -- start 2024-03-23T08:31:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f6329859700 1 -- start start 2024-03-23T08:31:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f6329859700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6324102e00 0x7f63241138c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f6329859700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 0x7f6324113e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f6329859700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6324114450 con 0x7f63241037a0 2024-03-23T08:31:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f6329859700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632410da50 con 0x7f6324102e00 2024-03-23T08:31:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f63227fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 0x7f6324113e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f63227fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 0x7f6324113e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43192/0 (socket says 172.21.15.70:43192) 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f63227fc700 1 -- 172.21.15.70:0/3845692943 learned_addr learned my addr 172.21.15.70:0/3845692943 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f6322ffd700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6324102e00 0x7f63241138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f63227fc700 1 -- 172.21.15.70:0/3845692943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6324102e00 msgr2=0x7f63241138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f63227fc700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6324102e00 0x7f63241138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.939+0000 7f63227fc700 1 -- 172.21.15.70:0/3845692943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6318005040 con 0x7f63241037a0 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.940+0000 7f63227fc700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 0x7f6324113e00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6318009380 tx=0x7f6318009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.940+0000 7f630b7fe700 1 -- 172.21.15.70:0/3845692943 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631800e040 con 0x7f63241037a0 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.940+0000 7f630b7fe700 1 -- 172.21.15.70:0/3845692943 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6318013420 con 0x7f63241037a0 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.940+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632410dcd0 con 0x7f63241037a0 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.940+0000 7f630b7fe700 1 -- 172.21.15.70:0/3845692943 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631801c630 con 0x7f63241037a0 2024-03-23T08:31:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.940+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632410e290 con 0x7f63241037a0 2024-03-23T08:31:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.941+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63240fe550 con 0x7f63241037a0 2024-03-23T08:31:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.941+0000 7f630b7fe700 1 -- 172.21.15.70:0/3845692943 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f63180139b0 con 0x7f63241037a0 2024-03-23T08:31:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.942+0000 7f630b7fe700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f630c06cc20 0x7f630c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.942+0000 7f630b7fe700 1 -- 172.21.15.70:0/3845692943 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f631808d100 con 0x7f63241037a0 2024-03-23T08:31:19.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.942+0000 7f6322ffd700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f630c06cc20 0x7f630c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:19.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.942+0000 7f6322ffd700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f630c06cc20 0x7f630c06f0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6314004f40 tx=0x7f631400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:19.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:19.946+0000 7f630b7fe700 1 -- 172.21.15.70:0/3845692943 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f631805ac70 con 0x7f63241037a0 2024-03-23T08:31:19.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:19 smithi193 ceph-mon[108533]: pgmap v130: 65 pgs: 6 stale+active+clean, 59 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-23T08:31:19.974 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:19 smithi193 ceph-mon[108533]: osdmap e73: 6 total, 5 up, 6 in 2024-03-23T08:31:20.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.158+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6324109da0 con 0x7f630c06cc20 2024-03-23T08:31:20.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.159+0000 7f630b7fe700 1 -- 172.21.15.70:0/3845692943 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6324109da0 con 0x7f630c06cc20 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f630c06cc20 msgr2=0x7f630c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f630c06cc20 0x7f630c06f0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6314004f40 tx=0x7f631400b040 comp rx=0 tx=0).stop 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 msgr2=0x7f6324113e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 0x7f6324113e00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6318009380 tx=0x7f6318009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 shutdown_connections 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f630c06cc20 0x7f630c06f0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6324102e00 0x7f63241138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 --2- 172.21.15.70:0/3845692943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63241037a0 0x7f6324113e00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 >> 172.21.15.70:0/3845692943 conn(0x7f63240f89b0 msgr2=0x7f63240f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 shutdown_connections 2024-03-23T08:31:20.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.162+0000 7f6329859700 1 -- 172.21.15.70:0/3845692943 wait complete. 2024-03-23T08:31:20.172 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:31:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:19 smithi070 ceph-mon[145869]: pgmap v130: 65 pgs: 6 stale+active+clean, 59 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-23T08:31:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:19 smithi070 ceph-mon[145869]: osdmap e73: 6 total, 5 up, 6 in 2024-03-23T08:31:20.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.262+0000 7fbec9237700 1 -- 172.21.15.70:0/3865119202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 msgr2=0x7fbec4109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.262+0000 7fbec9237700 1 --2- 172.21.15.70:0/3865119202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 0x7fbec4109260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4004940 tx=0x7fbeb4009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:20.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.262+0000 7fbec9237700 1 -- 172.21.15.70:0/3865119202 shutdown_connections 2024-03-23T08:31:20.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.262+0000 7fbec9237700 1 --2- 172.21.15.70:0/3865119202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 0x7fbec4109260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.262+0000 7fbec9237700 1 --2- 172.21.15.70:0/3865119202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbec40fbbf0 0x7fbec40fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.262+0000 7fbec9237700 1 -- 172.21.15.70:0/3865119202 >> 172.21.15.70:0/3865119202 conn(0x7fbec40f7880 msgr2=0x7fbec40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:20.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.262+0000 7fbec9237700 1 -- 172.21.15.70:0/3865119202 shutdown_connections 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.262+0000 7fbec9237700 1 -- 172.21.15.70:0/3865119202 wait complete. 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec9237700 1 Processor -- start 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec9237700 1 -- start start 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec9237700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbec40fbbf0 0x7fbec4115ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec9237700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 0x7fbec4116530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec9237700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4116b80 con 0x7fbec40fc500 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec9237700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec410ff50 con 0x7fbec40fbbf0 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 0x7fbec4116530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 0x7fbec4116530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43216/0 (socket says 172.21.15.70:43216) 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec2d9d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbec40fbbf0 0x7fbec4115ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.263+0000 7fbec259c700 1 -- 172.21.15.70:0/1417731048 learned_addr learned my addr 172.21.15.70:0/1417731048 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.264+0000 7fbec259c700 1 -- 172.21.15.70:0/1417731048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbec40fbbf0 msgr2=0x7fbec4115ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.264+0000 7fbec259c700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbec40fbbf0 0x7fbec4115ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.264+0000 7fbec259c700 1 -- 172.21.15.70:0/1417731048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb4005040 con 0x7fbec40fc500 2024-03-23T08:31:20.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.264+0000 7fbec259c700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 0x7fbec4116530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400b3b0 tx=0x7fbeb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.264+0000 7fbeab7fe700 1 -- 172.21.15.70:0/1417731048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb400e040 con 0x7fbec40fc500 2024-03-23T08:31:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.264+0000 7fbeab7fe700 1 -- 172.21.15.70:0/1417731048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbeb400bed0 con 0x7fbec40fc500 2024-03-23T08:31:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.264+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec41101d0 con 0x7fbec40fc500 2024-03-23T08:31:20.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.264+0000 7fbeab7fe700 1 -- 172.21.15.70:0/1417731048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb4013db0 con 0x7fbec40fc500 2024-03-23T08:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.264+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec4110790 con 0x7fbec40fc500 2024-03-23T08:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.265+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec4103d20 con 0x7fbec40fc500 2024-03-23T08:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.265+0000 7fbeab7fe700 1 -- 172.21.15.70:0/1417731048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbeb400ba20 con 0x7fbec40fc500 2024-03-23T08:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.266+0000 7fbeab7fe700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbeac06cc20 0x7fbeac06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.266+0000 7fbeab7fe700 1 -- 172.21.15.70:0/1417731048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbeb408d280 con 0x7fbec40fc500 2024-03-23T08:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.266+0000 7fbec2d9d700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbeac06cc20 0x7fbeac06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:20.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.266+0000 7fbec2d9d700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbeac06cc20 0x7fbeac06f0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8004f40 tx=0x7fbeb800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:20.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.271+0000 7fbeab7fe700 1 -- 172.21.15.70:0/1417731048 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbeb405ad50 con 0x7fbec40fc500 2024-03-23T08:31:20.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.479+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbec4111640 con 0x7fbeac06cc20 2024-03-23T08:31:20.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.480+0000 7fbeab7fe700 1 -- 172.21.15.70:0/1417731048 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbec4111640 con 0x7fbeac06cc20 2024-03-23T08:31:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbeac06cc20 msgr2=0x7fbeac06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbeac06cc20 0x7fbeac06f0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8004f40 tx=0x7fbeb800b040 comp rx=0 tx=0).stop 2024-03-23T08:31:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 msgr2=0x7fbec4116530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 0x7fbec4116530 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400b3b0 tx=0x7fbeb4009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 shutdown_connections 2024-03-23T08:31:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbeac06cc20 0x7fbeac06f0d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbec40fbbf0 0x7fbec4115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 --2- 172.21.15.70:0/1417731048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbec40fc500 0x7fbec4116530 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 >> 172.21.15.70:0/1417731048 conn(0x7fbec40f7880 msgr2=0x7fbec40f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:20.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 shutdown_connections 2024-03-23T08:31:20.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.482+0000 7fbec9237700 1 -- 172.21.15.70:0/1417731048 wait complete. 2024-03-23T08:31:20.575 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:20 smithi193 systemd[1]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.4.service: Succeeded. 2024-03-23T08:31:20.576 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:20 smithi193 systemd[1]: Stopped Ceph osd.4 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:31:20.576 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:20 smithi193 systemd[1]: Starting Ceph osd.4 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:31:20.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 -- 172.21.15.70:0/4004736682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 msgr2=0x7f6e580fcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 --2- 172.21.15.70:0/4004736682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 0x7f6e580fcdb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6e44004940 tx=0x7f6e44009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:20.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 -- 172.21.15.70:0/4004736682 shutdown_connections 2024-03-23T08:31:20.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 --2- 172.21.15.70:0/4004736682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 0x7f6e580fcdb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 --2- 172.21.15.70:0/4004736682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e58102da0 0x7f6e58103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 -- 172.21.15.70:0/4004736682 >> 172.21.15.70:0/4004736682 conn(0x7f6e580f8970 msgr2=0x7f6e580fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:20.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 -- 172.21.15.70:0/4004736682 shutdown_connections 2024-03-23T08:31:20.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 -- 172.21.15.70:0/4004736682 wait complete. 2024-03-23T08:31:20.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 Processor -- start 2024-03-23T08:31:20.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.582+0000 7f6e5f227700 1 -- start start 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e5f227700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e58102da0 0x7f6e58113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e5f227700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 0x7f6e58113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e5f227700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581146b0 con 0x7f6e58103740 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e5f227700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5810da10 con 0x7f6e58102da0 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e57fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 0x7f6e58113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e57fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 0x7f6e58113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43244/0 (socket says 172.21.15.70:43244) 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e57fff700 1 -- 172.21.15.70:0/4086545794 learned_addr learned my addr 172.21.15.70:0/4086545794 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e5cfc3700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e58102da0 0x7f6e58113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e57fff700 1 -- 172.21.15.70:0/4086545794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e58102da0 msgr2=0x7f6e58113a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e57fff700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e58102da0 0x7f6e58113a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e57fff700 1 -- 172.21.15.70:0/4086545794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e44005040 con 0x7f6e58103740 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e5cfc3700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e58102da0 0x7f6e58113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e57fff700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 0x7f6e58113fd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009380 tx=0x7f6e44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:20.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e557fa700 1 -- 172.21.15.70:0/4086545794 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e4400e040 con 0x7f6e58103740 2024-03-23T08:31:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e557fa700 1 -- 172.21.15.70:0/4086545794 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e44013420 con 0x7f6e58103740 2024-03-23T08:31:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5810dc90 con 0x7f6e58103740 2024-03-23T08:31:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e557fa700 1 -- 172.21.15.70:0/4086545794 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e4401c630 con 0x7f6e58103740 2024-03-23T08:31:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.583+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5810e250 con 0x7f6e58103740 2024-03-23T08:31:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.584+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e580fe4f0 con 0x7f6e58103740 2024-03-23T08:31:20.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.585+0000 7f6e557fa700 1 -- 172.21.15.70:0/4086545794 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6e440139e0 con 0x7f6e58103740 2024-03-23T08:31:20.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.585+0000 7f6e557fa700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6e4806cc20 0x7f6e4806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:20.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.585+0000 7f6e557fa700 1 -- 172.21.15.70:0/4086545794 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6e4408d180 con 0x7f6e58103740 2024-03-23T08:31:20.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.585+0000 7f6e5cfc3700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6e4806cc20 0x7f6e4806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:20.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.586+0000 7f6e5cfc3700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6e4806cc20 0x7f6e4806f0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009b10 tx=0x7f6e4c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:20.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.591+0000 7f6e557fa700 1 -- 172.21.15.70:0/4086545794 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6e4405acf0 con 0x7f6e58103740 2024-03-23T08:31:20.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.789+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e58109db0 con 0x7f6e4806cc20 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.799+0000 7f6e557fa700 1 -- 172.21.15.70:0/4086545794 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6e58109db0 con 0x7f6e4806cc20 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (4m) 60s ago 16m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (2m) 60s ago 16m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (2m) 28s ago 15m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (4m) 60s ago 15m 83.9M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (10m) 60s ago 10m 146M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (10m) 60s ago 10m 14.4M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (10m) 28s ago 10m 97.2M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (10m) 28s ago 10m 15.8M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:31:20.799 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (6m) 60s ago 17m 501M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:31:20.800 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (5m) 28s ago 15m 449M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:31:20.800 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (3m) 60s ago 17m 48.9M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:31:20.800 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (3m) 28s ago 15m 38.0M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:31:20.800 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (5m) 60s ago 16m 18.6M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:31:20.800 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (5m) 28s ago 15m 16.7M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:31:20.800 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (2m) 60s ago 14m 85.0M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:31:20.800 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (101s) 60s ago 13m 96.3M 1568M 18.2.2-954-g82316b04 b24e1baba637 13fce63c86eb 2024-03-23T08:31:20.800 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (66s) 60s ago 13m 10.7M 1568M 18.2.2-954-g82316b04 b24e1baba637 73e36e623491 2024-03-23T08:31:20.800 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (31s) 28s ago 13m 10.9M 2592M 18.2.2-954-g82316b04 b24e1baba637 8a7617f5541a 2024-03-23T08:31:20.801 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (12m) 28s ago 12m 243M 2592M 17.2.7-726-g9c4e0c34 837124da804f ee606c293fe0 2024-03-23T08:31:20.801 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (12m) 28s ago 12m 226M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:31:20.801 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (5m) 60s ago 15m 60.3M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:31:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6e4806cc20 msgr2=0x7f6e4806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6e4806cc20 0x7f6e4806f0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009b10 tx=0x7f6e4c008040 comp rx=0 tx=0).stop 2024-03-23T08:31:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 msgr2=0x7f6e58113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 0x7f6e58113fd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009380 tx=0x7f6e44009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 shutdown_connections 2024-03-23T08:31:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6e4806cc20 0x7f6e4806f0d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e58102da0 0x7f6e58113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 --2- 172.21.15.70:0/4086545794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e58103740 0x7f6e58113fd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 >> 172.21.15.70:0/4086545794 conn(0x7f6e580f8970 msgr2=0x7f6e580f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:20.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 shutdown_connections 2024-03-23T08:31:20.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.802+0000 7f6e5f227700 1 -- 172.21.15.70:0/4086545794 wait complete. 2024-03-23T08:31:20.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.901+0000 7f870b3dc700 1 -- 172.21.15.70:0/1264161435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704102710 msgr2=0x7f87040fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.901+0000 7f870b3dc700 1 --2- 172.21.15.70:0/1264161435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704102710 0x7f87040fbd80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8700004940 tx=0x7f8700009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.901+0000 7f870b3dc700 1 -- 172.21.15.70:0/1264161435 shutdown_connections 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.901+0000 7f870b3dc700 1 --2- 172.21.15.70:0/1264161435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704102710 0x7f87040fbd80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.901+0000 7f870b3dc700 1 --2- 172.21.15.70:0/1264161435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8704101d70 0x7f8704102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.901+0000 7f870b3dc700 1 -- 172.21.15.70:0/1264161435 >> 172.21.15.70:0/1264161435 conn(0x7f8704072bc0 msgr2=0x7f8704072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f870b3dc700 1 -- 172.21.15.70:0/1264161435 shutdown_connections 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f870b3dc700 1 -- 172.21.15.70:0/1264161435 wait complete. 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f870b3dc700 1 Processor -- start 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f870b3dc700 1 -- start start 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f870b3dc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704101d70 0x7f870410ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f870b3dc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8704102710 0x7f870410efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f870b3dc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870410f5f0 con 0x7f8704101d70 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f870b3dc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870410f730 con 0x7f8704102710 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f8709178700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704101d70 0x7f870410ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f8709178700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704101d70 0x7f870410ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43272/0 (socket says 172.21.15.70:43272) 2024-03-23T08:31:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f8709178700 1 -- 172.21.15.70:0/2384723252 learned_addr learned my addr 172.21.15.70:0/2384723252 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.902+0000 7f8708977700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8704102710 0x7f870410efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.903+0000 7f8709178700 1 -- 172.21.15.70:0/2384723252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8704102710 msgr2=0x7f870410efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.903+0000 7f8709178700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8704102710 0x7f870410efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.903+0000 7f8709178700 1 -- 172.21.15.70:0/2384723252 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700005040 con 0x7f8704101d70 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.903+0000 7f8709178700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704101d70 0x7f870410ea60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f86f4002a50 tx=0x7f86f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.903+0000 7f86f9ffb700 1 -- 172.21.15.70:0/2384723252 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86f4007cc0 con 0x7f8704101d70 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.903+0000 7f86f9ffb700 1 -- 172.21.15.70:0/2384723252 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f86f400f040 con 0x7f8704101d70 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.903+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8704117ad0 con 0x7f8704101d70 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.903+0000 7f86f9ffb700 1 -- 172.21.15.70:0/2384723252 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86f400d9c0 con 0x7f8704101d70 2024-03-23T08:31:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.903+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8704118030 con 0x7f8704101d70 2024-03-23T08:31:20.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.904+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87040ff080 con 0x7f8704101d70 2024-03-23T08:31:20.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.904+0000 7f86f9ffb700 1 -- 172.21.15.70:0/2384723252 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f86f400db20 con 0x7f8704101d70 2024-03-23T08:31:20.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.905+0000 7f86f9ffb700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f86f006cc30 0x7f86f006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:20.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.905+0000 7f86f9ffb700 1 -- 172.21.15.70:0/2384723252 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f86f408bc70 con 0x7f8704101d70 2024-03-23T08:31:20.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.905+0000 7f8708977700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f86f006cc30 0x7f86f006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:20.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.905+0000 7f8708977700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f86f006cc30 0x7f86f006f0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8704066000 tx=0x7f8700009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:20.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:20.909+0000 7f86f9ffb700 1 -- 172.21.15.70:0/2384723252 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f86f4055cf0 con 0x7f8704101d70 2024-03-23T08:31:21.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.178+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f870404bed0 con 0x7f8704101d70 2024-03-23T08:31:21.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.178+0000 7f86f9ffb700 1 -- 172.21.15.70:0/2384723252 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f86f4059310 con 0x7f8704101d70 2024-03-23T08:31:21.177 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 4 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5, 2024-03-23T08:31:21.178 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 8 2024-03-23T08:31:21.179 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:31:21.179 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:31:21.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f86f006cc30 msgr2=0x7f86f006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f86f006cc30 0x7f86f006f0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8704066000 tx=0x7f8700009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:21.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704101d70 msgr2=0x7f870410ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704101d70 0x7f870410ea60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f86f4002a50 tx=0x7f86f40046b0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 shutdown_connections 2024-03-23T08:31:21.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f86f006cc30 0x7f86f006f0e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8704101d70 0x7f870410ea60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 --2- 172.21.15.70:0/2384723252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8704102710 0x7f870410efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 >> 172.21.15.70:0/2384723252 conn(0x7f8704072bc0 msgr2=0x7f87040fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:21.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 shutdown_connections 2024-03-23T08:31:21.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.180+0000 7f870b3dc700 1 -- 172.21.15.70:0/2384723252 wait complete. 2024-03-23T08:31:21.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.279+0000 7fe438eaa700 1 -- 172.21.15.70:0/2268959410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 msgr2=0x7fe4341031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.279+0000 7fe438eaa700 1 --2- 172.21.15.70:0/2268959410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 0x7fe4341031f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe428004940 tx=0x7fe428009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:21.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.279+0000 7fe438eaa700 1 -- 172.21.15.70:0/2268959410 shutdown_connections 2024-03-23T08:31:21.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.279+0000 7fe438eaa700 1 --2- 172.21.15.70:0/2268959410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4341037c0 0x7fe4340fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.279+0000 7fe438eaa700 1 --2- 172.21.15.70:0/2268959410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 0x7fe4341031f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.279+0000 7fe438eaa700 1 -- 172.21.15.70:0/2268959410 >> 172.21.15.70:0/2268959410 conn(0x7fe4340f89d0 msgr2=0x7fe4340fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:21.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.279+0000 7fe438eaa700 1 -- 172.21.15.70:0/2268959410 shutdown_connections 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.279+0000 7fe438eaa700 1 -- 172.21.15.70:0/2268959410 wait complete. 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe438eaa700 1 Processor -- start 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe438eaa700 1 -- start start 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe438eaa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 0x7fe434113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe438eaa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4341037c0 0x7fe434114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe438eaa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434114670 con 0x7fe434102e20 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe438eaa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43410da60 con 0x7fe4341037c0 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe43259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 0x7fe434113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe43259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 0x7fe434113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43294/0 (socket says 172.21.15.70:43294) 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe43259c700 1 -- 172.21.15.70:0/1040346905 learned_addr learned my addr 172.21.15.70:0/1040346905 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.280+0000 7fe431d9b700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4341037c0 0x7fe434114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.281+0000 7fe43259c700 1 -- 172.21.15.70:0/1040346905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4341037c0 msgr2=0x7fe434114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.281+0000 7fe43259c700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4341037c0 0x7fe434114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.281+0000 7fe43259c700 1 -- 172.21.15.70:0/1040346905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe428005040 con 0x7fe434102e20 2024-03-23T08:31:21.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.281+0000 7fe43259c700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 0x7fe434113ae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe428009360 tx=0x7fe428009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:21.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.281+0000 7fe422ffd700 1 -- 172.21.15.70:0/1040346905 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe42800e040 con 0x7fe434102e20 2024-03-23T08:31:21.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.281+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43410dce0 con 0x7fe434102e20 2024-03-23T08:31:21.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.281+0000 7fe422ffd700 1 -- 172.21.15.70:0/1040346905 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe428013420 con 0x7fe434102e20 2024-03-23T08:31:21.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.281+0000 7fe422ffd700 1 -- 172.21.15.70:0/1040346905 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe42801c5d0 con 0x7fe434102e20 2024-03-23T08:31:21.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.281+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43410e2a0 con 0x7fe434102e20 2024-03-23T08:31:21.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.283+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4340fe570 con 0x7fe434102e20 2024-03-23T08:31:21.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.283+0000 7fe422ffd700 1 -- 172.21.15.70:0/1040346905 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe428013590 con 0x7fe434102e20 2024-03-23T08:31:21.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.283+0000 7fe422ffd700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe41c06cc30 0x7fe41c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:21.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.283+0000 7fe422ffd700 1 -- 172.21.15.70:0/1040346905 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe42808cef0 con 0x7fe434102e20 2024-03-23T08:31:21.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.283+0000 7fe431d9b700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe41c06cc30 0x7fe41c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:21.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.284+0000 7fe431d9b700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe41c06cc30 0x7fe41c06f0e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe43410ec20 tx=0x7fe42400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:21.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.288+0000 7fe422ffd700 1 -- 172.21.15.70:0/1040346905 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe42805a9e0 con 0x7fe434102e20 2024-03-23T08:31:21.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.520+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe43404bed0 con 0x7fe434102e20 2024-03-23T08:31:21.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.520+0000 7fe422ffd700 1 -- 172.21.15.70:0/1040346905 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fe42807a070 con 0x7fe434102e20 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:31:21.520 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:31:21.521 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:31:21.522 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:31:21.522 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:31:21.522 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:31:21.522 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:31:21.522 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:31:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:31:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:31:21.522 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:31:21.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:31:21.523 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:31:21.523 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:31:21.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe41c06cc30 msgr2=0x7fe41c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe41c06cc30 0x7fe41c06f0e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe43410ec20 tx=0x7fe42400b040 comp rx=0 tx=0).stop 2024-03-23T08:31:21.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 msgr2=0x7fe434113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 0x7fe434113ae0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe428009360 tx=0x7fe428009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 shutdown_connections 2024-03-23T08:31:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe41c06cc30 0x7fe41c06f0e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe434102e20 0x7fe434113ae0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 --2- 172.21.15.70:0/1040346905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4341037c0 0x7fe434114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 >> 172.21.15.70:0/1040346905 conn(0x7fe4340f89d0 msgr2=0x7fe4340f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.522+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 shutdown_connections 2024-03-23T08:31:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.523+0000 7fe438eaa700 1 -- 172.21.15.70:0/1040346905 wait complete. 2024-03-23T08:31:21.524 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:31:21.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 -- 172.21.15.70:0/2254895662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c501037b0 msgr2=0x7f4c500fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 --2- 172.21.15.70:0/2254895662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c501037b0 0x7f4c500fce20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4c40004940 tx=0x7f4c40009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 -- 172.21.15.70:0/2254895662 shutdown_connections 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 --2- 172.21.15.70:0/2254895662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c501037b0 0x7f4c500fce20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 --2- 172.21.15.70:0/2254895662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c50102e10 0x7f4c501031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 -- 172.21.15.70:0/2254895662 >> 172.21.15.70:0/2254895662 conn(0x7f4c500f8980 msgr2=0x7f4c500fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 -- 172.21.15.70:0/2254895662 shutdown_connections 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 -- 172.21.15.70:0/2254895662 wait complete. 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 Processor -- start 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.624+0000 7f4c579f7700 1 -- start start 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c579f7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c50102e10 0x7f4c50115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c579f7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c501037b0 0x7f4c50116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c579f7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50116890 con 0x7f4c50102e10 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c579f7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5010fc60 con 0x7f4c501037b0 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c55793700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c50102e10 0x7f4c50115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c55793700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c50102e10 0x7f4c50115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43316/0 (socket says 172.21.15.70:43316) 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c55793700 1 -- 172.21.15.70:0/3582133312 learned_addr learned my addr 172.21.15.70:0/3582133312 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:21.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c54f92700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c501037b0 0x7f4c50116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:21.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c55793700 1 -- 172.21.15.70:0/3582133312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c501037b0 msgr2=0x7f4c50116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c55793700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c501037b0 0x7f4c50116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c55793700 1 -- 172.21.15.70:0/3582133312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c40005040 con 0x7f4c50102e10 2024-03-23T08:31:21.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c55793700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c50102e10 0x7f4c50115d00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800cb10 tx=0x7f4c48007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:21.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c45ffb700 1 -- 172.21.15.70:0/3582133312 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c48004200 con 0x7f4c50102e10 2024-03-23T08:31:21.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c45ffb700 1 -- 172.21.15.70:0/3582133312 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c4800ac50 con 0x7f4c50102e10 2024-03-23T08:31:21.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5010ff40 con 0x7f4c50102e10 2024-03-23T08:31:21.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.625+0000 7f4c45ffb700 1 -- 172.21.15.70:0/3582133312 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c48003ce0 con 0x7f4c50102e10 2024-03-23T08:31:21.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.626+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c50110500 con 0x7f4c50102e10 2024-03-23T08:31:21.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.626+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c500fe560 con 0x7f4c50102e10 2024-03-23T08:31:21.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.627+0000 7f4c45ffb700 1 -- 172.21.15.70:0/3582133312 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4c4800a7a0 con 0x7f4c50102e10 2024-03-23T08:31:21.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.627+0000 7f4c45ffb700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f4c3c06cc20 0x7f4c3c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:21.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.627+0000 7f4c54f92700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f4c3c06cc20 0x7f4c3c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:21.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.627+0000 7f4c45ffb700 1 -- 172.21.15.70:0/3582133312 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4c4808c480 con 0x7f4c50102e10 2024-03-23T08:31:21.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.628+0000 7f4c54f92700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f4c3c06cc20 0x7f4c3c06f0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4c501115a0 tx=0x7f4c40009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:21.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.632+0000 7f4c45ffb700 1 -- 172.21.15.70:0/3582133312 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4c48059f70 con 0x7f4c50102e10 2024-03-23T08:31:21.699 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:21 smithi193 bash[127202]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-23T08:31:21.699 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:21 smithi193 bash[127202]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-23T08:31:21.699 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:21 smithi193 bash[127202]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-23T08:31:21.699 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:21 smithi193 bash[127202]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-23T08:31:21.699 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:21 smithi193 bash[127202]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-23T08:31:21.699 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:21 smithi193 bash[127202]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-23T08:31:21.699 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:21 smithi193 bash[127202]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-23T08:31:21.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.850+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c5004bed0 con 0x7f4c3c06cc20 2024-03-23T08:31:21.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.851+0000 7f4c45ffb700 1 -- 172.21.15.70:0/3582133312 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4c5004bed0 con 0x7f4c3c06cc20 2024-03-23T08:31:21.850 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:31:21.850 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:31:21.850 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:31:21.850 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:31:21.850 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:31:21.851 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:31:21.851 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-03-23T08:31:21.851 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-03-23T08:31:21.851 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:31:21.851 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "10/21 daemons upgraded", 2024-03-23T08:31:21.851 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-03-23T08:31:21.851 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:31:21.851 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:31:21.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f4c3c06cc20 msgr2=0x7f4c3c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f4c3c06cc20 0x7f4c3c06f0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4c501115a0 tx=0x7f4c40009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:21.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c50102e10 msgr2=0x7f4c50115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c50102e10 0x7f4c50115d00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800cb10 tx=0x7f4c48007f90 comp rx=0 tx=0).stop 2024-03-23T08:31:21.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 shutdown_connections 2024-03-23T08:31:21.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f4c3c06cc20 0x7f4c3c06f0d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c50102e10 0x7f4c50115d00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 --2- 172.21.15.70:0/3582133312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c501037b0 0x7f4c50116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 >> 172.21.15.70:0/3582133312 conn(0x7f4c500f8980 msgr2=0x7f4c500f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:21.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 shutdown_connections 2024-03-23T08:31:21.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.853+0000 7f4c579f7700 1 -- 172.21.15.70:0/3582133312 wait complete. 2024-03-23T08:31:21.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.954+0000 7f05b2654700 1 -- 172.21.15.70:0/3629071448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac102710 msgr2=0x7f05ac0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.954+0000 7f05b2654700 1 --2- 172.21.15.70:0/3629071448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac102710 0x7f05ac0fbd80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f059c004940 tx=0x7f059c009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:21.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.954+0000 7f05b2654700 1 -- 172.21.15.70:0/3629071448 shutdown_connections 2024-03-23T08:31:21.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.954+0000 7f05b2654700 1 --2- 172.21.15.70:0/3629071448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac102710 0x7f05ac0fbd80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.954+0000 7f05b2654700 1 --2- 172.21.15.70:0/3629071448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05ac101d70 0x7f05ac102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.954+0000 7f05b2654700 1 -- 172.21.15.70:0/3629071448 >> 172.21.15.70:0/3629071448 conn(0x7f05ac072bc0 msgr2=0x7f05ac072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:21.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.954+0000 7f05b2654700 1 -- 172.21.15.70:0/3629071448 shutdown_connections 2024-03-23T08:31:21.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.954+0000 7f05b2654700 1 -- 172.21.15.70:0/3629071448 wait complete. 2024-03-23T08:31:21.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05b2654700 1 Processor -- start 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05b2654700 1 -- start start 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05b2654700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac101d70 0x7f05ac103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05b2654700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05ac102710 0x7f05ac103880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05b2654700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac103e10 con 0x7f05ac101d70 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05b2654700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac103f50 con 0x7f05ac102710 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac101d70 0x7f05ac103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac101d70 0x7f05ac103340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43342/0 (socket says 172.21.15.70:43342) 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05abfff700 1 -- 172.21.15.70:0/4279870638 learned_addr learned my addr 172.21.15.70:0/4279870638 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05ab7fe700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05ac102710 0x7f05ac103880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05abfff700 1 -- 172.21.15.70:0/4279870638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05ac102710 msgr2=0x7f05ac103880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05abfff700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05ac102710 0x7f05ac103880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05abfff700 1 -- 172.21.15.70:0/4279870638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c005040 con 0x7f05ac101d70 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.955+0000 7f05ab7fe700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05ac102710 0x7f05ac103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.956+0000 7f05abfff700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac101d70 0x7f05ac103340 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f05a000c830 tx=0x7f05a0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:21.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.956+0000 7f05a8ff9700 1 -- 172.21.15.70:0/4279870638 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05a000a760 con 0x7f05ac101d70 2024-03-23T08:31:21.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.956+0000 7f05a8ff9700 1 -- 172.21.15.70:0/4279870638 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05a000ad70 con 0x7f05ac101d70 2024-03-23T08:31:21.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.956+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac119da0 con 0x7f05ac101d70 2024-03-23T08:31:21.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.956+0000 7f05a8ff9700 1 -- 172.21.15.70:0/4279870638 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05a00113a0 con 0x7f05ac101d70 2024-03-23T08:31:21.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.956+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac11a220 con 0x7f05ac101d70 2024-03-23T08:31:21.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.957+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac0fd4c0 con 0x7f05ac101d70 2024-03-23T08:31:21.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.957+0000 7f05a8ff9700 1 -- 172.21.15.70:0/4279870638 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f05a0012400 con 0x7f05ac101d70 2024-03-23T08:31:21.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.958+0000 7f05a8ff9700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f059406cc20 0x7f059406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:21.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.958+0000 7f05a8ff9700 1 -- 172.21.15.70:0/4279870638 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f05a008c000 con 0x7f05ac101d70 2024-03-23T08:31:21.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.958+0000 7f05ab7fe700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f059406cc20 0x7f059406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:21.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.958+0000 7f05ab7fe700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f059406cc20 0x7f059406f0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f059c00b3b0 tx=0x7f059c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:21.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:21.962+0000 7f05a8ff9700 1 -- 172.21.15.70:0/4279870638 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f05a00560e0 con 0x7f05ac101d70 2024-03-23T08:31:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:21 smithi193 ceph-mon[108533]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:21 smithi193 ceph-mon[108533]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:21 smithi193 ceph-mon[108533]: pgmap v132: 65 pgs: 5 stale+active+clean, 2 active+undersized, 58 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-23T08:31:22.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:21 smithi193 ceph-mon[108533]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:22.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:21 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2384723252' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:22.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:21 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1040346905' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:31:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:21 smithi070 ceph-mon[145869]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:21 smithi070 ceph-mon[145869]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:21 smithi070 ceph-mon[145869]: pgmap v132: 65 pgs: 5 stale+active+clean, 2 active+undersized, 58 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-23T08:31:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:21 smithi070 ceph-mon[145869]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:21 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2384723252' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:21 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1040346905' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:31:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.234+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f05ac05f5f0 con 0x7f05ac101d70 2024-03-23T08:31:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.234+0000 7f05a8ff9700 1 -- 172.21.15.70:0/4279870638 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f05a0059700 con 0x7f05ac101d70 2024-03-23T08:31:22.234 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN 1 osds down 2024-03-23T08:31:22.234 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-23T08:31:22.234 INFO:teuthology.orchestra.run.smithi070.stdout: osd.4 (root=default,host=smithi193) is down 2024-03-23T08:31:22.235 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f059406cc20 msgr2=0x7f059406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f059406cc20 0x7f059406f0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f059c00b3b0 tx=0x7f059c009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac101d70 msgr2=0x7f05ac103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac101d70 0x7f05ac103340 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f05a000c830 tx=0x7f05a0004470 comp rx=0 tx=0).stop 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 shutdown_connections 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f059406cc20 0x7f059406f0d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ac101d70 0x7f05ac103340 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 --2- 172.21.15.70:0/4279870638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05ac102710 0x7f05ac103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 >> 172.21.15.70:0/4279870638 conn(0x7f05ac072bc0 msgr2=0x7f05ac1058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 shutdown_connections 2024-03-23T08:31:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:22.236+0000 7f05b2654700 1 -- 172.21.15.70:0/4279870638 wait complete. 2024-03-23T08:31:22.810 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:22 smithi193 ceph-osd[127501]: -- 172.21.15.193:0/299703885 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b42ccdd1e0 con 0x55b42ccad000 2024-03-23T08:31:22.810 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:22 smithi193 bash[127463]: 76a25ee5af50a33def80ae333d1c72f4923e7c9a5b8463695a58ad3f118da4b2 2024-03-23T08:31:22.810 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:22 smithi193 systemd[1]: Started Ceph osd.4 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:31:23.075 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:23 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:31:23.053+0000 7ff336ae8640 -1 Falling back to public interface 2024-03-23T08:31:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:22 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/4279870638' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:31:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:23.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:31:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:22 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/4279870638' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:31:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:31:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:23 smithi193 ceph-mon[108533]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:23 smithi193 ceph-mon[108533]: pgmap v133: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:23 smithi193 ceph-mon[108533]: Health check failed: Degraded data redundancy: 39/222 objects degraded (17.568%), 15 pgs degraded (PG_DEGRADED) 2024-03-23T08:31:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:23 smithi070 ceph-mon[145869]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:23 smithi070 ceph-mon[145869]: pgmap v133: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:23 smithi070 ceph-mon[145869]: Health check failed: Degraded data redundancy: 39/222 objects degraded (17.568%), 15 pgs degraded (PG_DEGRADED) 2024-03-23T08:31:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:25 smithi193 ceph-mon[108533]: pgmap v134: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:25 smithi070 ceph-mon[145869]: pgmap v134: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:27.312 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:27 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:27.312 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:27 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:27.312 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:27 smithi193 ceph-mon[108533]: pgmap v135: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 581 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:27.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:27 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:27.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:27 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:27.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:27 smithi070 ceph-mon[145869]: pgmap v135: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 581 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:27.935 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:27 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:31:27.612+0000 7ff336ae8640 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-23T08:31:28.325 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:28 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:31:28.142+0000 7ff336ae8640 -1 osd.4 71 log_to_monitors true 2024-03-23T08:31:28.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:28 smithi193 ceph-mon[108533]: from='osd.4 [v2:172.21.15.193:6808/3936171891,v1:172.21.15.193:6809/3936171891]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T08:31:28.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:28 smithi193 ceph-mon[108533]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T08:31:28.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:28 smithi070 ceph-mon[145869]: from='osd.4 [v2:172.21.15.193:6808/3936171891,v1:172.21.15.193:6809/3936171891]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T08:31:28.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:28 smithi070 ceph-mon[145869]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-23T08:31:29.325 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:31:29 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:31:29.074+0000 7ff323f39700 -1 osd.4 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:31:29.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:29 smithi193 ceph-mon[108533]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T08:31:29.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:29 smithi193 ceph-mon[108533]: osdmap e74: 6 total, 5 up, 6 in 2024-03-23T08:31:29.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:29 smithi193 ceph-mon[108533]: from='osd.4 [v2:172.21.15.193:6808/3936171891,v1:172.21.15.193:6809/3936171891]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:31:29.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:29 smithi193 ceph-mon[108533]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:31:29.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:29 smithi193 ceph-mon[108533]: pgmap v137: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 522 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:29.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:29 smithi193 ceph-mon[108533]: from='osd.4 ' entity='osd.4' 2024-03-23T08:31:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:29 smithi070 ceph-mon[145869]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-23T08:31:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:29 smithi070 ceph-mon[145869]: osdmap e74: 6 total, 5 up, 6 in 2024-03-23T08:31:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:29 smithi070 ceph-mon[145869]: from='osd.4 [v2:172.21.15.193:6808/3936171891,v1:172.21.15.193:6809/3936171891]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:31:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:29 smithi070 ceph-mon[145869]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:31:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:29 smithi070 ceph-mon[145869]: pgmap v137: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 522 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:29.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:29 smithi070 ceph-mon[145869]: from='osd.4 ' entity='osd.4' 2024-03-23T08:31:31.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:31:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:31:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:31:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: osd.4 [v2:172.21.15.193:6808/3936171891,v1:172.21.15.193:6809/3936171891] boot 2024-03-23T08:31:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: osdmap e75: 6 total, 6 up, 6 in 2024-03-23T08:31:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:31:31.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:31:31.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:31.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:31.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:31.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-23T08:31:31.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-23T08:31:31.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-23T08:31:31.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:31 smithi193 ceph-mon[108533]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:31:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:31:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:31.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: osd.4 [v2:172.21.15.193:6808/3936171891,v1:172.21.15.193:6809/3936171891] boot 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: osdmap e75: 6 total, 6 up, 6 in 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-23T08:31:31.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:31 smithi070 ceph-mon[145869]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 39/222 objects degraded (17.568%) 2024-03-23T08:31:32.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:32 smithi070 ceph-mon[145869]: osdmap e76: 6 total, 6 up, 6 in 2024-03-23T08:31:32.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:32 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:32.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:32 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:31:32.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:32 smithi193 ceph-mon[108533]: osdmap e76: 6 total, 6 up, 6 in 2024-03-23T08:31:32.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:32 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:32.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:32 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:31:33.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:33 smithi070 ceph-mon[145869]: pgmap v141: 65 pgs: 23 peering, 2 active+undersized+degraded, 5 active+undersized, 35 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 4/222 objects degraded (1.802%) 2024-03-23T08:31:33.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:33 smithi193 ceph-mon[108533]: pgmap v141: 65 pgs: 23 peering, 2 active+undersized+degraded, 5 active+undersized, 35 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 4/222 objects degraded (1.802%) 2024-03-23T08:31:34.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:34 smithi070 ceph-mon[145869]: Health check update: Degraded data redundancy: 4/222 objects degraded (1.802%), 2 pgs degraded (PG_DEGRADED) 2024-03-23T08:31:34.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:34 smithi193 ceph-mon[108533]: Health check update: Degraded data redundancy: 4/222 objects degraded (1.802%), 2 pgs degraded (PG_DEGRADED) 2024-03-23T08:31:35.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:35 smithi070 ceph-mon[145869]: pgmap v142: 65 pgs: 23 peering, 1 active+undersized, 41 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 476 B/s rd, 1 op/s 2024-03-23T08:31:35.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:35 smithi193 ceph-mon[108533]: pgmap v142: 65 pgs: 23 peering, 1 active+undersized, 41 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 476 B/s rd, 1 op/s 2024-03-23T08:31:36.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:36 smithi070 ceph-mon[145869]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/222 objects degraded (1.802%), 2 pgs degraded) 2024-03-23T08:31:36.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:36 smithi070 ceph-mon[145869]: Cluster is now healthy 2024-03-23T08:31:36.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:36 smithi193 ceph-mon[108533]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/222 objects degraded (1.802%), 2 pgs degraded) 2024-03-23T08:31:36.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:36 smithi193 ceph-mon[108533]: Cluster is now healthy 2024-03-23T08:31:38.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:37 smithi193 ceph-mon[108533]: pgmap v143: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-03-23T08:31:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:37 smithi070 ceph-mon[145869]: pgmap v143: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-03-23T08:31:40.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:39 smithi193 ceph-mon[108533]: pgmap v144: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 959 B/s rd, 1 op/s 2024-03-23T08:31:40.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:39 smithi070 ceph-mon[145869]: pgmap v144: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 959 B/s rd, 1 op/s 2024-03-23T08:31:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:41 smithi193 ceph-mon[108533]: pgmap v145: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-03-23T08:31:42.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:41 smithi070 ceph-mon[145869]: pgmap v145: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 1 op/s 2024-03-23T08:31:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:44.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:43 smithi193 ceph-mon[108533]: pgmap v146: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 887 B/s rd, 1 op/s 2024-03-23T08:31:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:43 smithi070 ceph-mon[145869]: pgmap v146: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 887 B/s rd, 1 op/s 2024-03-23T08:31:46.029 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:45 smithi193 ceph-mon[108533]: pgmap v147: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:31:46.029 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-23T08:31:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:45 smithi070 ceph-mon[145869]: pgmap v147: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:31:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-23T08:31:47.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:46 smithi193 ceph-mon[108533]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-23T08:31:47.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:46 smithi193 ceph-mon[108533]: Upgrade: osd.5 is safe to restart 2024-03-23T08:31:47.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:47.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:31:47.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:47.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T08:31:47.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:31:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:46 smithi070 ceph-mon[145869]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-23T08:31:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:46 smithi070 ceph-mon[145869]: Upgrade: osd.5 is safe to restart 2024-03-23T08:31:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:31:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-23T08:31:47.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:31:48.014 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:47 smithi193 ceph-mon[108533]: pgmap v148: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:31:48.014 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:47 smithi193 ceph-mon[108533]: Upgrade: Updating osd.5 2024-03-23T08:31:48.014 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:47 smithi193 ceph-mon[108533]: Deploying daemon osd.5 on smithi193 2024-03-23T08:31:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:47 smithi070 ceph-mon[145869]: pgmap v148: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:31:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:47 smithi070 ceph-mon[145869]: Upgrade: Updating osd.5 2024-03-23T08:31:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:47 smithi070 ceph-mon[145869]: Deploying daemon osd.5 on smithi193 2024-03-23T08:31:49.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:49 smithi193 ceph-mon[108533]: pgmap v149: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:31:49.991 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:49 smithi193 systemd[1]: Stopping Ceph osd.5 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:31:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:49 smithi070 ceph-mon[145869]: pgmap v149: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:31:50.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:49 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[71571]: 2024-03-23T08:31:49.989+0000 7f97283cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:31:50.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:49 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[71571]: 2024-03-23T08:31:49.989+0000 7f97283cc700 -1 osd.5 76 *** Got signal Terminated *** 2024-03-23T08:31:50.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:49 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[71571]: 2024-03-23T08:31:49.989+0000 7f97283cc700 -1 osd.5 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:31:51.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:50 smithi193 ceph-mon[108533]: osd.5 marked itself down and dead 2024-03-23T08:31:51.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:50 smithi070 ceph-mon[145869]: osd.5 marked itself down and dead 2024-03-23T08:31:51.807 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:51 smithi193 bash[133042]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5 2024-03-23T08:31:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:51 smithi193 ceph-mon[108533]: pgmap v150: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:31:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:51 smithi193 ceph-mon[108533]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:31:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:51 smithi193 ceph-mon[108533]: osdmap e77: 6 total, 5 up, 6 in 2024-03-23T08:31:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:51 smithi070 ceph-mon[145869]: pgmap v150: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:31:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:51 smithi070 ceph-mon[145869]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-23T08:31:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:51 smithi070 ceph-mon[145869]: osdmap e77: 6 total, 5 up, 6 in 2024-03-23T08:31:52.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.349+0000 7f62afa4e700 1 -- 172.21.15.70:0/2580994529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8101d70 msgr2=0x7f62a8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.349+0000 7f62afa4e700 1 --2- 172.21.15.70:0/2580994529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8101d70 0x7f62a8102140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f62a4004940 tx=0x7f62a4009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.349+0000 7f62afa4e700 1 -- 172.21.15.70:0/2580994529 shutdown_connections 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.349+0000 7f62afa4e700 1 --2- 172.21.15.70:0/2580994529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62a8102710 0x7f62a80fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.349+0000 7f62afa4e700 1 --2- 172.21.15.70:0/2580994529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8101d70 0x7f62a8102140 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.349+0000 7f62afa4e700 1 -- 172.21.15.70:0/2580994529 >> 172.21.15.70:0/2580994529 conn(0x7f62a8072bc0 msgr2=0x7f62a8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.349+0000 7f62afa4e700 1 -- 172.21.15.70:0/2580994529 shutdown_connections 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.349+0000 7f62afa4e700 1 -- 172.21.15.70:0/2580994529 wait complete. 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.350+0000 7f62afa4e700 1 Processor -- start 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.350+0000 7f62afa4e700 1 -- start start 2024-03-23T08:31:52.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.350+0000 7f62afa4e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62a8101d70 0x7f62a810ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.350+0000 7f62afa4e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8102710 0x7f62a810f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.350+0000 7f62afa4e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a810f6f0 con 0x7f62a8102710 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.350+0000 7f62afa4e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a810f830 con 0x7f62a8101d70 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.350+0000 7f62acfe9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8102710 0x7f62a810f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.350+0000 7f62acfe9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8102710 0x7f62a810f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46422/0 (socket says 172.21.15.70:46422) 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.350+0000 7f62acfe9700 1 -- 172.21.15.70:0/2341162025 learned_addr learned my addr 172.21.15.70:0/2341162025 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f62ad7ea700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62a8101d70 0x7f62a810ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f62acfe9700 1 -- 172.21.15.70:0/2341162025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62a8101d70 msgr2=0x7f62a810ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f62acfe9700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62a8101d70 0x7f62a810ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f62acfe9700 1 -- 172.21.15.70:0/2341162025 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4005040 con 0x7f62a8102710 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f62acfe9700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8102710 0x7f62a810f1b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f629800b760 tx=0x7f62980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f629dffb700 1 -- 172.21.15.70:0/2341162025 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f629800f6f0 con 0x7f62a8102710 2024-03-23T08:31:52.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a8117b80 con 0x7f62a8102710 2024-03-23T08:31:52.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f629dffb700 1 -- 172.21.15.70:0/2341162025 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f629800fd00 con 0x7f62a8102710 2024-03-23T08:31:52.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f629dffb700 1 -- 172.21.15.70:0/2341162025 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6298011d90 con 0x7f62a8102710 2024-03-23T08:31:52.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.351+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a810cae0 con 0x7f62a8102710 2024-03-23T08:31:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.352+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a80ff080 con 0x7f62a8102710 2024-03-23T08:31:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.353+0000 7f629dffb700 1 -- 172.21.15.70:0/2341162025 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f629800f850 con 0x7f62a8102710 2024-03-23T08:31:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.353+0000 7f629dffb700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6294071040 0x7f62940734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.353+0000 7f629dffb700 1 -- 172.21.15.70:0/2341162025 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f629808bbe0 con 0x7f62a8102710 2024-03-23T08:31:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.353+0000 7f62ad7ea700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6294071040 0x7f62940734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:52.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.354+0000 7f62ad7ea700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6294071040 0x7f62940734f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62a400b840 tx=0x7f62a4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:52.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.358+0000 7f629dffb700 1 -- 172.21.15.70:0/2341162025 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6298055cc0 con 0x7f62a8102710 2024-03-23T08:31:52.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.570+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62a8065dd0 con 0x7f6294071040 2024-03-23T08:31:52.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.572+0000 7f629dffb700 1 -- 172.21.15.70:0/2341162025 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f62a8065dd0 con 0x7f6294071040 2024-03-23T08:31:52.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6294071040 msgr2=0x7f62940734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:52.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6294071040 0x7f62940734f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62a400b840 tx=0x7f62a4009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:52.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8102710 msgr2=0x7f62a810f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8102710 0x7f62a810f1b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f629800b760 tx=0x7f62980091a0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 shutdown_connections 2024-03-23T08:31:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6294071040 0x7f62940734f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62a8101d70 0x7f62a810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 --2- 172.21.15.70:0/2341162025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a8102710 0x7f62a810f1b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 >> 172.21.15.70:0/2341162025 conn(0x7f62a8072bc0 msgr2=0x7f62a80fb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 shutdown_connections 2024-03-23T08:31:52.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.574+0000 7f62afa4e700 1 -- 172.21.15.70:0/2341162025 wait complete. 2024-03-23T08:31:52.584 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:31:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 -- 172.21.15.70:0/1417908100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 msgr2=0x7f12700fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 --2- 172.21.15.70:0/1417908100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 0x7f12700fe260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1264004940 tx=0x7f1264009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 -- 172.21.15.70:0/1417908100 shutdown_connections 2024-03-23T08:31:52.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 --2- 172.21.15.70:0/1417908100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12700fe830 0x7f1270106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 --2- 172.21.15.70:0/1417908100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 0x7f12700fe260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 -- 172.21.15.70:0/1417908100 >> 172.21.15.70:0/1417908100 conn(0x7f1270072bc0 msgr2=0x7f1270072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 -- 172.21.15.70:0/1417908100 shutdown_connections 2024-03-23T08:31:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 -- 172.21.15.70:0/1417908100 wait complete. 2024-03-23T08:31:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 Processor -- start 2024-03-23T08:31:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.675+0000 7f12749ae700 1 -- start start 2024-03-23T08:31:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f12749ae700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 0x7f1270113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f12749ae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12700fe830 0x7f1270114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:52.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f12749ae700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270114680 con 0x7f12700fde90 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f12749ae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127010da70 con 0x7f12700fe830 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f126e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 0x7f1270113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f126e59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 0x7f1270113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46436/0 (socket says 172.21.15.70:46436) 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f126e59c700 1 -- 172.21.15.70:0/975408852 learned_addr learned my addr 172.21.15.70:0/975408852 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f126dd9b700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12700fe830 0x7f1270114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f126e59c700 1 -- 172.21.15.70:0/975408852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12700fe830 msgr2=0x7f1270114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f126e59c700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12700fe830 0x7f1270114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f126e59c700 1 -- 172.21.15.70:0/975408852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1264005040 con 0x7f12700fde90 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f126dd9b700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12700fe830 0x7f1270114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f126e59c700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 0x7f1270113af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f126400b3b0 tx=0x7f1264009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f125effd700 1 -- 172.21.15.70:0/975408852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f126400e040 con 0x7f12700fde90 2024-03-23T08:31:52.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f125effd700 1 -- 172.21.15.70:0/975408852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1264013420 con 0x7f12700fde90 2024-03-23T08:31:52.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f125effd700 1 -- 172.21.15.70:0/975408852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f126401c5d0 con 0x7f12700fde90 2024-03-23T08:31:52.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127010dcf0 con 0x7f12700fde90 2024-03-23T08:31:52.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.676+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127010e230 con 0x7f12700fde90 2024-03-23T08:31:52.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.677+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12701019b0 con 0x7f12700fde90 2024-03-23T08:31:52.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.678+0000 7f125effd700 1 -- 172.21.15.70:0/975408852 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1264013590 con 0x7f12700fde90 2024-03-23T08:31:52.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.678+0000 7f125effd700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f125806cc20 0x7f125806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:52.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.678+0000 7f125effd700 1 -- 172.21.15.70:0/975408852 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f126408d210 con 0x7f12700fde90 2024-03-23T08:31:52.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.678+0000 7f126dd9b700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f125806cc20 0x7f125806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:52.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.678+0000 7f126dd9b700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f125806cc20 0x7f125806f0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1260009fd0 tx=0x7f1260009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:52.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.683+0000 7f125effd700 1 -- 172.21.15.70:0/975408852 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f126405ad00 con 0x7f12700fde90 2024-03-23T08:31:52.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.895+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f127010f140 con 0x7f125806cc20 2024-03-23T08:31:52.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.896+0000 7f125effd700 1 -- 172.21.15.70:0/975408852 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f127010f140 con 0x7f125806cc20 2024-03-23T08:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f125806cc20 msgr2=0x7f125806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f125806cc20 0x7f125806f0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1260009fd0 tx=0x7f1260009ee0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 msgr2=0x7f1270113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 0x7f1270113af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f126400b3b0 tx=0x7f1264009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 shutdown_connections 2024-03-23T08:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f125806cc20 0x7f125806f0d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12700fde90 0x7f1270113af0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 --2- 172.21.15.70:0/975408852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12700fe830 0x7f1270114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 >> 172.21.15.70:0/975408852 conn(0x7f1270072bc0 msgr2=0x7f12700ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 shutdown_connections 2024-03-23T08:31:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.898+0000 7f12749ae700 1 -- 172.21.15.70:0/975408852 wait complete. 2024-03-23T08:31:53.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.998+0000 7f902b293700 1 -- 172.21.15.70:0/2400792740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 msgr2=0x7f9024102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.998+0000 7f902b293700 1 --2- 172.21.15.70:0/2400792740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 0x7f9024102140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9014004940 tx=0x7f9014009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:53.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.998+0000 7f902b293700 1 -- 172.21.15.70:0/2400792740 shutdown_connections 2024-03-23T08:31:53.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.998+0000 7f902b293700 1 --2- 172.21.15.70:0/2400792740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9024102710 0x7f90240fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.998+0000 7f902b293700 1 --2- 172.21.15.70:0/2400792740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 0x7f9024102140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.998+0000 7f902b293700 1 -- 172.21.15.70:0/2400792740 >> 172.21.15.70:0/2400792740 conn(0x7f9024072bc0 msgr2=0x7f9024072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:53.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.998+0000 7f902b293700 1 -- 172.21.15.70:0/2400792740 shutdown_connections 2024-03-23T08:31:53.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.998+0000 7f902b293700 1 -- 172.21.15.70:0/2400792740 wait complete. 2024-03-23T08:31:53.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902b293700 1 Processor -- start 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902b293700 1 -- start start 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902b293700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 0x7f9024113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902b293700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9024102710 0x7f9024113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902b293700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024114620 con 0x7f9024101d70 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902b293700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902410da60 con 0x7f9024102710 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902902f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 0x7f9024113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902902f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 0x7f9024113a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46450/0 (socket says 172.21.15.70:46450) 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902902f700 1 -- 172.21.15.70:0/2060771679 learned_addr learned my addr 172.21.15.70:0/2060771679 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902882e700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9024102710 0x7f9024113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902902f700 1 -- 172.21.15.70:0/2060771679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9024102710 msgr2=0x7f9024113fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902902f700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9024102710 0x7f9024113fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:52.999+0000 7f902902f700 1 -- 172.21.15.70:0/2060771679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014005040 con 0x7f9024101d70 2024-03-23T08:31:53.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.000+0000 7f902902f700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 0x7f9024113a90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f901400b3b0 tx=0x7f9014009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:53.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.000+0000 7f9019ffb700 1 -- 172.21.15.70:0/2060771679 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f901400e040 con 0x7f9024101d70 2024-03-23T08:31:53.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.000+0000 7f9019ffb700 1 -- 172.21.15.70:0/2060771679 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9014013420 con 0x7f9024101d70 2024-03-23T08:31:53.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.000+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902410dce0 con 0x7f9024101d70 2024-03-23T08:31:53.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.000+0000 7f9019ffb700 1 -- 172.21.15.70:0/2060771679 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f901401c630 con 0x7f9024101d70 2024-03-23T08:31:53.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.000+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902410e2a0 con 0x7f9024101d70 2024-03-23T08:31:53.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.001+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90240fd4c0 con 0x7f9024101d70 2024-03-23T08:31:53.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.001+0000 7f9019ffb700 1 -- 172.21.15.70:0/2060771679 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9014013970 con 0x7f9024101d70 2024-03-23T08:31:53.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.002+0000 7f9019ffb700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901006cc20 0x7f901006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:53.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.002+0000 7f9019ffb700 1 -- 172.21.15.70:0/2060771679 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f901408d0f0 con 0x7f9024101d70 2024-03-23T08:31:53.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.002+0000 7f902882e700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901006cc20 0x7f901006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:53.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.002+0000 7f902882e700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901006cc20 0x7f901006f0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9020007fd0 tx=0x7f9020009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:53.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.007+0000 7f9019ffb700 1 -- 172.21.15.70:0/2060771679 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f901405ad00 con 0x7f9024101d70 2024-03-23T08:31:53.069 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:52 smithi193 ceph-mon[108533]: osdmap e78: 6 total, 5 up, 6 in 2024-03-23T08:31:53.208 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.208+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9024108da0 con 0x7f901006cc20 2024-03-23T08:31:53.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.217+0000 7f9019ffb700 1 -- 172.21.15.70:0/2060771679 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f9024108da0 con 0x7f901006cc20 2024-03-23T08:31:53.217 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:31:53.217 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (5m) 93s ago 16m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:31:53.217 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (3m) 93s ago 16m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:31:53.217 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (3m) 27s ago 15m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (5m) 93s ago 16m 83.9M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (11m) 93s ago 11m 146M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (11m) 93s ago 11m 14.4M - 17.2.7-726-g9c4e0c34 837124da804f fe197428d779 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (11m) 27s ago 11m 97.3M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (11m) 27s ago 11m 15.9M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (7m) 93s ago 18m 501M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (6m) 27s ago 15m 449M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (4m) 93s ago 18m 48.9M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (3m) 27s ago 15m 39.3M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (6m) 93s ago 16m 18.6M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (6m) 27s ago 15m 16.6M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (2m) 93s ago 14m 85.0M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:31:53.218 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (2m) 93s ago 14m 96.3M 1568M 18.2.2-954-g82316b04 b24e1baba637 13fce63c86eb 2024-03-23T08:31:53.219 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (98s) 93s ago 13m 10.7M 1568M 18.2.2-954-g82316b04 b24e1baba637 73e36e623491 2024-03-23T08:31:53.219 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (63s) 27s ago 13m 100M 2592M 18.2.2-954-g82316b04 b24e1baba637 8a7617f5541a 2024-03-23T08:31:53.219 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (31s) 27s ago 13m 10.7M 2592M 18.2.2-954-g82316b04 b24e1baba637 76a25ee5af50 2024-03-23T08:31:53.219 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (12m) 27s ago 12m 227M 2592M 17.2.7-726-g9c4e0c34 837124da804f 47be9a22b04e 2024-03-23T08:31:53.219 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (5m) 93s ago 16m 60.3M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901006cc20 msgr2=0x7f901006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901006cc20 0x7f901006f0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9020007fd0 tx=0x7f9020009040 comp rx=0 tx=0).stop 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 msgr2=0x7f9024113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 0x7f9024113a90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f901400b3b0 tx=0x7f9014009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 shutdown_connections 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f901006cc20 0x7f901006f0d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9024101d70 0x7f9024113a90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 --2- 172.21.15.70:0/2060771679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9024102710 0x7f9024113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 >> 172.21.15.70:0/2060771679 conn(0x7f9024072bc0 msgr2=0x7f9024065a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 shutdown_connections 2024-03-23T08:31:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.219+0000 7f902b293700 1 -- 172.21.15.70:0/2060771679 wait complete. 2024-03-23T08:31:53.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:52 smithi070 ceph-mon[145869]: osdmap e78: 6 total, 5 up, 6 in 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/2640719007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0102710 msgr2=0x7fe5d00fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 --2- 172.21.15.70:0/2640719007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0102710 0x7fe5d00fbd80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0004940 tx=0x7fe5c0009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/2640719007 shutdown_connections 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 --2- 172.21.15.70:0/2640719007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0102710 0x7fe5d00fbd80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 --2- 172.21.15.70:0/2640719007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5d0101d70 0x7fe5d0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/2640719007 >> 172.21.15.70:0/2640719007 conn(0x7fe5d0072bc0 msgr2=0x7fe5d0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/2640719007 shutdown_connections 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/2640719007 wait complete. 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 Processor -- start 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.318+0000 7fe5d4f7e700 1 -- start start 2024-03-23T08:31:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5d4f7e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101d70 0x7fe5d010ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5d4f7e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5d0102710 0x7fe5d010efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5d4f7e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d010f5f0 con 0x7fe5d0101d70 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5d4f7e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d010f730 con 0x7fe5d0102710 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5ce59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101d70 0x7fe5d010ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5cdd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5d0102710 0x7fe5d010efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5ce59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101d70 0x7fe5d010ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46460/0 (socket says 172.21.15.70:46460) 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5cdd9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5d0102710 0x7fe5d010efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.70:52860/0 (socket says 172.21.15.70:52860) 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5ce59c700 1 -- 172.21.15.70:0/1716107152 learned_addr learned my addr 172.21.15.70:0/1716107152 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5ce59c700 1 -- 172.21.15.70:0/1716107152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5d0102710 msgr2=0x7fe5d010efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5ce59c700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5d0102710 0x7fe5d010efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5ce59c700 1 -- 172.21.15.70:0/1716107152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c0005040 con 0x7fe5d0101d70 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5ce59c700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101d70 0x7fe5d010ea60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4002a50 tx=0x7fe5c40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5beffd700 1 -- 172.21.15.70:0/1716107152 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5c4007cc0 con 0x7fe5d0101d70 2024-03-23T08:31:53.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5beffd700 1 -- 172.21.15.70:0/1716107152 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5c400f040 con 0x7fe5d0101d70 2024-03-23T08:31:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5beffd700 1 -- 172.21.15.70:0/1716107152 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5c400d9c0 con 0x7fe5d0101d70 2024-03-23T08:31:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.319+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d0117ad0 con 0x7fe5d0101d70 2024-03-23T08:31:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.320+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d0117fe0 con 0x7fe5d0101d70 2024-03-23T08:31:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.321+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d00ff080 con 0x7fe5d0101d70 2024-03-23T08:31:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.321+0000 7fe5beffd700 1 -- 172.21.15.70:0/1716107152 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe5c400db20 con 0x7fe5d0101d70 2024-03-23T08:31:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.321+0000 7fe5beffd700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe5b806cc20 0x7fe5b806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.321+0000 7fe5beffd700 1 -- 172.21.15.70:0/1716107152 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe5c408bcc0 con 0x7fe5d0101d70 2024-03-23T08:31:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.321+0000 7fe5cdd9b700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe5b806cc20 0x7fe5b806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:53.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.322+0000 7fe5cdd9b700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe5b806cc20 0x7fe5b806f0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0066000 tx=0x7fe5c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:53.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:53 smithi193 systemd[1]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.5.service: Succeeded. 2024-03-23T08:31:53.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:53 smithi193 systemd[1]: Stopped Ceph osd.5 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:31:53.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.326+0000 7fe5beffd700 1 -- 172.21.15.70:0/1716107152 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe5c4055d20 con 0x7fe5d0101d70 2024-03-23T08:31:53.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.600+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe5d004bed0 con 0x7fe5d0101d70 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.600+0000 7fe5beffd700 1 -- 172.21.15.70:0/1716107152 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fe5c4059340 con 0x7fe5d0101d70 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 5 2024-03-23T08:31:53.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:31:53.602 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:31:53.602 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-23T08:31:53.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:31:53.602 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:31:53.602 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-23T08:31:53.602 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 9 2024-03-23T08:31:53.602 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:31:53.602 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:31:53.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe5b806cc20 msgr2=0x7fe5b806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe5b806cc20 0x7fe5b806f0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0066000 tx=0x7fe5c0009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:53.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101d70 msgr2=0x7fe5d010ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101d70 0x7fe5d010ea60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4002a50 tx=0x7fe5c40046b0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 shutdown_connections 2024-03-23T08:31:53.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe5b806cc20 0x7fe5b806f0d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5d0101d70 0x7fe5d010ea60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 --2- 172.21.15.70:0/1716107152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5d0102710 0x7fe5d010efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 >> 172.21.15.70:0/1716107152 conn(0x7fe5d0072bc0 msgr2=0x7fe5d00fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:53.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 shutdown_connections 2024-03-23T08:31:53.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.603+0000 7fe5d4f7e700 1 -- 172.21.15.70:0/1716107152 wait complete. 2024-03-23T08:31:53.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.703+0000 7f35b0455700 1 -- 172.21.15.70:0/3364793729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a8102710 msgr2=0x7f35a80fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.703+0000 7f35b0455700 1 --2- 172.21.15.70:0/3364793729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a8102710 0x7f35a80fbd80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3598004940 tx=0x7f3598009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:53.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.703+0000 7f35b0455700 1 -- 172.21.15.70:0/3364793729 shutdown_connections 2024-03-23T08:31:53.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.703+0000 7f35b0455700 1 --2- 172.21.15.70:0/3364793729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a8102710 0x7f35a80fbd80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.703 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.703+0000 7f35b0455700 1 --2- 172.21.15.70:0/3364793729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35a8101d70 0x7f35a8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.703+0000 7f35b0455700 1 -- 172.21.15.70:0/3364793729 >> 172.21.15.70:0/3364793729 conn(0x7f35a8072bc0 msgr2=0x7f35a8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:53.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.703+0000 7f35b0455700 1 -- 172.21.15.70:0/3364793729 shutdown_connections 2024-03-23T08:31:53.704 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.703+0000 7f35b0455700 1 -- 172.21.15.70:0/3364793729 wait complete. 2024-03-23T08:31:53.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35b0455700 1 Processor -- start 2024-03-23T08:31:53.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35b0455700 1 -- start start 2024-03-23T08:31:53.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35b0455700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a8101d70 0x7f35a81128f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:53.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35b0455700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35a8102710 0x7f35a8112e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:53.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35b0455700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a8113480 con 0x7f35a8102710 2024-03-23T08:31:53.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35b0455700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a810c8c0 con 0x7f35a8101d70 2024-03-23T08:31:53.705 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35ad9f0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35a8102710 0x7f35a8112e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35ad9f0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35a8102710 0x7f35a8112e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46482/0 (socket says 172.21.15.70:46482) 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35ad9f0700 1 -- 172.21.15.70:0/427313351 learned_addr learned my addr 172.21.15.70:0/427313351 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35ae1f1700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a8101d70 0x7f35a81128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35ad9f0700 1 -- 172.21.15.70:0/427313351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a8101d70 msgr2=0x7f35a81128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35ad9f0700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a8101d70 0x7f35a81128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.704+0000 7f35ad9f0700 1 -- 172.21.15.70:0/427313351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598005040 con 0x7f35a8102710 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.705+0000 7f35ad9f0700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35a8102710 0x7f35a8112e30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3598000c00 tx=0x7f3598009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.705+0000 7f359effd700 1 -- 172.21.15.70:0/427313351 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3598021070 con 0x7f35a8102710 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.705+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a810cb40 con 0x7f35a8102710 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.705+0000 7f359effd700 1 -- 172.21.15.70:0/427313351 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3598013de0 con 0x7f35a8102710 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.705+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a810d0a0 con 0x7f35a8102710 2024-03-23T08:31:53.706 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.705+0000 7f359effd700 1 -- 172.21.15.70:0/427313351 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f359800e070 con 0x7f35a8102710 2024-03-23T08:31:53.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.706+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a80ff080 con 0x7f35a8102710 2024-03-23T08:31:53.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.706+0000 7f359effd700 1 -- 172.21.15.70:0/427313351 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f359801c600 con 0x7f35a8102710 2024-03-23T08:31:53.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.707+0000 7f359effd700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f359406cc20 0x7f359406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:53.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.707+0000 7f359effd700 1 -- 172.21.15.70:0/427313351 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f359808d000 con 0x7f35a8102710 2024-03-23T08:31:53.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.707+0000 7f35ae1f1700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f359406cc20 0x7f359406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:53.708 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.707+0000 7f35ae1f1700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f359406cc20 0x7f359406f0d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009b80 tx=0x7f35a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:53.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.711+0000 7f359effd700 1 -- 172.21.15.70:0/427313351 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f359805ab70 con 0x7f35a8102710 2024-03-23T08:31:53.752 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:53 smithi193 systemd[1]: Starting Ceph osd.5 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.948+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f35a810df40 con 0x7f35a8102710 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.949+0000 7f359effd700 1 -- 172.21.15.70:0/427313351 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f3598004490 con 0x7f35a8102710 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:e17 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:31:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 17 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:20:54.403320+0000 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 48 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:31:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:31:53.951 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:31:53.952 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:31:53.952 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{0:14474} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/4066762341,v1:172.21.15.193:6827/4066762341] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:31:53.952 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:31:53.952 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:31:53.952 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:31:53.952 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:31:53.952 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1586743298,v1:172.21.15.70:6827/1586743298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:31:53.952 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f359406cc20 msgr2=0x7f359406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f359406cc20 0x7f359406f0d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009b80 tx=0x7f35a400b040 comp rx=0 tx=0).stop 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35a8102710 msgr2=0x7f35a8112e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35a8102710 0x7f35a8112e30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3598000c00 tx=0x7f3598009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 shutdown_connections 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f359406cc20 0x7f359406f0d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35a8101d70 0x7f35a81128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 --2- 172.21.15.70:0/427313351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35a8102710 0x7f35a8112e30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 >> 172.21.15.70:0/427313351 conn(0x7f35a8072bc0 msgr2=0x7f35a80faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 shutdown_connections 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:53.951+0000 7f35b0455700 1 -- 172.21.15.70:0/427313351 wait complete. 2024-03-23T08:31:53.953 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 17 2024-03-23T08:31:54.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.052+0000 7f1d1d75f700 1 -- 172.21.15.70:0/2791433009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 msgr2=0x7f1d18102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:54.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.052+0000 7f1d1d75f700 1 --2- 172.21.15.70:0/2791433009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 0x7f1d18102070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c004940 tx=0x7f1d0c009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:54.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.052+0000 7f1d1d75f700 1 -- 172.21.15.70:0/2791433009 shutdown_connections 2024-03-23T08:31:54.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.052+0000 7f1d1d75f700 1 --2- 172.21.15.70:0/2791433009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18102640 0x7f1d180fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.052+0000 7f1d1d75f700 1 --2- 172.21.15.70:0/2791433009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 0x7f1d18102070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.052+0000 7f1d1d75f700 1 -- 172.21.15.70:0/2791433009 >> 172.21.15.70:0/2791433009 conn(0x7f1d180f7850 msgr2=0x7f1d180f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:54.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.052+0000 7f1d1d75f700 1 -- 172.21.15.70:0/2791433009 shutdown_connections 2024-03-23T08:31:54.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.052+0000 7f1d1d75f700 1 -- 172.21.15.70:0/2791433009 wait complete. 2024-03-23T08:31:54.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d1d75f700 1 Processor -- start 2024-03-23T08:31:54.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d1d75f700 1 -- start start 2024-03-23T08:31:54.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d1d75f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 0x7f1d1810e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:54.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d1d75f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18102640 0x7f1d1810eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:54.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d1d75f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1810f4f0 con 0x7f1d18101ca0 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d1d75f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1810f630 con 0x7f1d18102640 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d16ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 0x7f1d1810e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d16ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 0x7f1d1810e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46510/0 (socket says 172.21.15.70:46510) 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d16ffd700 1 -- 172.21.15.70:0/1899406600 learned_addr learned my addr 172.21.15.70:0/1899406600 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d167fc700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18102640 0x7f1d1810eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d16ffd700 1 -- 172.21.15.70:0/1899406600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18102640 msgr2=0x7f1d1810eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d16ffd700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18102640 0x7f1d1810eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.053+0000 7f1d16ffd700 1 -- 172.21.15.70:0/1899406600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c005040 con 0x7f1d18101ca0 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.054+0000 7f1d16ffd700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 0x7f1d1810e960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.054+0000 7f1cff7fe700 1 -- 172.21.15.70:0/1899406600 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0c021070 con 0x7f1d18101ca0 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.054+0000 7f1cff7fe700 1 -- 172.21.15.70:0/1899406600 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d0c0093b0 con 0x7f1d18101ca0 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.054+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d181179d0 con 0x7f1d18101ca0 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.054+0000 7f1cff7fe700 1 -- 172.21.15.70:0/1899406600 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0c00e070 con 0x7f1d18101ca0 2024-03-23T08:31:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.054+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d18117ee0 con 0x7f1d18101ca0 2024-03-23T08:31:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.055+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d180fd3f0 con 0x7f1d18101ca0 2024-03-23T08:31:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.055+0000 7f1cff7fe700 1 -- 172.21.15.70:0/1899406600 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1d0c01c490 con 0x7f1d18101ca0 2024-03-23T08:31:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.056+0000 7f1cff7fe700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1d0006cc20 0x7f1d0006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.056+0000 7f1cff7fe700 1 -- 172.21.15.70:0/1899406600 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1d0c08d050 con 0x7f1d18101ca0 2024-03-23T08:31:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.056+0000 7f1d167fc700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1d0006cc20 0x7f1d0006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.056+0000 7f1d167fc700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1d0006cc20 0x7f1d0006f0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d18066b30 tx=0x7f1d0800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:54.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.060+0000 7f1cff7fe700 1 -- 172.21.15.70:0/1899406600 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1d0c057130 con 0x7f1d18101ca0 2024-03-23T08:31:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:53 smithi193 ceph-mon[108533]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:53 smithi193 ceph-mon[108533]: pgmap v153: 65 pgs: 6 peering, 10 stale+active+clean, 49 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:31:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:53 smithi193 ceph-mon[108533]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-23T08:31:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:53 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1716107152' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:53 smithi070 ceph-mon[145869]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:53 smithi070 ceph-mon[145869]: pgmap v153: 65 pgs: 6 peering, 10 stale+active+clean, 49 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:31:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:53 smithi070 ceph-mon[145869]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-23T08:31:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:53 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1716107152' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:31:54.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.269+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d180668e0 con 0x7f1d0006cc20 2024-03-23T08:31:54.270 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.270+0000 7f1cff7fe700 1 -- 172.21.15.70:0/1899406600 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f1d180668e0 con 0x7f1d0006cc20 2024-03-23T08:31:54.270 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:31:54.270 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:31:54.270 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:31:54.270 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:31:54.270 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:31:54.271 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:31:54.271 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-03-23T08:31:54.271 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-03-23T08:31:54.271 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:31:54.271 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "11/21 daemons upgraded", 2024-03-23T08:31:54.271 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading osd daemons", 2024-03-23T08:31:54.271 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:31:54.271 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1d0006cc20 msgr2=0x7f1d0006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1d0006cc20 0x7f1d0006f0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d18066b30 tx=0x7f1d0800b040 comp rx=0 tx=0).stop 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 msgr2=0x7f1d1810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 0x7f1d1810e960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 shutdown_connections 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f1d0006cc20 0x7f1d0006f0d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d18101ca0 0x7f1d1810e960 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 --2- 172.21.15.70:0/1899406600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d18102640 0x7f1d1810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 >> 172.21.15.70:0/1899406600 conn(0x7f1d180f7850 msgr2=0x7f1d180f8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 shutdown_connections 2024-03-23T08:31:54.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.272+0000 7f1d1d75f700 1 -- 172.21.15.70:0/1899406600 wait complete. 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.371+0000 7f48d30de700 1 -- 172.21.15.70:0/3596343986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc101d70 msgr2=0x7f48cc102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.371+0000 7f48d30de700 1 --2- 172.21.15.70:0/3596343986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc101d70 0x7f48cc102140 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f48c0004940 tx=0x7f48c0009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.371+0000 7f48d30de700 1 -- 172.21.15.70:0/3596343986 shutdown_connections 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.371+0000 7f48d30de700 1 --2- 172.21.15.70:0/3596343986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc102710 0x7f48cc0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.371+0000 7f48d30de700 1 --2- 172.21.15.70:0/3596343986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc101d70 0x7f48cc102140 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.371+0000 7f48d30de700 1 -- 172.21.15.70:0/3596343986 >> 172.21.15.70:0/3596343986 conn(0x7f48cc072bc0 msgr2=0x7f48cc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.371+0000 7f48d30de700 1 -- 172.21.15.70:0/3596343986 shutdown_connections 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.371+0000 7f48d30de700 1 -- 172.21.15.70:0/3596343986 wait complete. 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.372+0000 7f48d30de700 1 Processor -- start 2024-03-23T08:31:54.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.372+0000 7f48d30de700 1 -- start start 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.372+0000 7f48d30de700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc101d70 0x7f48cc1129a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.372+0000 7f48d30de700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc102710 0x7f48cc112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.372+0000 7f48d30de700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc113530 con 0x7f48cc102710 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.372+0000 7f48d30de700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc10c920 con 0x7f48cc101d70 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.372+0000 7f48cbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc102710 0x7f48cc112ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.372+0000 7f48cbfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc102710 0x7f48cc112ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46522/0 (socket says 172.21.15.70:46522) 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.372+0000 7f48cbfff700 1 -- 172.21.15.70:0/740128887 learned_addr learned my addr 172.21.15.70:0/740128887 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48d0e7a700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc101d70 0x7f48cc1129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48cbfff700 1 -- 172.21.15.70:0/740128887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc101d70 msgr2=0x7f48cc1129a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48cbfff700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc101d70 0x7f48cc1129a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48cbfff700 1 -- 172.21.15.70:0/740128887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0005040 con 0x7f48cc102710 2024-03-23T08:31:54.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48cbfff700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc102710 0x7f48cc112ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48b8002a50 tx=0x7f48b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:54.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48c97fa700 1 -- 172.21.15.70:0/740128887 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48b8007cc0 con 0x7f48cc102710 2024-03-23T08:31:54.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc10cc00 con 0x7f48cc102710 2024-03-23T08:31:54.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48c97fa700 1 -- 172.21.15.70:0/740128887 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f48b800f040 con 0x7f48cc102710 2024-03-23T08:31:54.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48c97fa700 1 -- 172.21.15.70:0/740128887 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48b800d920 con 0x7f48cc102710 2024-03-23T08:31:54.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.373+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc10d1c0 con 0x7f48cc102710 2024-03-23T08:31:54.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.374+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc0ff080 con 0x7f48cc102710 2024-03-23T08:31:54.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.374+0000 7f48c97fa700 1 -- 172.21.15.70:0/740128887 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f48b8009680 con 0x7f48cc102710 2024-03-23T08:31:54.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.375+0000 7f48c97fa700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f48bc06cc20 0x7f48bc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:31:54.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.375+0000 7f48d0e7a700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f48bc06cc20 0x7f48bc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:31:54.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.375+0000 7f48c97fa700 1 -- 172.21.15.70:0/740128887 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f48b808bdd0 con 0x7f48cc102710 2024-03-23T08:31:54.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.375+0000 7f48d0e7a700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f48bc06cc20 0x7f48bc06f0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f48c0004f90 tx=0x7f48c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:31:54.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.380+0000 7f48c97fa700 1 -- 172.21.15.70:0/740128887 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f48b8059820 con 0x7f48cc102710 2024-03-23T08:31:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.651+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f48cc0fd4c0 con 0x7f48cc102710 2024-03-23T08:31:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.651+0000 7f48c97fa700 1 -- 172.21.15.70:0/740128887 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+438 (secure 0 0 0) 0x7f48b80593b0 con 0x7f48cc102710 2024-03-23T08:31:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 3 pgs peering 2024-03-23T08:31:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-23T08:31:54.652 INFO:teuthology.orchestra.run.smithi070.stdout: osd.5 (root=default,host=smithi193) is down 2024-03-23T08:31:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 3 pgs peering 2024-03-23T08:31:54.652 INFO:teuthology.orchestra.run.smithi070.stdout: pg 2.2 is stuck peering for 6m, current state peering, last acting [1,0] 2024-03-23T08:31:54.652 INFO:teuthology.orchestra.run.smithi070.stdout: pg 3.c is stuck peering for 6m, current state peering, last acting [0,3] 2024-03-23T08:31:54.652 INFO:teuthology.orchestra.run.smithi070.stdout: pg 3.10 is stuck peering for 6m, current state peering, last acting [0,1] 2024-03-23T08:31:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f48bc06cc20 msgr2=0x7f48bc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f48bc06cc20 0x7f48bc06f0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f48c0004f90 tx=0x7f48c0009e30 comp rx=0 tx=0).stop 2024-03-23T08:31:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc102710 msgr2=0x7f48cc112ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:31:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc102710 0x7f48cc112ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48b8002a50 tx=0x7f48b8004600 comp rx=0 tx=0).stop 2024-03-23T08:31:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 shutdown_connections 2024-03-23T08:31:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f48bc06cc20 0x7f48bc06f0d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc101d70 0x7f48cc1129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 --2- 172.21.15.70:0/740128887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48cc102710 0x7f48cc112ee0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:31:54.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 >> 172.21.15.70:0/740128887 conn(0x7f48cc072bc0 msgr2=0x7f48cc0fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:31:54.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 shutdown_connections 2024-03-23T08:31:54.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:31:54.654+0000 7f48d30de700 1 -- 172.21.15.70:0/740128887 wait complete. 2024-03-23T08:31:54.743 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:54 smithi193 bash[133318]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-23T08:31:54.743 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:54 smithi193 bash[133318]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-23T08:31:54.743 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:54 smithi193 bash[133318]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-23T08:31:54.743 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:54 smithi193 bash[133318]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-23T08:31:54.743 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:54 smithi193 bash[133318]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-23T08:31:54.743 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:54 smithi193 bash[133318]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-23T08:31:54.743 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:54 smithi193 bash[133318]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-23T08:31:55.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:54 smithi193 ceph-mon[108533]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:55.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:54 smithi193 ceph-mon[108533]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:55.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:54 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/427313351' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:31:55.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:54 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/740128887' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:31:55.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:54 smithi070 ceph-mon[145869]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:54 smithi070 ceph-mon[145869]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:54 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/427313351' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:31:55.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:54 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/740128887' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:31:55.664 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:55 smithi193 ceph-osd[133614]: -- 172.21.15.193:0/3235369556 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559fa9f971e0 con 0x559fa9f66c00 2024-03-23T08:31:56.068 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:55 smithi193 bash[133576]: 58bbe6abe8b3ff501950cb244046c47ea89dbb52df0b7840f6c96fcc3191e663 2024-03-23T08:31:56.068 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:55 smithi193 systemd[1]: Started Ceph osd.5 for 2762dd54-e8ed-11ee-95cd-87774f69a715. 2024-03-23T08:31:56.068 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:55 smithi193 ceph-mon[108533]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:56.068 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:55 smithi193 ceph-mon[108533]: pgmap v154: 65 pgs: 3 active+undersized+degraded, 6 peering, 6 stale+active+clean, 4 active+undersized, 46 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 9/222 objects degraded (4.054%) 2024-03-23T08:31:56.068 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:55 smithi193 ceph-mon[108533]: Health check failed: Degraded data redundancy: 9/222 objects degraded (4.054%), 3 pgs degraded (PG_DEGRADED) 2024-03-23T08:31:56.068 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:56.068 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:56.068 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:31:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:55 smithi070 ceph-mon[145869]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:31:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:55 smithi070 ceph-mon[145869]: pgmap v154: 65 pgs: 3 active+undersized+degraded, 6 peering, 6 stale+active+clean, 4 active+undersized, 46 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 9/222 objects degraded (4.054%) 2024-03-23T08:31:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:55 smithi070 ceph-mon[145869]: Health check failed: Degraded data redundancy: 9/222 objects degraded (4.054%), 3 pgs degraded (PG_DEGRADED) 2024-03-23T08:31:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:31:56.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:31:56 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:31:56.066+0000 7f6fde5b9640 -1 Falling back to public interface 2024-03-23T08:31:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:57 smithi193 ceph-mon[108533]: pgmap v155: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:31:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:57 smithi193 ceph-mon[108533]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-23T08:31:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:57 smithi070 ceph-mon[145869]: pgmap v155: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:31:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:57 smithi070 ceph-mon[145869]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-23T08:31:59.845 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:59 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:59.845 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:59 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:59.845 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:59 smithi193 ceph-mon[108533]: pgmap v156: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:31:59.845 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:59 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:59.845 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:31:59 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:59.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:59.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:59.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:59 smithi070 ceph-mon[145869]: pgmap v156: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:31:59.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:31:59.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:31:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:00.825 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:32:00 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:32:00.558+0000 7f6fde5b9640 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-23T08:32:01.268 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:32:00 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:32:00.993+0000 7f6fde5b9640 -1 osd.5 76 log_to_monitors true 2024-03-23T08:32:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:01 smithi070 ceph-mon[145869]: pgmap v157: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:32:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:01 smithi070 ceph-mon[145869]: from='osd.5 [v2:172.21.15.193:6816/2211659369,v1:172.21.15.193:6817/2211659369]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T08:32:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:01 smithi070 ceph-mon[145869]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T08:32:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:01.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:32:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:01 smithi193 ceph-mon[108533]: pgmap v157: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:32:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:01 smithi193 ceph-mon[108533]: from='osd.5 [v2:172.21.15.193:6816/2211659369,v1:172.21.15.193:6817/2211659369]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T08:32:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:01 smithi193 ceph-mon[108533]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-23T08:32:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:32:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:02 smithi193 ceph-mon[108533]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T08:32:02.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:02 smithi193 ceph-mon[108533]: osdmap e79: 6 total, 5 up, 6 in 2024-03-23T08:32:02.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:02 smithi193 ceph-mon[108533]: from='osd.5 [v2:172.21.15.193:6816/2211659369,v1:172.21.15.193:6817/2211659369]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:32:02.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:02 smithi193 ceph-mon[108533]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:32:02.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:02 smithi193 ceph-mon[108533]: Health check update: Degraded data redundancy: 35/222 objects degraded (15.766%), 12 pgs degraded (PG_DEGRADED) 2024-03-23T08:32:02.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:02 smithi193 ceph-mon[108533]: from='osd.5 ' entity='osd.5' 2024-03-23T08:32:02.826 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:32:02 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:32:02.572+0000 7f6fcba0a700 -1 osd.5 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T08:32:02.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:02 smithi070 ceph-mon[145869]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-23T08:32:02.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:02 smithi070 ceph-mon[145869]: osdmap e79: 6 total, 5 up, 6 in 2024-03-23T08:32:02.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:02 smithi070 ceph-mon[145869]: from='osd.5 [v2:172.21.15.193:6816/2211659369,v1:172.21.15.193:6817/2211659369]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:32:02.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:02 smithi070 ceph-mon[145869]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-23T08:32:02.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:02 smithi070 ceph-mon[145869]: Health check update: Degraded data redundancy: 35/222 objects degraded (15.766%), 12 pgs degraded (PG_DEGRADED) 2024-03-23T08:32:02.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:02 smithi070 ceph-mon[145869]: from='osd.5 ' entity='osd.5' 2024-03-23T08:32:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-23T08:32:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd require-osd-release", "release": "reef"}]: dispatch 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: osd.5 [v2:172.21.15.193:6816/2211659369,v1:172.21.15.193:6817/2211659369] boot 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "osd require-osd-release", "release": "reef"}]': finished 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: osdmap e80: 6 total, 6 up, 6 in 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-23T08:32:03.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:03 smithi070 ceph-mon[145869]: osdmap e81: 6 total, 6 up, 6 in 2024-03-23T08:32:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:32:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:04.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-23T08:32:04.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd require-osd-release", "release": "reef"}]: dispatch 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: osd.5 [v2:172.21.15.193:6816/2211659369,v1:172.21.15.193:6817/2211659369] boot 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "osd require-osd-release", "release": "reef"}]': finished 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: osdmap e80: 6 total, 6 up, 6 in 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-23T08:32:04.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:03 smithi193 ceph-mon[108533]: osdmap e81: 6 total, 6 up, 6 in 2024-03-23T08:32:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:04 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all osd 2024-03-23T08:32:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:04 smithi070 ceph-mon[145869]: Upgrade: Setting require_osd_release to 18 reef 2024-03-23T08:32:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:04 smithi070 ceph-mon[145869]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-23T08:32:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-23T08:32:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:04 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 2 up:standby 2024-03-23T08:32:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:32:04.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:04 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:05.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:04 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all osd 2024-03-23T08:32:05.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:04 smithi193 ceph-mon[108533]: Upgrade: Setting require_osd_release to 18 reef 2024-03-23T08:32:05.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:04 smithi193 ceph-mon[108533]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-23T08:32:05.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-23T08:32:05.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:04 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 2 up:standby 2024-03-23T08:32:05.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:05.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:05.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:32:05.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:04 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:05.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:05 smithi070 ceph-mon[145869]: pgmap v162: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:32:05.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:05.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:05.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:05.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:05.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:05.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:05 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.193:6826/2042142064,v1:172.21.15.193:6827/2042142064] up:boot 2024-03-23T08:32:05.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:05 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 3 up:standby 2024-03-23T08:32:05.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:32:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:05 smithi193 ceph-mon[108533]: pgmap v162: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 201 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 35/222 objects degraded (15.766%) 2024-03-23T08:32:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:05 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.193:6826/2042142064,v1:172.21.15.193:6827/2042142064] up:boot 2024-03-23T08:32:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:05 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 3 up:standby 2024-03-23T08:32:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:32:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:07 smithi070 ceph-mon[145869]: Upgrade: Updating mds.cephfs.smithi070.iziydh 2024-03-23T08:32:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:07 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:07 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.iziydh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:32:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:07 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:07 smithi070 ceph-mon[145869]: Deploying daemon mds.cephfs.smithi070.iziydh on smithi070 2024-03-23T08:32:07.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:07 smithi070 ceph-mon[145869]: pgmap v163: 65 pgs: 6 active+undersized+degraded, 10 active+undersized, 49 active+clean; 201 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 15/222 objects degraded (6.757%) 2024-03-23T08:32:07.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:07 smithi193 ceph-mon[108533]: Upgrade: Updating mds.cephfs.smithi070.iziydh 2024-03-23T08:32:07.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:07 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:07.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:07 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.iziydh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:32:07.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:07 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:07.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:07 smithi193 ceph-mon[108533]: Deploying daemon mds.cephfs.smithi070.iziydh on smithi070 2024-03-23T08:32:07.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:07 smithi193 ceph-mon[108533]: pgmap v163: 65 pgs: 6 active+undersized+degraded, 10 active+undersized, 49 active+clean; 201 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 15/222 objects degraded (6.757%) 2024-03-23T08:32:08.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:08 smithi070 ceph-mon[145869]: Health check update: Degraded data redundancy: 15/222 objects degraded (6.757%), 6 pgs degraded (PG_DEGRADED) 2024-03-23T08:32:08.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:08 smithi070 ceph-mon[145869]: osdmap e82: 6 total, 6 up, 6 in 2024-03-23T08:32:08.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:08 smithi193 ceph-mon[108533]: Health check update: Degraded data redundancy: 15/222 objects degraded (6.757%), 6 pgs degraded (PG_DEGRADED) 2024-03-23T08:32:08.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:08 smithi193 ceph-mon[108533]: osdmap e82: 6 total, 6 up, 6 in 2024-03-23T08:32:09.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:09 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 2 up:standby 2024-03-23T08:32:09.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:09 smithi070 ceph-mon[145869]: pgmap v165: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 341 B/s rd, 1 op/s 2024-03-23T08:32:09.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:09 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 2 up:standby 2024-03-23T08:32:09.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:09 smithi193 ceph-mon[108533]: pgmap v165: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 341 B/s rd, 1 op/s 2024-03-23T08:32:10.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:10 smithi070 ceph-mon[145869]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/222 objects degraded (6.757%), 6 pgs degraded) 2024-03-23T08:32:10.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:10 smithi070 ceph-mon[145869]: Cluster is now healthy 2024-03-23T08:32:10.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:10 smithi193 ceph-mon[108533]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/222 objects degraded (6.757%), 6 pgs degraded) 2024-03-23T08:32:10.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:10 smithi193 ceph-mon[108533]: Cluster is now healthy 2024-03-23T08:32:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:11 smithi070 ceph-mon[145869]: pgmap v166: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 290 B/s rd, 0 op/s 2024-03-23T08:32:11.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:11 smithi193 ceph-mon[108533]: pgmap v166: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 290 B/s rd, 0 op/s 2024-03-23T08:32:13.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:13 smithi070 ceph-mon[145869]: pgmap v167: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 226 B/s rd, 0 op/s 2024-03-23T08:32:14.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:13 smithi193 ceph-mon[108533]: pgmap v167: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 226 B/s rd, 0 op/s 2024-03-23T08:32:15.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:14 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] up:boot 2024-03-23T08:32:15.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:14 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 3 up:standby 2024-03-23T08:32:15.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:14 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:32:15.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:14 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:15.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:14 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:15.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:14 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:14 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] up:boot 2024-03-23T08:32:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:14 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.aymzqb=up:active} 3 up:standby 2024-03-23T08:32:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:14 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:32:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:14 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:14 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:15.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:14 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:16.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:15 smithi193 ceph-mon[108533]: pgmap v168: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 204 B/s rd, 0 op/s 2024-03-23T08:32:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:15 smithi070 ceph-mon[145869]: pgmap v168: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 204 B/s rd, 0 op/s 2024-03-23T08:32:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:17 smithi070 ceph-mon[145869]: pgmap v169: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:32:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:17 smithi193 ceph-mon[108533]: pgmap v169: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:32:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:19.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:19 smithi070 ceph-mon[145869]: pgmap v170: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:19.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:19.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:19 smithi193 ceph-mon[108533]: pgmap v170: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:20.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:20 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:20 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:21.113 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:20 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:21.113 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:20 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:21 smithi070 ceph-mon[145869]: pgmap v171: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:21 smithi193 ceph-mon[108533]: pgmap v171: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: pgmap v172: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:32:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:23.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:23.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:23.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: pgmap v172: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:32:24.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:24.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:24.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:24.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:24.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:24.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:24.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 -- 172.21.15.70:0/1641269622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48941037c0 msgr2=0x7f48940fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:24.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 --2- 172.21.15.70:0/1641269622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48941037c0 0x7f48940fce30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4888004940 tx=0x7f4888009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 -- 172.21.15.70:0/1641269622 shutdown_connections 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 --2- 172.21.15.70:0/1641269622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48941037c0 0x7f48940fce30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 --2- 172.21.15.70:0/1641269622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4894102e20 0x7f48941031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 -- 172.21.15.70:0/1641269622 >> 172.21.15.70:0/1641269622 conn(0x7f48940f89d0 msgr2=0x7f48940fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 -- 172.21.15.70:0/1641269622 shutdown_connections 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 -- 172.21.15.70:0/1641269622 wait complete. 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 Processor -- start 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 -- start start 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4894102e20 0x7f4894113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48941037c0 0x7f4894114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894114660 con 0x7f4894102e20 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.757+0000 7f4898f77700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489410da50 con 0x7f48941037c0 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f489259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4894102e20 0x7f4894113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f489259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4894102e20 0x7f4894113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54124/0 (socket says 172.21.15.70:54124) 2024-03-23T08:32:24.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f489259c700 1 -- 172.21.15.70:0/4212838693 learned_addr learned my addr 172.21.15.70:0/4212838693 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f4891d9b700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48941037c0 0x7f4894114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f489259c700 1 -- 172.21.15.70:0/4212838693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48941037c0 msgr2=0x7f4894114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f489259c700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48941037c0 0x7f4894114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f489259c700 1 -- 172.21.15.70:0/4212838693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4888005040 con 0x7f4894102e20 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f489259c700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4894102e20 0x7f4894113ad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f488400c830 tx=0x7f4884004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f4882ffd700 1 -- 172.21.15.70:0/4212838693 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48840106f0 con 0x7f4894102e20 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489410dd30 con 0x7f4894102e20 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f4882ffd700 1 -- 172.21.15.70:0/4212838693 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4884010d00 con 0x7f4894102e20 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f4882ffd700 1 -- 172.21.15.70:0/4212838693 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4884012e90 con 0x7f4894102e20 2024-03-23T08:32:24.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.758+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489410e2c0 con 0x7f4894102e20 2024-03-23T08:32:24.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.759+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48940fe570 con 0x7f4894102e20 2024-03-23T08:32:24.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.760+0000 7f4882ffd700 1 -- 172.21.15.70:0/4212838693 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f4884010850 con 0x7f4894102e20 2024-03-23T08:32:24.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.760+0000 7f4882ffd700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f487c06cc20 0x7f487c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:24.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.760+0000 7f4882ffd700 1 -- 172.21.15.70:0/4212838693 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f488408bf60 con 0x7f4894102e20 2024-03-23T08:32:24.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.760+0000 7f4891d9b700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f487c06cc20 0x7f487c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:24.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.760+0000 7f4891d9b700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f487c06cc20 0x7f487c06f0d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4888004890 tx=0x7f4888009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:24.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.765+0000 7f4882ffd700 1 -- 172.21.15.70:0/4212838693 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f48840bf9f0 con 0x7f4894102e20 2024-03-23T08:32:24.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.988+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4894109e10 con 0x7f487c06cc20 2024-03-23T08:32:24.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.990+0000 7f4882ffd700 1 -- 172.21.15.70:0/4212838693 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4894109e10 con 0x7f487c06cc20 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f487c06cc20 msgr2=0x7f487c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f487c06cc20 0x7f487c06f0d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4888004890 tx=0x7f4888009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4894102e20 msgr2=0x7f4894113ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4894102e20 0x7f4894113ad0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f488400c830 tx=0x7f4884004470 comp rx=0 tx=0).stop 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 shutdown_connections 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f487c06cc20 0x7f487c06f0d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4894102e20 0x7f4894113ad0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 --2- 172.21.15.70:0/4212838693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48941037c0 0x7f4894114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 >> 172.21.15.70:0/4212838693 conn(0x7f48940f89d0 msgr2=0x7f48940f9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 shutdown_connections 2024-03-23T08:32:24.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:24.992+0000 7f4898f77700 1 -- 172.21.15.70:0/4212838693 wait complete. 2024-03-23T08:32:25.002 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:32:25.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 -- 172.21.15.70:0/3396695014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 msgr2=0x7fe7980fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 --2- 172.21.15.70:0/3396695014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 0x7fe7980fe260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe788004940 tx=0x7fe788009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 -- 172.21.15.70:0/3396695014 shutdown_connections 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 --2- 172.21.15.70:0/3396695014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7980fe830 0x7fe798106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 --2- 172.21.15.70:0/3396695014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 0x7fe7980fe260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 -- 172.21.15.70:0/3396695014 >> 172.21.15.70:0/3396695014 conn(0x7fe798072bc0 msgr2=0x7fe798072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 -- 172.21.15.70:0/3396695014 shutdown_connections 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 -- 172.21.15.70:0/3396695014 wait complete. 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 Processor -- start 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.094+0000 7fe79fb66700 1 -- start start 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79fb66700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 0x7fe798113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79fb66700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7980fe830 0x7fe798114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79fb66700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798114690 con 0x7fe7980fde90 2024-03-23T08:32:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79fb66700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79810da80 con 0x7fe7980fe830 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79d902700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 0x7fe798113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79d902700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 0x7fe798113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54146/0 (socket says 172.21.15.70:54146) 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79d902700 1 -- 172.21.15.70:0/4070941120 learned_addr learned my addr 172.21.15.70:0/4070941120 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79d101700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7980fe830 0x7fe798114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79d902700 1 -- 172.21.15.70:0/4070941120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7980fe830 msgr2=0x7fe798114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79d902700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7980fe830 0x7fe798114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79d902700 1 -- 172.21.15.70:0/4070941120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788005040 con 0x7fe7980fde90 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79d902700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 0x7fe798113b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe788009390 tx=0x7fe788009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe78e7fc700 1 -- 172.21.15.70:0/4070941120 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe78800e040 con 0x7fe7980fde90 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe78e7fc700 1 -- 172.21.15.70:0/4070941120 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe788004290 con 0x7fe7980fde90 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79810dd00 con 0x7fe7980fde90 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe78e7fc700 1 -- 172.21.15.70:0/4070941120 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe78801c3f0 con 0x7fe7980fde90 2024-03-23T08:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.095+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79810e240 con 0x7fe7980fde90 2024-03-23T08:32:25.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.096+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe798065e30 con 0x7fe7980fde90 2024-03-23T08:32:25.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.097+0000 7fe78e7fc700 1 -- 172.21.15.70:0/4070941120 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe7880136b0 con 0x7fe7980fde90 2024-03-23T08:32:25.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.097+0000 7fe78e7fc700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe78406cc20 0x7fe78406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:25.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.097+0000 7fe78e7fc700 1 -- 172.21.15.70:0/4070941120 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe78808d050 con 0x7fe7980fde90 2024-03-23T08:32:25.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.097+0000 7fe79d101700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe78406cc20 0x7fe78406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:25.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.098+0000 7fe79d101700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe78406cc20 0x7fe78406f0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe79810ec40 tx=0x7fe7940043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:25.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.102+0000 7fe78e7fc700 1 -- 172.21.15.70:0/4070941120 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe78805ac60 con 0x7fe7980fde90 2024-03-23T08:32:25.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.334+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe798109df0 con 0x7fe78406cc20 2024-03-23T08:32:25.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.335+0000 7fe78e7fc700 1 -- 172.21.15.70:0/4070941120 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe798109df0 con 0x7fe78406cc20 2024-03-23T08:32:25.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe78406cc20 msgr2=0x7fe78406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe78406cc20 0x7fe78406f0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe79810ec40 tx=0x7fe7940043c0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 msgr2=0x7fe798113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 0x7fe798113b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe788009390 tx=0x7fe788009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 shutdown_connections 2024-03-23T08:32:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe78406cc20 0x7fe78406f0d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7980fde90 0x7fe798113b00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 --2- 172.21.15.70:0/4070941120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7980fe830 0x7fe798114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 >> 172.21.15.70:0/4070941120 conn(0x7fe798072bc0 msgr2=0x7fe798103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 shutdown_connections 2024-03-23T08:32:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.337+0000 7fe79fb66700 1 -- 172.21.15.70:0/4070941120 wait complete. 2024-03-23T08:32:25.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.443+0000 7f2579f53700 1 -- 172.21.15.70:0/1582836944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 msgr2=0x7f2574106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.443+0000 7f2579f53700 1 --2- 172.21.15.70:0/1582836944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 0x7f2574106f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2568004940 tx=0x7f2568009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:25.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.443+0000 7f2579f53700 1 -- 172.21.15.70:0/1582836944 shutdown_connections 2024-03-23T08:32:25.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.443+0000 7f2579f53700 1 --2- 172.21.15.70:0/1582836944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 0x7f2574106f00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.443+0000 7f2579f53700 1 --2- 172.21.15.70:0/1582836944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25740fde90 0x7f25740fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.443+0000 7f2579f53700 1 -- 172.21.15.70:0/1582836944 >> 172.21.15.70:0/1582836944 conn(0x7f2574072bc0 msgr2=0x7f2574072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:25.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.443+0000 7f2579f53700 1 -- 172.21.15.70:0/1582836944 shutdown_connections 2024-03-23T08:32:25.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.443+0000 7f2579f53700 1 -- 172.21.15.70:0/1582836944 wait complete. 2024-03-23T08:32:25.444 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.444+0000 7f2579f53700 1 Processor -- start 2024-03-23T08:32:25.445 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.444+0000 7f2579f53700 1 -- start start 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.444+0000 7f2579f53700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25740fde90 0x7f25740fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.444+0000 7f2579f53700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 0x7f25740fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.444+0000 7f2579f53700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740fc820 con 0x7f25740fe830 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.444+0000 7f2579f53700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740fc960 con 0x7f25740fde90 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.444+0000 7f2572ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 0x7f25740fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.444+0000 7f2572ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 0x7f25740fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54170/0 (socket says 172.21.15.70:54170) 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.444+0000 7f2572ffd700 1 -- 172.21.15.70:0/2857224184 learned_addr learned my addr 172.21.15.70:0/2857224184 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f25737fe700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25740fde90 0x7f25740fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f2572ffd700 1 -- 172.21.15.70:0/2857224184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25740fde90 msgr2=0x7f25740fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f2572ffd700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25740fde90 0x7f25740fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f2572ffd700 1 -- 172.21.15.70:0/2857224184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2568005040 con 0x7f25740fe830 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f25737fe700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25740fde90 0x7f25740fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f2572ffd700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 0x7f25740fc290 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f25680093b0 tx=0x7f2568009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f255bfff700 1 -- 172.21.15.70:0/2857224184 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256800e040 con 0x7f25740fe830 2024-03-23T08:32:25.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f255bfff700 1 -- 172.21.15.70:0/2857224184 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2568004290 con 0x7f25740fe830 2024-03-23T08:32:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25740fcbb0 con 0x7f25740fe830 2024-03-23T08:32:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f255bfff700 1 -- 172.21.15.70:0/2857224184 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256801c3f0 con 0x7f25740fe830 2024-03-23T08:32:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.445+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257411a190 con 0x7f25740fe830 2024-03-23T08:32:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.446+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2574065e30 con 0x7f25740fe830 2024-03-23T08:32:25.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.448+0000 7f255bfff700 1 -- 172.21.15.70:0/2857224184 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2568013790 con 0x7f25740fe830 2024-03-23T08:32:25.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.448+0000 7f255bfff700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f255c06cc30 0x7f255c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:25.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.448+0000 7f255bfff700 1 -- 172.21.15.70:0/2857224184 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f256808de10 con 0x7f25740fe830 2024-03-23T08:32:25.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.448+0000 7f25737fe700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f255c06cc30 0x7f255c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:25.448 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.449+0000 7f25737fe700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f255c06cc30 0x7f255c06f0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2564009fd0 tx=0x7f256400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:25.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.452+0000 7f255bfff700 1 -- 172.21.15.70:0/2857224184 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2568057ed0 con 0x7f25740fe830 2024-03-23T08:32:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.661+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f257404bed0 con 0x7f255c06cc30 2024-03-23T08:32:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.671+0000 7f255bfff700 1 -- 172.21.15.70:0/2857224184 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f257404bed0 con 0x7f255c06cc30 2024-03-23T08:32:25.671 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:32:25.671 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (6m) 6s ago 17m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:32:25.671 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (3m) 6s ago 17m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (3m) 27s ago 16m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (5m) 6s ago 16m 90.9M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (11m) 6s ago 11m 146M - 17.2.7-726-g9c4e0c34 837124da804f a9de03bd7afb 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (12s) 6s ago 11m 14.5M - 18.2.2-954-g82316b04 b24e1baba637 1066f2091682 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (11m) 27s ago 11m 97.3M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (11m) 27s ago 11m 16.1M - 17.2.7-726-g9c4e0c34 837124da804f 78dfc2140504 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (7m) 6s ago 18m 504M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (6m) 27s ago 16m 449M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (4m) 6s ago 18m 54.3M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (4m) 27s ago 16m 38.5M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (6m) 6s ago 17m 18.5M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (6m) 27s ago 16m 16.5M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (3m) 6s ago 15m 89.6M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (2m) 6s ago 15m 100M 1568M 18.2.2-954-g82316b04 b24e1baba637 13fce63c86eb 2024-03-23T08:32:25.672 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (2m) 6s ago 14m 89.9M 1568M 18.2.2-954-g82316b04 b24e1baba637 73e36e623491 2024-03-23T08:32:25.673 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (96s) 27s ago 14m 102M 2592M 18.2.2-954-g82316b04 b24e1baba637 8a7617f5541a 2024-03-23T08:32:25.673 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (63s) 27s ago 13m 102M 2592M 18.2.2-954-g82316b04 b24e1baba637 76a25ee5af50 2024-03-23T08:32:25.673 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (30s) 27s ago 13m 11.4M 2592M 18.2.2-954-g82316b04 b24e1baba637 58bbe6abe8b3 2024-03-23T08:32:25.673 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (6m) 6s ago 16m 60.4M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:32:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f255c06cc30 msgr2=0x7f255c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f255c06cc30 0x7f255c06f0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2564009fd0 tx=0x7f256400b040 comp rx=0 tx=0).stop 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 msgr2=0x7f25740fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 0x7f25740fc290 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f25680093b0 tx=0x7f2568009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 shutdown_connections 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f255c06cc30 0x7f255c06f0e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25740fde90 0x7f25740fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 --2- 172.21.15.70:0/2857224184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25740fe830 0x7f25740fc290 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 >> 172.21.15.70:0/2857224184 conn(0x7f2574072bc0 msgr2=0x7f2574103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 shutdown_connections 2024-03-23T08:32:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.673+0000 7f2579f53700 1 -- 172.21.15.70:0/2857224184 wait complete. 2024-03-23T08:32:25.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.779+0000 7fc7b31cd700 1 -- 172.21.15.70:0/3193611946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac102e10 msgr2=0x7fc7ac1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.779+0000 7fc7b31cd700 1 --2- 172.21.15.70:0/3193611946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac102e10 0x7fc7ac1031e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0004940 tx=0x7fc7a0009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.779+0000 7fc7b31cd700 1 -- 172.21.15.70:0/3193611946 shutdown_connections 2024-03-23T08:32:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.779+0000 7fc7b31cd700 1 --2- 172.21.15.70:0/3193611946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7ac1037b0 0x7fc7ac0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.779+0000 7fc7b31cd700 1 --2- 172.21.15.70:0/3193611946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac102e10 0x7fc7ac1031e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.779+0000 7fc7b31cd700 1 -- 172.21.15.70:0/3193611946 >> 172.21.15.70:0/3193611946 conn(0x7fc7ac0f89a0 msgr2=0x7fc7ac0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.779+0000 7fc7b31cd700 1 -- 172.21.15.70:0/3193611946 shutdown_connections 2024-03-23T08:32:25.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.779+0000 7fc7b31cd700 1 -- 172.21.15.70:0/3193611946 wait complete. 2024-03-23T08:32:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.780+0000 7fc7b31cd700 1 Processor -- start 2024-03-23T08:32:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.780+0000 7fc7b31cd700 1 -- start start 2024-03-23T08:32:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.780+0000 7fc7b31cd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7ac102e10 0x7fc7ac113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.780+0000 7fc7b31cd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac1037b0 0x7fc7ac114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.780+0000 7fc7b31cd700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac114680 con 0x7fc7ac1037b0 2024-03-23T08:32:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.780+0000 7fc7b31cd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac10da70 con 0x7fc7ac102e10 2024-03-23T08:32:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.780+0000 7fc7abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac1037b0 0x7fc7ac114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.780+0000 7fc7abfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac1037b0 0x7fc7ac114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54196/0 (socket says 172.21.15.70:54196) 2024-03-23T08:32:25.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.780+0000 7fc7abfff700 1 -- 172.21.15.70:0/671402066 learned_addr learned my addr 172.21.15.70:0/671402066 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7b0f69700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7ac102e10 0x7fc7ac113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7abfff700 1 -- 172.21.15.70:0/671402066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7ac102e10 msgr2=0x7fc7ac113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7abfff700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7ac102e10 0x7fc7ac113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7abfff700 1 -- 172.21.15.70:0/671402066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0005040 con 0x7fc7ac1037b0 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7abfff700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac1037b0 0x7fc7ac114030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc79800c800 tx=0x7fc798004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7a97fa700 1 -- 172.21.15.70:0/671402066 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7980106f0 con 0x7fc7ac1037b0 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac10dd50 con 0x7fc7ac1037b0 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7a97fa700 1 -- 172.21.15.70:0/671402066 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc798010d00 con 0x7fc7ac1037b0 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7a97fa700 1 -- 172.21.15.70:0/671402066 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7980113a0 con 0x7fc7ac1037b0 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.781+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac10e310 con 0x7fc7ac1037b0 2024-03-23T08:32:25.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.782+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac0fe560 con 0x7fc7ac1037b0 2024-03-23T08:32:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.782+0000 7fc7a97fa700 1 -- 172.21.15.70:0/671402066 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc798012400 con 0x7fc7ac1037b0 2024-03-23T08:32:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.783+0000 7fc7a97fa700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc79c06cc20 0x7fc79c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.783+0000 7fc7a97fa700 1 -- 172.21.15.70:0/671402066 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc79808bf30 con 0x7fc7ac1037b0 2024-03-23T08:32:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.783+0000 7fc7b0f69700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc79c06cc20 0x7fc79c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:25.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.783+0000 7fc7b0f69700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc79c06cc20 0x7fc79c06f0d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0004f90 tx=0x7fc7a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:25.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:25.787+0000 7fc7a97fa700 1 -- 172.21.15.70:0/671402066 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc7980bf9f0 con 0x7fc7ac1037b0 2024-03-23T08:32:26.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.081+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc7ac109dd0 con 0x7fc7ac1037b0 2024-03-23T08:32:26.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.082+0000 7fc7a97fa700 1 -- 172.21.15.70:0/671402066 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+842 (secure 0 0 0) 0x7fc798059aa0 con 0x7fc7ac1037b0 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 6 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 1 2024-03-23T08:32:26.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:32:26.084 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:32:26.084 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-23T08:32:26.084 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 11 2024-03-23T08:32:26.084 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:32:26.084 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.084+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc79c06cc20 msgr2=0x7fc79c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.084+0000 7fc7b31cd700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc79c06cc20 0x7fc79c06f0d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0004f90 tx=0x7fc7a0009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.084+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac1037b0 msgr2=0x7fc7ac114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.084+0000 7fc7b31cd700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac1037b0 0x7fc7ac114030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc79800c800 tx=0x7fc798004470 comp rx=0 tx=0).stop 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.085+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 shutdown_connections 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.085+0000 7fc7b31cd700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc79c06cc20 0x7fc79c06f0d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.085+0000 7fc7b31cd700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7ac102e10 0x7fc7ac113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.085+0000 7fc7b31cd700 1 --2- 172.21.15.70:0/671402066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7ac1037b0 0x7fc7ac114030 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.085+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 >> 172.21.15.70:0/671402066 conn(0x7fc7ac0f89a0 msgr2=0x7fc7ac0f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.085+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 shutdown_connections 2024-03-23T08:32:26.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.085+0000 7fc7b31cd700 1 -- 172.21.15.70:0/671402066 wait complete. 2024-03-23T08:32:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.193+0000 7fc5528b2700 1 -- 172.21.15.70:0/4016391737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c101d70 msgr2=0x7fc54c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.193+0000 7fc5528b2700 1 --2- 172.21.15.70:0/4016391737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c101d70 0x7fc54c102140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc53c004940 tx=0x7fc53c009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.193+0000 7fc5528b2700 1 -- 172.21.15.70:0/4016391737 shutdown_connections 2024-03-23T08:32:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.193+0000 7fc5528b2700 1 --2- 172.21.15.70:0/4016391737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc54c102710 0x7fc54c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.193+0000 7fc5528b2700 1 --2- 172.21.15.70:0/4016391737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c101d70 0x7fc54c102140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.194+0000 7fc5528b2700 1 -- 172.21.15.70:0/4016391737 >> 172.21.15.70:0/4016391737 conn(0x7fc54c072bc0 msgr2=0x7fc54c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.194+0000 7fc5528b2700 1 -- 172.21.15.70:0/4016391737 shutdown_connections 2024-03-23T08:32:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.194+0000 7fc5528b2700 1 -- 172.21.15.70:0/4016391737 wait complete. 2024-03-23T08:32:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.194+0000 7fc5528b2700 1 Processor -- start 2024-03-23T08:32:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.194+0000 7fc5528b2700 1 -- start start 2024-03-23T08:32:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.194+0000 7fc5528b2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc54c101d70 0x7fc54c102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.194+0000 7fc5528b2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c102710 0x7fc54c103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.194+0000 7fc5528b2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c103650 con 0x7fc54c102710 2024-03-23T08:32:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.194+0000 7fc5528b2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c103790 con 0x7fc54c101d70 2024-03-23T08:32:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc54b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c102710 0x7fc54c103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc54b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c102710 0x7fc54c103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54226/0 (socket says 172.21.15.70:54226) 2024-03-23T08:32:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc54b7fe700 1 -- 172.21.15.70:0/2498112589 learned_addr learned my addr 172.21.15.70:0/2498112589 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:26.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc54bfff700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc54c101d70 0x7fc54c102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc54b7fe700 1 -- 172.21.15.70:0/2498112589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc54c101d70 msgr2=0x7fc54c102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc54b7fe700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc54c101d70 0x7fc54c102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc54b7fe700 1 -- 172.21.15.70:0/2498112589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c005040 con 0x7fc54c102710 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc54b7fe700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c102710 0x7fc54c103110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc54000c890 tx=0x7fc5400044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc548ff9700 1 -- 172.21.15.70:0/2498112589 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc54000a700 con 0x7fc54c102710 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc548ff9700 1 -- 172.21.15.70:0/2498112589 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc54000ad10 con 0x7fc54c102710 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c103a40 con 0x7fc54c102710 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc548ff9700 1 -- 172.21.15.70:0/2498112589 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc540012e70 con 0x7fc54c102710 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.195+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c11a210 con 0x7fc54c102710 2024-03-23T08:32:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.196+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c0fd4c0 con 0x7fc54c102710 2024-03-23T08:32:26.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.197+0000 7fc548ff9700 1 -- 172.21.15.70:0/2498112589 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc54000a860 con 0x7fc54c102710 2024-03-23T08:32:26.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.197+0000 7fc548ff9700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc53406cc30 0x7fc53406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:26.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.197+0000 7fc54bfff700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc53406cc30 0x7fc53406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:26.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.197+0000 7fc548ff9700 1 -- 172.21.15.70:0/2498112589 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc54008be20 con 0x7fc54c102710 2024-03-23T08:32:26.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.198+0000 7fc54bfff700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc53406cc30 0x7fc53406f0e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00b840 tx=0x7fc53c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:26.201 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.202+0000 7fc548ff9700 1 -- 172.21.15.70:0/2498112589 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc540055f00 con 0x7fc54c102710 2024-03-23T08:32:26.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:25 smithi070 ceph-mon[145869]: pgmap v173: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.aymzqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:32:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:25 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:25 smithi193 ceph-mon[108533]: pgmap v173: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.aymzqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:32:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:25 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:26.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.442+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc54c108d60 con 0x7fc54c102710 2024-03-23T08:32:26.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.442+0000 7fc548ff9700 1 -- 172.21.15.70:0/2498112589 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1849 (secure 0 0 0) 0x7fc540059520 con 0x7fc54c102710 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:e21 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 18 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:32:04.587715+0000 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:32:26.443 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 0 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=14462} 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:32:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:14462} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.70:6828/2658800706,v1:172.21.15.70:6829/2658800706] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:24249} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{-1:34376} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:32:26.445 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{-1:44255} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/2042142064,v1:172.21.15.193:6827/2042142064] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc53406cc30 msgr2=0x7fc53406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc53406cc30 0x7fc53406f0e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00b840 tx=0x7fc53c009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c102710 msgr2=0x7fc54c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c102710 0x7fc54c103110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc54000c890 tx=0x7fc5400044c0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 shutdown_connections 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc53406cc30 0x7fc53406f0e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc54c101d70 0x7fc54c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 --2- 172.21.15.70:0/2498112589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c102710 0x7fc54c103110 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 >> 172.21.15.70:0/2498112589 conn(0x7fc54c072bc0 msgr2=0x7fc54c065a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 shutdown_connections 2024-03-23T08:32:26.447 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.444+0000 7fc5528b2700 1 -- 172.21.15.70:0/2498112589 wait complete. 2024-03-23T08:32:26.447 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 21 2024-03-23T08:32:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 -- 172.21.15.70:0/1843182710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec101d10 msgr2=0x7f0dec1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 --2- 172.21.15.70:0/1843182710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec101d10 0x7f0dec1020e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc004940 tx=0x7f0ddc009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 -- 172.21.15.70:0/1843182710 shutdown_connections 2024-03-23T08:32:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 --2- 172.21.15.70:0/1843182710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec1026b0 0x7f0dec0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 --2- 172.21.15.70:0/1843182710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec101d10 0x7f0dec1020e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 -- 172.21.15.70:0/1843182710 >> 172.21.15.70:0/1843182710 conn(0x7f0dec0f7880 msgr2=0x7f0dec0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 -- 172.21.15.70:0/1843182710 shutdown_connections 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 -- 172.21.15.70:0/1843182710 wait complete. 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 Processor -- start 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 -- start start 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec101d10 0x7f0dec102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec1026b0 0x7f0dec1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec103640 con 0x7f0dec1026b0 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.546+0000 7f0df0db9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec103780 con 0x7f0dec101d10 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0de9d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec1026b0 0x7f0dec1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0de9d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec1026b0 0x7f0dec1030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54244/0 (socket says 172.21.15.70:54244) 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0de9d9b700 1 -- 172.21.15.70:0/549695746 learned_addr learned my addr 172.21.15.70:0/549695746 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0dea59c700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec101d10 0x7f0dec102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0de9d9b700 1 -- 172.21.15.70:0/549695746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec101d10 msgr2=0x7f0dec102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0de9d9b700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec101d10 0x7f0dec102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0de9d9b700 1 -- 172.21.15.70:0/549695746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc005040 con 0x7f0dec1026b0 2024-03-23T08:32:26.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0de9d9b700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec1026b0 0x7f0dec1030b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0de000c800 tx=0x7f0de0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:26.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0ddaffd700 1 -- 172.21.15.70:0/549695746 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0de00106f0 con 0x7f0dec1026b0 2024-03-23T08:32:26.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0ddaffd700 1 -- 172.21.15.70:0/549695746 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0de0010d00 con 0x7f0dec1026b0 2024-03-23T08:32:26.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec11a090 con 0x7f0dec1026b0 2024-03-23T08:32:26.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0ddaffd700 1 -- 172.21.15.70:0/549695746 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0de00113a0 con 0x7f0dec1026b0 2024-03-23T08:32:26.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.547+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec11a510 con 0x7f0dec1026b0 2024-03-23T08:32:26.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.548+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec0fd460 con 0x7f0dec1026b0 2024-03-23T08:32:26.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.549+0000 7f0ddaffd700 1 -- 172.21.15.70:0/549695746 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f0de0012400 con 0x7f0dec1026b0 2024-03-23T08:32:26.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.549+0000 7f0ddaffd700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0dd406ccf0 0x7f0dd406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:26.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.549+0000 7f0ddaffd700 1 -- 172.21.15.70:0/549695746 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0de008bf00 con 0x7f0dec1026b0 2024-03-23T08:32:26.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.549+0000 7f0dea59c700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0dd406ccf0 0x7f0dd406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:26.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.550+0000 7f0dea59c700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0dd406ccf0 0x7f0dd406f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc004f90 tx=0x7f0ddc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:26.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.554+0000 7f0ddaffd700 1 -- 172.21.15.70:0/549695746 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0de0055fc0 con 0x7f0dec1026b0 2024-03-23T08:32:26.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.768+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dec04bed0 con 0x7f0dd406ccf0 2024-03-23T08:32:26.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.769+0000 7f0ddaffd700 1 -- 172.21.15.70:0/549695746 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0dec04bed0 con 0x7f0dd406ccf0 2024-03-23T08:32:26.769 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:32:26.769 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:32:26.769 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd", 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "13/21 daemons upgraded", 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mds daemons", 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:32:26.770 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:32:26.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0dd406ccf0 msgr2=0x7f0dd406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0dd406ccf0 0x7f0dd406f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc004f90 tx=0x7f0ddc009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec1026b0 msgr2=0x7f0dec1030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec1026b0 0x7f0dec1030b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0de000c800 tx=0x7f0de0004470 comp rx=0 tx=0).stop 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 shutdown_connections 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f0dd406ccf0 0x7f0dd406f1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dec101d10 0x7f0dec102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 --2- 172.21.15.70:0/549695746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dec1026b0 0x7f0dec1030b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 >> 172.21.15.70:0/549695746 conn(0x7f0dec0f7880 msgr2=0x7f0dec0f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 shutdown_connections 2024-03-23T08:32:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.771+0000 7f0df0db9700 1 -- 172.21.15.70:0/549695746 wait complete. 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.876+0000 7f6744c8e700 1 -- 172.21.15.70:0/3699430148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 msgr2=0x7f67400fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.876+0000 7f6744c8e700 1 --2- 172.21.15.70:0/3699430148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 0x7f67400fbd80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6730004940 tx=0x7f6730009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.876+0000 7f6744c8e700 1 -- 172.21.15.70:0/3699430148 shutdown_connections 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.876+0000 7f6744c8e700 1 --2- 172.21.15.70:0/3699430148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 0x7f67400fbd80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.876+0000 7f6744c8e700 1 --2- 172.21.15.70:0/3699430148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6740101d70 0x7f6740102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.876+0000 7f6744c8e700 1 -- 172.21.15.70:0/3699430148 >> 172.21.15.70:0/3699430148 conn(0x7f6740072bc0 msgr2=0x7f6740072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.876+0000 7f6744c8e700 1 -- 172.21.15.70:0/3699430148 shutdown_connections 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.876+0000 7f6744c8e700 1 -- 172.21.15.70:0/3699430148 wait complete. 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f6744c8e700 1 Processor -- start 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f6744c8e700 1 -- start start 2024-03-23T08:32:26.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f6744c8e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6740101d70 0x7f674010c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f6744c8e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 0x7f674010cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f6744c8e700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674010d330 con 0x7f6740102710 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f6744c8e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674010d4a0 con 0x7f6740101d70 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f673dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 0x7f674010cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f673dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 0x7f674010cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54268/0 (socket says 172.21.15.70:54268) 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f673dd9b700 1 -- 172.21.15.70:0/3020111090 learned_addr learned my addr 172.21.15.70:0/3020111090 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f673e59c700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6740101d70 0x7f674010c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f673dd9b700 1 -- 172.21.15.70:0/3020111090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6740101d70 msgr2=0x7f674010c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f673dd9b700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6740101d70 0x7f674010c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f673dd9b700 1 -- 172.21.15.70:0/3020111090 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6730005040 con 0x7f6740102710 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f673e59c700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6740101d70 0x7f674010c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f673dd9b700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 0x7f674010cd60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6730009380 tx=0x7f6730009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f6736ffd700 1 -- 172.21.15.70:0/3020111090 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f673000e040 con 0x7f6740102710 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.877+0000 7f6736ffd700 1 -- 172.21.15.70:0/3020111090 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6730013420 con 0x7f6740102710 2024-03-23T08:32:26.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.878+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6740119e90 con 0x7f6740102710 2024-03-23T08:32:26.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.878+0000 7f6736ffd700 1 -- 172.21.15.70:0/3020111090 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f673001c630 con 0x7f6740102710 2024-03-23T08:32:26.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.878+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674011a310 con 0x7f6740102710 2024-03-23T08:32:26.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.878+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67400ff080 con 0x7f6740102710 2024-03-23T08:32:26.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.879+0000 7f6736ffd700 1 -- 172.21.15.70:0/3020111090 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f67300139e0 con 0x7f6740102710 2024-03-23T08:32:26.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.879+0000 7f6736ffd700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f672c06cc20 0x7f672c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:26.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.879+0000 7f6736ffd700 1 -- 172.21.15.70:0/3020111090 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f673008d130 con 0x7f6740102710 2024-03-23T08:32:26.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.879+0000 7f673e59c700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f672c06cc20 0x7f672c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:26.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.880+0000 7f673e59c700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f672c06cc20 0x7f672c06f0d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6728009b10 tx=0x7f6728008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:26.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:26.884+0000 7f6736ffd700 1 -- 172.21.15.70:0/3020111090 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f673005aca0 con 0x7f6740102710 2024-03-23T08:32:27.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:26 smithi070 ceph-mon[145869]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:27.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:26 smithi070 ceph-mon[145869]: Upgrade: Updating mds.cephfs.smithi070.aymzqb 2024-03-23T08:32:27.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:26 smithi070 ceph-mon[145869]: Deploying daemon mds.cephfs.smithi070.aymzqb on smithi070 2024-03-23T08:32:27.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:26 smithi070 ceph-mon[145869]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:27.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:26 smithi070 ceph-mon[145869]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:27.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:26 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/671402066' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:27.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:26 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2498112589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:32:27.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.189+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f67400fd4c0 con 0x7f6740102710 2024-03-23T08:32:27.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.190+0000 7f6736ffd700 1 -- 172.21.15.70:0/3020111090 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f673005a830 con 0x7f6740102710 2024-03-23T08:32:27.190 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f672c06cc20 msgr2=0x7f672c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f672c06cc20 0x7f672c06f0d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6728009b10 tx=0x7f6728008040 comp rx=0 tx=0).stop 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 msgr2=0x7f674010cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 0x7f674010cd60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6730009380 tx=0x7f6730009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 shutdown_connections 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f672c06cc20 0x7f672c06f0d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6740101d70 0x7f674010c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 --2- 172.21.15.70:0/3020111090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6740102710 0x7f674010cd60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 >> 172.21.15.70:0/3020111090 conn(0x7f6740072bc0 msgr2=0x7f67400fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 shutdown_connections 2024-03-23T08:32:27.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:27.192+0000 7f6744c8e700 1 -- 172.21.15.70:0/3020111090 wait complete. 2024-03-23T08:32:27.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:26 smithi193 ceph-mon[108533]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:27.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:26 smithi193 ceph-mon[108533]: Upgrade: Updating mds.cephfs.smithi070.aymzqb 2024-03-23T08:32:27.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:26 smithi193 ceph-mon[108533]: Deploying daemon mds.cephfs.smithi070.aymzqb on smithi070 2024-03-23T08:32:27.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:26 smithi193 ceph-mon[108533]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:27.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:26 smithi193 ceph-mon[108533]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:27.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:26 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/671402066' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:27.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:26 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2498112589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:32:27.732 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070[145846]: 2024-03-23T08:32:27.286+0000 7f3c746c7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T08:32:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: pgmap v174: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3020111090' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:32:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T08:32:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T08:32:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: osdmap e83: 6 total, 6 up, 6 in 2024-03-23T08:32:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: Standby daemon mds.cephfs.smithi193.sdlbdd assigned to filesystem cephfs as rank 0 2024-03-23T08:32:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T08:32:28.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-23T08:32:28.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: fsmap cephfs:1/1 {0=cephfs.smithi193.sdlbdd=up:replay} 2 up:standby 2024-03-23T08:32:28.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:27 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:32:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: pgmap v174: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:32:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3020111090' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:32:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T08:32:28.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T08:32:28.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: osdmap e83: 6 total, 6 up, 6 in 2024-03-23T08:32:28.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: Standby daemon mds.cephfs.smithi193.sdlbdd assigned to filesystem cephfs as rank 0 2024-03-23T08:32:28.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T08:32:28.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-23T08:32:28.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: fsmap cephfs:1/1 {0=cephfs.smithi193.sdlbdd=up:replay} 2 up:standby 2024-03-23T08:32:28.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:27 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:32:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:29 smithi070 ceph-mon[145869]: pgmap v176: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-03-23T08:32:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:29 smithi193 ceph-mon[108533]: pgmap v176: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-03-23T08:32:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:31 smithi070 ceph-mon[145869]: pgmap v177: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 0 op/s 2024-03-23T08:32:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:32:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:31 smithi193 ceph-mon[108533]: pgmap v177: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 0 op/s 2024-03-23T08:32:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:32:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:33 smithi070 ceph-mon[145869]: pgmap v178: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-23T08:32:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:33 smithi193 ceph-mon[108533]: pgmap v178: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-23T08:32:36.100 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:35 smithi070 ceph-mon[145869]: pgmap v179: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-23T08:32:36.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:35 smithi193 ceph-mon[108533]: pgmap v179: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-23T08:32:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: pgmap v180: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-23T08:32:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: reconnect by client.24291 192.168.0.1:0/2176118012 after 0.000999985 2024-03-23T08:32:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: reconnect by client.14522 192.168.0.1:0/1817993757 after 0.000999985 2024-03-23T08:32:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:reconnect 2024-03-23T08:32:37.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.70:6828/470900113,v1:172.21.15.70:6829/470900113] up:boot 2024-03-23T08:32:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: fsmap cephfs:1/1 {0=cephfs.smithi193.sdlbdd=up:reconnect} 3 up:standby 2024-03-23T08:32:37.484 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:37 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:32:37.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:37.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:37.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:37.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: pgmap v180: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-23T08:32:37.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: reconnect by client.24291 192.168.0.1:0/2176118012 after 0.000999985 2024-03-23T08:32:37.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: reconnect by client.14522 192.168.0.1:0/1817993757 after 0.000999985 2024-03-23T08:32:37.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:reconnect 2024-03-23T08:32:37.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.70:6828/470900113,v1:172.21.15.70:6829/470900113] up:boot 2024-03-23T08:32:37.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: fsmap cephfs:1/1 {0=cephfs.smithi193.sdlbdd=up:reconnect} 3 up:standby 2024-03-23T08:32:37.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:37 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:32:38.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:38 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:rejoin 2024-03-23T08:32:38.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:38 smithi070 ceph-mon[145869]: fsmap cephfs:1/1 {0=cephfs.smithi193.sdlbdd=up:rejoin} 3 up:standby 2024-03-23T08:32:38.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:38 smithi070 ceph-mon[145869]: daemon mds.cephfs.smithi193.sdlbdd is now active in filesystem cephfs as rank 0 2024-03-23T08:32:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:38 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:rejoin 2024-03-23T08:32:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:38 smithi193 ceph-mon[108533]: fsmap cephfs:1/1 {0=cephfs.smithi193.sdlbdd=up:rejoin} 3 up:standby 2024-03-23T08:32:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:38 smithi193 ceph-mon[108533]: daemon mds.cephfs.smithi193.sdlbdd is now active in filesystem cephfs as rank 0 2024-03-23T08:32:40.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:39 smithi070 ceph-mon[145869]: pgmap v181: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-23T08:32:40.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:39 smithi070 ceph-mon[145869]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T08:32:40.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:39 smithi070 ceph-mon[145869]: Cluster is now healthy 2024-03-23T08:32:40.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:39 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:active 2024-03-23T08:32:40.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:39 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi193.sdlbdd=up:active} 3 up:standby 2024-03-23T08:32:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:39 smithi193 ceph-mon[108533]: pgmap v181: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-23T08:32:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:39 smithi193 ceph-mon[108533]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T08:32:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:39 smithi193 ceph-mon[108533]: Cluster is now healthy 2024-03-23T08:32:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:39 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.193:6824/3335188435,v1:172.21.15.193:6825/3335188435] up:active 2024-03-23T08:32:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:39 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi193.sdlbdd=up:active} 3 up:standby 2024-03-23T08:32:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:41 smithi070 ceph-mon[145869]: pgmap v182: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 6 op/s 2024-03-23T08:32:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:41 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:41 smithi193 ceph-mon[108533]: pgmap v182: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 6 op/s 2024-03-23T08:32:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:41 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:44.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:43 smithi070 ceph-mon[145869]: pgmap v183: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T08:32:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:43 smithi193 ceph-mon[108533]: pgmap v183: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T08:32:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: pgmap v184: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:46.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:45 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: pgmap v184: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:45 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:47.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:47 smithi193 ceph-mon[108533]: pgmap v185: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-23T08:32:47.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:47 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:47.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:47 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:32:47.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:47 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:47.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:47 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sdlbdd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:32:47.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:47 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:47 smithi070 ceph-mon[145869]: pgmap v185: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-23T08:32:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:32:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.sdlbdd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:32:47.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:47 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:32:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:48 smithi193 ceph-mon[108533]: Upgrade: Updating mds.cephfs.smithi193.sdlbdd 2024-03-23T08:32:48.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:48 smithi193 ceph-mon[108533]: Deploying daemon mds.cephfs.smithi193.sdlbdd on smithi193 2024-03-23T08:32:48.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:48 smithi070 ceph-mon[145869]: Upgrade: Updating mds.cephfs.smithi193.sdlbdd 2024-03-23T08:32:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:48 smithi070 ceph-mon[145869]: Deploying daemon mds.cephfs.smithi193.sdlbdd on smithi193 2024-03-23T08:32:48.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:48 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070[145846]: 2024-03-23T08:32:48.850+0000 7f3c746c7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T08:32:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:49 smithi070 ceph-mon[145869]: pgmap v186: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 686 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-23T08:32:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:49 smithi070 ceph-mon[145869]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T08:32:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:49 smithi070 ceph-mon[145869]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T08:32:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:49 smithi070 ceph-mon[145869]: osdmap e84: 6 total, 6 up, 6 in 2024-03-23T08:32:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:49 smithi070 ceph-mon[145869]: Standby daemon mds.cephfs.smithi070.iziydh assigned to filesystem cephfs as rank 0 2024-03-23T08:32:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:49 smithi070 ceph-mon[145869]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T08:32:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:49 smithi070 ceph-mon[145869]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-23T08:32:49.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:49 smithi070 ceph-mon[145869]: fsmap cephfs:1/1 {0=cephfs.smithi070.iziydh=up:replay} 2 up:standby 2024-03-23T08:32:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:49 smithi193 ceph-mon[108533]: pgmap v186: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 686 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-23T08:32:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:49 smithi193 ceph-mon[108533]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-23T08:32:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:49 smithi193 ceph-mon[108533]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T08:32:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:49 smithi193 ceph-mon[108533]: osdmap e84: 6 total, 6 up, 6 in 2024-03-23T08:32:50.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:49 smithi193 ceph-mon[108533]: Standby daemon mds.cephfs.smithi070.iziydh assigned to filesystem cephfs as rank 0 2024-03-23T08:32:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:49 smithi193 ceph-mon[108533]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-23T08:32:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:49 smithi193 ceph-mon[108533]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-23T08:32:50.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:49 smithi193 ceph-mon[108533]: fsmap cephfs:1/1 {0=cephfs.smithi070.iziydh=up:replay} 2 up:standby 2024-03-23T08:32:51.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:51 smithi070 ceph-mon[145869]: pgmap v188: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 412 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-03-23T08:32:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:51 smithi193 ceph-mon[108533]: pgmap v188: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 412 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-03-23T08:32:54.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:53 smithi193 ceph-mon[108533]: pgmap v189: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-03-23T08:32:54.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:53 smithi070 ceph-mon[145869]: pgmap v189: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-03-23T08:32:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:55 smithi193 ceph-mon[108533]: pgmap v190: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 9.8 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-03-23T08:32:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:56.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:55 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:55 smithi070 ceph-mon[145869]: pgmap v190: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 9.8 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-03-23T08:32:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:55 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:32:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:56 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.193:6824/4097359817,v1:172.21.15.193:6825/4097359817] up:boot 2024-03-23T08:32:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:56 smithi193 ceph-mon[108533]: fsmap cephfs:1/1 {0=cephfs.smithi070.iziydh=up:replay} 3 up:standby 2024-03-23T08:32:57.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:56 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:32:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:56 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.193:6824/4097359817,v1:172.21.15.193:6825/4097359817] up:boot 2024-03-23T08:32:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:56 smithi070 ceph-mon[145869]: fsmap cephfs:1/1 {0=cephfs.smithi070.iziydh=up:replay} 3 up:standby 2024-03-23T08:32:57.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:56 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:32:57.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.298+0000 7fb891671700 1 -- 172.21.15.70:0/2037881921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c102710 msgr2=0x7fb88c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.298+0000 7fb891671700 1 --2- 172.21.15.70:0/2037881921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c102710 0x7fb88c0fbd80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb880004940 tx=0x7fb880009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:57.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.298+0000 7fb891671700 1 -- 172.21.15.70:0/2037881921 shutdown_connections 2024-03-23T08:32:57.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.298+0000 7fb891671700 1 --2- 172.21.15.70:0/2037881921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c102710 0x7fb88c0fbd80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.298+0000 7fb891671700 1 --2- 172.21.15.70:0/2037881921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c101d70 0x7fb88c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.298+0000 7fb891671700 1 -- 172.21.15.70:0/2037881921 >> 172.21.15.70:0/2037881921 conn(0x7fb88c072bc0 msgr2=0x7fb88c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:57.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.298+0000 7fb891671700 1 -- 172.21.15.70:0/2037881921 shutdown_connections 2024-03-23T08:32:57.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.298+0000 7fb891671700 1 -- 172.21.15.70:0/2037881921 wait complete. 2024-03-23T08:32:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.299+0000 7fb891671700 1 Processor -- start 2024-03-23T08:32:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.299+0000 7fb891671700 1 -- start start 2024-03-23T08:32:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb891671700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c101d70 0x7fb88c113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb891671700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102710 0x7fb88c113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb891671700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c1145d0 con 0x7fb88c101d70 2024-03-23T08:32:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb891671700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c10d9c0 con 0x7fb88c102710 2024-03-23T08:32:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb88affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c101d70 0x7fb88c113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb88affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c101d70 0x7fb88c113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40676/0 (socket says 172.21.15.70:40676) 2024-03-23T08:32:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb88affd700 1 -- 172.21.15.70:0/2150549420 learned_addr learned my addr 172.21.15.70:0/2150549420 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb88a7fc700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102710 0x7fb88c113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb88affd700 1 -- 172.21.15.70:0/2150549420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102710 msgr2=0x7fb88c113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb88affd700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102710 0x7fb88c113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb88affd700 1 -- 172.21.15.70:0/2150549420 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb880005040 con 0x7fb88c101d70 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb88affd700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c101d70 0x7fb88c113a40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb87c002a80 tx=0x7fb87c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb8737fe700 1 -- 172.21.15.70:0/2150549420 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb87c007cc0 con 0x7fb88c101d70 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb8737fe700 1 -- 172.21.15.70:0/2150549420 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb87c00f040 con 0x7fb88c101d70 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb8737fe700 1 -- 172.21.15.70:0/2150549420 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb87c00da60 con 0x7fb88c101d70 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.300+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c10dca0 con 0x7fb88c101d70 2024-03-23T08:32:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.301+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c10e230 con 0x7fb88c101d70 2024-03-23T08:32:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.302+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c100090 con 0x7fb88c101d70 2024-03-23T08:32:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.302+0000 7fb8737fe700 1 -- 172.21.15.70:0/2150549420 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb87c00dbc0 con 0x7fb88c101d70 2024-03-23T08:32:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.302+0000 7fb8737fe700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb87406cc20 0x7fb87406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.303+0000 7fb8737fe700 1 -- 172.21.15.70:0/2150549420 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb87c08bd70 con 0x7fb88c101d70 2024-03-23T08:32:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.303+0000 7fb88a7fc700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb87406cc20 0x7fb87406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.303+0000 7fb88a7fc700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb87406cc20 0x7fb87406f0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb88c10eca0 tx=0x7fb880009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:57.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.311+0000 7fb8737fe700 1 -- 172.21.15.70:0/2150549420 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb87c059800 con 0x7fb88c101d70 2024-03-23T08:32:57.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.521+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb88c10de30 con 0x7fb87406cc20 2024-03-23T08:32:57.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.523+0000 7fb8737fe700 1 -- 172.21.15.70:0/2150549420 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb88c10de30 con 0x7fb87406cc20 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb87406cc20 msgr2=0x7fb87406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb87406cc20 0x7fb87406f0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb88c10eca0 tx=0x7fb880009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c101d70 msgr2=0x7fb88c113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c101d70 0x7fb88c113a40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb87c002a80 tx=0x7fb87c0046b0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 shutdown_connections 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb87406cc20 0x7fb87406f0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb88c101d70 0x7fb88c113a40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 --2- 172.21.15.70:0/2150549420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102710 0x7fb88c113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 >> 172.21.15.70:0/2150549420 conn(0x7fb88c072bc0 msgr2=0x7fb88c065a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 shutdown_connections 2024-03-23T08:32:57.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.525+0000 7fb891671700 1 -- 172.21.15.70:0/2150549420 wait complete. 2024-03-23T08:32:57.535 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.626+0000 7f119e843700 1 -- 172.21.15.70:0/1049114252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 msgr2=0x7f11980fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.626+0000 7f119e843700 1 --2- 172.21.15.70:0/1049114252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 0x7f11980fe260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1188004940 tx=0x7f1188009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.626+0000 7f119e843700 1 -- 172.21.15.70:0/1049114252 shutdown_connections 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.626+0000 7f119e843700 1 --2- 172.21.15.70:0/1049114252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11980fe830 0x7f1198106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.626+0000 7f119e843700 1 --2- 172.21.15.70:0/1049114252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 0x7f11980fe260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.626+0000 7f119e843700 1 -- 172.21.15.70:0/1049114252 >> 172.21.15.70:0/1049114252 conn(0x7f1198072bc0 msgr2=0x7f1198072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.626+0000 7f119e843700 1 -- 172.21.15.70:0/1049114252 shutdown_connections 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.626+0000 7f119e843700 1 -- 172.21.15.70:0/1049114252 wait complete. 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f119e843700 1 Processor -- start 2024-03-23T08:32:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f119e843700 1 -- start start 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f119e843700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 0x7f11980fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f119e843700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11980fe830 0x7f11980fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f119e843700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11980fc820 con 0x7f11980fde90 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f119e843700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11980fc960 con 0x7f11980fe830 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f1197fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 0x7f11980fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f1197fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 0x7f11980fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40700/0 (socket says 172.21.15.70:40700) 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f1197fff700 1 -- 172.21.15.70:0/4011991982 learned_addr learned my addr 172.21.15.70:0/4011991982 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f11977fe700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11980fe830 0x7f11980fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f1197fff700 1 -- 172.21.15.70:0/4011991982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11980fe830 msgr2=0x7f11980fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f1197fff700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11980fe830 0x7f11980fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.627+0000 7f1197fff700 1 -- 172.21.15.70:0/4011991982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1188005040 con 0x7f11980fde90 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.628+0000 7f1197fff700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 0x7f11980fbd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f118800b3b0 tx=0x7f1188009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.628+0000 7f1194ff9700 1 -- 172.21.15.70:0/4011991982 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118800e040 con 0x7f11980fde90 2024-03-23T08:32:57.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.628+0000 7f1194ff9700 1 -- 172.21.15.70:0/4011991982 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1188013420 con 0x7f11980fde90 2024-03-23T08:32:57.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.628+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11980fcbb0 con 0x7f11980fde90 2024-03-23T08:32:57.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.628+0000 7f1194ff9700 1 -- 172.21.15.70:0/4011991982 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118801c5f0 con 0x7f11980fde90 2024-03-23T08:32:57.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.628+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119811a1a0 con 0x7f11980fde90 2024-03-23T08:32:57.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.629+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1198065e30 con 0x7f11980fde90 2024-03-23T08:32:57.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.629+0000 7f1194ff9700 1 -- 172.21.15.70:0/4011991982 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1188013590 con 0x7f11980fde90 2024-03-23T08:32:57.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.629+0000 7f1194ff9700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f118006cc20 0x7f118006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:57.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.630+0000 7f1194ff9700 1 -- 172.21.15.70:0/4011991982 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f118808d330 con 0x7f11980fde90 2024-03-23T08:32:57.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.630+0000 7f11977fe700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f118006cc20 0x7f118006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:57.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.630+0000 7f11977fe700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f118006cc20 0x7f118006f0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11980fd690 tx=0x7f118c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:57.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.634+0000 7f1194ff9700 1 -- 172.21.15.70:0/4011991982 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1188057370 con 0x7f11980fde90 2024-03-23T08:32:57.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.851+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11980fd490 con 0x7f118006cc20 2024-03-23T08:32:57.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.852+0000 7f1194ff9700 1 -- 172.21.15.70:0/4011991982 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f11980fd490 con 0x7f118006cc20 2024-03-23T08:32:57.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f118006cc20 msgr2=0x7f118006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f118006cc20 0x7f118006f0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11980fd690 tx=0x7f118c00b040 comp rx=0 tx=0).stop 2024-03-23T08:32:57.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 msgr2=0x7f11980fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 0x7f11980fbd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f118800b3b0 tx=0x7f1188009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:57.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 shutdown_connections 2024-03-23T08:32:57.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f118006cc20 0x7f118006f0d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11980fde90 0x7f11980fbd50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 --2- 172.21.15.70:0/4011991982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11980fe830 0x7f11980fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 >> 172.21.15.70:0/4011991982 conn(0x7f1198072bc0 msgr2=0x7f11980ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:57.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.854+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 shutdown_connections 2024-03-23T08:32:57.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.855+0000 7f119e843700 1 -- 172.21.15.70:0/4011991982 wait complete. 2024-03-23T08:32:57.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/4281172100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fde90 msgr2=0x7f8c940fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 --2- 172.21.15.70:0/4281172100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fde90 0x7f8c940fe260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8c90004940 tx=0x7f8c90009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/4281172100 shutdown_connections 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 --2- 172.21.15.70:0/4281172100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c940fe830 0x7f8c94106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 --2- 172.21.15.70:0/4281172100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fde90 0x7f8c940fe260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/4281172100 >> 172.21.15.70:0/4281172100 conn(0x7f8c94072bc0 msgr2=0x7f8c94072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/4281172100 shutdown_connections 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/4281172100 wait complete. 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 Processor -- start 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 -- start start 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c940fde90 0x7f8c94113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fe830 0x7f8c94114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c94114680 con 0x7f8c940fe830 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c9ba7c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9410da70 con 0x7f8c940fde90 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.955+0000 7f8c99017700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fe830 0x7f8c94114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:57.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c99017700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fe830 0x7f8c94114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40724/0 (socket says 172.21.15.70:40724) 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c99017700 1 -- 172.21.15.70:0/1559225704 learned_addr learned my addr 172.21.15.70:0/1559225704 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c99818700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c940fde90 0x7f8c94113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c99017700 1 -- 172.21.15.70:0/1559225704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c940fde90 msgr2=0x7f8c94113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c99017700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c940fde90 0x7f8c94113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c99017700 1 -- 172.21.15.70:0/1559225704 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c90005040 con 0x7f8c940fe830 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c99017700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fe830 0x7f8c94114030 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c84002a50 tx=0x7f8c840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c8a7fc700 1 -- 172.21.15.70:0/1559225704 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c84007cc0 con 0x7f8c940fe830 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9410dd50 con 0x7f8c940fe830 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c8a7fc700 1 -- 172.21.15.70:0/1559225704 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c8400f040 con 0x7f8c940fe830 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c8a7fc700 1 -- 172.21.15.70:0/1559225704 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c8400d9c0 con 0x7f8c940fe830 2024-03-23T08:32:57.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.956+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9410e290 con 0x7f8c940fe830 2024-03-23T08:32:57.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.957+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c941019b0 con 0x7f8c940fe830 2024-03-23T08:32:57.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.957+0000 7f8c8a7fc700 1 -- 172.21.15.70:0/1559225704 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8c8400db20 con 0x7f8c940fe830 2024-03-23T08:32:57.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.958+0000 7f8c8a7fc700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8006cc20 0x7f8c8006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:57.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.958+0000 7f8c8a7fc700 1 -- 172.21.15.70:0/1559225704 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8c8408bd60 con 0x7f8c940fe830 2024-03-23T08:32:57.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.958+0000 7f8c99818700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8006cc20 0x7f8c8006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:57.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.958+0000 7f8c99818700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8006cc20 0x7f8c8006f0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000b7e0 tx=0x7f8c90009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:57.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:57.962+0000 7f8c8a7fc700 1 -- 172.21.15.70:0/1559225704 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8c840597b0 con 0x7f8c940fe830 2024-03-23T08:32:58.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:57 smithi193 ceph-mon[108533]: pgmap v191: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-23T08:32:58.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.163+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c941137c0 con 0x7f8c8006cc20 2024-03-23T08:32:58.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.173+0000 7f8c8a7fc700 1 -- 172.21.15.70:0/1559225704 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f8c941137c0 con 0x7f8c8006cc20 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (6m) 17s ago 17m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (4m) 17s ago 17m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (4m) 59s ago 16m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (6m) 17s ago 17m 90.9M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (22s) 17s ago 12m 10.9M - 18.2.2-954-g82316b04 b24e1baba637 3aedd00ed88c 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (44s) 17s ago 12m 14.7M - 18.2.2-954-g82316b04 b24e1baba637 1066f2091682 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (12m) 59s ago 12m 97.3M - 17.2.7-726-g9c4e0c34 837124da804f a1106b4c2dc9 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 starting - - - - 2024-03-23T08:32:58.173 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (8m) 17s ago 19m 504M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (7m) 59s ago 16m 449M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (5m) 17s ago 19m 55.7M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (4m) 59s ago 16m 38.5M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 17s ago 17m 18.8M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (7m) 59s ago 16m 16.5M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (3m) 17s ago 16m 92.6M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (3m) 17s ago 15m 107M 1568M 18.2.2-954-g82316b04 b24e1baba637 13fce63c86eb 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (2m) 17s ago 15m 96.6M 1568M 18.2.2-954-g82316b04 b24e1baba637 73e36e623491 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (2m) 59s ago 14m 102M 2592M 18.2.2-954-g82316b04 b24e1baba637 8a7617f5541a 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (96s) 59s ago 14m 102M 2592M 18.2.2-954-g82316b04 b24e1baba637 76a25ee5af50 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (63s) 59s ago 13m 11.4M 2592M 18.2.2-954-g82316b04 b24e1baba637 58bbe6abe8b3 2024-03-23T08:32:58.174 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (6m) 17s ago 17m 60.5M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8006cc20 msgr2=0x7f8c8006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8006cc20 0x7f8c8006f0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000b7e0 tx=0x7f8c90009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fe830 msgr2=0x7f8c94114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fe830 0x7f8c94114030 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c84002a50 tx=0x7f8c840046b0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 shutdown_connections 2024-03-23T08:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8006cc20 0x7f8c8006f0d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c940fde90 0x7f8c94113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 --2- 172.21.15.70:0/1559225704 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c940fe830 0x7f8c94114030 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 >> 172.21.15.70:0/1559225704 conn(0x7f8c94072bc0 msgr2=0x7f8c940ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:58.176 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 shutdown_connections 2024-03-23T08:32:58.176 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.175+0000 7f8c9ba7c700 1 -- 172.21.15.70:0/1559225704 wait complete. 2024-03-23T08:32:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:57 smithi070 ceph-mon[145869]: pgmap v191: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.274+0000 7fc538e64700 1 -- 172.21.15.70:0/3205678331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 msgr2=0x7fc5340fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.274+0000 7fc538e64700 1 --2- 172.21.15.70:0/3205678331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 0x7fc5340fbd80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc528004940 tx=0x7fc528009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.274+0000 7fc538e64700 1 -- 172.21.15.70:0/3205678331 shutdown_connections 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.274+0000 7fc538e64700 1 --2- 172.21.15.70:0/3205678331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 0x7fc5340fbd80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.274+0000 7fc538e64700 1 --2- 172.21.15.70:0/3205678331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc534101d70 0x7fc534102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc538e64700 1 -- 172.21.15.70:0/3205678331 >> 172.21.15.70:0/3205678331 conn(0x7fc534072bc0 msgr2=0x7fc534072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc538e64700 1 -- 172.21.15.70:0/3205678331 shutdown_connections 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc538e64700 1 -- 172.21.15.70:0/3205678331 wait complete. 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc538e64700 1 Processor -- start 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc538e64700 1 -- start start 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc538e64700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc534101d70 0x7fc534112980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:58.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc538e64700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 0x7fc534112ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc538e64700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc534113510 con 0x7fc534102710 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc538e64700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53410c900 con 0x7fc534101d70 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc531d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 0x7fc534112ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc531d9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 0x7fc534112ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40734/0 (socket says 172.21.15.70:40734) 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc531d9b700 1 -- 172.21.15.70:0/223443099 learned_addr learned my addr 172.21.15.70:0/223443099 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.275+0000 7fc53259c700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc534101d70 0x7fc534112980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.276+0000 7fc531d9b700 1 -- 172.21.15.70:0/223443099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc534101d70 msgr2=0x7fc534112980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.276+0000 7fc531d9b700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc534101d70 0x7fc534112980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.276+0000 7fc531d9b700 1 -- 172.21.15.70:0/223443099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc528005040 con 0x7fc534102710 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.276+0000 7fc531d9b700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 0x7fc534112ec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5280041d0 tx=0x7fc528009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.276+0000 7fc522ffd700 1 -- 172.21.15.70:0/223443099 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52800e070 con 0x7fc534102710 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.276+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53410cb80 con 0x7fc534102710 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.276+0000 7fc522ffd700 1 -- 172.21.15.70:0/223443099 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc528004290 con 0x7fc534102710 2024-03-23T08:32:58.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.276+0000 7fc522ffd700 1 -- 172.21.15.70:0/223443099 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52801cc00 con 0x7fc534102710 2024-03-23T08:32:58.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.276+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53410d0e0 con 0x7fc534102710 2024-03-23T08:32:58.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.277+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc534104240 con 0x7fc534102710 2024-03-23T08:32:58.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.277+0000 7fc522ffd700 1 -- 172.21.15.70:0/223443099 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fc528013930 con 0x7fc534102710 2024-03-23T08:32:58.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.278+0000 7fc522ffd700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc51c06cc30 0x7fc51c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:58.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.278+0000 7fc53259c700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc51c06cc30 0x7fc51c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:58.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.278+0000 7fc522ffd700 1 -- 172.21.15.70:0/223443099 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc52808d4d0 con 0x7fc534102710 2024-03-23T08:32:58.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.278+0000 7fc53259c700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc51c06cc30 0x7fc51c06f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc524009b80 tx=0x7fc52400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:58.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.283+0000 7fc522ffd700 1 -- 172.21.15.70:0/223443099 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc52805afa0 con 0x7fc534102710 2024-03-23T08:32:58.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.562+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc5340fd4c0 con 0x7fc534102710 2024-03-23T08:32:58.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.562+0000 7fc522ffd700 1 -- 172.21.15.70:0/223443099 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+842 (secure 0 0 0) 0x7fc52805ab30 con 0x7fc534102710 2024-03-23T08:32:58.562 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:32:58.562 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 6 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 3 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 13 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:32:58.563 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:32:58.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc51c06cc30 msgr2=0x7fc51c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc51c06cc30 0x7fc51c06f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc524009b80 tx=0x7fc52400b040 comp rx=0 tx=0).stop 2024-03-23T08:32:58.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 msgr2=0x7fc534112ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 0x7fc534112ec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5280041d0 tx=0x7fc528009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:58.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 shutdown_connections 2024-03-23T08:32:58.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fc51c06cc30 0x7fc51c06f0e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc534101d70 0x7fc534112980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 --2- 172.21.15.70:0/223443099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc534102710 0x7fc534112ec0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 >> 172.21.15.70:0/223443099 conn(0x7fc534072bc0 msgr2=0x7fc5340fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:58.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 shutdown_connections 2024-03-23T08:32:58.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.564+0000 7fc538e64700 1 -- 172.21.15.70:0/223443099 wait complete. 2024-03-23T08:32:58.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 -- 172.21.15.70:0/2665110639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 msgr2=0x7fb320102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 --2- 172.21.15.70:0/2665110639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 0x7fb320102140 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb310004940 tx=0x7fb310009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:58.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 -- 172.21.15.70:0/2665110639 shutdown_connections 2024-03-23T08:32:58.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 --2- 172.21.15.70:0/2665110639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb320102710 0x7fb3200fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 --2- 172.21.15.70:0/2665110639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 0x7fb320102140 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 -- 172.21.15.70:0/2665110639 >> 172.21.15.70:0/2665110639 conn(0x7fb320072bc0 msgr2=0x7fb320072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 -- 172.21.15.70:0/2665110639 shutdown_connections 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 -- 172.21.15.70:0/2665110639 wait complete. 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 Processor -- start 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.665+0000 7fb3284a2700 1 -- start start 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb3284a2700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 0x7fb320115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb3284a2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb320102710 0x7fb3201161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb3284a2700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb320116820 con 0x7fb320101d70 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb3284a2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32010fc40 con 0x7fb320102710 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb32623e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 0x7fb320115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb32623e700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 0x7fb320115c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48570/0 (socket says 172.21.15.70:48570) 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb32623e700 1 -- 172.21.15.70:0/1670245423 learned_addr learned my addr 172.21.15.70:0/1670245423 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb325a3d700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb320102710 0x7fb3201161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb32623e700 1 -- 172.21.15.70:0/1670245423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb320102710 msgr2=0x7fb3201161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb32623e700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb320102710 0x7fb3201161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb32623e700 1 -- 172.21.15.70:0/1670245423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310005040 con 0x7fb320101d70 2024-03-23T08:32:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb32623e700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 0x7fb320115c90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb31000b3b0 tx=0x7fb310009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb316ffd700 1 -- 172.21.15.70:0/1670245423 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31000e040 con 0x7fb320101d70 2024-03-23T08:32:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32010fec0 con 0x7fb320101d70 2024-03-23T08:32:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb316ffd700 1 -- 172.21.15.70:0/1670245423 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb310013420 con 0x7fb320101d70 2024-03-23T08:32:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb316ffd700 1 -- 172.21.15.70:0/1670245423 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31001c610 con 0x7fb320101d70 2024-03-23T08:32:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.666+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb320110480 con 0x7fb320101d70 2024-03-23T08:32:58.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.667+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3200fd4c0 con 0x7fb320101d70 2024-03-23T08:32:58.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.669+0000 7fb316ffd700 1 -- 172.21.15.70:0/1670245423 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb310013a00 con 0x7fb320101d70 2024-03-23T08:32:58.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.669+0000 7fb316ffd700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb30c06cc20 0x7fb30c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:58.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.669+0000 7fb316ffd700 1 -- 172.21.15.70:0/1670245423 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb31008dc00 con 0x7fb320101d70 2024-03-23T08:32:58.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.669+0000 7fb325a3d700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb30c06cc20 0x7fb30c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:58.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.670+0000 7fb325a3d700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb30c06cc20 0x7fb30c06f0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb320111530 tx=0x7fb31c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:58.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.675+0000 7fb316ffd700 1 -- 172.21.15.70:0/1670245423 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb31005b790 con 0x7fb320101d70 2024-03-23T08:32:58.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.919+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb32004bf00 con 0x7fb320101d70 2024-03-23T08:32:58.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.919+0000 7fb316ffd700 1 -- 172.21.15.70:0/1670245423 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1849 (secure 0 0 0) 0x7fb31005b320 con 0x7fb320101d70 2024-03-23T08:32:58.919 INFO:teuthology.orchestra.run.smithi070.stdout:e31 2024-03-23T08:32:58.919 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:32:58.919 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:32:58.919 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:32:58.919 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:32:58.919 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:32:58.919 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 31 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:32:58.824297+0000 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 84 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:32:58.920 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=34376} 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{0:34376} state up:rejoin seq 13 join_fscid=1 addr [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6828/470900113,v1:172.21.15.70:6829/470900113] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:32:58.921 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{-1:44255} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/2042142064,v1:172.21.15.193:6827/2042142064] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:32:58.922 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:44279} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/4097359817,v1:172.21.15.193:6825/4097359817] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:32:58.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb30c06cc20 msgr2=0x7fb30c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb30c06cc20 0x7fb30c06f0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb320111530 tx=0x7fb31c00b040 comp rx=0 tx=0).stop 2024-03-23T08:32:58.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 msgr2=0x7fb320115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:58.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 0x7fb320115c90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb31000b3b0 tx=0x7fb310009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:58.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 shutdown_connections 2024-03-23T08:32:58.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb30c06cc20 0x7fb30c06f0d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb320101d70 0x7fb320115c90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 --2- 172.21.15.70:0/1670245423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb320102710 0x7fb3201161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:58.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 >> 172.21.15.70:0/1670245423 conn(0x7fb320072bc0 msgr2=0x7fb320065a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:58.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 shutdown_connections 2024-03-23T08:32:58.923 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:58.921+0000 7fb3284a2700 1 -- 172.21.15.70:0/1670245423 wait complete. 2024-03-23T08:32:58.923 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 31 2024-03-23T08:32:59.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.027+0000 7fe72bf19700 1 -- 172.21.15.70:0/2602030680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe724101d10 msgr2=0x7fe7241020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:59.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.027+0000 7fe72bf19700 1 --2- 172.21.15.70:0/2602030680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe724101d10 0x7fe7241020e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe720004940 tx=0x7fe720009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:59.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.027+0000 7fe72bf19700 1 -- 172.21.15.70:0/2602030680 shutdown_connections 2024-03-23T08:32:59.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.027+0000 7fe72bf19700 1 --2- 172.21.15.70:0/2602030680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7241026b0 0x7fe7240fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.027+0000 7fe72bf19700 1 --2- 172.21.15.70:0/2602030680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe724101d10 0x7fe7241020e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.027+0000 7fe72bf19700 1 -- 172.21.15.70:0/2602030680 >> 172.21.15.70:0/2602030680 conn(0x7fe7240f7860 msgr2=0x7fe7240f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:59.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.027+0000 7fe72bf19700 1 -- 172.21.15.70:0/2602030680 shutdown_connections 2024-03-23T08:32:59.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.027+0000 7fe72bf19700 1 -- 172.21.15.70:0/2602030680 wait complete. 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.027+0000 7fe72bf19700 1 Processor -- start 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe72bf19700 1 -- start start 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe72bf19700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe724101d10 0x7fe724113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe72bf19700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7241026b0 0x7fe724113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe72bf19700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7241145b0 con 0x7fe7241026b0 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe72bf19700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72410d9a0 con 0x7fe724101d10 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe7294b4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7241026b0 0x7fe724113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe729cb5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe724101d10 0x7fe724113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe7294b4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7241026b0 0x7fe724113f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48586/0 (socket says 172.21.15.70:48586) 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe7294b4700 1 -- 172.21.15.70:0/1742791987 learned_addr learned my addr 172.21.15.70:0/1742791987 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe7294b4700 1 -- 172.21.15.70:0/1742791987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe724101d10 msgr2=0x7fe724113a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe7294b4700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe724101d10 0x7fe724113a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe7294b4700 1 -- 172.21.15.70:0/1742791987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe720005040 con 0x7fe7241026b0 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.028+0000 7fe7294b4700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7241026b0 0x7fe724113f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe71400eb40 tx=0x7fe71400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:59.029 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.029+0000 7fe71a7fc700 1 -- 172.21.15.70:0/1742791987 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe714007cc0 con 0x7fe7241026b0 2024-03-23T08:32:59.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.029+0000 7fe71a7fc700 1 -- 172.21.15.70:0/1742791987 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe71400f040 con 0x7fe7241026b0 2024-03-23T08:32:59.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.029+0000 7fe71a7fc700 1 -- 172.21.15.70:0/1742791987 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe71400c8f0 con 0x7fe7241026b0 2024-03-23T08:32:59.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.029+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72410dc20 con 0x7fe7241026b0 2024-03-23T08:32:59.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.029+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72410e1e0 con 0x7fe7241026b0 2024-03-23T08:32:59.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.029+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7240fd460 con 0x7fe7241026b0 2024-03-23T08:32:59.030 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.030+0000 7fe71a7fc700 1 -- 172.21.15.70:0/1742791987 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe7140090d0 con 0x7fe7241026b0 2024-03-23T08:32:59.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.031+0000 7fe71a7fc700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe71006cc20 0x7fe71006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:59.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.031+0000 7fe71a7fc700 1 -- 172.21.15.70:0/1742791987 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe71408bc90 con 0x7fe7241026b0 2024-03-23T08:32:59.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.031+0000 7fe729cb5700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe71006cc20 0x7fe71006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:59.031 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.031+0000 7fe729cb5700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe71006cc20 0x7fe71006f0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b0d0 tx=0x7fe720009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:59.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.035+0000 7fe71a7fc700 1 -- 172.21.15.70:0/1742791987 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe714059810 con 0x7fe7241026b0 2024-03-23T08:32:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:58 smithi193 ceph-mon[108533]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:58 smithi193 ceph-mon[108533]: reconnect by client.14522 192.168.0.1:0/1817993757 after 0 2024-03-23T08:32:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:58 smithi193 ceph-mon[108533]: reconnect by client.24291 192.168.0.1:0/2176118012 after 0.000999971 2024-03-23T08:32:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:58 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] up:reconnect 2024-03-23T08:32:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:58 smithi193 ceph-mon[108533]: fsmap cephfs:1/1 {0=cephfs.smithi070.iziydh=up:reconnect} 3 up:standby 2024-03-23T08:32:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:58 smithi193 ceph-mon[108533]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:58 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:58 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:59.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:58 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/223443099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:59.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:58 smithi070 ceph-mon[145869]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:58 smithi070 ceph-mon[145869]: reconnect by client.14522 192.168.0.1:0/1817993757 after 0 2024-03-23T08:32:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:58 smithi070 ceph-mon[145869]: reconnect by client.24291 192.168.0.1:0/2176118012 after 0.000999971 2024-03-23T08:32:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:58 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] up:reconnect 2024-03-23T08:32:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:58 smithi070 ceph-mon[145869]: fsmap cephfs:1/1 {0=cephfs.smithi070.iziydh=up:reconnect} 3 up:standby 2024-03-23T08:32:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:58 smithi070 ceph-mon[145869]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:32:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:58 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:32:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:58 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/223443099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:32:59.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.249+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe72410f110 con 0x7fe71006cc20 2024-03-23T08:32:59.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.250+0000 7fe71a7fc700 1 -- 172.21.15.70:0/1742791987 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe72410f110 con 0x7fe71006cc20 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:4026324c507519c5a82fd7010cfd052f0faff768cf9b4dba85fa6998f59d8d2f", 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": true, 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [ 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "mon", 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "osd", 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr", 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "crash" 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": "15/21 daemons upgraded", 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "Currently upgrading mds daemons", 2024-03-23T08:32:59.251 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:32:59.252 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.252+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe71006cc20 msgr2=0x7fe71006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.252+0000 7fe72bf19700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe71006cc20 0x7fe71006f0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b0d0 tx=0x7fe720009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.252+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7241026b0 msgr2=0x7fe724113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.252+0000 7fe72bf19700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7241026b0 0x7fe724113f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe71400eb40 tx=0x7fe71400b040 comp rx=0 tx=0).stop 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.253+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 shutdown_connections 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.253+0000 7fe72bf19700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe71006cc20 0x7fe71006f0d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.253+0000 7fe72bf19700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe724101d10 0x7fe724113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.253+0000 7fe72bf19700 1 --2- 172.21.15.70:0/1742791987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7241026b0 0x7fe724113f60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.253+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 >> 172.21.15.70:0/1742791987 conn(0x7fe7240f7860 msgr2=0x7fe7240f8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.253+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 shutdown_connections 2024-03-23T08:32:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.253+0000 7fe72bf19700 1 -- 172.21.15.70:0/1742791987 wait complete. 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.352+0000 7f7d48df6700 1 -- 172.21.15.70:0/3213733236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44102710 msgr2=0x7f7d440fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.352+0000 7f7d48df6700 1 --2- 172.21.15.70:0/3213733236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44102710 0x7f7d440fbd80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7d38004940 tx=0x7f7d38009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.352+0000 7f7d48df6700 1 -- 172.21.15.70:0/3213733236 shutdown_connections 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.352+0000 7f7d48df6700 1 --2- 172.21.15.70:0/3213733236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44102710 0x7f7d440fbd80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.352+0000 7f7d48df6700 1 --2- 172.21.15.70:0/3213733236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d44101d70 0x7f7d44102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.352+0000 7f7d48df6700 1 -- 172.21.15.70:0/3213733236 >> 172.21.15.70:0/3213733236 conn(0x7f7d44072bc0 msgr2=0x7f7d44072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.352+0000 7f7d48df6700 1 -- 172.21.15.70:0/3213733236 shutdown_connections 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d48df6700 1 -- 172.21.15.70:0/3213733236 wait complete. 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d48df6700 1 Processor -- start 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d48df6700 1 -- start start 2024-03-23T08:32:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d48df6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44101d70 0x7f7d44112a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d48df6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d44102710 0x7f7d44112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d48df6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d441135b0 con 0x7f7d44101d70 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d48df6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4410c9a0 con 0x7f7d44102710 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d4259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44101d70 0x7f7d44112a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d4259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44101d70 0x7f7d44112a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48604/0 (socket says 172.21.15.70:48604) 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d4259c700 1 -- 172.21.15.70:0/698123628 learned_addr learned my addr 172.21.15.70:0/698123628 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.353+0000 7f7d41d9b700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d44102710 0x7f7d44112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.354+0000 7f7d4259c700 1 -- 172.21.15.70:0/698123628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d44102710 msgr2=0x7f7d44112f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.354+0000 7f7d4259c700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d44102710 0x7f7d44112f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.354+0000 7f7d4259c700 1 -- 172.21.15.70:0/698123628 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38005040 con 0x7f7d44101d70 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.354+0000 7f7d4259c700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44101d70 0x7f7d44112a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7d34002a50 tx=0x7f7d340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.354+0000 7f7d32ffd700 1 -- 172.21.15.70:0/698123628 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d34007cc0 con 0x7f7d44101d70 2024-03-23T08:32:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.354+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4410cc80 con 0x7f7d44101d70 2024-03-23T08:32:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.354+0000 7f7d32ffd700 1 -- 172.21.15.70:0/698123628 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7d3400f040 con 0x7f7d44101d70 2024-03-23T08:32:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.354+0000 7f7d32ffd700 1 -- 172.21.15.70:0/698123628 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d3400d9c0 con 0x7f7d44101d70 2024-03-23T08:32:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.354+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4410d210 con 0x7f7d44101d70 2024-03-23T08:32:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.355+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d440fd4c0 con 0x7f7d44101d70 2024-03-23T08:32:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.355+0000 7f7d32ffd700 1 -- 172.21.15.70:0/698123628 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7d3400db60 con 0x7f7d44101d70 2024-03-23T08:32:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.356+0000 7f7d32ffd700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7d2c071040 0x7f7d2c0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:32:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.356+0000 7f7d32ffd700 1 -- 172.21.15.70:0/698123628 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7d3408b6e0 con 0x7f7d44101d70 2024-03-23T08:32:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.356+0000 7f7d41d9b700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7d2c071040 0x7f7d2c0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:32:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.356+0000 7f7d41d9b700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7d2c071040 0x7f7d2c0734f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7d38004890 tx=0x7f7d38009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:32:59.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.360+0000 7f7d32ffd700 1 -- 172.21.15.70:0/698123628 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7d34059130 con 0x7f7d44101d70 2024-03-23T08:32:59.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.632+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7d4404bed0 con 0x7f7d44101d70 2024-03-23T08:32:59.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.632+0000 7f7d32ffd700 1 -- 172.21.15.70:0/698123628 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7f7d34058cc0 con 0x7f7d44101d70 2024-03-23T08:32:59.632 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_WARN 1 filesystem is degraded 2024-03-23T08:32:59.632 INFO:teuthology.orchestra.run.smithi070.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-23T08:32:59.633 INFO:teuthology.orchestra.run.smithi070.stdout: fs cephfs is degraded 2024-03-23T08:32:59.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7d2c071040 msgr2=0x7f7d2c0734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7d2c071040 0x7f7d2c0734f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7d38004890 tx=0x7f7d38009e30 comp rx=0 tx=0).stop 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44101d70 msgr2=0x7f7d44112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44101d70 0x7f7d44112a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7d34002a50 tx=0x7f7d340046b0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 shutdown_connections 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7d2c071040 0x7f7d2c0734f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d44101d70 0x7f7d44112a20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 --2- 172.21.15.70:0/698123628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d44102710 0x7f7d44112f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 >> 172.21.15.70:0/698123628 conn(0x7f7d44072bc0 msgr2=0x7f7d440fb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 shutdown_connections 2024-03-23T08:32:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:32:59.634+0000 7f7d48df6700 1 -- 172.21.15.70:0/698123628 wait complete. 2024-03-23T08:33:00.164 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:59 smithi193 ceph-mon[108533]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:00.164 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:59 smithi193 ceph-mon[108533]: pgmap v192: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-23T08:33:00.164 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:59 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] up:rejoin 2024-03-23T08:33:00.164 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:59 smithi193 ceph-mon[108533]: fsmap cephfs:1/1 {0=cephfs.smithi070.iziydh=up:rejoin} 3 up:standby 2024-03-23T08:33:00.164 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:59 smithi193 ceph-mon[108533]: daemon mds.cephfs.smithi070.iziydh is now active in filesystem cephfs as rank 0 2024-03-23T08:33:00.164 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:59 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1670245423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:33:00.164 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:59 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:00.164 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:59 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:00.164 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:32:59 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/698123628' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:33:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:59 smithi070 ceph-mon[145869]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:59 smithi070 ceph-mon[145869]: pgmap v192: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-03-23T08:33:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:59 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] up:rejoin 2024-03-23T08:33:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:59 smithi070 ceph-mon[145869]: fsmap cephfs:1/1 {0=cephfs.smithi070.iziydh=up:rejoin} 3 up:standby 2024-03-23T08:33:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:59 smithi070 ceph-mon[145869]: daemon mds.cephfs.smithi070.iziydh is now active in filesystem cephfs as rank 0 2024-03-23T08:33:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:59 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1670245423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:33:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:59 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:32:59 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/698123628' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:33:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:00 smithi193 ceph-mon[108533]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:00 smithi193 ceph-mon[108533]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T08:33:01.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:00 smithi193 ceph-mon[108533]: Cluster is now healthy 2024-03-23T08:33:01.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:00 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] up:active 2024-03-23T08:33:01.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:00 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:33:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:00 smithi070 ceph-mon[145869]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:00 smithi070 ceph-mon[145869]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-23T08:33:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:00 smithi070 ceph-mon[145869]: Cluster is now healthy 2024-03-23T08:33:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:00 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] up:active 2024-03-23T08:33:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:00 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:33:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:01 smithi193 ceph-mon[108533]: pgmap v193: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 86 B/s wr, 5 op/s 2024-03-23T08:33:02.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:33:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:01 smithi070 ceph-mon[145869]: pgmap v193: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 86 B/s wr, 5 op/s 2024-03-23T08:33:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: pgmap v194: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:03.826 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:03 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:33:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: pgmap v194: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 7 op/s 2024-03-23T08:33:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:03.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:03.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:03 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:05.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:05 smithi070 ceph-mon[145869]: Upgrade: Updating mds.cephfs.smithi193.nucaub 2024-03-23T08:33:05.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:05.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.nucaub", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:33:05.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:05 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:05.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:05 smithi070 ceph-mon[145869]: Deploying daemon mds.cephfs.smithi193.nucaub on smithi193 2024-03-23T08:33:05.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:05 smithi070 ceph-mon[145869]: pgmap v195: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 8.8 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-23T08:33:05.565 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:05 smithi193 ceph-mon[108533]: Upgrade: Updating mds.cephfs.smithi193.nucaub 2024-03-23T08:33:05.565 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:05.565 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.nucaub", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-23T08:33:05.565 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:05 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:05.565 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:05 smithi193 ceph-mon[108533]: Deploying daemon mds.cephfs.smithi193.nucaub on smithi193 2024-03-23T08:33:05.565 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:05 smithi193 ceph-mon[108533]: pgmap v195: 65 pgs: 65 active+clean; 201 MiB data, 852 MiB used, 536 GiB / 536 GiB avail; 8.8 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-23T08:33:07.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:07 smithi193 ceph-mon[108533]: osdmap e85: 6 total, 6 up, 6 in 2024-03-23T08:33:07.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:07 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 2 up:standby 2024-03-23T08:33:07.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:07 smithi193 ceph-mon[108533]: pgmap v197: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-03-23T08:33:07.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:06 smithi070 ceph-mon[145869]: osdmap e85: 6 total, 6 up, 6 in 2024-03-23T08:33:07.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:06 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 2 up:standby 2024-03-23T08:33:07.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:06 smithi070 ceph-mon[145869]: pgmap v197: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-03-23T08:33:09.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:09 smithi070 ceph-mon[145869]: pgmap v198: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-03-23T08:33:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:09 smithi193 ceph-mon[108533]: pgmap v198: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-03-23T08:33:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:11 smithi193 ceph-mon[108533]: pgmap v199: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-03-23T08:33:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:11 smithi193 ceph-mon[108533]: mds.? [v2:172.21.15.193:6826/1000528415,v1:172.21.15.193:6827/1000528415] up:boot 2024-03-23T08:33:11.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:11 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:33:11.576 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:11 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:33:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:11 smithi070 ceph-mon[145869]: pgmap v199: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-03-23T08:33:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:11 smithi070 ceph-mon[145869]: mds.? [v2:172.21.15.193:6826/1000528415,v1:172.21.15.193:6827/1000528415] up:boot 2024-03-23T08:33:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:11 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:33:11.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:11 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:33:13.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:13 smithi070 ceph-mon[145869]: pgmap v200: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 4.9 KiB/s wr, 3 op/s 2024-03-23T08:33:13.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:13 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:13.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:13 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:13.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:13 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:13.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:13 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:14.029 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:13 smithi193 ceph-mon[108533]: pgmap v200: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 4.9 KiB/s wr, 3 op/s 2024-03-23T08:33:14.029 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:13 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:14.029 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:13 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:14.029 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:13 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:14.029 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:13 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:15.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:15 smithi070 ceph-mon[145869]: pgmap v201: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 921 B/s rd, 4.9 KiB/s wr, 3 op/s 2024-03-23T08:33:15.987 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:15 smithi193 ceph-mon[108533]: pgmap v201: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 921 B/s rd, 4.9 KiB/s wr, 3 op/s 2024-03-23T08:33:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:17.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: pgmap v202: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all mds 2024-03-23T08:33:17.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.aymzqb"}]': finished 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.iziydh"}]': finished 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.nucaub"}]': finished 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.sdlbdd"}]': finished 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: Upgrade: Setting filesystem cephfs Joinable 2024-03-23T08:33:17.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:17 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-23T08:33:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:18.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: pgmap v202: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all mds 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:18.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.aymzqb"}]: dispatch 2024-03-23T08:33:18.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.aymzqb"}]': finished 2024-03-23T08:33:18.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.iziydh"}]: dispatch 2024-03-23T08:33:18.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.iziydh"}]': finished 2024-03-23T08:33:18.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.nucaub"}]: dispatch 2024-03-23T08:33:18.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.nucaub"}]': finished 2024-03-23T08:33:18.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.sdlbdd"}]: dispatch 2024-03-23T08:33:18.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.sdlbdd"}]': finished 2024-03-23T08:33:18.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: Upgrade: Setting filesystem cephfs Joinable 2024-03-23T08:33:18.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:17 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-23T08:33:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:18 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-23T08:33:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:18 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:33:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:18 smithi193 ceph-mon[108533]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-23T08:33:19.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:18 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-23T08:33:19.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:18 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-23T08:33:19.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:18 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:33:19.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:18 smithi070 ceph-mon[145869]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-23T08:33:19.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:18 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: pgmap v203: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all rgw 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.076 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all rbd-mirror 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all ceph-exporter 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all iscsi 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all nfs 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: Upgrade: Setting container_image for all nvmeof 2024-03-23T08:33:20.077 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: Upgrade: Finalizing container_image settings 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-23T08:33:20.078 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-23T08:33:20.079 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:20.080 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:33:20.081 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: pgmap v203: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail 2024-03-23T08:33:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-23T08:33:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 3 up:standby 2024-03-23T08:33:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: fsmap cephfs:1 {0=cephfs.smithi070.iziydh=up:active} 1 up:standby-replay 2 up:standby 2024-03-23T08:33:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all rgw 2024-03-23T08:33:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all rbd-mirror 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all ceph-exporter 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all iscsi 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all nfs 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: Upgrade: Setting container_image for all nvmeof 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: Upgrade: Finalizing container_image settings 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-23T08:33:20.235 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:33:20.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:21.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:20 smithi193 ceph-mon[108533]: Upgrade: Complete! 2024-03-23T08:33:21.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:20 smithi070 ceph-mon[145869]: Upgrade: Complete! 2024-03-23T08:33:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:21 smithi193 ceph-mon[108533]: pgmap v204: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s rd, 0 op/s 2024-03-23T08:33:22.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:21 smithi070 ceph-mon[145869]: pgmap v204: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s rd, 0 op/s 2024-03-23T08:33:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:33:24.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:23 smithi193 ceph-mon[108533]: pgmap v205: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 7.5 MiB/s rd, 2 op/s 2024-03-23T08:33:24.234 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:23 smithi070 ceph-mon[145869]: pgmap v205: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 7.5 MiB/s rd, 2 op/s 2024-03-23T08:33:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:25 smithi193 ceph-mon[108533]: pgmap v206: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 8.2 MiB/s rd, 2 op/s 2024-03-23T08:33:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:25 smithi070 ceph-mon[145869]: pgmap v206: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 8.2 MiB/s rd, 2 op/s 2024-03-23T08:33:28.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:27 smithi193 ceph-mon[108533]: pgmap v207: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-23T08:33:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:27 smithi070 ceph-mon[145869]: pgmap v207: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.750+0000 7f2a4c132700 1 -- 172.21.15.70:0/1591504858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 msgr2=0x7f2a44102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.750+0000 7f2a4c132700 1 --2- 172.21.15.70:0/1591504858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 0x7f2a44102140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2a34004940 tx=0x7f2a34009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.750+0000 7f2a4c132700 1 -- 172.21.15.70:0/1591504858 shutdown_connections 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.750+0000 7f2a4c132700 1 --2- 172.21.15.70:0/1591504858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a44102710 0x7f2a440fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.750+0000 7f2a4c132700 1 --2- 172.21.15.70:0/1591504858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 0x7f2a44102140 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a4c132700 1 -- 172.21.15.70:0/1591504858 >> 172.21.15.70:0/1591504858 conn(0x7f2a44072bc0 msgr2=0x7f2a44072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a4c132700 1 -- 172.21.15.70:0/1591504858 shutdown_connections 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a4c132700 1 -- 172.21.15.70:0/1591504858 wait complete. 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a4c132700 1 Processor -- start 2024-03-23T08:33:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a4c132700 1 -- start start 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a4c132700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 0x7f2a4410ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a4c132700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a44102710 0x7f2a4410ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a4c132700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4410f5d0 con 0x7f2a44101d70 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a4c132700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4410f710 con 0x7f2a44102710 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a49ece700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 0x7f2a4410ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a49ece700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 0x7f2a4410ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36088/0 (socket says 172.21.15.70:36088) 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.751+0000 7f2a49ece700 1 -- 172.21.15.70:0/3205849416 learned_addr learned my addr 172.21.15.70:0/3205849416 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a496cd700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a44102710 0x7f2a4410ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a49ece700 1 -- 172.21.15.70:0/3205849416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a44102710 msgr2=0x7f2a4410ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a49ece700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a44102710 0x7f2a4410ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a49ece700 1 -- 172.21.15.70:0/3205849416 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34005040 con 0x7f2a44101d70 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a49ece700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 0x7f2a4410ea40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009360 tx=0x7f2a34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a3a7fc700 1 -- 172.21.15.70:0/3205849416 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3400e040 con 0x7f2a44101d70 2024-03-23T08:33:29.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a3a7fc700 1 -- 172.21.15.70:0/3205849416 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a34013420 con 0x7f2a44101d70 2024-03-23T08:33:29.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a44117ab0 con 0x7f2a44101d70 2024-03-23T08:33:29.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a3a7fc700 1 -- 172.21.15.70:0/3205849416 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3401c5f0 con 0x7f2a44101d70 2024-03-23T08:33:29.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.752+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a44117fc0 con 0x7f2a44101d70 2024-03-23T08:33:29.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.753+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a440ff080 con 0x7f2a44101d70 2024-03-23T08:33:29.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.754+0000 7f2a3a7fc700 1 -- 172.21.15.70:0/3205849416 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2a34013590 con 0x7f2a44101d70 2024-03-23T08:33:29.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.754+0000 7f2a3a7fc700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2a3006cc30 0x7f2a3006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:29.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.754+0000 7f2a3a7fc700 1 -- 172.21.15.70:0/3205849416 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2a3408d370 con 0x7f2a44101d70 2024-03-23T08:33:29.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.754+0000 7f2a496cd700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2a3006cc30 0x7f2a3006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:29.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.755+0000 7f2a496cd700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2a3006cc30 0x7f2a3006f0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a44066000 tx=0x7f2a40009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:29.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.759+0000 7f2a3a7fc700 1 -- 172.21.15.70:0/3205849416 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2a34057330 con 0x7f2a44101d70 2024-03-23T08:33:29.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.971+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a4404bed0 con 0x7f2a3006cc30 2024-03-23T08:33:29.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.972+0000 7f2a3a7fc700 1 -- 172.21.15.70:0/3205849416 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f2a4404bed0 con 0x7f2a3006cc30 2024-03-23T08:33:29.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.974+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2a3006cc30 msgr2=0x7f2a3006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:29.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.974+0000 7f2a4c132700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2a3006cc30 0x7f2a3006f0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a44066000 tx=0x7f2a40009040 comp rx=0 tx=0).stop 2024-03-23T08:33:29.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.974+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 msgr2=0x7f2a4410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:29.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.974+0000 7f2a4c132700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 0x7f2a4410ea40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009360 tx=0x7f2a34009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:29.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.974+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 shutdown_connections 2024-03-23T08:33:29.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.974+0000 7f2a4c132700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2a3006cc30 0x7f2a3006f0e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:29.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.974+0000 7f2a4c132700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a44101d70 0x7f2a4410ea40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:29.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.974+0000 7f2a4c132700 1 --2- 172.21.15.70:0/3205849416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a44102710 0x7f2a4410ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:29.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.974+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 >> 172.21.15.70:0/3205849416 conn(0x7f2a44072bc0 msgr2=0x7f2a440fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:29.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.975+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 shutdown_connections 2024-03-23T08:33:29.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:29.975+0000 7f2a4c132700 1 -- 172.21.15.70:0/3205849416 wait complete. 2024-03-23T08:33:30.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:29 smithi193 ceph-mon[108533]: pgmap v208: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-03-23T08:33:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:29 smithi070 ceph-mon[145869]: pgmap v208: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-03-23T08:33:30.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph orch ps' 2024-03-23T08:33:31.030 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:33:31.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:31 smithi070 ceph-mon[145869]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:31.818 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:31 smithi070 ceph-mon[145869]: pgmap v209: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-23T08:33:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:31 smithi193 ceph-mon[108533]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:31 smithi193 ceph-mon[108533]: pgmap v209: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-23T08:33:32.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:33:32.202 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:33:32.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.754+0000 7f02d42bb700 1 -- 172.21.15.70:0/3023451105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 msgr2=0x7f02cc0fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.754+0000 7f02d42bb700 1 --2- 172.21.15.70:0/3023451105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 0x7f02cc0fce20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f02c8004940 tx=0x7f02c8009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.754+0000 7f02d42bb700 1 -- 172.21.15.70:0/3023451105 shutdown_connections 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.754+0000 7f02d42bb700 1 --2- 172.21.15.70:0/3023451105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 0x7f02cc0fce20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.754+0000 7f02d42bb700 1 --2- 172.21.15.70:0/3023451105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02cc102e10 0x7f02cc1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.754+0000 7f02d42bb700 1 -- 172.21.15.70:0/3023451105 >> 172.21.15.70:0/3023451105 conn(0x7f02cc0f8980 msgr2=0x7f02cc0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.754+0000 7f02d42bb700 1 -- 172.21.15.70:0/3023451105 shutdown_connections 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.754+0000 7f02d42bb700 1 -- 172.21.15.70:0/3023451105 wait complete. 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d42bb700 1 Processor -- start 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d42bb700 1 -- start start 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d42bb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02cc102e10 0x7f02cc113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d42bb700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 0x7f02cc114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d42bb700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc114680 con 0x7f02cc1037b0 2024-03-23T08:33:32.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d42bb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc10da70 con 0x7f02cc102e10 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d1856700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 0x7f02cc114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d1856700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 0x7f02cc114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36118/0 (socket says 172.21.15.70:36118) 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d1856700 1 -- 172.21.15.70:0/804026 learned_addr learned my addr 172.21.15.70:0/804026 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d2057700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02cc102e10 0x7f02cc113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d1856700 1 -- 172.21.15.70:0/804026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02cc102e10 msgr2=0x7f02cc113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d1856700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02cc102e10 0x7f02cc113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d1856700 1 -- 172.21.15.70:0/804026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c8005040 con 0x7f02cc1037b0 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02d1856700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 0x7f02cc114030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009390 tx=0x7f02c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.755+0000 7f02c2ffd700 1 -- 172.21.15.70:0/804026 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c800e040 con 0x7f02cc1037b0 2024-03-23T08:33:32.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.756+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc10dcf0 con 0x7f02cc1037b0 2024-03-23T08:33:32.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.756+0000 7f02c2ffd700 1 -- 172.21.15.70:0/804026 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02c8004290 con 0x7f02cc1037b0 2024-03-23T08:33:32.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.756+0000 7f02c2ffd700 1 -- 172.21.15.70:0/804026 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c8013e70 con 0x7f02cc1037b0 2024-03-23T08:33:32.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.756+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc10e2b0 con 0x7f02cc1037b0 2024-03-23T08:33:32.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.756+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc0fe560 con 0x7f02cc1037b0 2024-03-23T08:33:32.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.757+0000 7f02c2ffd700 1 -- 172.21.15.70:0/804026 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f02c8013690 con 0x7f02cc1037b0 2024-03-23T08:33:32.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.757+0000 7f02c2ffd700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f02b806cbc0 0x7f02b806f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:32.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.758+0000 7f02c2ffd700 1 -- 172.21.15.70:0/804026 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f02c808d240 con 0x7f02cc1037b0 2024-03-23T08:33:32.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.758+0000 7f02d2057700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f02b806cbc0 0x7f02b806f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:32.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.758+0000 7f02d2057700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f02b806cbc0 0x7f02b806f070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02bc004f40 tx=0x7f02bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:32.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.762+0000 7f02c2ffd700 1 -- 172.21.15.70:0/804026 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f02c805adc0 con 0x7f02cc1037b0 2024-03-23T08:33:32.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.966+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f02cc109db0 con 0x7f02b806cbc0 2024-03-23T08:33:32.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.975+0000 7f02c2ffd700 1 -- 172.21.15.70:0/804026 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3476 (secure 0 0 0) 0x7f02cc109db0 con 0x7f02b806cbc0 2024-03-23T08:33:32.975 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (7m) 52s ago 18m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (4m) 52s ago 18m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (4m) 20s ago 17m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (6m) 52s ago 18m 90.9M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (57s) 52s ago 12m 10.9M - 18.2.2-954-g82316b04 b24e1baba637 3aedd00ed88c 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (79s) 52s ago 12m 14.7M - 18.2.2-954-g82316b04 b24e1baba637 1066f2091682 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (23s) 20s ago 12m 13.6M - 18.2.2-954-g82316b04 b24e1baba637 ea117e71556f 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (37s) 20s ago 12m 11.4M - 18.2.2-954-g82316b04 b24e1baba637 5621af1e28f9 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (9m) 52s ago 19m 504M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (8m) 20s ago 17m 450M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (5m) 52s ago 19m 55.7M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (5m) 20s ago 17m 43.9M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 52s ago 18m 18.8M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:33:32.976 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (7m) 20s ago 17m 16.5M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:33:32.977 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (4m) 52s ago 16m 92.6M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:33:32.977 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (3m) 52s ago 16m 107M 1568M 18.2.2-954-g82316b04 b24e1baba637 13fce63c86eb 2024-03-23T08:33:32.977 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (3m) 52s ago 15m 96.6M 1568M 18.2.2-954-g82316b04 b24e1baba637 73e36e623491 2024-03-23T08:33:32.977 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (2m) 20s ago 15m 114M 2592M 18.2.2-954-g82316b04 b24e1baba637 8a7617f5541a 2024-03-23T08:33:32.977 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (2m) 20s ago 14m 104M 2592M 18.2.2-954-g82316b04 b24e1baba637 76a25ee5af50 2024-03-23T08:33:32.977 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (97s) 20s ago 14m 107M 2592M 18.2.2-954-g82316b04 b24e1baba637 58bbe6abe8b3 2024-03-23T08:33:32.977 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (7m) 52s ago 17m 60.5M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.977+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f02b806cbc0 msgr2=0x7f02b806f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.977+0000 7f02d42bb700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f02b806cbc0 0x7f02b806f070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02bc004f40 tx=0x7f02bc00b040 comp rx=0 tx=0).stop 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.977+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 msgr2=0x7f02cc114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.977+0000 7f02d42bb700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 0x7f02cc114030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009390 tx=0x7f02c8009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.977+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 shutdown_connections 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.977+0000 7f02d42bb700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f02b806cbc0 0x7f02b806f070 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.977+0000 7f02d42bb700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02cc102e10 0x7f02cc113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.977+0000 7f02d42bb700 1 --2- 172.21.15.70:0/804026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02cc1037b0 0x7f02cc114030 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.977+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 >> 172.21.15.70:0/804026 conn(0x7f02cc0f8980 msgr2=0x7f02cc0f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.978+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 shutdown_connections 2024-03-23T08:33:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:32.978+0000 7f02d42bb700 1 -- 172.21.15.70:0/804026 wait complete. 2024-03-23T08:33:33.672 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph orch upgrade status' 2024-03-23T08:33:33.942 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:33:33.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:33 smithi070 ceph-mon[145869]: pgmap v210: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-23T08:33:34.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:33 smithi193 ceph-mon[108533]: pgmap v210: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-23T08:33:35.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:34 smithi193 ceph-mon[108533]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:35.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:34 smithi070 ceph-mon[145869]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:36.203 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:35 smithi070 ceph-mon[145869]: pgmap v211: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 8.8 MiB/s rd, 2 op/s 2024-03-23T08:33:36.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:35 smithi193 ceph-mon[108533]: pgmap v211: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 8.8 MiB/s rd, 2 op/s 2024-03-23T08:33:37.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 -- 172.21.15.70:0/3593440392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 msgr2=0x7f7ce00fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 --2- 172.21.15.70:0/3593440392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 0x7f7ce00fbd80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0004940 tx=0x7f7cd0009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 -- 172.21.15.70:0/3593440392 shutdown_connections 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 --2- 172.21.15.70:0/3593440392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 0x7f7ce00fbd80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 --2- 172.21.15.70:0/3593440392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ce0101d70 0x7f7ce0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 -- 172.21.15.70:0/3593440392 >> 172.21.15.70:0/3593440392 conn(0x7f7ce0072bc0 msgr2=0x7f7ce0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 -- 172.21.15.70:0/3593440392 shutdown_connections 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 -- 172.21.15.70:0/3593440392 wait complete. 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 Processor -- start 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.197+0000 7f7ce62c0700 1 -- start start 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7ce62c0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ce0101d70 0x7f7ce01129c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:37.198 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7ce62c0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 0x7f7ce0112f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7ce62c0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce0113550 con 0x7f7ce0102710 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7ce62c0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce010c940 con 0x7f7ce0101d70 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 0x7f7ce0112f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdf7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 0x7f7ce0112f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36140/0 (socket says 172.21.15.70:36140) 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdf7fe700 1 -- 172.21.15.70:0/4024702095 learned_addr learned my addr 172.21.15.70:0/4024702095 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdffff700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ce0101d70 0x7f7ce01129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdf7fe700 1 -- 172.21.15.70:0/4024702095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ce0101d70 msgr2=0x7f7ce01129c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdf7fe700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ce0101d70 0x7f7ce01129c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdf7fe700 1 -- 172.21.15.70:0/4024702095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd0005040 con 0x7f7ce0102710 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdf7fe700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 0x7f7ce0112f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000b3b0 tx=0x7f7cd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdcff9700 1 -- 172.21.15.70:0/4024702095 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd000e040 con 0x7f7ce0102710 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdcff9700 1 -- 172.21.15.70:0/4024702095 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7cd000bd60 con 0x7f7ce0102710 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce010cbc0 con 0x7f7ce0102710 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7cdcff9700 1 -- 172.21.15.70:0/4024702095 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd0013660 con 0x7f7ce0102710 2024-03-23T08:33:37.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.198+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce010d120 con 0x7f7ce0102710 2024-03-23T08:33:37.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.199+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce00fd4c0 con 0x7f7ce0102710 2024-03-23T08:33:37.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.200+0000 7f7cdcff9700 1 -- 172.21.15.70:0/4024702095 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7cd00138d0 con 0x7f7ce0102710 2024-03-23T08:33:37.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.200+0000 7f7cdcff9700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7cc806cc20 0x7f7cc806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:37.201 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.200+0000 7f7cdcff9700 1 -- 172.21.15.70:0/4024702095 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7cd008ff50 con 0x7f7ce0102710 2024-03-23T08:33:37.201 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.200+0000 7f7cdffff700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7cc806cc20 0x7f7cc806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:37.201 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.201+0000 7f7cdffff700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7cc806cc20 0x7f7cc806f0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4007fd0 tx=0x7f7cd4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:37.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.205+0000 7f7cdcff9700 1 -- 172.21.15.70:0/4024702095 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7cd005da20 con 0x7f7ce0102710 2024-03-23T08:33:37.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.417+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ce0108d10 con 0x7f7cc806cc20 2024-03-23T08:33:37.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.418+0000 7f7cdcff9700 1 -- 172.21.15.70:0/4024702095 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f7ce0108d10 con 0x7f7cc806cc20 2024-03-23T08:33:37.417 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:33:37.417 INFO:teuthology.orchestra.run.smithi070.stdout: "target_image": null, 2024-03-23T08:33:37.417 INFO:teuthology.orchestra.run.smithi070.stdout: "in_progress": false, 2024-03-23T08:33:37.417 INFO:teuthology.orchestra.run.smithi070.stdout: "which": "", 2024-03-23T08:33:37.417 INFO:teuthology.orchestra.run.smithi070.stdout: "services_complete": [], 2024-03-23T08:33:37.417 INFO:teuthology.orchestra.run.smithi070.stdout: "progress": null, 2024-03-23T08:33:37.417 INFO:teuthology.orchestra.run.smithi070.stdout: "message": "", 2024-03-23T08:33:37.417 INFO:teuthology.orchestra.run.smithi070.stdout: "is_paused": false 2024-03-23T08:33:37.417 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7cc806cc20 msgr2=0x7f7cc806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7cc806cc20 0x7f7cc806f0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4007fd0 tx=0x7f7cd4009040 comp rx=0 tx=0).stop 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 msgr2=0x7f7ce0112f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 0x7f7ce0112f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000b3b0 tx=0x7f7cd0009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 shutdown_connections 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7cc806cc20 0x7f7cc806f0d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ce0101d70 0x7f7ce01129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 --2- 172.21.15.70:0/4024702095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ce0102710 0x7f7ce0112f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 >> 172.21.15.70:0/4024702095 conn(0x7f7ce0072bc0 msgr2=0x7f7ce00fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 shutdown_connections 2024-03-23T08:33:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:37.420+0000 7f7ce62c0700 1 -- 172.21.15.70:0/4024702095 wait complete. 2024-03-23T08:33:38.183 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:37 smithi070 ceph-mon[145869]: pgmap v212: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 3 op/s 2024-03-23T08:33:38.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph health detail' 2024-03-23T08:33:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:37 smithi193 ceph-mon[108533]: pgmap v212: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 8.1 MiB/s rd, 3 op/s 2024-03-23T08:33:38.475 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:33:38.945 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:38 smithi070 ceph-mon[145869]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:39.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:38 smithi193 ceph-mon[108533]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:33:39.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:39 smithi070 ceph-mon[145869]: pgmap v213: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 1 op/s 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 -- 172.21.15.70:0/4151522686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fde90 msgr2=0x7f5d180fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 --2- 172.21.15.70:0/4151522686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fde90 0x7f5d180fe260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5d14004940 tx=0x7f5d14009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 -- 172.21.15.70:0/4151522686 shutdown_connections 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 --2- 172.21.15.70:0/4151522686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d180fe830 0x7f5d18106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 --2- 172.21.15.70:0/4151522686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fde90 0x7f5d180fe260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 -- 172.21.15.70:0/4151522686 >> 172.21.15.70:0/4151522686 conn(0x7f5d18072bc0 msgr2=0x7f5d18072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 -- 172.21.15.70:0/4151522686 shutdown_connections 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 -- 172.21.15.70:0/4151522686 wait complete. 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 Processor -- start 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.286+0000 7f5d206b0700 1 -- start start 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d206b0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d180fde90 0x7f5d180fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d206b0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fe830 0x7f5d180fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d206b0700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d180fc820 con 0x7f5d180fe830 2024-03-23T08:33:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d206b0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d180fc960 con 0x7f5d180fde90 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d1dc4b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fe830 0x7f5d180fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d1dc4b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fe830 0x7f5d180fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59504/0 (socket says 172.21.15.70:59504) 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d1dc4b700 1 -- 172.21.15.70:0/1573243245 learned_addr learned my addr 172.21.15.70:0/1573243245 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d1e44c700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d180fde90 0x7f5d180fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d1dc4b700 1 -- 172.21.15.70:0/1573243245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d180fde90 msgr2=0x7f5d180fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d1dc4b700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d180fde90 0x7f5d180fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d1dc4b700 1 -- 172.21.15.70:0/1573243245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d14005040 con 0x7f5d180fe830 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d1dc4b700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fe830 0x7f5d180fc290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c8c0 tx=0x7f5d080044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d0effd700 1 -- 172.21.15.70:0/1573243245 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d0800a980 con 0x7f5d180fe830 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d0effd700 1 -- 172.21.15.70:0/1573243245 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d0800ce60 con 0x7f5d180fe830 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d18119df0 con 0x7f5d180fe830 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d0effd700 1 -- 172.21.15.70:0/1573243245 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d080113a0 con 0x7f5d180fe830 2024-03-23T08:33:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.287+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1811a1f0 con 0x7f5d180fe830 2024-03-23T08:33:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.288+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d18065e30 con 0x7f5d180fe830 2024-03-23T08:33:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.289+0000 7f5d0effd700 1 -- 172.21.15.70:0/1573243245 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5d08012400 con 0x7f5d180fe830 2024-03-23T08:33:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.289+0000 7f5d0effd700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5d0406cc30 0x7f5d0406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.289+0000 7f5d1e44c700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5d0406cc30 0x7f5d0406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.289+0000 7f5d0effd700 1 -- 172.21.15.70:0/1573243245 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5d0808c160 con 0x7f5d180fe830 2024-03-23T08:33:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.290+0000 7f5d1e44c700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5d0406cc30 0x7f5d0406f0e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400b3b0 tx=0x7f5d14009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:40.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.294+0000 7f5d0effd700 1 -- 172.21.15.70:0/1573243245 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5d080561a0 con 0x7f5d180fe830 2024-03-23T08:33:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:39 smithi193 ceph-mon[108533]: pgmap v213: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 1 op/s 2024-03-23T08:33:40.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.563+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5d1805f5f0 con 0x7f5d180fe830 2024-03-23T08:33:40.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.563+0000 7f5d0effd700 1 -- 172.21.15.70:0/1573243245 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5d080597c0 con 0x7f5d180fe830 2024-03-23T08:33:40.563 INFO:teuthology.orchestra.run.smithi070.stdout:HEALTH_OK 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.565+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5d0406cc30 msgr2=0x7f5d0406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.565+0000 7f5d206b0700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5d0406cc30 0x7f5d0406f0e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400b3b0 tx=0x7f5d14009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.565+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fe830 msgr2=0x7f5d180fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.565+0000 7f5d206b0700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fe830 0x7f5d180fc290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c8c0 tx=0x7f5d080044c0 comp rx=0 tx=0).stop 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.565+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 shutdown_connections 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.565+0000 7f5d206b0700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5d0406cc30 0x7f5d0406f0e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.565+0000 7f5d206b0700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d180fde90 0x7f5d180fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.565+0000 7f5d206b0700 1 --2- 172.21.15.70:0/1573243245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d180fe830 0x7f5d180fc290 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.565+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 >> 172.21.15.70:0/1573243245 conn(0x7f5d18072bc0 msgr2=0x7f5d180ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.566+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 shutdown_connections 2024-03-23T08:33:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:40.566+0000 7f5d206b0700 1 -- 172.21.15.70:0/1573243245 wait complete. 2024-03-23T08:33:41.188 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:40 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1573243245' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:33:41.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph versions' 2024-03-23T08:33:41.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:40 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1573243245' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-23T08:33:41.477 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:33:41.975 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:41 smithi070 ceph-mon[145869]: pgmap v214: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s rd, 1 op/s 2024-03-23T08:33:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:41 smithi193 ceph-mon[108533]: pgmap v214: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s rd, 1 op/s 2024-03-23T08:33:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.248+0000 7fef839df700 1 -- 172.21.15.70:0/3208901208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c1026c0 msgr2=0x7fef7c0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.248+0000 7fef839df700 1 --2- 172.21.15.70:0/3208901208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c1026c0 0x7fef7c0fbd30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fef78004940 tx=0x7fef78009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.248+0000 7fef839df700 1 -- 172.21.15.70:0/3208901208 shutdown_connections 2024-03-23T08:33:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.248+0000 7fef839df700 1 --2- 172.21.15.70:0/3208901208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c1026c0 0x7fef7c0fbd30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.248+0000 7fef839df700 1 --2- 172.21.15.70:0/3208901208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7c101d20 0x7fef7c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.248+0000 7fef839df700 1 -- 172.21.15.70:0/3208901208 >> 172.21.15.70:0/3208901208 conn(0x7fef7c0f7870 msgr2=0x7fef7c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.248+0000 7fef839df700 1 -- 172.21.15.70:0/3208901208 shutdown_connections 2024-03-23T08:33:43.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.248+0000 7fef839df700 1 -- 172.21.15.70:0/3208901208 wait complete. 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef839df700 1 Processor -- start 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef839df700 1 -- start start 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef839df700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c101d20 0x7fef7c10ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef839df700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7c1026c0 0x7fef7c10efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef839df700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c10f4f0 con 0x7fef7c101d20 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef839df700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c10f630 con 0x7fef7c1026c0 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef8177b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c101d20 0x7fef7c10ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef8177b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c101d20 0x7fef7c10ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59522/0 (socket says 172.21.15.70:59522) 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef8177b700 1 -- 172.21.15.70:0/4023598548 learned_addr learned my addr 172.21.15.70:0/4023598548 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef80f7a700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7c1026c0 0x7fef7c10efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef8177b700 1 -- 172.21.15.70:0/4023598548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7c1026c0 msgr2=0x7fef7c10efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef8177b700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7c1026c0 0x7fef7c10efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:43.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef8177b700 1 -- 172.21.15.70:0/4023598548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78005040 con 0x7fef7c101d20 2024-03-23T08:33:43.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.249+0000 7fef80f7a700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7c1026c0 0x7fef7c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:33:43.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.250+0000 7fef8177b700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c101d20 0x7fef7c10ea70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fef6c002a50 tx=0x7fef6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:43.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.250+0000 7fef71ffb700 1 -- 172.21.15.70:0/4023598548 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6c007cc0 con 0x7fef7c101d20 2024-03-23T08:33:43.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.250+0000 7fef71ffb700 1 -- 172.21.15.70:0/4023598548 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef6c00f040 con 0x7fef7c101d20 2024-03-23T08:33:43.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.250+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c117ae0 con 0x7fef7c101d20 2024-03-23T08:33:43.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.250+0000 7fef71ffb700 1 -- 172.21.15.70:0/4023598548 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6c009540 con 0x7fef7c101d20 2024-03-23T08:33:43.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.250+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c117eb0 con 0x7fef7c101d20 2024-03-23T08:33:43.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.251+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c0fd470 con 0x7fef7c101d20 2024-03-23T08:33:43.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.251+0000 7fef71ffb700 1 -- 172.21.15.70:0/4023598548 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fef6c0096a0 con 0x7fef7c101d20 2024-03-23T08:33:43.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.252+0000 7fef71ffb700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fef6806cc20 0x7fef6806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:33:43.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.252+0000 7fef71ffb700 1 -- 172.21.15.70:0/4023598548 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fef6c08be60 con 0x7fef7c101d20 2024-03-23T08:33:43.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.252+0000 7fef80f7a700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fef6806cc20 0x7fef6806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:33:43.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.252+0000 7fef80f7a700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fef6806cc20 0x7fef6806f0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef7800b3b0 tx=0x7fef78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:33:43.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.256+0000 7fef71ffb700 1 -- 172.21.15.70:0/4023598548 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fef6c055e20 con 0x7fef7c101d20 2024-03-23T08:33:43.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.529+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fef7c066830 con 0x7fef7c101d20 2024-03-23T08:33:43.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.530+0000 7fef71ffb700 1 -- 172.21.15.70:0/4023598548 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fef6c059440 con 0x7fef7c101d20 2024-03-23T08:33:43.529 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:33:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:33:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:33:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:33:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:33:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:33:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:33:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:33:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 6 2024-03-23T08:33:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:33:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:33:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 4 2024-03-23T08:33:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:33:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:33:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 14 2024-03-23T08:33:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:33:43.530 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:33:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fef6806cc20 msgr2=0x7fef6806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fef6806cc20 0x7fef6806f0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef7800b3b0 tx=0x7fef78009e30 comp rx=0 tx=0).stop 2024-03-23T08:33:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c101d20 msgr2=0x7fef7c10ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:33:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c101d20 0x7fef7c10ea70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fef6c002a50 tx=0x7fef6c0046b0 comp rx=0 tx=0).stop 2024-03-23T08:33:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 shutdown_connections 2024-03-23T08:33:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fef6806cc20 0x7fef6806f0d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef7c101d20 0x7fef7c10ea70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 --2- 172.21.15.70:0/4023598548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7c1026c0 0x7fef7c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:33:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 >> 172.21.15.70:0/4023598548 conn(0x7fef7c0f7870 msgr2=0x7fef7c0f8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:33:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 shutdown_connections 2024-03-23T08:33:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:33:43.532+0000 7fef839df700 1 -- 172.21.15.70:0/4023598548 wait complete. 2024-03-23T08:33:44.187 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:43 smithi070 ceph-mon[145869]: pgmap v215: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-23T08:33:44.187 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:43 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/4023598548' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:44.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-23T08:33:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:43 smithi193 ceph-mon[108533]: pgmap v215: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-23T08:33:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:43 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/4023598548' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:33:44.477 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:33:45.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:45 smithi070 ceph-mon[145869]: pgmap v216: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:33:46.316 INFO:teuthology.orchestra.run.smithi070.stdout:wait for servicemap items w/ changing names to refresh 2024-03-23T08:33:46.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:45 smithi193 ceph-mon[108533]: pgmap v216: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:33:46.950 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:33:46.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'sleep 60' 2024-03-23T08:33:47.239 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:33:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:33:48.134 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:47 smithi070 ceph-mon[145869]: pgmap v217: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:33:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:47 smithi193 ceph-mon[108533]: pgmap v217: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:33:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:49 smithi070 ceph-mon[145869]: pgmap v218: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:33:50.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:49 smithi193 ceph-mon[108533]: pgmap v218: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:33:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:51 smithi070 ceph-mon[145869]: pgmap v219: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:33:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:51 smithi193 ceph-mon[108533]: pgmap v219: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:33:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:53 smithi070 ceph-mon[145869]: pgmap v220: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:33:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:53 smithi193 ceph-mon[108533]: pgmap v220: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:33:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:55 smithi070 ceph-mon[145869]: pgmap v221: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:33:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:55 smithi193 ceph-mon[108533]: pgmap v221: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:33:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:57 smithi070 ceph-mon[145869]: pgmap v222: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:33:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:57 smithi193 ceph-mon[108533]: pgmap v222: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:33:59 smithi070 ceph-mon[145869]: pgmap v223: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:00.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:33:59 smithi193 ceph-mon[108533]: pgmap v223: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:01 smithi070 ceph-mon[145869]: pgmap v224: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:34:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:01 smithi193 ceph-mon[108533]: pgmap v224: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:34:03.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:02 smithi070 ceph-mon[145869]: pgmap v225: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:03.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:02 smithi193 ceph-mon[108533]: pgmap v225: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:05.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:05 smithi070 ceph-mon[145869]: pgmap v226: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:05 smithi193 ceph-mon[108533]: pgmap v226: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:08.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:07 smithi193 ceph-mon[108533]: pgmap v227: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:07 smithi070 ceph-mon[145869]: pgmap v227: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:10.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:09 smithi193 ceph-mon[108533]: pgmap v228: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:09 smithi070 ceph-mon[145869]: pgmap v228: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:12.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:11 smithi193 ceph-mon[108533]: pgmap v229: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:11 smithi070 ceph-mon[145869]: pgmap v229: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:14.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:13 smithi070 ceph-mon[145869]: pgmap v230: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:13 smithi193 ceph-mon[108533]: pgmap v230: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:15 smithi070 ceph-mon[145869]: pgmap v231: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:15 smithi193 ceph-mon[108533]: pgmap v231: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:16 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:34:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:16 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:34:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:17 smithi070 ceph-mon[145869]: pgmap v232: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:18.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:17 smithi193 ceph-mon[108533]: pgmap v232: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:19 smithi070 ceph-mon[145869]: pgmap v233: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:34:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:34:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:34:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:34:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:19 smithi193 ceph-mon[108533]: pgmap v233: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:34:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:34:20.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:34:20.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:34:22.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:21 smithi070 ceph-mon[145869]: pgmap v234: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:21 smithi193 ceph-mon[108533]: pgmap v234: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:23 smithi070 ceph-mon[145869]: pgmap v235: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:23 smithi193 ceph-mon[108533]: pgmap v235: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:26.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:25 smithi070 ceph-mon[145869]: pgmap v236: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:25 smithi193 ceph-mon[108533]: pgmap v236: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:27 smithi070 ceph-mon[145869]: pgmap v237: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:27 smithi193 ceph-mon[108533]: pgmap v237: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:29 smithi070 ceph-mon[145869]: pgmap v238: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:29 smithi193 ceph-mon[108533]: pgmap v238: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:31 smithi070 ceph-mon[145869]: pgmap v239: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:34:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:31 smithi193 ceph-mon[108533]: pgmap v239: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:34:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:33 smithi070 ceph-mon[145869]: pgmap v240: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:33 smithi193 ceph-mon[108533]: pgmap v240: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:36.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:35 smithi070 ceph-mon[145869]: pgmap v241: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:36.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:35 smithi193 ceph-mon[108533]: pgmap v241: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:37 smithi070 ceph-mon[145869]: pgmap v242: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:37 smithi193 ceph-mon[108533]: pgmap v242: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:40.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:39 smithi070 ceph-mon[145869]: pgmap v243: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:39 smithi193 ceph-mon[108533]: pgmap v243: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:41 smithi070 ceph-mon[145869]: pgmap v244: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:41 smithi193 ceph-mon[108533]: pgmap v244: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:43 smithi070 ceph-mon[145869]: pgmap v245: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:43 smithi193 ceph-mon[108533]: pgmap v245: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:45 smithi070 ceph-mon[145869]: pgmap v246: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:46.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:45 smithi193 ceph-mon[108533]: pgmap v246: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:34:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:34:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:47 smithi070 ceph-mon[145869]: pgmap v247: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:47 smithi193 ceph-mon[108533]: pgmap v247: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:49.190 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:48 smithi070 ceph-mon[145869]: pgmap v248: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:49.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:48 smithi193 ceph-mon[108533]: pgmap v248: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:49.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph orch ps' 2024-03-23T08:34:49.915 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.698+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3813231494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c102710 msgr2=0x7f8c9c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.698+0000 7f8ca0a66700 1 --2- 172.21.15.70:0/3813231494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c102710 0x7f8c9c0fbd80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c004940 tx=0x7f8c8c009e30 comp rx=0 tx=0).stop 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.698+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3813231494 shutdown_connections 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.698+0000 7f8ca0a66700 1 --2- 172.21.15.70:0/3813231494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c102710 0x7f8c9c0fbd80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.698+0000 7f8ca0a66700 1 --2- 172.21.15.70:0/3813231494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c9c101d70 0x7f8c9c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.698+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3813231494 >> 172.21.15.70:0/3813231494 conn(0x7f8c9c072bc0 msgr2=0x7f8c9c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.698+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3813231494 shutdown_connections 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.698+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3813231494 wait complete. 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8ca0a66700 1 Processor -- start 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8ca0a66700 1 -- start start 2024-03-23T08:34:51.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8ca0a66700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c101d70 0x7f8c9c10ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8ca0a66700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c9c102710 0x7f8c9c10efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8ca0a66700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c10f6b0 con 0x7f8c9c101d70 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8ca0a66700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c10f7f0 con 0x7f8c9c102710 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8c9a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c101d70 0x7f8c9c10ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8c9a59c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c101d70 0x7f8c9c10ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60420/0 (socket says 172.21.15.70:60420) 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8c9a59c700 1 -- 172.21.15.70:0/3400057900 learned_addr learned my addr 172.21.15.70:0/3400057900 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8c99d9b700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c9c102710 0x7f8c9c10efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8c9a59c700 1 -- 172.21.15.70:0/3400057900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c9c102710 msgr2=0x7f8c9c10efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8c9a59c700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c9c102710 0x7f8c9c10efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8c9a59c700 1 -- 172.21.15.70:0/3400057900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c005040 con 0x7f8c9c101d70 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8c99d9b700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c9c102710 0x7f8c9c10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.699+0000 7f8c9a59c700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c101d70 0x7f8c9c10ea90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000b790 tx=0x7f8c90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.700+0000 7f8c8affd700 1 -- 172.21.15.70:0/3400057900 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c9000f770 con 0x7f8c9c101d70 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.700+0000 7f8c8affd700 1 -- 172.21.15.70:0/3400057900 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c9000fd80 con 0x7f8c9c101d70 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.700+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c117b60 con 0x7f8c9c101d70 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.700+0000 7f8c8affd700 1 -- 172.21.15.70:0/3400057900 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c9000d310 con 0x7f8c9c101d70 2024-03-23T08:34:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.700+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c1180f0 con 0x7f8c9c101d70 2024-03-23T08:34:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.701+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c0ff080 con 0x7f8c9c101d70 2024-03-23T08:34:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.701+0000 7f8c8affd700 1 -- 172.21.15.70:0/3400057900 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8c90010420 con 0x7f8c9c101d70 2024-03-23T08:34:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.701+0000 7f8c8affd700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8406cc20 0x7f8c8406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:34:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.702+0000 7f8c8affd700 1 -- 172.21.15.70:0/3400057900 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8c9008bef0 con 0x7f8c9c101d70 2024-03-23T08:34:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.702+0000 7f8c99d9b700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8406cc20 0x7f8c8406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:34:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.702+0000 7f8c99d9b700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8406cc20 0x7f8c8406f0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c000c00 tx=0x7f8c8c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:34:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.706+0000 7f8c8affd700 1 -- 172.21.15.70:0/3400057900 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8c90055ff0 con 0x7f8c9c101d70 2024-03-23T08:34:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.909+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 --> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c9c04bed0 con 0x7f8c8406cc20 2024-03-23T08:34:51.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.919+0000 7f8c8affd700 1 -- 172.21.15.70:0/3400057900 <== mgr.34104 v2:172.21.15.70:6800/227946788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8c9c04bed0 con 0x7f8c8406cc20 2024-03-23T08:34:51.919 INFO:teuthology.orchestra.run.smithi070.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-23T08:34:51.919 INFO:teuthology.orchestra.run.smithi070.stdout:alertmanager.smithi070 smithi070 *:9093,9094 running (8m) 2m ago 19m 26.3M - 0.25.0 c8568f914cd2 7ff293abd805 2024-03-23T08:34:51.919 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi070 smithi070 running (6m) 2m ago 19m 6656k - 18.2.2-954-g82316b04 b24e1baba637 50f5b7335667 2024-03-23T08:34:51.919 INFO:teuthology.orchestra.run.smithi070.stdout:crash.smithi193 smithi193 running (6m) 99s ago 18m 6656k - 18.2.2-954-g82316b04 b24e1baba637 de0729e28c83 2024-03-23T08:34:51.919 INFO:teuthology.orchestra.run.smithi070.stdout:grafana.smithi070 smithi070 *:3000 running (8m) 2m ago 19m 90.9M - 9.4.7 954c08fa6188 d92549d28689 2024-03-23T08:34:51.919 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.aymzqb smithi070 running (2m) 2m ago 14m 10.9M - 18.2.2-954-g82316b04 b24e1baba637 3aedd00ed88c 2024-03-23T08:34:51.919 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi070.iziydh smithi070 running (2m) 2m ago 14m 14.7M - 18.2.2-954-g82316b04 b24e1baba637 1066f2091682 2024-03-23T08:34:51.919 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.nucaub smithi193 running (102s) 99s ago 14m 13.6M - 18.2.2-954-g82316b04 b24e1baba637 ea117e71556f 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:mds.cephfs.smithi193.sdlbdd smithi193 running (116s) 99s ago 14m 11.4M - 18.2.2-954-g82316b04 b24e1baba637 5621af1e28f9 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi070.jhsbch smithi070 *:8443,9283,8765 running (10m) 2m ago 21m 504M - 18.2.2-954-g82316b04 b24e1baba637 b5c218330e6b 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:mgr.smithi193.zgpdbu smithi193 *:8443,9283,8765 running (9m) 99s ago 18m 450M - 18.2.2-954-g82316b04 b24e1baba637 ad6843bd2193 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi070 smithi070 running (7m) 2m ago 21m 55.7M 2048M 18.2.2-954-g82316b04 b24e1baba637 a020dc442bcf 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:mon.smithi193 smithi193 running (6m) 99s ago 18m 43.9M 2048M 18.2.2-954-g82316b04 b24e1baba637 e49914f9c03e 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi070 smithi070 *:9100 running (9m) 2m ago 19m 18.8M - 1.5.0 0da6a335fe13 0a59818f00b9 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:node-exporter.smithi193 smithi193 *:9100 running (9m) 99s ago 18m 16.5M - 1.5.0 0da6a335fe13 4d296227e09d 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 smithi070 running (5m) 2m ago 17m 92.6M 1568M 18.2.2-954-g82316b04 b24e1baba637 9396a3debb40 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 smithi070 running (5m) 2m ago 17m 107M 1568M 18.2.2-954-g82316b04 b24e1baba637 13fce63c86eb 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 smithi070 running (4m) 2m ago 16m 96.6M 1568M 18.2.2-954-g82316b04 b24e1baba637 73e36e623491 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 smithi193 running (4m) 99s ago 16m 114M 2592M 18.2.2-954-g82316b04 b24e1baba637 8a7617f5541a 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 smithi193 running (3m) 99s ago 16m 104M 2592M 18.2.2-954-g82316b04 b24e1baba637 76a25ee5af50 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 smithi193 running (2m) 99s ago 15m 107M 2592M 18.2.2-954-g82316b04 b24e1baba637 58bbe6abe8b3 2024-03-23T08:34:51.920 INFO:teuthology.orchestra.run.smithi070.stdout:prometheus.smithi070 smithi070 *:9095 running (8m) 2m ago 19m 60.5M - 2.43.0 a07b618ecd1d c4864ea243d4 2024-03-23T08:34:51.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8406cc20 msgr2=0x7f8c8406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:51.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8406cc20 0x7f8c8406f0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c000c00 tx=0x7f8c8c009e30 comp rx=0 tx=0).stop 2024-03-23T08:34:51.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c101d70 msgr2=0x7f8c9c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:51.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c101d70 0x7f8c9c10ea90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000b790 tx=0x7f8c90009250 comp rx=0 tx=0).stop 2024-03-23T08:34:51.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 shutdown_connections 2024-03-23T08:34:51.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f8c8406cc20 0x7f8c8406f0d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:51.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c9c101d70 0x7f8c9c10ea90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:51.921 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 --2- 172.21.15.70:0/3400057900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c9c102710 0x7f8c9c10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:51.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 >> 172.21.15.70:0/3400057900 conn(0x7f8c9c072bc0 msgr2=0x7f8c9c0fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:34:51.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 shutdown_connections 2024-03-23T08:34:51.922 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:51.921+0000 7f8ca0a66700 1 -- 172.21.15.70:0/3400057900 wait complete. 2024-03-23T08:34:51.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:51 smithi070 ceph-mon[145869]: pgmap v249: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:51 smithi193 ceph-mon[108533]: pgmap v249: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:34:52.666 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph versions' 2024-03-23T08:34:52.944 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:34:54.133 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:53 smithi070 ceph-mon[145869]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:34:54.133 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:53 smithi070 ceph-mon[145869]: pgmap v250: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:53 smithi193 ceph-mon[108533]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-23T08:34:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:53 smithi193 ceph-mon[108533]: pgmap v250: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:54.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.691+0000 7f49f2762700 1 -- 172.21.15.70:0/2380043187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec101d70 msgr2=0x7f49ec102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.692+0000 7f49f2762700 1 --2- 172.21.15.70:0/2380043187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec101d70 0x7f49ec102140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49dc004940 tx=0x7f49dc009e30 comp rx=0 tx=0).stop 2024-03-23T08:34:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.692+0000 7f49f2762700 1 -- 172.21.15.70:0/2380043187 shutdown_connections 2024-03-23T08:34:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.692+0000 7f49f2762700 1 --2- 172.21.15.70:0/2380043187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49ec102710 0x7f49ec0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.692+0000 7f49f2762700 1 --2- 172.21.15.70:0/2380043187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec101d70 0x7f49ec102140 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.692+0000 7f49f2762700 1 -- 172.21.15.70:0/2380043187 >> 172.21.15.70:0/2380043187 conn(0x7f49ec072bc0 msgr2=0x7f49ec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:34:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.692+0000 7f49f2762700 1 -- 172.21.15.70:0/2380043187 shutdown_connections 2024-03-23T08:34:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.692+0000 7f49f2762700 1 -- 172.21.15.70:0/2380043187 wait complete. 2024-03-23T08:34:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.692+0000 7f49f2762700 1 Processor -- start 2024-03-23T08:34:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49f2762700 1 -- start start 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49f2762700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49ec101d70 0x7f49ec1129a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49f2762700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec102710 0x7f49ec112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49f2762700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec113530 con 0x7f49ec102710 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49f2762700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec10c920 con 0x7f49ec101d70 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec102710 0x7f49ec112ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49eb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec102710 0x7f49ec112ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60432/0 (socket says 172.21.15.70:60432) 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49eb7fe700 1 -- 172.21.15.70:0/994373934 learned_addr learned my addr 172.21.15.70:0/994373934 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49ebfff700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49ec101d70 0x7f49ec1129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49eb7fe700 1 -- 172.21.15.70:0/994373934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49ec101d70 msgr2=0x7f49ec1129a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49eb7fe700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49ec101d70 0x7f49ec1129a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49eb7fe700 1 -- 172.21.15.70:0/994373934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc005040 con 0x7f49ec102710 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49eb7fe700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec102710 0x7f49ec112ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f49e000c890 tx=0x7f49e00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:34:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.693+0000 7f49e8ff9700 1 -- 172.21.15.70:0/994373934 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e000a700 con 0x7f49ec102710 2024-03-23T08:34:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.694+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec10cc00 con 0x7f49ec102710 2024-03-23T08:34:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.694+0000 7f49e8ff9700 1 -- 172.21.15.70:0/994373934 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49e000ad10 con 0x7f49ec102710 2024-03-23T08:34:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.694+0000 7f49e8ff9700 1 -- 172.21.15.70:0/994373934 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e0012e70 con 0x7f49ec102710 2024-03-23T08:34:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.694+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec10d1c0 con 0x7f49ec102710 2024-03-23T08:34:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.694+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ec0ff080 con 0x7f49ec102710 2024-03-23T08:34:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.695+0000 7f49e8ff9700 1 -- 172.21.15.70:0/994373934 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f49e000a860 con 0x7f49ec102710 2024-03-23T08:34:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.695+0000 7f49e8ff9700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f49d406cc20 0x7f49d406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:34:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.696+0000 7f49ebfff700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f49d406cc20 0x7f49d406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:34:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.696+0000 7f49e8ff9700 1 -- 172.21.15.70:0/994373934 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f49e008c0a0 con 0x7f49ec102710 2024-03-23T08:34:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.696+0000 7f49ebfff700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f49d406cc20 0x7f49d406f0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49dc004f90 tx=0x7f49dc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:34:54.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.700+0000 7f49e8ff9700 1 -- 172.21.15.70:0/994373934 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f49e0059b70 con 0x7f49ec102710 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.977+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f49ec10dff0 con 0x7f49ec102710 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.978+0000 7f49e8ff9700 1 -- 172.21.15.70:0/994373934 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f49e0059700 con 0x7f49ec102710 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 2 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-03-23T08:34:54.978 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 6 2024-03-23T08:34:54.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:34:54.979 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": { 2024-03-23T08:34:54.979 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 4 2024-03-23T08:34:54.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-03-23T08:34:54.979 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-03-23T08:34:54.979 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)": 14 2024-03-23T08:34:54.979 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-03-23T08:34:54.979 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-03-23T08:34:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f49d406cc20 msgr2=0x7f49d406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f49d406cc20 0x7f49d406f0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49dc004f90 tx=0x7f49dc009e30 comp rx=0 tx=0).stop 2024-03-23T08:34:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec102710 msgr2=0x7f49ec112ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec102710 0x7f49ec112ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f49e000c890 tx=0x7f49e00044c0 comp rx=0 tx=0).stop 2024-03-23T08:34:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 shutdown_connections 2024-03-23T08:34:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f49d406cc20 0x7f49d406f0d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49ec101d70 0x7f49ec1129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 --2- 172.21.15.70:0/994373934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec102710 0x7f49ec112ee0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 >> 172.21.15.70:0/994373934 conn(0x7f49ec072bc0 msgr2=0x7f49ec0fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:34:54.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 shutdown_connections 2024-03-23T08:34:54.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:54.980+0000 7f49f2762700 1 -- 172.21.15.70:0/994373934 wait complete. 2024-03-23T08:34:55.600 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-23T08:34:55.872 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:34:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:55 smithi070 ceph-mon[145869]: pgmap v251: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:55.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:55 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/994373934' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:34:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:55 smithi193 ceph-mon[108533]: pgmap v251: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:34:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:55 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/994373934' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:34:57.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.647+0000 7f59e678f700 1 -- 172.21.15.70:0/1124956111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 msgr2=0x7f59e00fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:57.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.647+0000 7f59e678f700 1 --2- 172.21.15.70:0/1124956111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 0x7f59e00fbd80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f59d4004940 tx=0x7f59d4009e30 comp rx=0 tx=0).stop 2024-03-23T08:34:57.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.647+0000 7f59e678f700 1 -- 172.21.15.70:0/1124956111 shutdown_connections 2024-03-23T08:34:57.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.647+0000 7f59e678f700 1 --2- 172.21.15.70:0/1124956111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 0x7f59e00fbd80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.647+0000 7f59e678f700 1 --2- 172.21.15.70:0/1124956111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59e0101d70 0x7f59e0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.647+0000 7f59e678f700 1 -- 172.21.15.70:0/1124956111 >> 172.21.15.70:0/1124956111 conn(0x7f59e0072bc0 msgr2=0x7f59e0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:34:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.647+0000 7f59e678f700 1 -- 172.21.15.70:0/1124956111 shutdown_connections 2024-03-23T08:34:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.647+0000 7f59e678f700 1 -- 172.21.15.70:0/1124956111 wait complete. 2024-03-23T08:34:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.647+0000 7f59e678f700 1 Processor -- start 2024-03-23T08:34:57.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59e678f700 1 -- start start 2024-03-23T08:34:57.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59e678f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59e0101d70 0x7f59e01129c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:34:57.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59e678f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 0x7f59e0112f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:34:57.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59e678f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e0113550 con 0x7f59e0102710 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59e678f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e010c940 con 0x7f59e0101d70 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59df7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 0x7f59e0112f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59df7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 0x7f59e0112f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60464/0 (socket says 172.21.15.70:60464) 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59df7fe700 1 -- 172.21.15.70:0/2424406073 learned_addr learned my addr 172.21.15.70:0/2424406073 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59dffff700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59e0101d70 0x7f59e01129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59df7fe700 1 -- 172.21.15.70:0/2424406073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59e0101d70 msgr2=0x7f59e01129c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59df7fe700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59e0101d70 0x7f59e01129c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59df7fe700 1 -- 172.21.15.70:0/2424406073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d4005040 con 0x7f59e0102710 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59df7fe700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 0x7f59e0112f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009380 tx=0x7f59d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.648+0000 7f59dcff9700 1 -- 172.21.15.70:0/2424406073 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59d400e040 con 0x7f59e0102710 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.649+0000 7f59dcff9700 1 -- 172.21.15.70:0/2424406073 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59d4013420 con 0x7f59e0102710 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.649+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e010cbc0 con 0x7f59e0102710 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.649+0000 7f59dcff9700 1 -- 172.21.15.70:0/2424406073 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59d401c630 con 0x7f59e0102710 2024-03-23T08:34:57.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.649+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e010d180 con 0x7f59e0102710 2024-03-23T08:34:57.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.649+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e00ff080 con 0x7f59e0102710 2024-03-23T08:34:57.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.650+0000 7f59dcff9700 1 -- 172.21.15.70:0/2424406073 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f59d40139b0 con 0x7f59e0102710 2024-03-23T08:34:57.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.650+0000 7f59dcff9700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f59c806cc20 0x7f59c806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:34:57.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.650+0000 7f59dcff9700 1 -- 172.21.15.70:0/2424406073 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f59d408d1a0 con 0x7f59e0102710 2024-03-23T08:34:57.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.650+0000 7f59dffff700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f59c806cc20 0x7f59c806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:34:57.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.651+0000 7f59dffff700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f59c806cc20 0x7f59c806f0d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59d0007fd0 tx=0x7f59d0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:34:57.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.655+0000 7f59dcff9700 1 -- 172.21.15.70:0/2424406073 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f59d405ac70 con 0x7f59e0102710 2024-03-23T08:34:57.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.929+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f59e010e010 con 0x7f59e0102710 2024-03-23T08:34:57.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.929+0000 7f59dcff9700 1 -- 172.21.15.70:0/2424406073 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f59d405a800 con 0x7f59e0102710 2024-03-23T08:34:57.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f59c806cc20 msgr2=0x7f59c806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:57.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f59c806cc20 0x7f59c806f0d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f59d0007fd0 tx=0x7f59d0009040 comp rx=0 tx=0).stop 2024-03-23T08:34:57.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 msgr2=0x7f59e0112f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:34:57.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 0x7f59e0112f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009380 tx=0x7f59d4009e30 comp rx=0 tx=0).stop 2024-03-23T08:34:57.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 shutdown_connections 2024-03-23T08:34:57.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f59c806cc20 0x7f59c806f0d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:57.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59e0101d70 0x7f59e01129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:57.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 --2- 172.21.15.70:0/2424406073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59e0102710 0x7f59e0112f00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:34:57.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 >> 172.21.15.70:0/2424406073 conn(0x7f59e0072bc0 msgr2=0x7f59e00fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:34:57.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 shutdown_connections 2024-03-23T08:34:57.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:34:57.931+0000 7f59e678f700 1 -- 172.21.15.70:0/2424406073 wait complete. 2024-03-23T08:34:57.941 INFO:teuthology.orchestra.run.smithi070.stdout:true 2024-03-23T08:34:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:57 smithi070 ceph-mon[145869]: pgmap v252: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:57 smithi193 ceph-mon[108533]: pgmap v252: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:34:58.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -e sha1=82316b04821a6c2c1d458a881920454bbadb8837 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-23T08:34:58.750 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:34:59.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:58 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2424406073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:34:59.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:58 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2424406073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:34:59.918 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:34:59 smithi070 ceph-mon[145869]: pgmap v253: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:00.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:34:59 smithi193 ceph-mon[108533]: pgmap v253: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 -- 172.21.15.70:0/41295652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b801026c0 msgr2=0x7f6b800fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 --2- 172.21.15.70:0/41295652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b801026c0 0x7f6b800fbd30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b74004940 tx=0x7f6b74009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 -- 172.21.15.70:0/41295652 shutdown_connections 2024-03-23T08:35:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 --2- 172.21.15.70:0/41295652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b801026c0 0x7f6b800fbd30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 --2- 172.21.15.70:0/41295652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b80101d20 0x7f6b801020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 -- 172.21.15.70:0/41295652 >> 172.21.15.70:0/41295652 conn(0x7f6b800f7870 msgr2=0x7f6b800f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 -- 172.21.15.70:0/41295652 shutdown_connections 2024-03-23T08:35:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 -- 172.21.15.70:0/41295652 wait complete. 2024-03-23T08:35:00.503 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 Processor -- start 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.502+0000 7f6b85424700 1 -- start start 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b85424700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80101d20 0x7f6b8010ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b85424700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b801026c0 0x7f6b8010efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b85424700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8010f4f0 con 0x7f6b80101d20 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b85424700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8010f660 con 0x7f6b801026c0 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b7effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80101d20 0x7f6b8010ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b7effd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80101d20 0x7f6b8010ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44590/0 (socket says 172.21.15.70:44590) 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b7effd700 1 -- 172.21.15.70:0/4130691937 learned_addr learned my addr 172.21.15.70:0/4130691937 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b7e7fc700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b801026c0 0x7f6b8010efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b7effd700 1 -- 172.21.15.70:0/4130691937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b801026c0 msgr2=0x7f6b8010efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b7effd700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b801026c0 0x7f6b8010efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b7effd700 1 -- 172.21.15.70:0/4130691937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b74005040 con 0x7f6b80101d20 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b7effd700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80101d20 0x7f6b8010ea70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000b790 tx=0x7f6b70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b677fe700 1 -- 172.21.15.70:0/4130691937 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7000f770 con 0x7f6b80101d20 2024-03-23T08:35:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b80117ae0 con 0x7f6b80101d20 2024-03-23T08:35:00.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b677fe700 1 -- 172.21.15.70:0/4130691937 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b7000fd80 con 0x7f6b80101d20 2024-03-23T08:35:00.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b677fe700 1 -- 172.21.15.70:0/4130691937 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7000d310 con 0x7f6b80101d20 2024-03-23T08:35:00.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.503+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8010ca20 con 0x7f6b80101d20 2024-03-23T08:35:00.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.505+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b800fd470 con 0x7f6b80101d20 2024-03-23T08:35:00.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.505+0000 7f6b677fe700 1 -- 172.21.15.70:0/4130691937 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6b7000d470 con 0x7f6b80101d20 2024-03-23T08:35:00.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.505+0000 7f6b677fe700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6b68071030 0x7f6b680734e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:00.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.505+0000 7f6b677fe700 1 -- 172.21.15.70:0/4130691937 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6b7008c040 con 0x7f6b80101d20 2024-03-23T08:35:00.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.505+0000 7f6b7e7fc700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6b68071030 0x7f6b680734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:00.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.506+0000 7f6b7e7fc700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6b68071030 0x7f6b680734e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400b3b0 tx=0x7f6b74009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:00.509 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.510+0000 7f6b677fe700 1 -- 172.21.15.70:0/4130691937 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6b70056080 con 0x7f6b80101d20 2024-03-23T08:35:00.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.782+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b8010cca0 con 0x7f6b80101d20 2024-03-23T08:35:00.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.783+0000 7f6b677fe700 1 -- 172.21.15.70:0/4130691937 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f6b700596a0 con 0x7f6b80101d20 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6b68071030 msgr2=0x7f6b680734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6b68071030 0x7f6b680734e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400b3b0 tx=0x7f6b74009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80101d20 msgr2=0x7f6b8010ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80101d20 0x7f6b8010ea70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000b790 tx=0x7f6b70009250 comp rx=0 tx=0).stop 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 shutdown_connections 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6b68071030 0x7f6b680734e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b80101d20 0x7f6b8010ea70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 --2- 172.21.15.70:0/4130691937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b801026c0 0x7f6b8010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 >> 172.21.15.70:0/4130691937 conn(0x7f6b800f7870 msgr2=0x7f6b800f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 shutdown_connections 2024-03-23T08:35:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:00.785+0000 7f6b85424700 1 -- 172.21.15.70:0/4130691937 wait complete. 2024-03-23T08:35:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.2-954-g82316b04 (82316b04821a6c2c1d458a881920454bbadb8837) reef (stable)" 2024-03-23T08:35:01.178 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:00 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/4130691937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:35:01.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:00 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/4130691937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-23T08:35:01.469 DEBUG:teuthology.parallel:result is None 2024-03-23T08:35:01.469 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-23T08:35:01.482 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:35:01.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-23T08:35:01.757 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/mon.smithi070/config 2024-03-23T08:35:02.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:01 smithi070 ceph-mon[145869]: pgmap v254: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:02.230 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:35:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:01 smithi193 ceph-mon[108533]: pgmap v254: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:35:03.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.547+0000 7f5c8c902700 1 -- 172.21.15.70:0/3374814660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 msgr2=0x7f5c880fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:03.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.547+0000 7f5c8c902700 1 --2- 172.21.15.70:0/3374814660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 0x7f5c880fe260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c004940 tx=0x7f5c7c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:03.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.547+0000 7f5c8c902700 1 -- 172.21.15.70:0/3374814660 shutdown_connections 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.547+0000 7f5c8c902700 1 --2- 172.21.15.70:0/3374814660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c880fe830 0x7f5c88106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.547+0000 7f5c8c902700 1 --2- 172.21.15.70:0/3374814660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 0x7f5c880fe260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.547+0000 7f5c8c902700 1 -- 172.21.15.70:0/3374814660 >> 172.21.15.70:0/3374814660 conn(0x7f5c88072bc0 msgr2=0x7f5c88072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.547+0000 7f5c8c902700 1 -- 172.21.15.70:0/3374814660 shutdown_connections 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.547+0000 7f5c8c902700 1 -- 172.21.15.70:0/3374814660 wait complete. 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8c902700 1 Processor -- start 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8c902700 1 -- start start 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8c902700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 0x7f5c88113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8c902700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c880fe830 0x7f5c88114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8c902700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88114710 con 0x7f5c880fde90 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8c902700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8810da70 con 0x7f5c880fe830 2024-03-23T08:35:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 0x7f5c88113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8659c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 0x7f5c88113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44600/0 (socket says 172.21.15.70:44600) 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8659c700 1 -- 172.21.15.70:0/2686513971 learned_addr learned my addr 172.21.15.70:0/2686513971 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c85d9b700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c880fe830 0x7f5c88114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8659c700 1 -- 172.21.15.70:0/2686513971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c880fe830 msgr2=0x7f5c88114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8659c700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c880fe830 0x7f5c88114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8659c700 1 -- 172.21.15.70:0/2686513971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c005040 con 0x7f5c880fde90 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.548+0000 7f5c8659c700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 0x7f5c88113af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009360 tx=0x7f5c7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.549+0000 7f5c76ffd700 1 -- 172.21.15.70:0/2686513971 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7c00e040 con 0x7f5c880fde90 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.549+0000 7f5c76ffd700 1 -- 172.21.15.70:0/2686513971 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c7c013420 con 0x7f5c880fde90 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.549+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8810dcf0 con 0x7f5c880fde90 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.549+0000 7f5c76ffd700 1 -- 172.21.15.70:0/2686513971 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7c01c630 con 0x7f5c880fde90 2024-03-23T08:35:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.549+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8810e230 con 0x7f5c880fde90 2024-03-23T08:35:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.550+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c88065e30 con 0x7f5c880fde90 2024-03-23T08:35:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.551+0000 7f5c76ffd700 1 -- 172.21.15.70:0/2686513971 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5c7c0139e0 con 0x7f5c880fde90 2024-03-23T08:35:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.551+0000 7f5c76ffd700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5c7006cc20 0x7f5c7006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.551+0000 7f5c76ffd700 1 -- 172.21.15.70:0/2686513971 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5c7c08dc90 con 0x7f5c880fde90 2024-03-23T08:35:03.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.551+0000 7f5c85d9b700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5c7006cc20 0x7f5c7006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:03.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.551+0000 7f5c85d9b700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5c7006cc20 0x7f5c7006f0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5c78006fd0 tx=0x7f5c78008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:03.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.555+0000 7f5c76ffd700 1 -- 172.21.15.70:0/2686513971 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5c7c05b820 con 0x7f5c880fde90 2024-03-23T08:35:03.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.789+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5c88109ed0 con 0x7f5c880fde90 2024-03-23T08:35:03.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.790+0000 7f5c76ffd700 1 -- 172.21.15.70:0/2686513971 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f5c7c05b3b0 con 0x7f5c880fde90 2024-03-23T08:35:03.790 INFO:teuthology.orchestra.run.smithi070.stdout:e37 2024-03-23T08:35:03.790 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-23T08:35:03.790 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:35:03.790 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: 1 2024-03-23T08:35:03.790 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:Filesystem 'cephfs' (1) 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:fs_name cephfs 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 37 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-03-23T08:20:37.818350+0000 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-03-23T08:33:18.781347+0000 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:tableserver 0 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:root 0 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:session_timeout 60 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:session_autoclose 300 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:max_file_size 1099511627776 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:max_xattr_size 65536 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:required_client_features {} 2024-03-23T08:35:03.791 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure 0 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:last_failure_osd_epoch 84 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:max_mds 1 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:in 0 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:up {0=34376} 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:failed 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:damaged 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:stopped 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:data_pools [3] 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:metadata_pool 2 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:inline_data disabled 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:balancer 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:bal_rank_mask -1 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:standby_count_wanted 1 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.iziydh{0:34376} state up:active seq 14 join_fscid=1 addr [v2:172.21.15.70:6826/1449136189,v1:172.21.15.70:6827/1449136189] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:35:03.792 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi070.aymzqb{0:34408} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.70:6828/470900113,v1:172.21.15.70:6829/470900113] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:35:03.793 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:03.793 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:03.793 INFO:teuthology.orchestra.run.smithi070.stdout:Standby daemons: 2024-03-23T08:35:03.793 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:03.793 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.sdlbdd{-1:44279} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/4097359817,v1:172.21.15.193:6825/4097359817] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:35:03.793 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.cephfs.smithi193.nucaub{-1:44311} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/1000528415,v1:172.21.15.193:6827/1000528415] compat {c=[1],r=[1],i=[7ff]}] 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5c7006cc20 msgr2=0x7f5c7006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5c7006cc20 0x7f5c7006f0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5c78006fd0 tx=0x7f5c78008040 comp rx=0 tx=0).stop 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 msgr2=0x7f5c88113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 0x7f5c88113af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009360 tx=0x7f5c7c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 shutdown_connections 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5c7006cc20 0x7f5c7006f0d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c880fde90 0x7f5c88113af0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 --2- 172.21.15.70:0/2686513971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c880fe830 0x7f5c88114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 >> 172.21.15.70:0/2686513971 conn(0x7f5c88072bc0 msgr2=0x7f5c881037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 shutdown_connections 2024-03-23T08:35:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:03.792+0000 7f5c8c902700 1 -- 172.21.15.70:0/2686513971 wait complete. 2024-03-23T08:35:03.795 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 37 2024-03-23T08:35:04.141 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:03 smithi070 ceph-mon[145869]: pgmap v255: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:04.142 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:03 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2686513971' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:35:04.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:03 smithi193 ceph-mon[108533]: pgmap v255: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:04.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:03 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2686513971' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-23T08:35:04.416 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-23T08:35:04.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T08:35:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.550+0000 7ffa110cf700 1 -- 172.21.15.70:0/2970375434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fecd0 msgr2=0x7ffa0c0ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.550+0000 7ffa110cf700 1 --2- 172.21.15.70:0/2970375434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fecd0 0x7ffa0c0ff0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa00004940 tx=0x7ffa00009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.550+0000 7ffa110cf700 1 -- 172.21.15.70:0/2970375434 shutdown_connections 2024-03-23T08:35:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.550+0000 7ffa110cf700 1 --2- 172.21.15.70:0/2970375434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa0c0fb360 0x7ffa0c0fb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.550+0000 7ffa110cf700 1 --2- 172.21.15.70:0/2970375434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fecd0 0x7ffa0c0ff0a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.550+0000 7ffa110cf700 1 -- 172.21.15.70:0/2970375434 >> 172.21.15.70:0/2970375434 conn(0x7ffa0c071370 msgr2=0x7ffa0c071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.550+0000 7ffa110cf700 1 -- 172.21.15.70:0/2970375434 shutdown_connections 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.550+0000 7ffa110cf700 1 -- 172.21.15.70:0/2970375434 wait complete. 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa110cf700 1 Processor -- start 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa110cf700 1 -- start start 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa110cf700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fb360 0x7ffa0c1137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa110cf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa0c0fecd0 0x7ffa0c113d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa110cf700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c114350 con 0x7ffa0c0fb360 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa110cf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c10d950 con 0x7ffa0c0fecd0 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fb360 0x7ffa0c1137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fb360 0x7ffa0c1137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44628/0 (socket says 172.21.15.70:44628) 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa0ad9d700 1 -- 172.21.15.70:0/2822115043 learned_addr learned my addr 172.21.15.70:0/2822115043 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa0a59c700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa0c0fecd0 0x7ffa0c113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa0ad9d700 1 -- 172.21.15.70:0/2822115043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa0c0fecd0 msgr2=0x7ffa0c113d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa0ad9d700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa0c0fecd0 0x7ffa0c113d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:04.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa0ad9d700 1 -- 172.21.15.70:0/2822115043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00005040 con 0x7ffa0c0fb360 2024-03-23T08:35:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa0a59c700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa0c0fecd0 0x7ffa0c113d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:35:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa0ad9d700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fb360 0x7ffa0c1137c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000b3b0 tx=0x7ffa00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ff9f37fe700 1 -- 172.21.15.70:0/2822115043 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0000e040 con 0x7ffa0c0fb360 2024-03-23T08:35:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ff9f37fe700 1 -- 172.21.15.70:0/2822115043 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffa00013420 con 0x7ffa0c0fb360 2024-03-23T08:35:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c10dbd0 con 0x7ffa0c0fb360 2024-03-23T08:35:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ff9f37fe700 1 -- 172.21.15.70:0/2822115043 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0001c630 con 0x7ffa0c0fb360 2024-03-23T08:35:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.551+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c10e110 con 0x7ffa0c0fb360 2024-03-23T08:35:04.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.552+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c1052e0 con 0x7ffa0c0fb360 2024-03-23T08:35:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.554+0000 7ff9f37fe700 1 -- 172.21.15.70:0/2822115043 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ffa00013970 con 0x7ffa0c0fb360 2024-03-23T08:35:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.554+0000 7ff9f37fe700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff9f406cc20 0x7ff9f406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.554+0000 7ff9f37fe700 1 -- 172.21.15.70:0/2822115043 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffa0008dd30 con 0x7ffa0c0fb360 2024-03-23T08:35:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.554+0000 7ffa0a59c700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff9f406cc20 0x7ff9f406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.554+0000 7ffa0a59c700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff9f406cc20 0x7ff9f406f0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009fd0 tx=0x7ff9fc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.558+0000 7ff9f37fe700 1 -- 172.21.15.70:0/2822115043 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffa0005b800 con 0x7ffa0c0fb360 2024-03-23T08:35:04.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.755+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa0c064e20 con 0x7ffa0c0fb360 2024-03-23T08:35:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.756+0000 7ff9f37fe700 1 -- 172.21.15.70:0/2822115043 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16545 (secure 0 0 0) 0x7ffa00011370 con 0x7ffa0c0fb360 2024-03-23T08:35:04.756 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:04.756 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":85,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","created":"2024-03-23T08:13:36.916195+0000","modified":"2024-03-23T08:33:05.994824+0000","last_up_change":"2024-03-23T08:32:03.566601+0000","last_in_change":"2024-03-23T08:18:58.641957+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T08:18:02.704319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-23T08:20:36.515704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-23T08:20:37.038000+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a71163c1-e6c6-490d-ade2-f345e4b22ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":51,"up_from":55,"up_thru":80,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6803","nonce":2783750869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6805","nonce":2783750869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6809","nonce":2783750869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6807","nonce":2783750869}]},"public_addr":"172.21.15.70:6803/2783750869","cluster_addr":"172.21.15.70:6805/2783750869","heartbeat_back_addr":"172.21.15.70:6809/2783750869","heartbeat_front_addr":"172.21.15.70:6807/2783750869","state":["exists","up"]},{"osd":1,"uuid":"0054cfce-6cdc-46f2-b434-ecd02d1ad193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":56,"up_from":60,"up_thru":80,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6811","nonce":1925098063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6813","nonce":1925098063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6817","nonce":1925098063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6815","nonce":1925098063}]},"public_addr":"172.21.15.70:6811/1925098063","cluster_addr":"172.21.15.70:6813/1925098063","heartbeat_back_addr":"172.21.15.70:6817/1925098063","heartbeat_front_addr":"172.21.15.70:6815/1925098063","state":["exists","up"]},{"osd":2,"uuid":"24c96238-b75f-47c4-b8ef-3794e45e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":61,"up_from":65,"up_thru":80,"down_at":62,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6819","nonce":357933512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6821","nonce":357933512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6825","nonce":357933512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6823","nonce":357933512}]},"public_addr":"172.21.15.70:6819/357933512","cluster_addr":"172.21.15.70:6821/357933512","heartbeat_back_addr":"172.21.15.70:6825/357933512","heartbeat_front_addr":"172.21.15.70:6823/357933512","state":["exists","up"]},{"osd":3,"uuid":"64e802c4-a037-4a42-8f0f-14756b62251b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":28,"last_clean_end":66,"up_from":70,"up_thru":80,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6801","nonce":457144182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6803","nonce":457144182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6807","nonce":457144182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6805","nonce":457144182}]},"public_addr":"172.21.15.193:6801/457144182","cluster_addr":"172.21.15.193:6803/457144182","heartbeat_back_addr":"172.21.15.193:6807/457144182","heartbeat_front_addr":"172.21.15.193:6805/457144182","state":["exists","up"]},{"osd":4,"uuid":"3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":36,"last_clean_end":71,"up_from":75,"up_thru":80,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6809","nonce":3936171891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6811","nonce":3936171891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6815","nonce":3936171891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6813","nonce":3936171891}]},"public_addr":"172.21.15.193:6809/3936171891","cluster_addr":"172.21.15.193:6811/3936171891","heartbeat_back_addr":"172.21.15.193:6815/3936171891","heartbeat_front_addr":"172.21.15.193:6813/3936171891","state":["exists","up"]},{"osd":5,"uuid":"57e8b767-4e62-41b1-8d96-caeaad28c929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":41,"last_clean_end":76,"up_from":80,"up_thru":80,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6817","nonce":2211659369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6819","nonce":2211659369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6823","nonce":2211659369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6821","nonce":2211659369}]},"public_addr":"172.21.15.193:6817/2211659369","cluster_addr":"172.21.15.193:6819/2211659369","heartbeat_back_addr":"172.21.15.193:6823/2211659369","heartbeat_front_addr":"172.21.15.193:6821/2211659369","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:01.224711+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:29.865012+0000","dead_epoch":56},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:58.894993+0000","dead_epoch":61},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:23.061992+0000","dead_epoch":66},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:49.665476+0000","dead_epoch":71},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:19:16.008523+0000","dead_epoch":76}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6825/3335188435":"2024-03-24T08:32:48.851367+0000","172.21.15.70:6829/2658800706":"2024-03-24T08:32:27.286778+0000","172.21.15.70:6800/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.193:6824/3335188435":"2024-03-24T08:32:48.851367+0000","172.21.15.70:6801/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/3721968086":"2024-03-24T08:15:48.390242+0000","172.21.15.193:0/4291429451":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/2658612358":"2024-03-24T08:27:46.544317+0000","172.21.15.70:0/3991559747":"2024-03-24T08:15:48.390242+0000","172.21.15.193:0/1987061818":"2024-03-24T08:24:45.131405+0000","172.21.15.70:6801/244960840":"2024-03-24T08:27:46.544317+0000","172.21.15.70:0/70452311":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/562426678":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6826/1612292110":"2024-03-24T08:20:52.080956+0000","172.21.15.193:6828/758671107":"2024-03-24T08:24:45.131405+0000","172.21.15.193:6829/758671107":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/3004659310":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.193:0/413036228":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/3009100678":"2024-03-24T08:22:46.280738+0000","172.21.15.70:6800/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/2209675146":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/1281398424":"2024-03-24T08:22:46.280738+0000","172.21.15.70:6827/1612292110":"2024-03-24T08:20:52.080956+0000","172.21.15.193:0/1836969396":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/370857700":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/104037264":"2024-03-24T08:27:46.544317+0000","172.21.15.70:6801/3694824274":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/3484360896":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/1608386260":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/400165096":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/3894371824":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6800/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6800/244960840":"2024-03-24T08:27:46.544317+0000","172.21.15.70:6800/3694824274":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/1984770541":"2024-03-24T08:27:46.544317+0000","172.21.15.70:0/2973224698":"2024-03-24T08:27:46.544317+0000","172.21.15.70:6828/2658800706":"2024-03-24T08:32:27.286778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T08:35:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff9f406cc20 msgr2=0x7ff9f406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff9f406cc20 0x7ff9f406f0d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009fd0 tx=0x7ff9fc00b040 comp rx=0 tx=0).stop 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fb360 msgr2=0x7ffa0c1137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fb360 0x7ffa0c1137c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000b3b0 tx=0x7ffa00009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 shutdown_connections 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff9f406cc20 0x7ff9f406f0d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa0c0fb360 0x7ffa0c1137c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 --2- 172.21.15.70:0/2822115043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa0c0fecd0 0x7ffa0c113d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 >> 172.21.15.70:0/2822115043 conn(0x7ffa0c071370 msgr2=0x7ffa0c0fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 shutdown_connections 2024-03-23T08:35:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.758+0000 7ffa110cf700 1 -- 172.21.15.70:0/2822115043 wait complete. 2024-03-23T08:35:04.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T08:18:02.704319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-23T08:20:36.515704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-23T08:20:37.038000+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T08:35:04.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T08:35:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.894+0000 7fe80b7c9700 1 -- 172.21.15.70:0/4065537653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 msgr2=0x7fe804103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.894+0000 7fe80b7c9700 1 --2- 172.21.15.70:0/4065537653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 0x7fe804103090 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe800004940 tx=0x7fe800009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.894+0000 7fe80b7c9700 1 -- 172.21.15.70:0/4065537653 shutdown_connections 2024-03-23T08:35:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.894+0000 7fe80b7c9700 1 --2- 172.21.15.70:0/4065537653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804103660 0x7fe8040fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.894+0000 7fe80b7c9700 1 --2- 172.21.15.70:0/4065537653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 0x7fe804103090 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.894+0000 7fe80b7c9700 1 -- 172.21.15.70:0/4065537653 >> 172.21.15.70:0/4065537653 conn(0x7fe8040f8830 msgr2=0x7fe8040fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.894+0000 7fe80b7c9700 1 -- 172.21.15.70:0/4065537653 shutdown_connections 2024-03-23T08:35:04.896 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.894+0000 7fe80b7c9700 1 -- 172.21.15.70:0/4065537653 wait complete. 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.894+0000 7fe80b7c9700 1 Processor -- start 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe80b7c9700 1 -- start start 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe80b7c9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 0x7fe804113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe80b7c9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804103660 0x7fe804113cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe80b7c9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe804114300 con 0x7fe804102cc0 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe80b7c9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80410d900 con 0x7fe804103660 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe809565700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 0x7fe804113770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe809565700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 0x7fe804113770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44652/0 (socket says 172.21.15.70:44652) 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe809565700 1 -- 172.21.15.70:0/362243796 learned_addr learned my addr 172.21.15.70:0/362243796 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe808d64700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804103660 0x7fe804113cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe809565700 1 -- 172.21.15.70:0/362243796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804103660 msgr2=0x7fe804113cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe809565700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804103660 0x7fe804113cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:04.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe809565700 1 -- 172.21.15.70:0/362243796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800005040 con 0x7fe804102cc0 2024-03-23T08:35:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe808d64700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804103660 0x7fe804113cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:35:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe809565700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 0x7fe804113770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe800009180 tx=0x7fe800009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe7f9ffb700 1 -- 172.21.15.70:0/362243796 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe80000e040 con 0x7fe804102cc0 2024-03-23T08:35:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe7f9ffb700 1 -- 172.21.15.70:0/362243796 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe800004290 con 0x7fe804102cc0 2024-03-23T08:35:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe7f9ffb700 1 -- 172.21.15.70:0/362243796 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe80001c3f0 con 0x7fe804102cc0 2024-03-23T08:35:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.895+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80410db80 con 0x7fe804102cc0 2024-03-23T08:35:04.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.896+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80410e140 con 0x7fe804102cc0 2024-03-23T08:35:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.897+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8040fe410 con 0x7fe804102cc0 2024-03-23T08:35:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.897+0000 7fe7f9ffb700 1 -- 172.21.15.70:0/362243796 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe800013790 con 0x7fe804102cc0 2024-03-23T08:35:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.897+0000 7fe7f9ffb700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cc20 0x7fe7f006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.898+0000 7fe7f9ffb700 1 -- 172.21.15.70:0/362243796 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe80008d2f0 con 0x7fe804102cc0 2024-03-23T08:35:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.898+0000 7fe808d64700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cc20 0x7fe7f006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:04.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.898+0000 7fe808d64700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cc20 0x7fe7f006f0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009fd0 tx=0x7fe7f400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:04.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:04.902+0000 7fe7f9ffb700 1 -- 172.21.15.70:0/362243796 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe80005adc0 con 0x7fe804102cc0 2024-03-23T08:35:05.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.094+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe804109c70 con 0x7fe804102cc0 2024-03-23T08:35:05.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.095+0000 7fe7f9ffb700 1 -- 172.21.15.70:0/362243796 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe800013420 con 0x7fe804102cc0 2024-03-23T08:35:05.095 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-03-23T08:35:05.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cc20 msgr2=0x7fe7f006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cc20 0x7fe7f006f0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009fd0 tx=0x7fe7f400b040 comp rx=0 tx=0).stop 2024-03-23T08:35:05.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 msgr2=0x7fe804113770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 0x7fe804113770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe800009180 tx=0x7fe800009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:05.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 shutdown_connections 2024-03-23T08:35:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe7f006cc20 0x7fe7f006f0d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe804102cc0 0x7fe804113770 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 --2- 172.21.15.70:0/362243796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804103660 0x7fe804113cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 >> 172.21.15.70:0/362243796 conn(0x7fe8040f8830 msgr2=0x7fe8040f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 shutdown_connections 2024-03-23T08:35:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.097+0000 7fe80b7c9700 1 -- 172.21.15.70:0/362243796 wait complete. 2024-03-23T08:35:05.107 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:04 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2822115043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:35:05.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-23T08:35:05.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.229+0000 7fd75d47f700 1 -- 172.21.15.70:0/1411414664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 msgr2=0x7fd7580fe0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.229+0000 7fd75d47f700 1 --2- 172.21.15.70:0/1411414664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 0x7fd7580fe0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd74c004940 tx=0x7fd74c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:05.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.229+0000 7fd75d47f700 1 -- 172.21.15.70:0/1411414664 shutdown_connections 2024-03-23T08:35:05.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.229+0000 7fd75d47f700 1 --2- 172.21.15.70:0/1411414664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7580fe690 0x7fd758106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.229+0000 7fd75d47f700 1 --2- 172.21.15.70:0/1411414664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 0x7fd7580fe0c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.229+0000 7fd75d47f700 1 -- 172.21.15.70:0/1411414664 >> 172.21.15.70:0/1411414664 conn(0x7fd7580f7730 msgr2=0x7fd7580f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:05.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.229+0000 7fd75d47f700 1 -- 172.21.15.70:0/1411414664 shutdown_connections 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.229+0000 7fd75d47f700 1 -- 172.21.15.70:0/1411414664 wait complete. 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd75d47f700 1 Processor -- start 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd75d47f700 1 -- start start 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd75d47f700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 0x7fd7580fbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd75d47f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7580fe690 0x7fd7580fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd75d47f700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580fdb00 con 0x7fd7580fdcf0 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd75d47f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580fc630 con 0x7fd7580fe690 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd756ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 0x7fd7580fbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd756ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 0x7fd7580fbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44658/0 (socket says 172.21.15.70:44658) 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd756ffd700 1 -- 172.21.15.70:0/3025552427 learned_addr learned my addr 172.21.15.70:0/3025552427 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd7567fc700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7580fe690 0x7fd7580fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd756ffd700 1 -- 172.21.15.70:0/3025552427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7580fe690 msgr2=0x7fd7580fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd756ffd700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7580fe690 0x7fd7580fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.230+0000 7fd756ffd700 1 -- 172.21.15.70:0/3025552427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c005040 con 0x7fd7580fdcf0 2024-03-23T08:35:05.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.231+0000 7fd756ffd700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 0x7fd7580fbbb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00b3b0 tx=0x7fd74c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:05.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.231+0000 7fd73f7fe700 1 -- 172.21.15.70:0/3025552427 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd74c00e040 con 0x7fd7580fdcf0 2024-03-23T08:35:05.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.231+0000 7fd73f7fe700 1 -- 172.21.15.70:0/3025552427 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd74c00bed0 con 0x7fd7580fdcf0 2024-03-23T08:35:05.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.231+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7580fc8b0 con 0x7fd7580fdcf0 2024-03-23T08:35:05.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.231+0000 7fd73f7fe700 1 -- 172.21.15.70:0/3025552427 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd74c013db0 con 0x7fd7580fdcf0 2024-03-23T08:35:05.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.231+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd758119eb0 con 0x7fd7580fdcf0 2024-03-23T08:35:05.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.232+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7580669f0 con 0x7fd7580fdcf0 2024-03-23T08:35:05.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.232+0000 7fd73f7fe700 1 -- 172.21.15.70:0/3025552427 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd74c00ba20 con 0x7fd7580fdcf0 2024-03-23T08:35:05.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.233+0000 7fd73f7fe700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd74006cc20 0x7fd74006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:05.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.233+0000 7fd7567fc700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd74006cc20 0x7fd74006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:05.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.233+0000 7fd73f7fe700 1 -- 172.21.15.70:0/3025552427 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd74c08d120 con 0x7fd7580fdcf0 2024-03-23T08:35:05.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.233+0000 7fd7567fc700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd74006cc20 0x7fd74006f0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd7580fd2e0 tx=0x7fd74800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:05.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.237+0000 7fd73f7fe700 1 -- 172.21.15.70:0/3025552427 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd74c057220 con 0x7fd7580fdcf0 2024-03-23T08:35:05.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:04 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2822115043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:35:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.429+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd758064dc0 con 0x7fd7580fdcf0 2024-03-23T08:35:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.429+0000 7fd73f7fe700 1 -- 172.21.15.70:0/3025552427 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd74c05a840 con 0x7fd7580fdcf0 2024-03-23T08:35:05.429 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd74006cc20 msgr2=0x7fd74006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd74006cc20 0x7fd74006f0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd7580fd2e0 tx=0x7fd74800b040 comp rx=0 tx=0).stop 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 msgr2=0x7fd7580fbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 0x7fd7580fbbb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00b3b0 tx=0x7fd74c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 shutdown_connections 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fd74006cc20 0x7fd74006f0d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7580fdcf0 0x7fd7580fbbb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 --2- 172.21.15.70:0/3025552427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7580fe690 0x7fd7580fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 >> 172.21.15.70:0/3025552427 conn(0x7fd7580f7730 msgr2=0x7fd7580f8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 shutdown_connections 2024-03-23T08:35:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.431+0000 7fd75d47f700 1 -- 172.21.15.70:0/3025552427 wait complete. 2024-03-23T08:35:05.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-23T08:35:05.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.564+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/201701083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 msgr2=0x7ff1ec106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.564+0000 7ff1f2bd7700 1 --2- 172.21.15.70:0/201701083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 0x7ff1ec106d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8004940 tx=0x7ff1d8009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:05.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.564+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/201701083 shutdown_connections 2024-03-23T08:35:05.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.564+0000 7ff1f2bd7700 1 --2- 172.21.15.70:0/201701083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 0x7ff1ec106d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.564+0000 7ff1f2bd7700 1 --2- 172.21.15.70:0/201701083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1ec0fdcd0 0x7ff1ec0fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.564+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/201701083 >> 172.21.15.70:0/201701083 conn(0x7ff1ec0f7710 msgr2=0x7ff1ec0f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:05.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.564+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/201701083 shutdown_connections 2024-03-23T08:35:05.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.564+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/201701083 wait complete. 2024-03-23T08:35:05.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.565+0000 7ff1f2bd7700 1 Processor -- start 2024-03-23T08:35:05.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.565+0000 7ff1f2bd7700 1 -- start start 2024-03-23T08:35:05.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.565+0000 7ff1f2bd7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1ec0fdcd0 0x7ff1ec113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.565+0000 7ff1f2bd7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 0x7ff1ec113cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.565+0000 7ff1f2bd7700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec114310 con 0x7ff1ec0fe670 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.565+0000 7ff1f2bd7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec10d910 con 0x7ff1ec0fdcd0 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.565+0000 7ff1ebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 0x7ff1ec113cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.565+0000 7ff1ebfff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 0x7ff1ec113cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44668/0 (socket says 172.21.15.70:44668) 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.565+0000 7ff1ebfff700 1 -- 172.21.15.70:0/539552954 learned_addr learned my addr 172.21.15.70:0/539552954 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1f0973700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1ec0fdcd0 0x7ff1ec113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1ebfff700 1 -- 172.21.15.70:0/539552954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1ec0fdcd0 msgr2=0x7ff1ec113780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1ebfff700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1ec0fdcd0 0x7ff1ec113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1ebfff700 1 -- 172.21.15.70:0/539552954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d8005040 con 0x7ff1ec0fe670 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1ebfff700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 0x7ff1ec113cc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8004890 tx=0x7ff1d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:05.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1e97fa700 1 -- 172.21.15.70:0/539552954 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d800e040 con 0x7ff1ec0fe670 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec10db90 con 0x7ff1ec0fe670 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1e97fa700 1 -- 172.21.15.70:0/539552954 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1d8009af0 con 0x7ff1ec0fe670 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1e97fa700 1 -- 172.21.15.70:0/539552954 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d801c3f0 con 0x7ff1ec0fe670 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.566+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec10e0d0 con 0x7ff1ec0fe670 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.567+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec066490 con 0x7ff1ec0fe670 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.567+0000 7ff1e97fa700 1 -- 172.21.15.70:0/539552954 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff1d8013420 con 0x7ff1ec0fe670 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.568+0000 7ff1e97fa700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff1dc06cc70 0x7ff1dc06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.568+0000 7ff1e97fa700 1 -- 172.21.15.70:0/539552954 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff1d808d4b0 con 0x7ff1ec0fe670 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.568+0000 7ff1f0973700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff1dc06cc70 0x7ff1dc06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.568+0000 7ff1f0973700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff1dc06cc70 0x7ff1dc06f120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0004c40 tx=0x7ff1e0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:05.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.572+0000 7ff1e97fa700 1 -- 172.21.15.70:0/539552954 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff1d805af00 con 0x7ff1ec0fe670 2024-03-23T08:35:05.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.766+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff1ec0640e0 con 0x7ff1ec0fe670 2024-03-23T08:35:05.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.767+0000 7ff1e97fa700 1 -- 172.21.15.70:0/539552954 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff1d805aa90 con 0x7ff1ec0fe670 2024-03-23T08:35:05.767 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-03-23T08:35:05.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff1dc06cc70 msgr2=0x7ff1dc06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff1dc06cc70 0x7ff1dc06f120 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0004c40 tx=0x7ff1e0009040 comp rx=0 tx=0).stop 2024-03-23T08:35:05.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 msgr2=0x7ff1ec113cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 0x7ff1ec113cc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8004890 tx=0x7ff1d8009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:05.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 shutdown_connections 2024-03-23T08:35:05.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ff1dc06cc70 0x7ff1dc06f120 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1ec0fdcd0 0x7ff1ec113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 --2- 172.21.15.70:0/539552954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1ec0fe670 0x7ff1ec113cc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 >> 172.21.15.70:0/539552954 conn(0x7ff1ec0f7710 msgr2=0x7ff1ec0f8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:05.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 shutdown_connections 2024-03-23T08:35:05.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.769+0000 7ff1f2bd7700 1 -- 172.21.15.70:0/539552954 wait complete. 2024-03-23T08:35:05.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.899+0000 7f6625ed4700 1 -- 172.21.15.70:0/2124186215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66201025a0 msgr2=0x7f66200fbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.899+0000 7f6625ed4700 1 --2- 172.21.15.70:0/2124186215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66201025a0 0x7f66200fbc10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6614004940 tx=0x7f6614009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.899+0000 7f6625ed4700 1 -- 172.21.15.70:0/2124186215 shutdown_connections 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.899+0000 7f6625ed4700 1 --2- 172.21.15.70:0/2124186215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66201025a0 0x7f66200fbc10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.899+0000 7f6625ed4700 1 --2- 172.21.15.70:0/2124186215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6620101c00 0x7f6620101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.899+0000 7f6625ed4700 1 -- 172.21.15.70:0/2124186215 >> 172.21.15.70:0/2124186215 conn(0x7f66200f7730 msgr2=0x7f66200f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.899+0000 7f6625ed4700 1 -- 172.21.15.70:0/2124186215 shutdown_connections 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.899+0000 7f6625ed4700 1 -- 172.21.15.70:0/2124186215 wait complete. 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.899+0000 7f6625ed4700 1 Processor -- start 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f6625ed4700 1 -- start start 2024-03-23T08:35:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f6625ed4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6620101c00 0x7f6620112690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f6625ed4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66201025a0 0x7f6620112bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f6625ed4700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6620113220 con 0x7f6620101c00 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f6625ed4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662010c840 con 0x7f66201025a0 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f661f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6620101c00 0x7f6620112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f661f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6620101c00 0x7f6620112690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44696/0 (socket says 172.21.15.70:44696) 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f661f7fe700 1 -- 172.21.15.70:0/258787157 learned_addr learned my addr 172.21.15.70:0/258787157 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f661effd700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66201025a0 0x7f6620112bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f661f7fe700 1 -- 172.21.15.70:0/258787157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66201025a0 msgr2=0x7f6620112bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f661f7fe700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66201025a0 0x7f6620112bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f661f7fe700 1 -- 172.21.15.70:0/258787157 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614005040 con 0x7f6620101c00 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f661f7fe700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6620101c00 0x7f6620112690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6610002a50 tx=0x7f66100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.900+0000 7f6607fff700 1 -- 172.21.15.70:0/258787157 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6610007cc0 con 0x7f6620101c00 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.901+0000 7f6607fff700 1 -- 172.21.15.70:0/258787157 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f661000f040 con 0x7f6620101c00 2024-03-23T08:35:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.901+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662010cb20 con 0x7f6620101c00 2024-03-23T08:35:05.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.901+0000 7f6607fff700 1 -- 172.21.15.70:0/258787157 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f661000d9c0 con 0x7f6620101c00 2024-03-23T08:35:05.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.901+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662010d0b0 con 0x7f6620101c00 2024-03-23T08:35:05.903 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.901+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66200fd350 con 0x7f6620101c00 2024-03-23T08:35:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.902+0000 7f6607fff700 1 -- 172.21.15.70:0/258787157 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f661000db60 con 0x7f6620101c00 2024-03-23T08:35:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.902+0000 7f6607fff700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f660806cc20 0x7f660806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.902+0000 7f6607fff700 1 -- 172.21.15.70:0/258787157 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f661008b6e0 con 0x7f6620101c00 2024-03-23T08:35:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.902+0000 7f661effd700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f660806cc20 0x7f660806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.903+0000 7f661effd700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f660806cc20 0x7f660806f0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6614004890 tx=0x7f6614009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:05.907+0000 7f6607fff700 1 -- 172.21.15.70:0/258787157 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6610059130 con 0x7f6620101c00 2024-03-23T08:35:06.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.142+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f662010dd20 con 0x7f6620101c00 2024-03-23T08:35:06.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.143+0000 7f6607fff700 1 -- 172.21.15.70:0/258787157 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4837 (secure 0 0 0) 0x7f6610058cc0 con 0x7f6620101c00 2024-03-23T08:35:06.143 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:06.143 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44279,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/4097359817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4097359817},{"type":"v1","addr":"172.21.15.193:6825","nonce":4097359817}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/1000528415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1000528415},{"type":"v1","addr":"172.21.15.193:6827","nonce":1000528415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:33:18.781347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:active","state_seq":14,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34408":{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.145+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f660806cc20 msgr2=0x7f660806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.145+0000 7f6625ed4700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f660806cc20 0x7f660806f0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6614004890 tx=0x7f6614009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.145+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6620101c00 msgr2=0x7f6620112690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.145+0000 7f6625ed4700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6620101c00 0x7f6620112690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6610002a50 tx=0x7f66100046b0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.145+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 shutdown_connections 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.145+0000 7f6625ed4700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f660806cc20 0x7f660806f0d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.145+0000 7f6625ed4700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6620101c00 0x7f6620112690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.145+0000 7f6625ed4700 1 --2- 172.21.15.70:0/258787157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66201025a0 0x7f6620112bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.145+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 >> 172.21.15.70:0/258787157 conn(0x7f66200f7730 msgr2=0x7f66200f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.146+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 shutdown_connections 2024-03-23T08:35:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.146+0000 7f6625ed4700 1 -- 172.21.15.70:0/258787157 wait complete. 2024-03-23T08:35:06.147 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 37 2024-03-23T08:35:06.156 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:05 smithi070 ceph-mon[145869]: pgmap v256: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:06.156 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:05 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/362243796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:35:06.156 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:05 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3025552427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-23T08:35:06.156 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:05 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/539552954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-23T08:35:06.157 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 1, 'flags': 50} 2024-03-23T08:35:06.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.276+0000 7f7851423700 1 -- 172.21.15.70:0/1950856593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c102580 msgr2=0x7f784c0fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.276+0000 7f7851423700 1 --2- 172.21.15.70:0/1950856593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c102580 0x7f784c0fbbf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f783c004940 tx=0x7f783c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.276+0000 7f7851423700 1 -- 172.21.15.70:0/1950856593 shutdown_connections 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.276+0000 7f7851423700 1 --2- 172.21.15.70:0/1950856593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c102580 0x7f784c0fbbf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.276+0000 7f7851423700 1 --2- 172.21.15.70:0/1950856593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f784c101be0 0x7f784c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.276+0000 7f7851423700 1 -- 172.21.15.70:0/1950856593 >> 172.21.15.70:0/1950856593 conn(0x7f784c072ad0 msgr2=0x7f784c072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.276+0000 7f7851423700 1 -- 172.21.15.70:0/1950856593 shutdown_connections 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.276+0000 7f7851423700 1 -- 172.21.15.70:0/1950856593 wait complete. 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f7851423700 1 Processor -- start 2024-03-23T08:35:06.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f7851423700 1 -- start start 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f7851423700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c101be0 0x7f784c10e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f7851423700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f784c102580 0x7f784c10edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f7851423700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c10f330 con 0x7f784c101be0 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f7851423700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c10f4a0 con 0x7f784c102580 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f784affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c101be0 0x7f784c10e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f784affd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c101be0 0x7f784c10e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44712/0 (socket says 172.21.15.70:44712) 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f784affd700 1 -- 172.21.15.70:0/522383641 learned_addr learned my addr 172.21.15.70:0/522383641 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f784a7fc700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f784c102580 0x7f784c10edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f784affd700 1 -- 172.21.15.70:0/522383641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f784c102580 msgr2=0x7f784c10edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f784affd700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f784c102580 0x7f784c10edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f784affd700 1 -- 172.21.15.70:0/522383641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c005040 con 0x7f784c101be0 2024-03-23T08:35:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f784affd700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c101be0 0x7f784c10e8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f784000c830 tx=0x7f7840004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:06.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.277+0000 7f78337fe700 1 -- 172.21.15.70:0/522383641 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784000a760 con 0x7f784c101be0 2024-03-23T08:35:06.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.278+0000 7f78337fe700 1 -- 172.21.15.70:0/522383641 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f784000ad70 con 0x7f784c101be0 2024-03-23T08:35:06.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.278+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784c10f730 con 0x7f784c101be0 2024-03-23T08:35:06.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.278+0000 7f78337fe700 1 -- 172.21.15.70:0/522383641 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78400113a0 con 0x7f784c101be0 2024-03-23T08:35:06.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.278+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c117cd0 con 0x7f784c101be0 2024-03-23T08:35:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.279+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c0fd330 con 0x7f784c101be0 2024-03-23T08:35:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.279+0000 7f78337fe700 1 -- 172.21.15.70:0/522383641 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7840011500 con 0x7f784c101be0 2024-03-23T08:35:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.279+0000 7f78337fe700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f783406cc20 0x7f783406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.280+0000 7f784a7fc700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f783406cc20 0x7f783406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.280+0000 7f78337fe700 1 -- 172.21.15.70:0/522383641 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f784008c0e0 con 0x7f784c101be0 2024-03-23T08:35:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.280+0000 7f784a7fc700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f783406cc20 0x7f783406f0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f783c000c00 tx=0x7f783c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:06.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.284+0000 7f78337fe700 1 -- 172.21.15.70:0/522383641 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7840056120 con 0x7f784c101be0 2024-03-23T08:35:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:05 smithi193 ceph-mon[108533]: pgmap v256: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:05 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/362243796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:35:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:05 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3025552427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-23T08:35:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:05 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/539552954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-23T08:35:06.519 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.519+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f784c065450 con 0x7f784c101be0 2024-03-23T08:35:06.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.520+0000 7f78337fe700 1 -- 172.21.15.70:0/522383641 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v37) v1 ==== 107+0+4050 (secure 0 0 0) 0x7f7840059740 con 0x7f784c101be0 2024-03-23T08:35:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:06.520 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1586743298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1586743298},{"type":"v1","addr":"172.21.15.70:6827","nonce":1586743298}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:04.587715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14462":{"gid":14462,"name":"cephfs.smithi070.aymzqb","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6829/2658800706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2658800706},{"type":"v1","addr":"172.21.15.70:6829","nonce":2658800706}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f783406cc20 msgr2=0x7f783406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f783406cc20 0x7f783406f0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f783c000c00 tx=0x7f783c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c101be0 msgr2=0x7f784c10e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c101be0 0x7f784c10e8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f784000c830 tx=0x7f7840004470 comp rx=0 tx=0).stop 2024-03-23T08:35:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 shutdown_connections 2024-03-23T08:35:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f783406cc20 0x7f783406f0d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784c101be0 0x7f784c10e8b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 --2- 172.21.15.70:0/522383641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f784c102580 0x7f784c10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 >> 172.21.15.70:0/522383641 conn(0x7f784c072ad0 msgr2=0x7f784c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 shutdown_connections 2024-03-23T08:35:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.522+0000 7f7851423700 1 -- 172.21.15.70:0/522383641 wait complete. 2024-03-23T08:35:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 18 2024-03-23T08:35:06.534 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-03-23T08:35:06.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-23T08:35:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.655+0000 7f6064812700 1 -- 172.21.15.70:0/1659087171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 msgr2=0x7f6060064e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.655+0000 7f6064812700 1 --2- 172.21.15.70:0/1659087171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 0x7f6060064e40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6054004940 tx=0x7f6054009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.655+0000 7f6064812700 1 -- 172.21.15.70:0/1659087171 shutdown_connections 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.655+0000 7f6064812700 1 --2- 172.21.15.70:0/1659087171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 0x7f6060064e40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.655+0000 7f6064812700 1 --2- 172.21.15.70:0/1659087171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6060102ce0 0x7f60601030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.655+0000 7f6064812700 1 -- 172.21.15.70:0/1659087171 >> 172.21.15.70:0/1659087171 conn(0x7f6060072ad0 msgr2=0x7f6060072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.655+0000 7f6064812700 1 -- 172.21.15.70:0/1659087171 shutdown_connections 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.655+0000 7f6064812700 1 -- 172.21.15.70:0/1659087171 wait complete. 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f6064812700 1 Processor -- start 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f6064812700 1 -- start start 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f6064812700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6060102ce0 0x7f6060113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f6064812700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 0x7f6060113cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f6064812700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6060114320 con 0x7f6060103680 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f6064812700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606010d920 con 0x7f6060102ce0 2024-03-23T08:35:06.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f605dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 0x7f6060113cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f605dd9b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 0x7f6060113cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44720/0 (socket says 172.21.15.70:44720) 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f605dd9b700 1 -- 172.21.15.70:0/1797546819 learned_addr learned my addr 172.21.15.70:0/1797546819 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.656+0000 7f605e59c700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6060102ce0 0x7f6060113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.657+0000 7f605dd9b700 1 -- 172.21.15.70:0/1797546819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6060102ce0 msgr2=0x7f6060113790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.657+0000 7f605dd9b700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6060102ce0 0x7f6060113790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.657+0000 7f605dd9b700 1 -- 172.21.15.70:0/1797546819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6054005040 con 0x7f6060103680 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.657+0000 7f605dd9b700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 0x7f6060113cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f60540093b0 tx=0x7f6054009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.657+0000 7f604effd700 1 -- 172.21.15.70:0/1797546819 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605400e040 con 0x7f6060103680 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.657+0000 7f604effd700 1 -- 172.21.15.70:0/1797546819 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6054013420 con 0x7f6060103680 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.657+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606010dba0 con 0x7f6060103680 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.657+0000 7f604effd700 1 -- 172.21.15.70:0/1797546819 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605401c630 con 0x7f6060103680 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.657+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606010e160 con 0x7f6060103680 2024-03-23T08:35:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.658+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60600fe430 con 0x7f6060103680 2024-03-23T08:35:06.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.658+0000 7f604effd700 1 -- 172.21.15.70:0/1797546819 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f60540139e0 con 0x7f6060103680 2024-03-23T08:35:06.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.659+0000 7f604effd700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f604806cc20 0x7f604806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:06.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.659+0000 7f604effd700 1 -- 172.21.15.70:0/1797546819 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f605408d050 con 0x7f6060103680 2024-03-23T08:35:06.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.659+0000 7f605e59c700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f604806cc20 0x7f604806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:06.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.659+0000 7f605e59c700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f604806cc20 0x7f604806f0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6050007fd0 tx=0x7f6050009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:06.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.663+0000 7f604effd700 1 -- 172.21.15.70:0/1797546819 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f605405ac90 con 0x7f6060103680 2024-03-23T08:35:06.901 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.901+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f6060109c80 con 0x7f6060103680 2024-03-23T08:35:06.902 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.902+0000 7f604effd700 1 -- 172.21.15.70:0/1797546819 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v37) v1 ==== 107+0+4832 (secure 0 0 0) 0x7f605405a820 con 0x7f6060103680 2024-03-23T08:35:06.902 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:06.903 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1586743298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1586743298},{"type":"v1","addr":"172.21.15.70:6827","nonce":1586743298}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:04.587715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14462":{"gid":14462,"name":"cephfs.smithi070.aymzqb","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6829/2658800706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2658800706},{"type":"v1","addr":"172.21.15.70:6829","nonce":2658800706}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:06.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f604806cc20 msgr2=0x7f604806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f604806cc20 0x7f604806f0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6050007fd0 tx=0x7f6050009040 comp rx=0 tx=0).stop 2024-03-23T08:35:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 msgr2=0x7f6060113cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 0x7f6060113cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f60540093b0 tx=0x7f6054009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 shutdown_connections 2024-03-23T08:35:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f604806cc20 0x7f604806f0d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6060102ce0 0x7f6060113790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 --2- 172.21.15.70:0/1797546819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6060103680 0x7f6060113cd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 >> 172.21.15.70:0/1797546819 conn(0x7f6060072ad0 msgr2=0x7f6060105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 shutdown_connections 2024-03-23T08:35:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:06.904+0000 7f6064812700 1 -- 172.21.15.70:0/1797546819 wait complete. 2024-03-23T08:35:06.906 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 19 2024-03-23T08:35:06.916 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-03-23T08:35:06.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-23T08:35:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.035+0000 7f698f6c9700 1 -- 172.21.15.70:0/4018257321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988103660 msgr2=0x7f69880fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.035+0000 7f698f6c9700 1 --2- 172.21.15.70:0/4018257321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988103660 0x7f69880fccd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6978004940 tx=0x7f6978009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.035+0000 7f698f6c9700 1 -- 172.21.15.70:0/4018257321 shutdown_connections 2024-03-23T08:35:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.035+0000 7f698f6c9700 1 --2- 172.21.15.70:0/4018257321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988103660 0x7f69880fccd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 --2- 172.21.15.70:0/4018257321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6988102cc0 0x7f6988103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 -- 172.21.15.70:0/4018257321 >> 172.21.15.70:0/4018257321 conn(0x7f69880f8830 msgr2=0x7f69880fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 -- 172.21.15.70:0/4018257321 shutdown_connections 2024-03-23T08:35:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 -- 172.21.15.70:0/4018257321 wait complete. 2024-03-23T08:35:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 Processor -- start 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 -- start start 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988102cc0 0x7f6988113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6988103660 0x7f6988113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6988114380 con 0x7f6988102cc0 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698f6c9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698810d8f0 con 0x7f6988103660 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698d465700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988102cc0 0x7f6988113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698d465700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988102cc0 0x7f6988113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44740/0 (socket says 172.21.15.70:44740) 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.036+0000 7f698d465700 1 -- 172.21.15.70:0/3595606537 learned_addr learned my addr 172.21.15.70:0/3595606537 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f698cc64700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6988103660 0x7f6988113ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f698d465700 1 -- 172.21.15.70:0/3595606537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6988103660 msgr2=0x7f6988113ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f698d465700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6988103660 0x7f6988113ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f698d465700 1 -- 172.21.15.70:0/3595606537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6978005040 con 0x7f6988102cc0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f698d465700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988102cc0 0x7f6988113760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f698400b790 tx=0x7f6984009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f697dffb700 1 -- 172.21.15.70:0/3595606537 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f698400f6f0 con 0x7f6988102cc0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f697dffb700 1 -- 172.21.15.70:0/3595606537 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f698400fd00 con 0x7f6988102cc0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698810dbd0 con 0x7f6988102cc0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f697dffb700 1 -- 172.21.15.70:0/3595606537 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69840103a0 con 0x7f6988102cc0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.037+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698810e160 con 0x7f6988102cc0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.038+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69880fe410 con 0x7f6988102cc0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.038+0000 7f697dffb700 1 -- 172.21.15.70:0/3595606537 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f698400f850 con 0x7f6988102cc0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.039+0000 7f697dffb700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f697406cc20 0x7f697406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.039+0000 7f697dffb700 1 -- 172.21.15.70:0/3595606537 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f698408beb0 con 0x7f6988102cc0 2024-03-23T08:35:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.039+0000 7f698cc64700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f697406cc20 0x7f697406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.039+0000 7f698cc64700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f697406cc20 0x7f697406f0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f698810ebd0 tx=0x7f6978009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.043+0000 7f697dffb700 1 -- 172.21.15.70:0/3595606537 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6984059980 con 0x7f6988102cc0 2024-03-23T08:35:07.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:06 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/258787157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T08:35:07.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:06 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/522383641' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-23T08:35:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.284+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f6988109cd0 con 0x7f6988102cc0 2024-03-23T08:35:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.285+0000 7f697dffb700 1 -- 172.21.15.70:0/3595606537 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v37) v1 ==== 107+0+4053 (secure 0 0 0) 0x7f6984059510 con 0x7f6988102cc0 2024-03-23T08:35:07.285 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:07.285 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:04.587715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14462":{"gid":14462,"name":"cephfs.smithi070.aymzqb","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6829/2658800706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2658800706},{"type":"v1","addr":"172.21.15.70:6829","nonce":2658800706}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f697406cc20 msgr2=0x7f697406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f697406cc20 0x7f697406f0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f698810ebd0 tx=0x7f6978009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988102cc0 msgr2=0x7f6988113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988102cc0 0x7f6988113760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f698400b790 tx=0x7f6984009250 comp rx=0 tx=0).stop 2024-03-23T08:35:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 shutdown_connections 2024-03-23T08:35:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f697406cc20 0x7f697406f0d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6988102cc0 0x7f6988113760 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 --2- 172.21.15.70:0/3595606537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6988103660 0x7f6988113ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 >> 172.21.15.70:0/3595606537 conn(0x7f69880f8830 msgr2=0x7f69880f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 shutdown_connections 2024-03-23T08:35:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.287+0000 7f698f6c9700 1 -- 172.21.15.70:0/3595606537 wait complete. 2024-03-23T08:35:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2024-03-23T08:35:07.299 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-03-23T08:35:07.299 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-23T08:35:07.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:06 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/258787157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-23T08:35:07.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:06 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/522383641' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-23T08:35:07.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 -- 172.21.15.70:0/1598607771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 msgr2=0x7faf740fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 --2- 172.21.15.70:0/1598607771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 0x7faf740fbbf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faf64004940 tx=0x7faf64009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:07.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 -- 172.21.15.70:0/1598607771 shutdown_connections 2024-03-23T08:35:07.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 --2- 172.21.15.70:0/1598607771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 0x7faf740fbbf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 --2- 172.21.15.70:0/1598607771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf74101be0 0x7faf74101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 -- 172.21.15.70:0/1598607771 >> 172.21.15.70:0/1598607771 conn(0x7faf74072ad0 msgr2=0x7faf74072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:07.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 -- 172.21.15.70:0/1598607771 shutdown_connections 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 -- 172.21.15.70:0/1598607771 wait complete. 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 Processor -- start 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.417+0000 7faf79030700 1 -- start start 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf79030700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf74101be0 0x7faf741126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf79030700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 0x7faf74112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf79030700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf74113230 con 0x7faf74102580 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf79030700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7410c7f0 con 0x7faf74101be0 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf7259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 0x7faf74112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf7259c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 0x7faf74112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44756/0 (socket says 172.21.15.70:44756) 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf7259c700 1 -- 172.21.15.70:0/2917846705 learned_addr learned my addr 172.21.15.70:0/2917846705 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf72d9d700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf74101be0 0x7faf741126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf7259c700 1 -- 172.21.15.70:0/2917846705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf74101be0 msgr2=0x7faf741126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf7259c700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf74101be0 0x7faf741126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf7259c700 1 -- 172.21.15.70:0/2917846705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf64005040 con 0x7faf74102580 2024-03-23T08:35:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf7259c700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 0x7faf74112be0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faf6400b3b0 tx=0x7faf64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:07.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf5b7fe700 1 -- 172.21.15.70:0/2917846705 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf6400e040 con 0x7faf74102580 2024-03-23T08:35:07.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf5b7fe700 1 -- 172.21.15.70:0/2917846705 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf6400bc90 con 0x7faf74102580 2024-03-23T08:35:07.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7410ca70 con 0x7faf74102580 2024-03-23T08:35:07.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf5b7fe700 1 -- 172.21.15.70:0/2917846705 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf64013660 con 0x7faf74102580 2024-03-23T08:35:07.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.418+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7410cf50 con 0x7faf74102580 2024-03-23T08:35:07.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.419+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf74112330 con 0x7faf74102580 2024-03-23T08:35:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.420+0000 7faf5b7fe700 1 -- 172.21.15.70:0/2917846705 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7faf64013920 con 0x7faf74102580 2024-03-23T08:35:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.420+0000 7faf5b7fe700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7faf5c06cc20 0x7faf5c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.420+0000 7faf5b7fe700 1 -- 172.21.15.70:0/2917846705 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faf6408fef0 con 0x7faf74102580 2024-03-23T08:35:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.420+0000 7faf72d9d700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7faf5c06cc20 0x7faf5c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.421+0000 7faf72d9d700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7faf5c06cc20 0x7faf5c06f0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faf68004f40 tx=0x7faf6800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.425+0000 7faf5b7fe700 1 -- 172.21.15.70:0/2917846705 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faf6405d940 con 0x7faf74102580 2024-03-23T08:35:07.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.663+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7faf740fd330 con 0x7faf74102580 2024-03-23T08:35:07.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.664+0000 7faf5b7fe700 1 -- 172.21.15.70:0/2917846705 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v37) v1 ==== 107+0+4832 (secure 0 0 0) 0x7faf6405d4d0 con 0x7faf74102580 2024-03-23T08:35:07.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:07.664 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:04.587715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14462":{"gid":14462,"name":"cephfs.smithi070.aymzqb","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6829/2658800706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2658800706},{"type":"v1","addr":"172.21.15.70:6829","nonce":2658800706}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.666+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7faf5c06cc20 msgr2=0x7faf5c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.666+0000 7faf79030700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7faf5c06cc20 0x7faf5c06f0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faf68004f40 tx=0x7faf6800b040 comp rx=0 tx=0).stop 2024-03-23T08:35:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.666+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 msgr2=0x7faf74112be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.666+0000 7faf79030700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 0x7faf74112be0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faf6400b3b0 tx=0x7faf64009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.666+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 shutdown_connections 2024-03-23T08:35:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.666+0000 7faf79030700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7faf5c06cc20 0x7faf5c06f0d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.667+0000 7faf79030700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf74101be0 0x7faf741126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.667+0000 7faf79030700 1 --2- 172.21.15.70:0/2917846705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf74102580 0x7faf74112be0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.667+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 >> 172.21.15.70:0/2917846705 conn(0x7faf74072ad0 msgr2=0x7faf74104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:07.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.667+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 shutdown_connections 2024-03-23T08:35:07.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.667+0000 7faf79030700 1 -- 172.21.15.70:0/2917846705 wait complete. 2024-03-23T08:35:07.668 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 21 2024-03-23T08:35:07.679 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-03-23T08:35:07.679 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-23T08:35:07.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.798+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/1665859906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 msgr2=0x7fb3a0101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.798+0000 7fb3a5dd1700 1 --2- 172.21.15.70:0/1665859906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 0x7fb3a0101fb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb390004940 tx=0x7fb390009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.798+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/1665859906 shutdown_connections 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.798+0000 7fb3a5dd1700 1 --2- 172.21.15.70:0/1665859906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3a0102580 0x7fb3a00fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.798+0000 7fb3a5dd1700 1 --2- 172.21.15.70:0/1665859906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 0x7fb3a0101fb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.798+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/1665859906 >> 172.21.15.70:0/1665859906 conn(0x7fb3a0072ad0 msgr2=0x7fb3a0072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/1665859906 shutdown_connections 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/1665859906 wait complete. 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb3a5dd1700 1 Processor -- start 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb3a5dd1700 1 -- start start 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb3a5dd1700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 0x7fb3a01136e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb3a5dd1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3a0102580 0x7fb3a0113c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb3a5dd1700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a0114270 con 0x7fb3a0101be0 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb3a5dd1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a010d890 con 0x7fb3a0102580 2024-03-23T08:35:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb39f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 0x7fb3a01136e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb39f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 0x7fb3a01136e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44782/0 (socket says 172.21.15.70:44782) 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.799+0000 7fb39f7fe700 1 -- 172.21.15.70:0/3400906564 learned_addr learned my addr 172.21.15.70:0/3400906564 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb39effd700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3a0102580 0x7fb3a0113c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb39f7fe700 1 -- 172.21.15.70:0/3400906564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3a0102580 msgr2=0x7fb3a0113c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb39f7fe700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3a0102580 0x7fb3a0113c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb39f7fe700 1 -- 172.21.15.70:0/3400906564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390005040 con 0x7fb3a0101be0 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb39f7fe700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 0x7fb3a01136e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb390008d80 tx=0x7fb390009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb387fff700 1 -- 172.21.15.70:0/3400906564 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb39000e030 con 0x7fb3a0101be0 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a010db10 con 0x7fb3a0101be0 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb387fff700 1 -- 172.21.15.70:0/3400906564 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb390013de0 con 0x7fb3a0101be0 2024-03-23T08:35:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb387fff700 1 -- 172.21.15.70:0/3400906564 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb39001cd60 con 0x7fb3a0101be0 2024-03-23T08:35:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.800+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a010e0d0 con 0x7fb3a0101be0 2024-03-23T08:35:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.801+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a00fd330 con 0x7fb3a0101be0 2024-03-23T08:35:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.801+0000 7fb387fff700 1 -- 172.21.15.70:0/3400906564 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb39001c590 con 0x7fb3a0101be0 2024-03-23T08:35:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.802+0000 7fb387fff700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb38806cc20 0x7fb38806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.802+0000 7fb387fff700 1 -- 172.21.15.70:0/3400906564 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb39008d930 con 0x7fb3a0101be0 2024-03-23T08:35:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.802+0000 7fb39effd700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb38806cc20 0x7fb38806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.802+0000 7fb39effd700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb38806cc20 0x7fb38806f0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb394004600 tx=0x7fb39400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:07.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:07.806+0000 7fb387fff700 1 -- 172.21.15.70:0/3400906564 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb39005b400 con 0x7fb3a0101be0 2024-03-23T08:35:07.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:07 smithi070 ceph-mon[145869]: pgmap v257: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:07.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:07 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1797546819' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-23T08:35:07.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:07 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3595606537' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-23T08:35:07.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:07 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2917846705' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-23T08:35:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.042+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fb3a010ed70 con 0x7fb3a0101be0 2024-03-23T08:35:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.043+0000 7fb387fff700 1 -- 172.21.15.70:0/3400906564 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v37) v1 ==== 107+0+4043 (secure 0 0 0) 0x7fb39005af90 con 0x7fb3a0101be0 2024-03-23T08:35:08.043 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:08.043 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:27.287167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb38806cc20 msgr2=0x7fb38806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb38806cc20 0x7fb38806f0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb394004600 tx=0x7fb39400b040 comp rx=0 tx=0).stop 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 msgr2=0x7fb3a01136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 0x7fb3a01136e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb390008d80 tx=0x7fb390009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 shutdown_connections 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb38806cc20 0x7fb38806f0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3a0101be0 0x7fb3a01136e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 --2- 172.21.15.70:0/3400906564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3a0102580 0x7fb3a0113c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 >> 172.21.15.70:0/3400906564 conn(0x7fb3a0072ad0 msgr2=0x7fb3a0064dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 shutdown_connections 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.045+0000 7fb3a5dd1700 1 -- 172.21.15.70:0/3400906564 wait complete. 2024-03-23T08:35:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 22 2024-03-23T08:35:08.057 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-03-23T08:35:08.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-23T08:35:08.179 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.176+0000 7fb56bec8700 1 -- 172.21.15.70:0/1180501809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564102cc0 msgr2=0x7fb564103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.176+0000 7fb56bec8700 1 --2- 172.21.15.70:0/1180501809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564102cc0 0x7fb564103090 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb558004940 tx=0x7fb558009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:08.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.176+0000 7fb56bec8700 1 -- 172.21.15.70:0/1180501809 shutdown_connections 2024-03-23T08:35:08.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.176+0000 7fb56bec8700 1 --2- 172.21.15.70:0/1180501809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb564103660 0x7fb5640fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.176+0000 7fb56bec8700 1 --2- 172.21.15.70:0/1180501809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564102cc0 0x7fb564103090 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.176+0000 7fb56bec8700 1 -- 172.21.15.70:0/1180501809 >> 172.21.15.70:0/1180501809 conn(0x7fb5640f8830 msgr2=0x7fb5640fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:08.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.176+0000 7fb56bec8700 1 -- 172.21.15.70:0/1180501809 shutdown_connections 2024-03-23T08:35:08.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.176+0000 7fb56bec8700 1 -- 172.21.15.70:0/1180501809 wait complete. 2024-03-23T08:35:08.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb56bec8700 1 Processor -- start 2024-03-23T08:35:08.180 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb56bec8700 1 -- start start 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb56bec8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb564102cc0 0x7fb5640702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb56bec8700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564103660 0x7fb564070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb56bec8700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb564072210 con 0x7fb564103660 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb56bec8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb564070d40 con 0x7fb564102cc0 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb569463700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564103660 0x7fb564070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb569463700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564103660 0x7fb564070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44800/0 (socket says 172.21.15.70:44800) 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb569463700 1 -- 172.21.15.70:0/494037058 learned_addr learned my addr 172.21.15.70:0/494037058 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb569c64700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb564102cc0 0x7fb5640702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb569463700 1 -- 172.21.15.70:0/494037058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb564102cc0 msgr2=0x7fb5640702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb569463700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb564102cc0 0x7fb5640702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb569463700 1 -- 172.21.15.70:0/494037058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb558005040 con 0x7fb564103660 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb569c64700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb564102cc0 0x7fb5640702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb569463700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564103660 0x7fb564070800 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb560002a50 tx=0x7fb560004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb5567fc700 1 -- 172.21.15.70:0/494037058 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb560007cc0 con 0x7fb564103660 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb5567fc700 1 -- 172.21.15.70:0/494037058 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb56000f040 con 0x7fb564103660 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb5567fc700 1 -- 172.21.15.70:0/494037058 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56000d990 con 0x7fb564103660 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.177+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb564071020 con 0x7fb564103660 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.178+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb564119f50 con 0x7fb564103660 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.178+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5640fe410 con 0x7fb564103660 2024-03-23T08:35:08.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.179+0000 7fb5567fc700 1 -- 172.21.15.70:0/494037058 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb560009710 con 0x7fb564103660 2024-03-23T08:35:08.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.179+0000 7fb5567fc700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb55006cc20 0x7fb55006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:08.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.179+0000 7fb5567fc700 1 -- 172.21.15.70:0/494037058 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb56008bd70 con 0x7fb564103660 2024-03-23T08:35:08.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.179+0000 7fb569c64700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb55006cc20 0x7fb55006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:08.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.180+0000 7fb569c64700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb55006cc20 0x7fb55006f0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb558004560 tx=0x7fb558009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:08.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.184+0000 7fb5567fc700 1 -- 172.21.15.70:0/494037058 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb560059840 con 0x7fb564103660 2024-03-23T08:35:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:07 smithi193 ceph-mon[108533]: pgmap v257: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:07 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1797546819' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-23T08:35:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:07 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3595606537' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-23T08:35:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:07 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2917846705' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-23T08:35:08.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.419+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fb5640640e0 con 0x7fb564103660 2024-03-23T08:35:08.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.420+0000 7fb5567fc700 1 -- 172.21.15.70:0/494037058 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v37) v1 ==== 107+0+4054 (secure 0 0 0) 0x7fb5600593d0 con 0x7fb564103660 2024-03-23T08:35:08.420 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:08.421 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:27.293909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":0,"incarnation":23,"state":"up:replay","state_seq":3,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:08.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb55006cc20 msgr2=0x7fb55006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb55006cc20 0x7fb55006f0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb558004560 tx=0x7fb558009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:08.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564103660 msgr2=0x7fb564070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564103660 0x7fb564070800 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb560002a50 tx=0x7fb560004600 comp rx=0 tx=0).stop 2024-03-23T08:35:08.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 shutdown_connections 2024-03-23T08:35:08.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb55006cc20 0x7fb55006f0d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb564102cc0 0x7fb5640702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 --2- 172.21.15.70:0/494037058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb564103660 0x7fb564070800 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 >> 172.21.15.70:0/494037058 conn(0x7fb5640f8830 msgr2=0x7fb5640f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:08.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 shutdown_connections 2024-03-23T08:35:08.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.422+0000 7fb56bec8700 1 -- 172.21.15.70:0/494037058 wait complete. 2024-03-23T08:35:08.424 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 23 2024-03-23T08:35:08.434 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-03-23T08:35:08.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-23T08:35:08.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.553+0000 7f2d124bc700 1 -- 172.21.15.70:0/418919782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c101be0 msgr2=0x7f2d0c101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.553+0000 7f2d124bc700 1 --2- 172.21.15.70:0/418919782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c101be0 0x7f2d0c101fb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc004940 tx=0x7f2cfc009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:08.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.553+0000 7f2d124bc700 1 -- 172.21.15.70:0/418919782 shutdown_connections 2024-03-23T08:35:08.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.553+0000 7f2d124bc700 1 --2- 172.21.15.70:0/418919782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d0c102580 0x7f2d0c0fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.553+0000 7f2d124bc700 1 --2- 172.21.15.70:0/418919782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c101be0 0x7f2d0c101fb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.553+0000 7f2d124bc700 1 -- 172.21.15.70:0/418919782 >> 172.21.15.70:0/418919782 conn(0x7f2d0c072ad0 msgr2=0x7f2d0c072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:08.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.553+0000 7f2d124bc700 1 -- 172.21.15.70:0/418919782 shutdown_connections 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.553+0000 7f2d124bc700 1 -- 172.21.15.70:0/418919782 wait complete. 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d124bc700 1 Processor -- start 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d124bc700 1 -- start start 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d124bc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d0c101be0 0x7f2d0c112690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d124bc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c102580 0x7f2d0c112bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d124bc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c113220 con 0x7f2d0c102580 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d124bc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c10c850 con 0x7f2d0c101be0 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d0b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c102580 0x7f2d0c112bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d0b7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c102580 0x7f2d0c112bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59408/0 (socket says 172.21.15.70:59408) 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d0b7fe700 1 -- 172.21.15.70:0/2258154609 learned_addr learned my addr 172.21.15.70:0/2258154609 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d0bfff700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d0c101be0 0x7f2d0c112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d0b7fe700 1 -- 172.21.15.70:0/2258154609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d0c101be0 msgr2=0x7f2d0c112690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d0b7fe700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d0c101be0 0x7f2d0c112690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d0b7fe700 1 -- 172.21.15.70:0/2258154609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc005040 con 0x7f2d0c102580 2024-03-23T08:35:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.554+0000 7f2d0b7fe700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c102580 0x7f2d0c112bd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c800 tx=0x7f2cf4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.555+0000 7f2d08ff9700 1 -- 172.21.15.70:0/2258154609 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cf400a760 con 0x7f2d0c102580 2024-03-23T08:35:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.555+0000 7f2d08ff9700 1 -- 172.21.15.70:0/2258154609 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cf400ad70 con 0x7f2d0c102580 2024-03-23T08:35:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.555+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c10cb30 con 0x7f2d0c102580 2024-03-23T08:35:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.555+0000 7f2d08ff9700 1 -- 172.21.15.70:0/2258154609 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cf40113a0 con 0x7f2d0c102580 2024-03-23T08:35:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.555+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c10d070 con 0x7f2d0c102580 2024-03-23T08:35:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.556+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c112320 con 0x7f2d0c102580 2024-03-23T08:35:08.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.557+0000 7f2d08ff9700 1 -- 172.21.15.70:0/2258154609 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2cf4012400 con 0x7f2d0c102580 2024-03-23T08:35:08.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.557+0000 7f2d08ff9700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2cf806cc20 0x7f2cf806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:08.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.557+0000 7f2d08ff9700 1 -- 172.21.15.70:0/2258154609 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2cf408ccd0 con 0x7f2d0c102580 2024-03-23T08:35:08.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.557+0000 7f2d0bfff700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2cf806cc20 0x7f2cf806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:08.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.558+0000 7f2d0bfff700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2cf806cc20 0x7f2cf806f0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00b3b0 tx=0x7f2cfc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:08.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.561+0000 7f2d08ff9700 1 -- 172.21.15.70:0/2258154609 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2cf405a7a0 con 0x7f2d0c102580 2024-03-23T08:35:08.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.799+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f2d0c0fd330 con 0x7f2d0c102580 2024-03-23T08:35:08.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.800+0000 7f2d08ff9700 1 -- 172.21.15.70:0/2258154609 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v37) v1 ==== 107+0+4835 (secure 0 0 0) 0x7f2cf405a330 con 0x7f2d0c102580 2024-03-23T08:35:08.800 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:08.800 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:36.849871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":0,"incarnation":23,"state":"up:reconnect","state_seq":180,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2cf806cc20 msgr2=0x7f2cf806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2cf806cc20 0x7f2cf806f0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00b3b0 tx=0x7f2cfc009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c102580 msgr2=0x7f2d0c112bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c102580 0x7f2d0c112bd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c800 tx=0x7f2cf4004470 comp rx=0 tx=0).stop 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 shutdown_connections 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f2cf806cc20 0x7f2cf806f0d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d0c101be0 0x7f2d0c112690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 --2- 172.21.15.70:0/2258154609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d0c102580 0x7f2d0c112bd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 >> 172.21.15.70:0/2258154609 conn(0x7f2d0c072ad0 msgr2=0x7f2d0c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 shutdown_connections 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.802+0000 7f2d124bc700 1 -- 172.21.15.70:0/2258154609 wait complete. 2024-03-23T08:35:08.803 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2024-03-23T08:35:08.814 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-03-23T08:35:08.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-23T08:35:08.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.934+0000 7fe717e90700 1 -- 172.21.15.70:0/3695409423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 msgr2=0x7fe7100fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.934+0000 7fe717e90700 1 --2- 172.21.15.70:0/3695409423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 0x7fe7100fbbf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe704004940 tx=0x7fe704009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:08.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.934+0000 7fe717e90700 1 -- 172.21.15.70:0/3695409423 shutdown_connections 2024-03-23T08:35:08.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.934+0000 7fe717e90700 1 --2- 172.21.15.70:0/3695409423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 0x7fe7100fbbf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.934+0000 7fe717e90700 1 --2- 172.21.15.70:0/3695409423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe710101be0 0x7fe710101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.934+0000 7fe717e90700 1 -- 172.21.15.70:0/3695409423 >> 172.21.15.70:0/3695409423 conn(0x7fe710072ad0 msgr2=0x7fe710072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:08.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.934+0000 7fe717e90700 1 -- 172.21.15.70:0/3695409423 shutdown_connections 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.934+0000 7fe717e90700 1 -- 172.21.15.70:0/3695409423 wait complete. 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe717e90700 1 Processor -- start 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe717e90700 1 -- start start 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe717e90700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe710101be0 0x7fe7101126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe717e90700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 0x7fe710112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe717e90700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710113230 con 0x7fe710102580 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe717e90700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71010c7f0 con 0x7fe710101be0 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe71542b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 0x7fe710112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe71542b700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 0x7fe710112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59416/0 (socket says 172.21.15.70:59416) 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe71542b700 1 -- 172.21.15.70:0/1473293074 learned_addr learned my addr 172.21.15.70:0/1473293074 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe715c2c700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe710101be0 0x7fe7101126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe71542b700 1 -- 172.21.15.70:0/1473293074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe710101be0 msgr2=0x7fe7101126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:08.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe71542b700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe710101be0 0x7fe7101126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:08.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe71542b700 1 -- 172.21.15.70:0/1473293074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704005040 con 0x7fe710102580 2024-03-23T08:35:08.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.935+0000 7fe715c2c700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe710101be0 0x7fe7101126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:35:08.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.936+0000 7fe71542b700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 0x7fe710112be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe704009390 tx=0x7fe704009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:08.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.936+0000 7fe70e7fc700 1 -- 172.21.15.70:0/1473293074 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe70400e040 con 0x7fe710102580 2024-03-23T08:35:08.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.936+0000 7fe70e7fc700 1 -- 172.21.15.70:0/1473293074 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe704013420 con 0x7fe710102580 2024-03-23T08:35:08.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.936+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71010ca70 con 0x7fe710102580 2024-03-23T08:35:08.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.936+0000 7fe70e7fc700 1 -- 172.21.15.70:0/1473293074 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe70401c630 con 0x7fe710102580 2024-03-23T08:35:08.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.936+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71010cfb0 con 0x7fe710102580 2024-03-23T08:35:08.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.937+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe710112330 con 0x7fe710102580 2024-03-23T08:35:08.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.937+0000 7fe70e7fc700 1 -- 172.21.15.70:0/1473293074 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fe7040139e0 con 0x7fe710102580 2024-03-23T08:35:08.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.937+0000 7fe70e7fc700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe6f806cc20 0x7fe6f806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:08.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.937+0000 7fe70e7fc700 1 -- 172.21.15.70:0/1473293074 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe70408d1e0 con 0x7fe710102580 2024-03-23T08:35:08.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.938+0000 7fe715c2c700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe6f806cc20 0x7fe6f806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:08.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.938+0000 7fe715c2c700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe6f806cc20 0x7fe6f806f0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe700001250 tx=0x7fe70000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:08.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:08.942+0000 7fe70e7fc700 1 -- 172.21.15.70:0/1473293074 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe70405ad70 con 0x7fe710102580 2024-03-23T08:35:09.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.182+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fe7100fd330 con 0x7fe710102580 2024-03-23T08:35:09.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.182+0000 7fe70e7fc700 1 -- 172.21.15.70:0/1473293074 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v37) v1 ==== 107+0+4832 (secure 0 0 0) 0x7fe70405a900 con 0x7fe710102580 2024-03-23T08:35:09.183 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:09.183 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:37.851172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":0,"incarnation":23,"state":"up:rejoin","state_seq":181,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.184+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe6f806cc20 msgr2=0x7fe6f806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe6f806cc20 0x7fe6f806f0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe700001250 tx=0x7fe70000b040 comp rx=0 tx=0).stop 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 msgr2=0x7fe710112be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 0x7fe710112be0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe704009390 tx=0x7fe704009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 shutdown_connections 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fe6f806cc20 0x7fe6f806f0d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe710101be0 0x7fe7101126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 --2- 172.21.15.70:0/1473293074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe710102580 0x7fe710112be0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 >> 172.21.15.70:0/1473293074 conn(0x7fe710072ad0 msgr2=0x7fe710104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 shutdown_connections 2024-03-23T08:35:09.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.185+0000 7fe717e90700 1 -- 172.21.15.70:0/1473293074 wait complete. 2024-03-23T08:35:09.186 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 25 2024-03-23T08:35:09.196 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:08 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3400906564' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-23T08:35:09.196 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:08 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/494037058' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-23T08:35:09.196 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:08 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2258154609' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-23T08:35:09.197 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-03-23T08:35:09.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-23T08:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.316+0000 7f13200a5700 1 -- 172.21.15.70:0/2461216012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 msgr2=0x7f13181030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.316+0000 7f13200a5700 1 --2- 172.21.15.70:0/2461216012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 0x7f13181030b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1308004940 tx=0x7f1308009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.316+0000 7f13200a5700 1 -- 172.21.15.70:0/2461216012 shutdown_connections 2024-03-23T08:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.316+0000 7f13200a5700 1 --2- 172.21.15.70:0/2461216012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1318103680 0x7f1318064e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.316+0000 7f13200a5700 1 --2- 172.21.15.70:0/2461216012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 0x7f13181030b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.316+0000 7f13200a5700 1 -- 172.21.15.70:0/2461216012 >> 172.21.15.70:0/2461216012 conn(0x7f1318072ad0 msgr2=0x7f1318072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.316+0000 7f13200a5700 1 -- 172.21.15.70:0/2461216012 shutdown_connections 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.316+0000 7f13200a5700 1 -- 172.21.15.70:0/2461216012 wait complete. 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.316+0000 7f13200a5700 1 Processor -- start 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f13200a5700 1 -- start start 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f13200a5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 0x7f13180fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f13200a5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1318103680 0x7f13180fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f13200a5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180fdb90 con 0x7f1318102ce0 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f13200a5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180fc6c0 con 0x7f1318103680 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f131de41700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 0x7f13180fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f131de41700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 0x7f13180fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59434/0 (socket says 172.21.15.70:59434) 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f131de41700 1 -- 172.21.15.70:0/344371030 learned_addr learned my addr 172.21.15.70:0/344371030 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:09.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f131d640700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1318103680 0x7f13180fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f131de41700 1 -- 172.21.15.70:0/344371030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1318103680 msgr2=0x7f13180fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f131de41700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1318103680 0x7f13180fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f131de41700 1 -- 172.21.15.70:0/344371030 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308005040 con 0x7f1318102ce0 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f131de41700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 0x7f13180fbc40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1308000c00 tx=0x7f1308009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f130e7fc700 1 -- 172.21.15.70:0/344371030 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1308021070 con 0x7f1318102ce0 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f130e7fc700 1 -- 172.21.15.70:0/344371030 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1308013de0 con 0x7f1318102ce0 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13180fc940 con 0x7f1318102ce0 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f130e7fc700 1 -- 172.21.15.70:0/344371030 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130800e070 con 0x7f1318102ce0 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.317+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1318119e80 con 0x7f1318102ce0 2024-03-23T08:35:09.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.318+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13180fe430 con 0x7f1318102ce0 2024-03-23T08:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.319+0000 7f130e7fc700 1 -- 172.21.15.70:0/344371030 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f130801c600 con 0x7f1318102ce0 2024-03-23T08:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.319+0000 7f130e7fc700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f130406cc20 0x7f130406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.319+0000 7f130e7fc700 1 -- 172.21.15.70:0/344371030 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f130808cfe0 con 0x7f1318102ce0 2024-03-23T08:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.319+0000 7f131d640700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f130406cc20 0x7f130406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.320+0000 7f131d640700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f130406cc20 0x7f130406f0d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1314004600 tx=0x7f131400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:09.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.324+0000 7f130e7fc700 1 -- 172.21.15.70:0/344371030 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f13080570e0 con 0x7f1318102ce0 2024-03-23T08:35:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:08 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3400906564' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-23T08:35:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:08 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/494037058' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-23T08:35:09.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:08 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2258154609' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-23T08:35:09.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.560+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f1318064970 con 0x7f1318102ce0 2024-03-23T08:35:09.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.561+0000 7f130e7fc700 1 -- 172.21.15.70:0/344371030 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v37) v1 ==== 107+0+4832 (secure 0 0 0) 0x7f130805a700 con 0x7f1318102ce0 2024-03-23T08:35:09.561 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:09.561 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:38.853033+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi193.sdlbdd","rank":0,"incarnation":23,"state":"up:active","state_seq":182,"addr":"172.21.15.193:6825/3335188435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3335188435},{"type":"v1","addr":"172.21.15.193:6825","nonce":3335188435}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:09.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f130406cc20 msgr2=0x7f130406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f130406cc20 0x7f130406f0d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1314004600 tx=0x7f131400b040 comp rx=0 tx=0).stop 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 msgr2=0x7f13180fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 0x7f13180fbc40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1308000c00 tx=0x7f1308009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 shutdown_connections 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f130406cc20 0x7f130406f0d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1318102ce0 0x7f13180fbc40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 --2- 172.21.15.70:0/344371030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1318103680 0x7f13180fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 >> 172.21.15.70:0/344371030 conn(0x7f1318072ad0 msgr2=0x7f1318105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 shutdown_connections 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.563+0000 7f13200a5700 1 -- 172.21.15.70:0/344371030 wait complete. 2024-03-23T08:35:09.564 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 26 2024-03-23T08:35:09.575 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-03-23T08:35:09.575 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-23T08:35:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.694+0000 7f7c262ac700 1 -- 172.21.15.70:0/1307942367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20101be0 msgr2=0x7f7c20101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.694+0000 7f7c262ac700 1 --2- 172.21.15.70:0/1307942367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20101be0 0x7f7c20101fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7c10004940 tx=0x7f7c10009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.694+0000 7f7c262ac700 1 -- 172.21.15.70:0/1307942367 shutdown_connections 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.694+0000 7f7c262ac700 1 --2- 172.21.15.70:0/1307942367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c20102580 0x7f7c200fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.694+0000 7f7c262ac700 1 --2- 172.21.15.70:0/1307942367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20101be0 0x7f7c20101fb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.694+0000 7f7c262ac700 1 -- 172.21.15.70:0/1307942367 >> 172.21.15.70:0/1307942367 conn(0x7f7c20072ad0 msgr2=0x7f7c20072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.694+0000 7f7c262ac700 1 -- 172.21.15.70:0/1307942367 shutdown_connections 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.694+0000 7f7c262ac700 1 -- 172.21.15.70:0/1307942367 wait complete. 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.695+0000 7f7c262ac700 1 Processor -- start 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.695+0000 7f7c262ac700 1 -- start start 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.695+0000 7f7c262ac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c20101be0 0x7f7c201126c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.695+0000 7f7c262ac700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20102580 0x7f7c20112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.695+0000 7f7c262ac700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20113290 con 0x7f7c20102580 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.695+0000 7f7c262ac700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2010c850 con 0x7f7c20101be0 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.695+0000 7f7c1f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20102580 0x7f7c20112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.695+0000 7f7c1f7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20102580 0x7f7c20112c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59464/0 (socket says 172.21.15.70:59464) 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.695+0000 7f7c1f7fe700 1 -- 172.21.15.70:0/634559015 learned_addr learned my addr 172.21.15.70:0/634559015 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:09.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c1ffff700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c20101be0 0x7f7c201126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c1f7fe700 1 -- 172.21.15.70:0/634559015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c20101be0 msgr2=0x7f7c201126c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c1f7fe700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c20101be0 0x7f7c201126c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c1f7fe700 1 -- 172.21.15.70:0/634559015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10005040 con 0x7f7c20102580 2024-03-23T08:35:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c1f7fe700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20102580 0x7f7c20112c40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400c890 tx=0x7f7c140044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c1cff9700 1 -- 172.21.15.70:0/634559015 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c1400a7d0 con 0x7f7c20102580 2024-03-23T08:35:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c1cff9700 1 -- 172.21.15.70:0/634559015 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7c1400ade0 con 0x7f7c20102580 2024-03-23T08:35:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2010cae0 con 0x7f7c20102580 2024-03-23T08:35:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c1cff9700 1 -- 172.21.15.70:0/634559015 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c140113a0 con 0x7f7c20102580 2024-03-23T08:35:09.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.696+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2010d020 con 0x7f7c20102580 2024-03-23T08:35:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.697+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c200fd330 con 0x7f7c20102580 2024-03-23T08:35:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.697+0000 7f7c1cff9700 1 -- 172.21.15.70:0/634559015 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7c14012400 con 0x7f7c20102580 2024-03-23T08:35:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.698+0000 7f7c1cff9700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7c0806cc20 0x7f7c0806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.698+0000 7f7c1cff9700 1 -- 172.21.15.70:0/634559015 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7c1408c040 con 0x7f7c20102580 2024-03-23T08:35:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.698+0000 7f7c1ffff700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7c0806cc20 0x7f7c0806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.698+0000 7f7c1ffff700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7c0806cc20 0x7f7c0806f0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7c10004f90 tx=0x7f7c10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:09.702 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.702+0000 7f7c1cff9700 1 -- 172.21.15.70:0/634559015 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7c14059bd0 con 0x7f7c20102580 2024-03-23T08:35:09.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.940+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f7c20108b90 con 0x7f7c20102580 2024-03-23T08:35:09.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.941+0000 7f7c1cff9700 1 -- 172.21.15.70:0/634559015 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v37) v1 ==== 107+0+4037 (secure 0 0 0) 0x7f7c14059760 con 0x7f7c20102580 2024-03-23T08:35:09.941 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:09.941 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:48.851829+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:09.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7c0806cc20 msgr2=0x7f7c0806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7c0806cc20 0x7f7c0806f0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7c10004f90 tx=0x7f7c10009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:09.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20102580 msgr2=0x7f7c20112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:09.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20102580 0x7f7c20112c40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400c890 tx=0x7f7c140044c0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 shutdown_connections 2024-03-23T08:35:09.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f7c0806cc20 0x7f7c0806f0d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c20101be0 0x7f7c201126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 --2- 172.21.15.70:0/634559015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20102580 0x7f7c20112c40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:09.943 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 >> 172.21.15.70:0/634559015 conn(0x7f7c20072ad0 msgr2=0x7f7c20104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:09.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 shutdown_connections 2024-03-23T08:35:09.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:09.943+0000 7f7c262ac700 1 -- 172.21.15.70:0/634559015 wait complete. 2024-03-23T08:35:09.944 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 27 2024-03-23T08:35:09.955 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-03-23T08:35:09.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-23T08:35:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.073+0000 7fca3eac6700 1 -- 172.21.15.70:0/2097634386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 msgr2=0x7fca380fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.073+0000 7fca3eac6700 1 --2- 172.21.15.70:0/2097634386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 0x7fca380fbbf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fca24004940 tx=0x7fca24009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.073+0000 7fca3eac6700 1 -- 172.21.15.70:0/2097634386 shutdown_connections 2024-03-23T08:35:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.073+0000 7fca3eac6700 1 --2- 172.21.15.70:0/2097634386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 0x7fca380fbbf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.073+0000 7fca3eac6700 1 --2- 172.21.15.70:0/2097634386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca38101be0 0x7fca38101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.073+0000 7fca3eac6700 1 -- 172.21.15.70:0/2097634386 >> 172.21.15.70:0/2097634386 conn(0x7fca38072ad0 msgr2=0x7fca38072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.073+0000 7fca3eac6700 1 -- 172.21.15.70:0/2097634386 shutdown_connections 2024-03-23T08:35:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.073+0000 7fca3eac6700 1 -- 172.21.15.70:0/2097634386 wait complete. 2024-03-23T08:35:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.073+0000 7fca3eac6700 1 Processor -- start 2024-03-23T08:35:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca3eac6700 1 -- start start 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca3eac6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca38101be0 0x7fca3810e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca3eac6700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 0x7fca3810edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca3eac6700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3810f330 con 0x7fca38102580 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca3eac6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3810f4a0 con 0x7fca38101be0 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca37fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 0x7fca3810edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca37fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 0x7fca3810edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59472/0 (socket says 172.21.15.70:59472) 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca37fff700 1 -- 172.21.15.70:0/712299080 learned_addr learned my addr 172.21.15.70:0/712299080 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca3c862700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca38101be0 0x7fca3810e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca37fff700 1 -- 172.21.15.70:0/712299080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca38101be0 msgr2=0x7fca3810e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca37fff700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca38101be0 0x7fca3810e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca37fff700 1 -- 172.21.15.70:0/712299080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24005040 con 0x7fca38102580 2024-03-23T08:35:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca3c862700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca38101be0 0x7fca3810e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:35:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.074+0000 7fca37fff700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 0x7fca3810edf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fca24009380 tx=0x7fca24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.075+0000 7fca357fa700 1 -- 172.21.15.70:0/712299080 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca2400e040 con 0x7fca38102580 2024-03-23T08:35:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.075+0000 7fca357fa700 1 -- 172.21.15.70:0/712299080 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca24013420 con 0x7fca38102580 2024-03-23T08:35:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.075+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3810f6d0 con 0x7fca38102580 2024-03-23T08:35:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.075+0000 7fca357fa700 1 -- 172.21.15.70:0/712299080 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca2401c630 con 0x7fca38102580 2024-03-23T08:35:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.075+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca38117c70 con 0x7fca38102580 2024-03-23T08:35:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.076+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca380fd330 con 0x7fca38102580 2024-03-23T08:35:10.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.077+0000 7fca357fa700 1 -- 172.21.15.70:0/712299080 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fca240139e0 con 0x7fca38102580 2024-03-23T08:35:10.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.077+0000 7fca357fa700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fca2806cc20 0x7fca2806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:10.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.077+0000 7fca357fa700 1 -- 172.21.15.70:0/712299080 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fca2408dc70 con 0x7fca38102580 2024-03-23T08:35:10.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.077+0000 7fca3c862700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fca2806cc20 0x7fca2806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:10.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.077+0000 7fca3c862700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fca2806cc20 0x7fca2806f0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007fd0 tx=0x7fca2c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:10.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.081+0000 7fca357fa700 1 -- 172.21.15.70:0/712299080 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fca24057d70 con 0x7fca38102580 2024-03-23T08:35:10.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:09 smithi070 ceph-mon[145869]: pgmap v258: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:09 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1473293074' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-23T08:35:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:09 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/344371030' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-23T08:35:10.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.315+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fca38064970 con 0x7fca38102580 2024-03-23T08:35:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.316+0000 7fca357fa700 1 -- 172.21.15.70:0/712299080 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v37) v1 ==== 107+0+4048 (secure 0 0 0) 0x7fca2405b390 con 0x7fca38102580 2024-03-23T08:35:10.316 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:10.316 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:48.861616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:10.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fca2806cc20 msgr2=0x7fca2806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fca2806cc20 0x7fca2806f0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007fd0 tx=0x7fca2c009040 comp rx=0 tx=0).stop 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 msgr2=0x7fca3810edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 0x7fca3810edf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fca24009380 tx=0x7fca24009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 shutdown_connections 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fca2806cc20 0x7fca2806f0d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca38101be0 0x7fca3810e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 --2- 172.21.15.70:0/712299080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca38102580 0x7fca3810edf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 >> 172.21.15.70:0/712299080 conn(0x7fca38072ad0 msgr2=0x7fca38104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 shutdown_connections 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.318+0000 7fca3eac6700 1 -- 172.21.15.70:0/712299080 wait complete. 2024-03-23T08:35:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 28 2024-03-23T08:35:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:09 smithi193 ceph-mon[108533]: pgmap v258: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:09 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1473293074' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-23T08:35:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:09 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/344371030' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-23T08:35:10.330 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-03-23T08:35:10.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 -- 172.21.15.70:0/2786362046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88103660 msgr2=0x7eff880fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 --2- 172.21.15.70:0/2786362046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88103660 0x7eff880fccd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff78004940 tx=0x7eff78009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 -- 172.21.15.70:0/2786362046 shutdown_connections 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 --2- 172.21.15.70:0/2786362046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88103660 0x7eff880fccd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 --2- 172.21.15.70:0/2786362046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff88102cc0 0x7eff88103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 -- 172.21.15.70:0/2786362046 >> 172.21.15.70:0/2786362046 conn(0x7eff880f8830 msgr2=0x7eff880fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 -- 172.21.15.70:0/2786362046 shutdown_connections 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 -- 172.21.15.70:0/2786362046 wait complete. 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 Processor -- start 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.449+0000 7eff8e208700 1 -- start start 2024-03-23T08:35:10.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff8e208700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88102cc0 0x7eff88113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff8e208700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff88103660 0x7eff88113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff8e208700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff881142f0 con 0x7eff88102cc0 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff8e208700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8810d8f0 con 0x7eff88103660 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff877fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88102cc0 0x7eff88113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff877fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88102cc0 0x7eff88113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59484/0 (socket says 172.21.15.70:59484) 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff877fe700 1 -- 172.21.15.70:0/2036177249 learned_addr learned my addr 172.21.15.70:0/2036177249 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff86ffd700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff88103660 0x7eff88113ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff877fe700 1 -- 172.21.15.70:0/2036177249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff88103660 msgr2=0x7eff88113ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff877fe700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff88103660 0x7eff88113ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff877fe700 1 -- 172.21.15.70:0/2036177249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78005040 con 0x7eff88102cc0 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff877fe700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88102cc0 0x7eff88113760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c8c0 tx=0x7eff7c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff6ffff700 1 -- 172.21.15.70:0/2036177249 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7c00a760 con 0x7eff88102cc0 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff6ffff700 1 -- 172.21.15.70:0/2036177249 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7eff7c00ad70 con 0x7eff88102cc0 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8810dbd0 con 0x7eff88102cc0 2024-03-23T08:35:10.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff6ffff700 1 -- 172.21.15.70:0/2036177249 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7c0113a0 con 0x7eff88102cc0 2024-03-23T08:35:10.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.450+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8810e160 con 0x7eff88102cc0 2024-03-23T08:35:10.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.451+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff880fe410 con 0x7eff88102cc0 2024-03-23T08:35:10.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.452+0000 7eff6ffff700 1 -- 172.21.15.70:0/2036177249 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7eff7c012400 con 0x7eff88102cc0 2024-03-23T08:35:10.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.452+0000 7eff6ffff700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7eff7006cc20 0x7eff7006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:10.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.452+0000 7eff6ffff700 1 -- 172.21.15.70:0/2036177249 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7eff7c08c080 con 0x7eff88102cc0 2024-03-23T08:35:10.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.452+0000 7eff86ffd700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7eff7006cc20 0x7eff7006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:10.454 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.453+0000 7eff86ffd700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7eff7006cc20 0x7eff7006f0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff8810ebd0 tx=0x7eff78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:10.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.457+0000 7eff6ffff700 1 -- 172.21.15.70:0/2036177249 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7eff7c059b50 con 0x7eff88102cc0 2024-03-23T08:35:10.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.693+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7eff88109c50 con 0x7eff88102cc0 2024-03-23T08:35:10.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.694+0000 7eff6ffff700 1 -- 172.21.15.70:0/2036177249 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v37) v1 ==== 107+0+4830 (secure 0 0 0) 0x7eff7c0596e0 con 0x7eff88102cc0 2024-03-23T08:35:10.694 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:10.694 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44279,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/4097359817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4097359817},{"type":"v1","addr":"172.21.15.193:6825","nonce":4097359817}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:48.861616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7eff7006cc20 msgr2=0x7eff7006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7eff7006cc20 0x7eff7006f0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7eff8810ebd0 tx=0x7eff78009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88102cc0 msgr2=0x7eff88113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88102cc0 0x7eff88113760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c8c0 tx=0x7eff7c0044c0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 shutdown_connections 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7eff7006cc20 0x7eff7006f0d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88102cc0 0x7eff88113760 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 --2- 172.21.15.70:0/2036177249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff88103660 0x7eff88113ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 >> 172.21.15.70:0/2036177249 conn(0x7eff880f8830 msgr2=0x7eff880f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 shutdown_connections 2024-03-23T08:35:10.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.696+0000 7eff8e208700 1 -- 172.21.15.70:0/2036177249 wait complete. 2024-03-23T08:35:10.697 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 29 2024-03-23T08:35:10.708 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-03-23T08:35:10.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 -- 172.21.15.70:0/1934463780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 msgr2=0x7fee18101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 --2- 172.21.15.70:0/1934463780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 0x7fee18101fb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fee14004940 tx=0x7fee14009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 -- 172.21.15.70:0/1934463780 shutdown_connections 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 --2- 172.21.15.70:0/1934463780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee18102580 0x7fee180fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 --2- 172.21.15.70:0/1934463780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 0x7fee18101fb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 -- 172.21.15.70:0/1934463780 >> 172.21.15.70:0/1934463780 conn(0x7fee18072ad0 msgr2=0x7fee18072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 -- 172.21.15.70:0/1934463780 shutdown_connections 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 -- 172.21.15.70:0/1934463780 wait complete. 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 Processor -- start 2024-03-23T08:35:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.830+0000 7fee1ff14700 1 -- start start 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1ff14700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 0x7fee181126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1ff14700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee18102580 0x7fee18112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1ff14700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee18113270 con 0x7fee18101be0 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1ff14700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1810c830 con 0x7fee18102580 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1dcb0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 0x7fee181126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1dcb0700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 0x7fee181126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59504/0 (socket says 172.21.15.70:59504) 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1dcb0700 1 -- 172.21.15.70:0/2685022930 learned_addr learned my addr 172.21.15.70:0/2685022930 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1d4af700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee18102580 0x7fee18112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1dcb0700 1 -- 172.21.15.70:0/2685022930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee18102580 msgr2=0x7fee18112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1dcb0700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee18102580 0x7fee18112c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1dcb0700 1 -- 172.21.15.70:0/2685022930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee14005040 con 0x7fee18101be0 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1dcb0700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 0x7fee181126a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fee14009360 tx=0x7fee14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee0e7fc700 1 -- 172.21.15.70:0/2685022930 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1400e040 con 0x7fee18101be0 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee0e7fc700 1 -- 172.21.15.70:0/2685022930 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee14013420 con 0x7fee18101be0 2024-03-23T08:35:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1810ca60 con 0x7fee18101be0 2024-03-23T08:35:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee0e7fc700 1 -- 172.21.15.70:0/2685022930 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1401c5f0 con 0x7fee18101be0 2024-03-23T08:35:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.831+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1810cfa0 con 0x7fee18101be0 2024-03-23T08:35:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.832+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee180fd330 con 0x7fee18101be0 2024-03-23T08:35:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.833+0000 7fee0e7fc700 1 -- 172.21.15.70:0/2685022930 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fee14013590 con 0x7fee18101be0 2024-03-23T08:35:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.833+0000 7fee0e7fc700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fee0406cc20 0x7fee0406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:10.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.833+0000 7fee0e7fc700 1 -- 172.21.15.70:0/2685022930 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fee1408d2c0 con 0x7fee18101be0 2024-03-23T08:35:10.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.833+0000 7fee1d4af700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fee0406cc20 0x7fee0406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:10.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.834+0000 7fee1d4af700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fee0406cc20 0x7fee0406f0d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee08009fd0 tx=0x7fee08009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:10.838+0000 7fee0e7fc700 1 -- 172.21.15.70:0/2685022930 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee1405ad90 con 0x7fee18101be0 2024-03-23T08:35:10.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:10 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/634559015' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-23T08:35:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:10 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/712299080' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-23T08:35:10.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:10 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2036177249' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-23T08:35:11.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.074+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fee1810dd20 con 0x7fee18101be0 2024-03-23T08:35:11.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.075+0000 7fee0e7fc700 1 -- 172.21.15.70:0/2685022930 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v37) v1 ==== 107+0+4834 (secure 0 0 0) 0x7fee1405a920 con 0x7fee18101be0 2024-03-23T08:35:11.075 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:11.075 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44279,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/4097359817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4097359817},{"type":"v1","addr":"172.21.15.193:6825","nonce":4097359817}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:57.817550+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:11.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fee0406cc20 msgr2=0x7fee0406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fee0406cc20 0x7fee0406f0d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee08009fd0 tx=0x7fee08009f20 comp rx=0 tx=0).stop 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 msgr2=0x7fee181126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 0x7fee181126a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fee14009360 tx=0x7fee14009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 shutdown_connections 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fee0406cc20 0x7fee0406f0d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee18101be0 0x7fee181126a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 --2- 172.21.15.70:0/2685022930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee18102580 0x7fee18112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 >> 172.21.15.70:0/2685022930 conn(0x7fee18072ad0 msgr2=0x7fee18104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 shutdown_connections 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.077+0000 7fee1ff14700 1 -- 172.21.15.70:0/2685022930 wait complete. 2024-03-23T08:35:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 30 2024-03-23T08:35:11.089 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-03-23T08:35:11.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-23T08:35:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.208+0000 7f83141de700 1 -- 172.21.15.70:0/323210992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 msgr2=0x7f830c0fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.208+0000 7f83141de700 1 --2- 172.21.15.70:0/323210992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 0x7f830c0fbbf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8308004940 tx=0x7f8308009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.208+0000 7f83141de700 1 -- 172.21.15.70:0/323210992 shutdown_connections 2024-03-23T08:35:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.208+0000 7f83141de700 1 --2- 172.21.15.70:0/323210992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 0x7f830c0fbbf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.208+0000 7f83141de700 1 --2- 172.21.15.70:0/323210992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c101be0 0x7f830c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.208+0000 7f83141de700 1 -- 172.21.15.70:0/323210992 >> 172.21.15.70:0/323210992 conn(0x7f830c072ad0 msgr2=0x7f830c072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.208+0000 7f83141de700 1 -- 172.21.15.70:0/323210992 shutdown_connections 2024-03-23T08:35:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.208+0000 7f83141de700 1 -- 172.21.15.70:0/323210992 wait complete. 2024-03-23T08:35:11.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.208+0000 7f83141de700 1 Processor -- start 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83141de700 1 -- start start 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83141de700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c101be0 0x7f830c10e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83141de700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 0x7f830c10ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83141de700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c10f370 con 0x7f830c102580 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83141de700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c10f4e0 con 0x7f830c101be0 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f8311779700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 0x7f830c10ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f8311779700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 0x7f830c10ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59530/0 (socket says 172.21.15.70:59530) 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f8311779700 1 -- 172.21.15.70:0/2509682106 learned_addr learned my addr 172.21.15.70:0/2509682106 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f8311f7a700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c101be0 0x7f830c10e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f8311779700 1 -- 172.21.15.70:0/2509682106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c101be0 msgr2=0x7f830c10e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f8311779700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c101be0 0x7f830c10e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f8311779700 1 -- 172.21.15.70:0/2509682106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308005040 con 0x7f830c102580 2024-03-23T08:35:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f8311779700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 0x7f830c10ee30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f83080093b0 tx=0x7f8308009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83027fc700 1 -- 172.21.15.70:0/2509682106 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f830800e040 con 0x7f830c102580 2024-03-23T08:35:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c10f760 con 0x7f830c102580 2024-03-23T08:35:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83027fc700 1 -- 172.21.15.70:0/2509682106 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8308013420 con 0x7f830c102580 2024-03-23T08:35:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83027fc700 1 -- 172.21.15.70:0/2509682106 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f830801c630 con 0x7f830c102580 2024-03-23T08:35:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.209+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c117c80 con 0x7f830c102580 2024-03-23T08:35:11.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.210+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830c0fd330 con 0x7f830c102580 2024-03-23T08:35:11.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.211+0000 7f83027fc700 1 -- 172.21.15.70:0/2509682106 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f83080139e0 con 0x7f830c102580 2024-03-23T08:35:11.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.211+0000 7f83027fc700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f82f806cc20 0x7f82f806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:11.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.211+0000 7f83027fc700 1 -- 172.21.15.70:0/2509682106 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f830808d070 con 0x7f830c102580 2024-03-23T08:35:11.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.211+0000 7f8311f7a700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f82f806cc20 0x7f82f806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:11.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.212+0000 7f8311f7a700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f82f806cc20 0x7f82f806f0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82fc007fd0 tx=0x7f82fc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:11.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.216+0000 7f83027fc700 1 -- 172.21.15.70:0/2509682106 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8308057160 con 0x7f830c102580 2024-03-23T08:35:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:10 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/634559015' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-23T08:35:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:10 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/712299080' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-23T08:35:11.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:10 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2036177249' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-23T08:35:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.452+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f830c10f700 con 0x7f830c102580 2024-03-23T08:35:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.453+0000 7f83027fc700 1 -- 172.21.15.70:0/2509682106 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v37) v1 ==== 107+0+4831 (secure 0 0 0) 0x7f830805a780 con 0x7f830c102580 2024-03-23T08:35:11.453 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:11.453 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44279,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/4097359817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4097359817},{"type":"v1","addr":"172.21.15.193:6825","nonce":4097359817}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:58.824297+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f82f806cc20 msgr2=0x7f82f806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f82f806cc20 0x7f82f806f0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82fc007fd0 tx=0x7f82fc009040 comp rx=0 tx=0).stop 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 msgr2=0x7f830c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 0x7f830c10ee30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f83080093b0 tx=0x7f8308009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 shutdown_connections 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f82f806cc20 0x7f82f806f0d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f830c101be0 0x7f830c10e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 --2- 172.21.15.70:0/2509682106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f830c102580 0x7f830c10ee30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 >> 172.21.15.70:0/2509682106 conn(0x7f830c072ad0 msgr2=0x7f830c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 shutdown_connections 2024-03-23T08:35:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.455+0000 7f83141de700 1 -- 172.21.15.70:0/2509682106 wait complete. 2024-03-23T08:35:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 31 2024-03-23T08:35:11.467 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-03-23T08:35:11.467 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-23T08:35:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.585+0000 7f5e17b61700 1 -- 172.21.15.70:0/1866586598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10102580 msgr2=0x7f5e100fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.585+0000 7f5e17b61700 1 --2- 172.21.15.70:0/1866586598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10102580 0x7f5e100fbbf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c004940 tx=0x7f5e0c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.585+0000 7f5e17b61700 1 -- 172.21.15.70:0/1866586598 shutdown_connections 2024-03-23T08:35:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.585+0000 7f5e17b61700 1 --2- 172.21.15.70:0/1866586598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10102580 0x7f5e100fbbf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.585+0000 7f5e17b61700 1 --2- 172.21.15.70:0/1866586598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e10101be0 0x7f5e10101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.585+0000 7f5e17b61700 1 -- 172.21.15.70:0/1866586598 >> 172.21.15.70:0/1866586598 conn(0x7f5e10072ad0 msgr2=0x7f5e10072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.585+0000 7f5e17b61700 1 -- 172.21.15.70:0/1866586598 shutdown_connections 2024-03-23T08:35:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.585+0000 7f5e17b61700 1 -- 172.21.15.70:0/1866586598 wait complete. 2024-03-23T08:35:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.585+0000 7f5e17b61700 1 Processor -- start 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e17b61700 1 -- start start 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e17b61700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10101be0 0x7f5e1010e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e17b61700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e10102580 0x7f5e1010ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e17b61700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1010f370 con 0x7f5e10101be0 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e17b61700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1010f4e0 con 0x7f5e10102580 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e158fd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10101be0 0x7f5e1010e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e158fd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10101be0 0x7f5e1010e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59544/0 (socket says 172.21.15.70:59544) 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e158fd700 1 -- 172.21.15.70:0/972039434 learned_addr learned my addr 172.21.15.70:0/972039434 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e150fc700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e10102580 0x7f5e1010ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e158fd700 1 -- 172.21.15.70:0/972039434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e10102580 msgr2=0x7f5e1010ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e158fd700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e10102580 0x7f5e1010ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e158fd700 1 -- 172.21.15.70:0/972039434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c005040 con 0x7f5e10101be0 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e158fd700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10101be0 0x7f5e1010e8f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e00002a50 tx=0x7f5e000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e067fc700 1 -- 172.21.15.70:0/972039434 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e00007cc0 con 0x7f5e10101be0 2024-03-23T08:35:11.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e067fc700 1 -- 172.21.15.70:0/972039434 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e0000f040 con 0x7f5e10101be0 2024-03-23T08:35:11.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e10117960 con 0x7f5e10101be0 2024-03-23T08:35:11.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.586+0000 7f5e067fc700 1 -- 172.21.15.70:0/972039434 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e0000d9c0 con 0x7f5e10101be0 2024-03-23T08:35:11.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.587+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e10117d00 con 0x7f5e10101be0 2024-03-23T08:35:11.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.587+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e100fd330 con 0x7f5e10101be0 2024-03-23T08:35:11.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.588+0000 7f5e067fc700 1 -- 172.21.15.70:0/972039434 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f5e0000db20 con 0x7f5e10101be0 2024-03-23T08:35:11.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.588+0000 7f5e067fc700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5dfc06cc20 0x7f5dfc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:11.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.588+0000 7f5e067fc700 1 -- 172.21.15.70:0/972039434 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5e0008bd20 con 0x7f5e10101be0 2024-03-23T08:35:11.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.588+0000 7f5e150fc700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5dfc06cc20 0x7f5dfc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:11.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.589+0000 7f5e150fc700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5dfc06cc20 0x7f5dfc06f0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c004890 tx=0x7f5e0c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:11.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.593+0000 7f5e067fc700 1 -- 172.21.15.70:0/972039434 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5e00055ce0 con 0x7f5e10101be0 2024-03-23T08:35:11.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.830+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f5e10065380 con 0x7f5e10101be0 2024-03-23T08:35:11.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.830+0000 7f5e067fc700 1 -- 172.21.15.70:0/972039434 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v37) v1 ==== 107+0+4831 (secure 0 0 0) 0x7f5e00059300 con 0x7f5e10101be0 2024-03-23T08:35:11.830 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:11.831 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44255,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/2042142064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":2042142064},{"type":"v1","addr":"172.21.15.193:6827","nonce":2042142064}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44279,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/4097359817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4097359817},{"type":"v1","addr":"172.21.15.193:6825","nonce":4097359817}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:59.828224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:active","state_seq":14,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5dfc06cc20 msgr2=0x7f5dfc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5dfc06cc20 0x7f5dfc06f0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c004890 tx=0x7f5e0c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10101be0 msgr2=0x7f5e1010e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10101be0 0x7f5e1010e8f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e00002a50 tx=0x7f5e000046b0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 shutdown_connections 2024-03-23T08:35:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f5dfc06cc20 0x7f5dfc06f0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e10101be0 0x7f5e1010e8f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 --2- 172.21.15.70:0/972039434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e10102580 0x7f5e1010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 >> 172.21.15.70:0/972039434 conn(0x7f5e10072ad0 msgr2=0x7f5e10104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:11.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 shutdown_connections 2024-03-23T08:35:11.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.832+0000 7f5e17b61700 1 -- 172.21.15.70:0/972039434 wait complete. 2024-03-23T08:35:11.834 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2024-03-23T08:35:11.844 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-03-23T08:35:11.845 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-23T08:35:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.963+0000 7f357e0aa700 1 -- 172.21.15.70:0/3519395233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35781025b0 msgr2=0x7f35780fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.963+0000 7f357e0aa700 1 --2- 172.21.15.70:0/3519395233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35781025b0 0x7f35780fbc20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f356c004940 tx=0x7f356c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.963+0000 7f357e0aa700 1 -- 172.21.15.70:0/3519395233 shutdown_connections 2024-03-23T08:35:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.963+0000 7f357e0aa700 1 --2- 172.21.15.70:0/3519395233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35781025b0 0x7f35780fbc20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.963+0000 7f357e0aa700 1 --2- 172.21.15.70:0/3519395233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3578101c10 0x7f3578101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.963+0000 7f357e0aa700 1 -- 172.21.15.70:0/3519395233 >> 172.21.15.70:0/3519395233 conn(0x7f35780f7720 msgr2=0x7f35780f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.963+0000 7f357e0aa700 1 -- 172.21.15.70:0/3519395233 shutdown_connections 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.963+0000 7f357e0aa700 1 -- 172.21.15.70:0/3519395233 wait complete. 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f357e0aa700 1 Processor -- start 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f357e0aa700 1 -- start start 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f357e0aa700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578101c10 0x7f357810e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f357e0aa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35781025b0 0x7f357810ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f357e0aa700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357810f370 con 0x7f3578101c10 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f357e0aa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357810f4b0 con 0x7f35781025b0 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f35777fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578101c10 0x7f357810e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f35777fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578101c10 0x7f357810e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59552/0 (socket says 172.21.15.70:59552) 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f35777fe700 1 -- 172.21.15.70:0/3116897264 learned_addr learned my addr 172.21.15.70:0/3116897264 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f3576ffd700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35781025b0 0x7f357810ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f35777fe700 1 -- 172.21.15.70:0/3116897264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35781025b0 msgr2=0x7f357810ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f35777fe700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35781025b0 0x7f357810ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.964+0000 7f35777fe700 1 -- 172.21.15.70:0/3116897264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c005040 con 0x7f3578101c10 2024-03-23T08:35:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.965+0000 7f3576ffd700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35781025b0 0x7f357810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:35:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.965+0000 7f35777fe700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578101c10 0x7f357810e8f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3568002a50 tx=0x7f35680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.965+0000 7f355ffff700 1 -- 172.21.15.70:0/3116897264 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3568007cc0 con 0x7f3578101c10 2024-03-23T08:35:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.965+0000 7f355ffff700 1 -- 172.21.15.70:0/3116897264 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f356800f040 con 0x7f3578101c10 2024-03-23T08:35:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.965+0000 7f355ffff700 1 -- 172.21.15.70:0/3116897264 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3568009540 con 0x7f3578101c10 2024-03-23T08:35:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.965+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3578117960 con 0x7f3578101c10 2024-03-23T08:35:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.965+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3578117d80 con 0x7f3578101c10 2024-03-23T08:35:11.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.966+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35780fd360 con 0x7f3578101c10 2024-03-23T08:35:11.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.966+0000 7f355ffff700 1 -- 172.21.15.70:0/3116897264 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f35680096a0 con 0x7f3578101c10 2024-03-23T08:35:11.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.967+0000 7f355ffff700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f356006cc20 0x7f356006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:11.968 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.967+0000 7f355ffff700 1 -- 172.21.15.70:0/3116897264 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f356808be60 con 0x7f3578101c10 2024-03-23T08:35:11.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.967+0000 7f3576ffd700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f356006cc20 0x7f356006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:11.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.967+0000 7f3576ffd700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f356006cc20 0x7f356006f0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f356c00b3b0 tx=0x7f356c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:11.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:11.972+0000 7f355ffff700 1 -- 172.21.15.70:0/3116897264 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3568055e20 con 0x7f3578101c10 2024-03-23T08:35:12.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.209+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f35780640e0 con 0x7f3578101c10 2024-03-23T08:35:12.209 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.210+0000 7f355ffff700 1 -- 172.21.15.70:0/3116897264 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v37) v1 ==== 107+0+4049 (secure 0 0 0) 0x7f3568059440 con 0x7f3578101c10 2024-03-23T08:35:12.210 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:12.210 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44279,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/4097359817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4097359817},{"type":"v1","addr":"172.21.15.193:6825","nonce":4097359817}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:59.828224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:active","state_seq":14,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:12.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.211+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f356006cc20 msgr2=0x7f356006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.211+0000 7f357e0aa700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f356006cc20 0x7f356006f0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f356c00b3b0 tx=0x7f356c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:12.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.212+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578101c10 msgr2=0x7f357810e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.212+0000 7f357e0aa700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578101c10 0x7f357810e8f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3568002a50 tx=0x7f35680046b0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.212+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 shutdown_connections 2024-03-23T08:35:12.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.212+0000 7f357e0aa700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f356006cc20 0x7f356006f0d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.212+0000 7f357e0aa700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578101c10 0x7f357810e8f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.212+0000 7f357e0aa700 1 --2- 172.21.15.70:0/3116897264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35781025b0 0x7f357810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.212+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 >> 172.21.15.70:0/3116897264 conn(0x7f35780f7720 msgr2=0x7f35780f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:12.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.212+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 shutdown_connections 2024-03-23T08:35:12.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.212+0000 7f357e0aa700 1 -- 172.21.15.70:0/3116897264 wait complete. 2024-03-23T08:35:12.213 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 33 2024-03-23T08:35:12.222 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:11 smithi070 ceph-mon[145869]: pgmap v259: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:12.223 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:11 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2685022930' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-23T08:35:12.223 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:11 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2509682106' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-23T08:35:12.223 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:11 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/972039434' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-23T08:35:12.224 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-03-23T08:35:12.224 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-23T08:35:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:11 smithi193 ceph-mon[108533]: pgmap v259: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:11 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2685022930' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-23T08:35:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:11 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2509682106' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-23T08:35:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:11 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/972039434' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/4195187977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 msgr2=0x7f6c34101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 --2- 172.21.15.70:0/4195187977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 0x7f6c34101fb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6c30004940 tx=0x7f6c30009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/4195187977 shutdown_connections 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 --2- 172.21.15.70:0/4195187977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c34102580 0x7f6c340fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 --2- 172.21.15.70:0/4195187977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 0x7f6c34101fb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/4195187977 >> 172.21.15.70:0/4195187977 conn(0x7f6c34072ad0 msgr2=0x7f6c34072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/4195187977 shutdown_connections 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/4195187977 wait complete. 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 Processor -- start 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 -- start start 2024-03-23T08:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 0x7f6c341137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c34102580 0x7f6c34113d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34114380 con 0x7f6c34101be0 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.343+0000 7f6c3b6bc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3410d9d0 con 0x7f6c34102580 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c39458700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 0x7f6c341137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c39458700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 0x7f6c341137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59572/0 (socket says 172.21.15.70:59572) 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c39458700 1 -- 172.21.15.70:0/2067062065 learned_addr learned my addr 172.21.15.70:0/2067062065 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c38c57700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c34102580 0x7f6c34113d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c39458700 1 -- 172.21.15.70:0/2067062065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c34102580 msgr2=0x7f6c34113d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c39458700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c34102580 0x7f6c34113d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c39458700 1 -- 172.21.15.70:0/2067062065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30005040 con 0x7f6c34101be0 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c38c57700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c34102580 0x7f6c34113d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c39458700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 0x7f6c341137f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000b3b0 tx=0x7f6c30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:12.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c29ffb700 1 -- 172.21.15.70:0/2067062065 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c3000e040 con 0x7f6c34101be0 2024-03-23T08:35:12.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c29ffb700 1 -- 172.21.15.70:0/2067062065 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c30013420 con 0x7f6c34101be0 2024-03-23T08:35:12.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3410dc50 con 0x7f6c34101be0 2024-03-23T08:35:12.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c29ffb700 1 -- 172.21.15.70:0/2067062065 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c3001c630 con 0x7f6c34101be0 2024-03-23T08:35:12.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.344+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3410e210 con 0x7f6c34101be0 2024-03-23T08:35:12.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.345+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c340fd330 con 0x7f6c34101be0 2024-03-23T08:35:12.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.346+0000 7f6c29ffb700 1 -- 172.21.15.70:0/2067062065 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6c30013970 con 0x7f6c34101be0 2024-03-23T08:35:12.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.346+0000 7f6c29ffb700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6c2006cc20 0x7f6c2006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:12.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.346+0000 7f6c29ffb700 1 -- 172.21.15.70:0/2067062065 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6c3008d2a0 con 0x7f6c34101be0 2024-03-23T08:35:12.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.346+0000 7f6c38c57700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6c2006cc20 0x7f6c2006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:12.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.346+0000 7f6c38c57700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6c2006cc20 0x7f6c2006f0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009fd0 tx=0x7f6c24009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:12.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.351+0000 7f6c29ffb700 1 -- 172.21.15.70:0/2067062065 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6c3005ad70 con 0x7f6c34101be0 2024-03-23T08:35:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.590+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f6c34108be0 con 0x7f6c34101be0 2024-03-23T08:35:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.591+0000 7f6c29ffb700 1 -- 172.21.15.70:0/2067062065 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v37) v1 ==== 107+0+4831 (secure 0 0 0) 0x7f6c3005a900 con 0x7f6c34101be0 2024-03-23T08:35:12.592 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:12.592 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44279,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/4097359817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4097359817},{"type":"v1","addr":"172.21.15.193:6825","nonce":4097359817}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/1000528415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1000528415},{"type":"v1","addr":"172.21.15.193:6827","nonce":1000528415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:32:59.828224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:active","state_seq":14,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.593+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6c2006cc20 msgr2=0x7f6c2006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.593+0000 7f6c3b6bc700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6c2006cc20 0x7f6c2006f0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009fd0 tx=0x7f6c24009ee0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.593+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 msgr2=0x7f6c341137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.593+0000 7f6c3b6bc700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 0x7f6c341137f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000b3b0 tx=0x7f6c30009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.594+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 shutdown_connections 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.594+0000 7f6c3b6bc700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f6c2006cc20 0x7f6c2006f0d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.594+0000 7f6c3b6bc700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c34101be0 0x7f6c341137f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.594+0000 7f6c3b6bc700 1 --2- 172.21.15.70:0/2067062065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c34102580 0x7f6c34113d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.594+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 >> 172.21.15.70:0/2067062065 conn(0x7f6c34072ad0 msgr2=0x7f6c34064dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.594+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 shutdown_connections 2024-03-23T08:35:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.594+0000 7f6c3b6bc700 1 -- 172.21.15.70:0/2067062065 wait complete. 2024-03-23T08:35:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 34 2024-03-23T08:35:12.606 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-03-23T08:35:12.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.725+0000 7fcf5f282700 1 -- 172.21.15.70:0/3578390596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 msgr2=0x7fcf58064e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.725+0000 7fcf5f282700 1 --2- 172.21.15.70:0/3578390596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 0x7fcf58064e40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c004940 tx=0x7fcf4c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.725+0000 7fcf5f282700 1 -- 172.21.15.70:0/3578390596 shutdown_connections 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.725+0000 7fcf5f282700 1 --2- 172.21.15.70:0/3578390596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 0x7fcf58064e40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.725+0000 7fcf5f282700 1 --2- 172.21.15.70:0/3578390596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58102ce0 0x7fcf581030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.725+0000 7fcf5f282700 1 -- 172.21.15.70:0/3578390596 >> 172.21.15.70:0/3578390596 conn(0x7fcf58072ad0 msgr2=0x7fcf58072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.725+0000 7fcf5f282700 1 -- 172.21.15.70:0/3578390596 shutdown_connections 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.725+0000 7fcf5f282700 1 -- 172.21.15.70:0/3578390596 wait complete. 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.725+0000 7fcf5f282700 1 Processor -- start 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5f282700 1 -- start start 2024-03-23T08:35:12.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5f282700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58102ce0 0x7fcf58113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5f282700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 0x7fcf58113cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5f282700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf58114320 con 0x7fcf58103680 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5f282700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5810d920 con 0x7fcf58102ce0 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5c81d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 0x7fcf58113cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5c81d700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 0x7fcf58113cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59580/0 (socket says 172.21.15.70:59580) 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5c81d700 1 -- 172.21.15.70:0/1924525778 learned_addr learned my addr 172.21.15.70:0/1924525778 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5c81d700 1 -- 172.21.15.70:0/1924525778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58102ce0 msgr2=0x7fcf58113790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5c81d700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58102ce0 0x7fcf58113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5c81d700 1 -- 172.21.15.70:0/1924525778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c005040 con 0x7fcf58103680 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5c81d700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 0x7fcf58113cd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009380 tx=0x7fcf4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf49ffb700 1 -- 172.21.15.70:0/1924525778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4c00e040 con 0x7fcf58103680 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf49ffb700 1 -- 172.21.15.70:0/1924525778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcf4c013420 con 0x7fcf58103680 2024-03-23T08:35:12.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5810dba0 con 0x7fcf58103680 2024-03-23T08:35:12.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.726+0000 7fcf49ffb700 1 -- 172.21.15.70:0/1924525778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4c01c630 con 0x7fcf58103680 2024-03-23T08:35:12.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.727+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5810e160 con 0x7fcf58103680 2024-03-23T08:35:12.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.728+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf580fe430 con 0x7fcf58103680 2024-03-23T08:35:12.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.728+0000 7fcf49ffb700 1 -- 172.21.15.70:0/1924525778 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fcf4c0139e0 con 0x7fcf58103680 2024-03-23T08:35:12.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.728+0000 7fcf49ffb700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcf4406cc20 0x7fcf4406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:12.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.728+0000 7fcf49ffb700 1 -- 172.21.15.70:0/1924525778 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcf4c08d130 con 0x7fcf58103680 2024-03-23T08:35:12.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.728+0000 7fcf5d01e700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcf4406cc20 0x7fcf4406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:12.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.729+0000 7fcf5d01e700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcf4406cc20 0x7fcf4406f0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcf54004d00 tx=0x7fcf54009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.733+0000 7fcf49ffb700 1 -- 172.21.15.70:0/1924525778 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcf4c05acc0 con 0x7fcf58103680 2024-03-23T08:35:12.971 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.971+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fcf58109c80 con 0x7fcf58103680 2024-03-23T08:35:12.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.972+0000 7fcf49ffb700 1 -- 172.21.15.70:0/1924525778 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v37) v1 ==== 107+0+4831 (secure 0 0 0) 0x7fcf4c05a850 con 0x7fcf58103680 2024-03-23T08:35:12.972 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:12.972 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44279,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/4097359817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4097359817},{"type":"v1","addr":"172.21.15.193:6825","nonce":4097359817}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/1000528415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1000528415},{"type":"v1","addr":"172.21.15.193:6827","nonce":1000528415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:33:17.772269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:active","state_seq":14,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcf4406cc20 msgr2=0x7fcf4406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcf4406cc20 0x7fcf4406f0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcf54004d00 tx=0x7fcf54009ce0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 msgr2=0x7fcf58113cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 0x7fcf58113cd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009380 tx=0x7fcf4c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 shutdown_connections 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fcf4406cc20 0x7fcf4406f0d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf58102ce0 0x7fcf58113790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 --2- 172.21.15.70:0/1924525778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf58103680 0x7fcf58113cd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 >> 172.21.15.70:0/1924525778 conn(0x7fcf58072ad0 msgr2=0x7fcf58105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.974+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 shutdown_connections 2024-03-23T08:35:12.975 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:12.975+0000 7fcf5f282700 1 -- 172.21.15.70:0/1924525778 wait complete. 2024-03-23T08:35:12.976 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 35 2024-03-23T08:35:12.986 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-03-23T08:35:12.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.104+0000 7f22d774c700 1 -- 172.21.15.70:0/362474566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 msgr2=0x7f22d0064e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.104+0000 7f22d774c700 1 --2- 172.21.15.70:0/362474566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 0x7f22d0064e40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f22cc004940 tx=0x7f22cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.104+0000 7f22d774c700 1 -- 172.21.15.70:0/362474566 shutdown_connections 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.104+0000 7f22d774c700 1 --2- 172.21.15.70:0/362474566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 0x7f22d0064e40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.104+0000 7f22d774c700 1 --2- 172.21.15.70:0/362474566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d0102ce0 0x7f22d01030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.104+0000 7f22d774c700 1 -- 172.21.15.70:0/362474566 >> 172.21.15.70:0/362474566 conn(0x7f22d0072ad0 msgr2=0x7f22d0072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.104+0000 7f22d774c700 1 -- 172.21.15.70:0/362474566 shutdown_connections 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.104+0000 7f22d774c700 1 -- 172.21.15.70:0/362474566 wait complete. 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.105+0000 7f22d774c700 1 Processor -- start 2024-03-23T08:35:13.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.105+0000 7f22d774c700 1 -- start start 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.105+0000 7f22d774c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d0102ce0 0x7f22d01137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.105+0000 7f22d774c700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 0x7f22d0113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.105+0000 7f22d774c700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d01143c0 con 0x7f22d0103680 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.105+0000 7f22d774c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d010d930 con 0x7f22d0102ce0 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.105+0000 7f22d4ce7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 0x7f22d0113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.105+0000 7f22d4ce7700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 0x7f22d0113ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59608/0 (socket says 172.21.15.70:59608) 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.105+0000 7f22d4ce7700 1 -- 172.21.15.70:0/1543448464 learned_addr learned my addr 172.21.15.70:0/1543448464 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.106+0000 7f22d4ce7700 1 -- 172.21.15.70:0/1543448464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d0102ce0 msgr2=0x7f22d01137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.106+0000 7f22d4ce7700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d0102ce0 0x7f22d01137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.106+0000 7f22d4ce7700 1 -- 172.21.15.70:0/1543448464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc005040 con 0x7f22d0103680 2024-03-23T08:35:13.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.106+0000 7f22d4ce7700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 0x7f22d0113ce0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009380 tx=0x7f22cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:13.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.106+0000 7f22c1ffb700 1 -- 172.21.15.70:0/1543448464 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22cc00e040 con 0x7f22d0103680 2024-03-23T08:35:13.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.106+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d010dbb0 con 0x7f22d0103680 2024-03-23T08:35:13.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.106+0000 7f22c1ffb700 1 -- 172.21.15.70:0/1543448464 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22cc013420 con 0x7f22d0103680 2024-03-23T08:35:13.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.106+0000 7f22c1ffb700 1 -- 172.21.15.70:0/1543448464 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22cc01c630 con 0x7f22d0103680 2024-03-23T08:35:13.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.106+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d010e170 con 0x7f22d0103680 2024-03-23T08:35:13.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.107+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d00fe430 con 0x7f22d0103680 2024-03-23T08:35:13.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.108+0000 7f22c1ffb700 1 -- 172.21.15.70:0/1543448464 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f22cc0139e0 con 0x7f22d0103680 2024-03-23T08:35:13.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.108+0000 7f22c1ffb700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22bc06cc70 0x7f22bc06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:13.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.108+0000 7f22c1ffb700 1 -- 172.21.15.70:0/1543448464 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f22cc08d2a0 con 0x7f22d0103680 2024-03-23T08:35:13.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.108+0000 7f22d54e8700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22bc06cc70 0x7f22bc06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:13.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.108+0000 7f22d54e8700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22bc06cc70 0x7f22bc06f120 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:13.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.112+0000 7f22c1ffb700 1 -- 172.21.15.70:0/1543448464 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f22cc05acf0 con 0x7f22d0103680 2024-03-23T08:35:13.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:12 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3116897264' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-23T08:35:13.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:12 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2067062065' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-23T08:35:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:12 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3116897264' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-23T08:35:13.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:12 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2067062065' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-23T08:35:13.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.351+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f22d0109cf0 con 0x7f22d0103680 2024-03-23T08:35:13.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.352+0000 7f22c1ffb700 1 -- 172.21.15.70:0/1543448464 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v37) v1 ==== 107+0+4830 (secure 0 0 0) 0x7f22cc05a880 con 0x7f22d0103680 2024-03-23T08:35:13.352 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:13.352 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi070.aymzqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6829/470900113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":470900113},{"type":"v1","addr":"172.21.15.70:6829","nonce":470900113}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44279,"name":"cephfs.smithi193.sdlbdd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/4097359817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":4097359817},{"type":"v1","addr":"172.21.15.193:6825","nonce":4097359817}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":44311,"name":"cephfs.smithi193.nucaub","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/1000528415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1000528415},{"type":"v1","addr":"172.21.15.193:6827","nonce":1000528415}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-23T08:20:37.818350+0000","modified":"2024-03-23T08:33:18.773940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34376":{"gid":34376,"name":"cephfs.smithi070.iziydh","rank":0,"incarnation":28,"state":"up:active","state_seq":14,"addr":"172.21.15.70:6827/1449136189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1449136189},{"type":"v1","addr":"172.21.15.70:6827","nonce":1449136189}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-23T08:35:13.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22bc06cc70 msgr2=0x7f22bc06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22bc06cc70 0x7f22bc06f120 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f22c4006fd0 tx=0x7f22c4006db0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 msgr2=0x7f22d0113ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 0x7f22d0113ce0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009380 tx=0x7f22cc009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 shutdown_connections 2024-03-23T08:35:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f22bc06cc70 0x7f22bc06f120 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d0102ce0 0x7f22d01137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 --2- 172.21.15.70:0/1543448464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d0103680 0x7f22d0113ce0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 >> 172.21.15.70:0/1543448464 conn(0x7f22d0072ad0 msgr2=0x7f22d0105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 shutdown_connections 2024-03-23T08:35:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.354+0000 7f22d774c700 1 -- 172.21.15.70:0/1543448464 wait complete. 2024-03-23T08:35:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 36 2024-03-23T08:35:13.366 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-23T08:35:13.377 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-23T08:35:13.377 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:35:13.377 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T08:35:13.396 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:35:13.396 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T08:35:13.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T08:35:13.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.612+0000 7f260747a700 1 -- 172.21.15.70:0/162206448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600103690 msgr2=0x7f26000fcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.612+0000 7f260747a700 1 --2- 172.21.15.70:0/162206448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600103690 0x7f26000fcd00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25f0004940 tx=0x7f25f0009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 -- 172.21.15.70:0/162206448 shutdown_connections 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 --2- 172.21.15.70:0/162206448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600103690 0x7f26000fcd00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 --2- 172.21.15.70:0/162206448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2600102cf0 0x7f26001030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 -- 172.21.15.70:0/162206448 >> 172.21.15.70:0/162206448 conn(0x7f26000f8800 msgr2=0x7f26000fac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 -- 172.21.15.70:0/162206448 shutdown_connections 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 -- 172.21.15.70:0/162206448 wait complete. 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 Processor -- start 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 -- start start 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600102cf0 0x7f26001137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2600103690 0x7f2600113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2600114330 con 0x7f2600102cf0 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f260747a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260010d930 con 0x7f2600103690 2024-03-23T08:35:13.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f2605216700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600102cf0 0x7f26001137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f2605216700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600102cf0 0x7f26001137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59632/0 (socket says 172.21.15.70:59632) 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.613+0000 7f2605216700 1 -- 172.21.15.70:0/3803903327 learned_addr learned my addr 172.21.15.70:0/3803903327 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f2604a15700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2600103690 0x7f2600113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f2605216700 1 -- 172.21.15.70:0/3803903327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2600103690 msgr2=0x7f2600113ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f2605216700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2600103690 0x7f2600113ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f2605216700 1 -- 172.21.15.70:0/3803903327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f0005040 con 0x7f2600102cf0 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f2605216700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600102cf0 0x7f26001137a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c830 tx=0x7f25fc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f25f5ffb700 1 -- 172.21.15.70:0/3803903327 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25fc00a760 con 0x7f2600102cf0 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f25f5ffb700 1 -- 172.21.15.70:0/3803903327 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25fc00ad70 con 0x7f2600102cf0 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260010dc10 con 0x7f2600102cf0 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f25f5ffb700 1 -- 172.21.15.70:0/3803903327 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25fc0113a0 con 0x7f2600102cf0 2024-03-23T08:35:13.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.614+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260010e1d0 con 0x7f2600102cf0 2024-03-23T08:35:13.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.615+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26000fe440 con 0x7f2600102cf0 2024-03-23T08:35:13.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.615+0000 7f25f5ffb700 1 -- 172.21.15.70:0/3803903327 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f25fc012400 con 0x7f2600102cf0 2024-03-23T08:35:13.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.616+0000 7f25f5ffb700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f25ec06cc20 0x7f25ec06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:13.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.616+0000 7f25f5ffb700 1 -- 172.21.15.70:0/3803903327 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f25fc08c0a0 con 0x7f2600102cf0 2024-03-23T08:35:13.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.616+0000 7f2604a15700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f25ec06cc20 0x7f25ec06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:13.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.616+0000 7f2604a15700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f25ec06cc20 0x7f25ec06f0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f260010eaf0 tx=0x7f25f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:13.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.620+0000 7f25f5ffb700 1 -- 172.21.15.70:0/3803903327 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f25fc059b70 con 0x7f2600102cf0 2024-03-23T08:35:13.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.819+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2600109c90 con 0x7f2600102cf0 2024-03-23T08:35:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.820+0000 7f25f5ffb700 1 -- 172.21.15.70:0/3803903327 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f25fc059700 con 0x7f2600102cf0 2024-03-23T08:35:13.820 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:13.820 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":85,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","created":"2024-03-23T08:13:36.916195+0000","modified":"2024-03-23T08:33:05.994824+0000","last_up_change":"2024-03-23T08:32:03.566601+0000","last_in_change":"2024-03-23T08:18:58.641957+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T08:18:02.704319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-23T08:20:36.515704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-23T08:20:37.038000+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a71163c1-e6c6-490d-ade2-f345e4b22ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":51,"up_from":55,"up_thru":80,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6803","nonce":2783750869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6805","nonce":2783750869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6809","nonce":2783750869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6807","nonce":2783750869}]},"public_addr":"172.21.15.70:6803/2783750869","cluster_addr":"172.21.15.70:6805/2783750869","heartbeat_back_addr":"172.21.15.70:6809/2783750869","heartbeat_front_addr":"172.21.15.70:6807/2783750869","state":["exists","up"]},{"osd":1,"uuid":"0054cfce-6cdc-46f2-b434-ecd02d1ad193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":56,"up_from":60,"up_thru":80,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6811","nonce":1925098063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6813","nonce":1925098063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6817","nonce":1925098063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6815","nonce":1925098063}]},"public_addr":"172.21.15.70:6811/1925098063","cluster_addr":"172.21.15.70:6813/1925098063","heartbeat_back_addr":"172.21.15.70:6817/1925098063","heartbeat_front_addr":"172.21.15.70:6815/1925098063","state":["exists","up"]},{"osd":2,"uuid":"24c96238-b75f-47c4-b8ef-3794e45e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":61,"up_from":65,"up_thru":80,"down_at":62,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6819","nonce":357933512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6821","nonce":357933512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6825","nonce":357933512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6823","nonce":357933512}]},"public_addr":"172.21.15.70:6819/357933512","cluster_addr":"172.21.15.70:6821/357933512","heartbeat_back_addr":"172.21.15.70:6825/357933512","heartbeat_front_addr":"172.21.15.70:6823/357933512","state":["exists","up"]},{"osd":3,"uuid":"64e802c4-a037-4a42-8f0f-14756b62251b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":28,"last_clean_end":66,"up_from":70,"up_thru":80,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6801","nonce":457144182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6803","nonce":457144182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6807","nonce":457144182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6805","nonce":457144182}]},"public_addr":"172.21.15.193:6801/457144182","cluster_addr":"172.21.15.193:6803/457144182","heartbeat_back_addr":"172.21.15.193:6807/457144182","heartbeat_front_addr":"172.21.15.193:6805/457144182","state":["exists","up"]},{"osd":4,"uuid":"3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":36,"last_clean_end":71,"up_from":75,"up_thru":80,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6809","nonce":3936171891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6811","nonce":3936171891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6815","nonce":3936171891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6813","nonce":3936171891}]},"public_addr":"172.21.15.193:6809/3936171891","cluster_addr":"172.21.15.193:6811/3936171891","heartbeat_back_addr":"172.21.15.193:6815/3936171891","heartbeat_front_addr":"172.21.15.193:6813/3936171891","state":["exists","up"]},{"osd":5,"uuid":"57e8b767-4e62-41b1-8d96-caeaad28c929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":41,"last_clean_end":76,"up_from":80,"up_thru":80,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6817","nonce":2211659369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6819","nonce":2211659369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6823","nonce":2211659369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6821","nonce":2211659369}]},"public_addr":"172.21.15.193:6817/2211659369","cluster_addr":"172.21.15.193:6819/2211659369","heartbeat_back_addr":"172.21.15.193:6823/2211659369","heartbeat_front_addr":"172.21.15.193:6821/2211659369","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:01.224711+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:29.865012+0000","dead_epoch":56},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:58.894993+0000","dead_epoch":61},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:23.061992+0000","dead_epoch":66},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:49.665476+0000","dead_epoch":71},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:19:16.008523+0000","dead_epoch":76}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6825/3335188435":"2024-03-24T08:32:48.851367+0000","172.21.15.70:6829/2658800706":"2024-03-24T08:32:27.286778+0000","172.21.15.70:6800/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.193:6824/3335188435":"2024-03-24T08:32:48.851367+0000","172.21.15.70:6801/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/3721968086":"2024-03-24T08:15:48.390242+0000","172.21.15.193:0/4291429451":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/2658612358":"2024-03-24T08:27:46.544317+0000","172.21.15.70:0/3991559747":"2024-03-24T08:15:48.390242+0000","172.21.15.193:0/1987061818":"2024-03-24T08:24:45.131405+0000","172.21.15.70:6801/244960840":"2024-03-24T08:27:46.544317+0000","172.21.15.70:0/70452311":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/562426678":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6826/1612292110":"2024-03-24T08:20:52.080956+0000","172.21.15.193:6828/758671107":"2024-03-24T08:24:45.131405+0000","172.21.15.193:6829/758671107":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/3004659310":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.193:0/413036228":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/3009100678":"2024-03-24T08:22:46.280738+0000","172.21.15.70:6800/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/2209675146":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/1281398424":"2024-03-24T08:22:46.280738+0000","172.21.15.70:6827/1612292110":"2024-03-24T08:20:52.080956+0000","172.21.15.193:0/1836969396":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/370857700":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/104037264":"2024-03-24T08:27:46.544317+0000","172.21.15.70:6801/3694824274":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/3484360896":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/1608386260":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/400165096":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/3894371824":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6800/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6800/244960840":"2024-03-24T08:27:46.544317+0000","172.21.15.70:6800/3694824274":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/1984770541":"2024-03-24T08:27:46.544317+0000","172.21.15.70:0/2973224698":"2024-03-24T08:27:46.544317+0000","172.21.15.70:6828/2658800706":"2024-03-24T08:32:27.286778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.822+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f25ec06cc20 msgr2=0x7f25ec06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.822+0000 7f260747a700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f25ec06cc20 0x7f25ec06f0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f260010eaf0 tx=0x7f25f0009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.822+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600102cf0 msgr2=0x7f26001137a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.822+0000 7f260747a700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600102cf0 0x7f26001137a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c830 tx=0x7f25fc004470 comp rx=0 tx=0).stop 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.822+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 shutdown_connections 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.822+0000 7f260747a700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f25ec06cc20 0x7f25ec06f0d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.822+0000 7f260747a700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2600102cf0 0x7f26001137a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.822+0000 7f260747a700 1 --2- 172.21.15.70:0/3803903327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2600103690 0x7f2600113ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.823+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 >> 172.21.15.70:0/3803903327 conn(0x7f26000f8800 msgr2=0x7f26000f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.823+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 shutdown_connections 2024-03-23T08:35:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.823+0000 7f260747a700 1 -- 172.21.15.70:0/3803903327 wait complete. 2024-03-23T08:35:13.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T08:18:02.704319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-23T08:20:36.515704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-23T08:20:37.038000+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T08:35:13.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T08:35:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.955+0000 7ffb09734700 1 -- 172.21.15.70:0/2205474320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fe6e0 msgr2=0x7ffb04106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.955+0000 7ffb09734700 1 --2- 172.21.15.70:0/2205474320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fe6e0 0x7ffb04106d20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4004940 tx=0x7ffaf4009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.955+0000 7ffb09734700 1 -- 172.21.15.70:0/2205474320 shutdown_connections 2024-03-23T08:35:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.955+0000 7ffb09734700 1 --2- 172.21.15.70:0/2205474320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fe6e0 0x7ffb04106d20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.955+0000 7ffb09734700 1 --2- 172.21.15.70:0/2205474320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb040fdd40 0x7ffb040fe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.955+0000 7ffb09734700 1 -- 172.21.15.70:0/2205474320 >> 172.21.15.70:0/2205474320 conn(0x7ffb040f7710 msgr2=0x7ffb040f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.955+0000 7ffb09734700 1 -- 172.21.15.70:0/2205474320 shutdown_connections 2024-03-23T08:35:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.955+0000 7ffb09734700 1 -- 172.21.15.70:0/2205474320 wait complete. 2024-03-23T08:35:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb09734700 1 Processor -- start 2024-03-23T08:35:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb09734700 1 -- start start 2024-03-23T08:35:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb09734700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fdd40 0x7ffb041136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb09734700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb040fe6e0 0x7ffb04113c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb09734700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04114250 con 0x7ffb040fdd40 2024-03-23T08:35:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb09734700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0410d870 con 0x7ffb040fe6e0 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb02ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fdd40 0x7ffb041136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb02ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fdd40 0x7ffb041136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59648/0 (socket says 172.21.15.70:59648) 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb02ffd700 1 -- 172.21.15.70:0/4009855692 learned_addr learned my addr 172.21.15.70:0/4009855692 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb027fc700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb040fe6e0 0x7ffb04113c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb02ffd700 1 -- 172.21.15.70:0/4009855692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb040fe6e0 msgr2=0x7ffb04113c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb02ffd700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb040fe6e0 0x7ffb04113c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb02ffd700 1 -- 172.21.15.70:0/4009855692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4005040 con 0x7ffb040fdd40 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffb02ffd700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fdd40 0x7ffb041136c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8002a50 tx=0x7ffaf80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.956+0000 7ffaeb7fe700 1 -- 172.21.15.70:0/4009855692 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaf8007cc0 con 0x7ffb040fdd40 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.957+0000 7ffaeb7fe700 1 -- 172.21.15.70:0/4009855692 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffaf800f040 con 0x7ffb040fdd40 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.957+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0410db50 con 0x7ffb040fdd40 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.957+0000 7ffaeb7fe700 1 -- 172.21.15.70:0/4009855692 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaf8009540 con 0x7ffb040fdd40 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.957+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0410e090 con 0x7ffb040fdd40 2024-03-23T08:35:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.957+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb040664b0 con 0x7ffb040fdd40 2024-03-23T08:35:13.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.958+0000 7ffaeb7fe700 1 -- 172.21.15.70:0/4009855692 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ffaf80096a0 con 0x7ffb040fdd40 2024-03-23T08:35:13.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.958+0000 7ffaeb7fe700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ffaec06cc20 0x7ffaec06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:13.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.958+0000 7ffaeb7fe700 1 -- 172.21.15.70:0/4009855692 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffaf808be60 con 0x7ffb040fdd40 2024-03-23T08:35:13.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.958+0000 7ffb027fc700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ffaec06cc20 0x7ffaec06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:13.959 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.959+0000 7ffb027fc700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ffaec06cc20 0x7ffaec06f0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b3b0 tx=0x7ffaf4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:13.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:13.963+0000 7ffaeb7fe700 1 -- 172.21.15.70:0/4009855692 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffaf80598b0 con 0x7ffb040fdd40 2024-03-23T08:35:14.154 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.154+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb0410ed50 con 0x7ffb040fdd40 2024-03-23T08:35:14.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.155+0000 7ffaeb7fe700 1 -- 172.21.15.70:0/4009855692 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffaf8059440 con 0x7ffb040fdd40 2024-03-23T08:35:14.155 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-03-23T08:35:14.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ffaec06cc20 msgr2=0x7ffaec06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ffaec06cc20 0x7ffaec06f0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b3b0 tx=0x7ffaf4009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:14.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fdd40 msgr2=0x7ffb041136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fdd40 0x7ffb041136c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8002a50 tx=0x7ffaf80046b0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 shutdown_connections 2024-03-23T08:35:14.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7ffaec06cc20 0x7ffaec06f0d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb040fdd40 0x7ffb041136c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 --2- 172.21.15.70:0/4009855692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb040fe6e0 0x7ffb04113c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 >> 172.21.15.70:0/4009855692 conn(0x7ffb040f7710 msgr2=0x7ffb041068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:14.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 shutdown_connections 2024-03-23T08:35:14.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.157+0000 7ffb09734700 1 -- 172.21.15.70:0/4009855692 wait complete. 2024-03-23T08:35:14.168 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:13 smithi070 ceph-mon[145869]: pgmap v260: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:14.168 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:13 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1924525778' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-23T08:35:14.168 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:13 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1543448464' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-23T08:35:14.168 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:13 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3803903327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:35:14.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-23T08:35:14.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 -- 172.21.15.70:0/278529608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 msgr2=0x7fce54101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 --2- 172.21.15.70:0/278529608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 0x7fce54101fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fce3c004940 tx=0x7fce3c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:14.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 -- 172.21.15.70:0/278529608 shutdown_connections 2024-03-23T08:35:14.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 --2- 172.21.15.70:0/278529608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce54102580 0x7fce540fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 --2- 172.21.15.70:0/278529608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 0x7fce54101fb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 -- 172.21.15.70:0/278529608 >> 172.21.15.70:0/278529608 conn(0x7fce54072ad0 msgr2=0x7fce54072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:14.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 -- 172.21.15.70:0/278529608 shutdown_connections 2024-03-23T08:35:14.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 -- 172.21.15.70:0/278529608 wait complete. 2024-03-23T08:35:14.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 Processor -- start 2024-03-23T08:35:14.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 -- start start 2024-03-23T08:35:14.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.290+0000 7fce5a682700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 0x7fce541126c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:14.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce5a682700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce54102580 0x7fce54112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce5a682700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce54113290 con 0x7fce54101be0 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce5a682700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5410c850 con 0x7fce54102580 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce53fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 0x7fce541126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce53fff700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 0x7fce541126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59662/0 (socket says 172.21.15.70:59662) 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce53fff700 1 -- 172.21.15.70:0/1321178752 learned_addr learned my addr 172.21.15.70:0/1321178752 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce537fe700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce54102580 0x7fce54112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce53fff700 1 -- 172.21.15.70:0/1321178752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce54102580 msgr2=0x7fce54112c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce53fff700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce54102580 0x7fce54112c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce53fff700 1 -- 172.21.15.70:0/1321178752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c005040 con 0x7fce54101be0 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce53fff700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 0x7fce541126c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009180 tx=0x7fce3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce50ff9700 1 -- 172.21.15.70:0/1321178752 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3c00e040 con 0x7fce54101be0 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce50ff9700 1 -- 172.21.15.70:0/1321178752 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fce3c004290 con 0x7fce54101be0 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5410ca80 con 0x7fce54101be0 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce50ff9700 1 -- 172.21.15.70:0/1321178752 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3c01c3f0 con 0x7fce54101be0 2024-03-23T08:35:14.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.291+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5410cfc0 con 0x7fce54101be0 2024-03-23T08:35:14.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.292+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce540fd330 con 0x7fce54101be0 2024-03-23T08:35:14.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.293+0000 7fce50ff9700 1 -- 172.21.15.70:0/1321178752 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fce3c013790 con 0x7fce54101be0 2024-03-23T08:35:14.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.293+0000 7fce50ff9700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce4006cc20 0x7fce4006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:14.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.293+0000 7fce50ff9700 1 -- 172.21.15.70:0/1321178752 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fce3c08d2f0 con 0x7fce54101be0 2024-03-23T08:35:14.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.293+0000 7fce537fe700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce4006cc20 0x7fce4006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:14.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.294+0000 7fce537fe700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce4006cc20 0x7fce4006f0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fce44007fd0 tx=0x7fce44009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:14.297 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.298+0000 7fce50ff9700 1 -- 172.21.15.70:0/1321178752 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fce3c05adc0 con 0x7fce54101be0 2024-03-23T08:35:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:13 smithi193 ceph-mon[108533]: pgmap v260: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:13 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1924525778' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-23T08:35:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:13 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1543448464' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-23T08:35:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:13 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3803903327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:35:14.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.490+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fce5410dd10 con 0x7fce54101be0 2024-03-23T08:35:14.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.490+0000 7fce50ff9700 1 -- 172.21.15.70:0/1321178752 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fce3c013420 con 0x7fce54101be0 2024-03-23T08:35:14.490 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-03-23T08:35:14.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.492+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce4006cc20 msgr2=0x7fce4006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.492+0000 7fce5a682700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce4006cc20 0x7fce4006f0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fce44007fd0 tx=0x7fce44009040 comp rx=0 tx=0).stop 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.492+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 msgr2=0x7fce541126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.492+0000 7fce5a682700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 0x7fce541126c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009180 tx=0x7fce3c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.492+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 shutdown_connections 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.492+0000 7fce5a682700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fce4006cc20 0x7fce4006f0d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.492+0000 7fce5a682700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce54101be0 0x7fce541126c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.492+0000 7fce5a682700 1 --2- 172.21.15.70:0/1321178752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce54102580 0x7fce54112c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.492+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 >> 172.21.15.70:0/1321178752 conn(0x7fce54072ad0 msgr2=0x7fce54104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.493+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 shutdown_connections 2024-03-23T08:35:14.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.493+0000 7fce5a682700 1 -- 172.21.15.70:0/1321178752 wait complete. 2024-03-23T08:35:14.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-23T08:35:14.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.625+0000 7fb9d2224700 1 -- 172.21.15.70:0/350538500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 msgr2=0x7fb9cc101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.625+0000 7fb9d2224700 1 --2- 172.21.15.70:0/350538500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 0x7fb9cc101fb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0004940 tx=0x7fb9c0009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.625+0000 7fb9d2224700 1 -- 172.21.15.70:0/350538500 shutdown_connections 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.625+0000 7fb9d2224700 1 --2- 172.21.15.70:0/350538500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc102580 0x7fb9cc0fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.625+0000 7fb9d2224700 1 --2- 172.21.15.70:0/350538500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 0x7fb9cc101fb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.625+0000 7fb9d2224700 1 -- 172.21.15.70:0/350538500 >> 172.21.15.70:0/350538500 conn(0x7fb9cc072ad0 msgr2=0x7fb9cc072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.625+0000 7fb9d2224700 1 -- 172.21.15.70:0/350538500 shutdown_connections 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.625+0000 7fb9d2224700 1 -- 172.21.15.70:0/350538500 wait complete. 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.625+0000 7fb9d2224700 1 Processor -- start 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9d2224700 1 -- start start 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9d2224700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 0x7fb9cc10e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9d2224700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc102580 0x7fb9cc10ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9d2224700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc10f3b0 con 0x7fb9cc101be0 2024-03-23T08:35:14.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9d2224700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc10f4f0 con 0x7fb9cc102580 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9cb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 0x7fb9cc10e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9cb7fe700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 0x7fb9cc10e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59680/0 (socket says 172.21.15.70:59680) 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9cb7fe700 1 -- 172.21.15.70:0/3555780113 learned_addr learned my addr 172.21.15.70:0/3555780113 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9caffd700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc102580 0x7fb9cc10ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9cb7fe700 1 -- 172.21.15.70:0/3555780113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc102580 msgr2=0x7fb9cc10ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9cb7fe700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc102580 0x7fb9cc10ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9cb7fe700 1 -- 172.21.15.70:0/3555780113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0005040 con 0x7fb9cc101be0 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9cb7fe700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 0x7fb9cc10e930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009360 tx=0x7fb9c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.626+0000 7fb9b3fff700 1 -- 172.21.15.70:0/3555780113 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9c000e040 con 0x7fb9cc101be0 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.627+0000 7fb9b3fff700 1 -- 172.21.15.70:0/3555780113 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb9c0013420 con 0x7fb9cc101be0 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.627+0000 7fb9b3fff700 1 -- 172.21.15.70:0/3555780113 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9c001c5f0 con 0x7fb9cc101be0 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.627+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc10f770 con 0x7fb9cc101be0 2024-03-23T08:35:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.627+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc117cf0 con 0x7fb9cc101be0 2024-03-23T08:35:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.628+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc0fd330 con 0x7fb9cc101be0 2024-03-23T08:35:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.628+0000 7fb9b3fff700 1 -- 172.21.15.70:0/3555780113 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb9c0013590 con 0x7fb9cc101be0 2024-03-23T08:35:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.628+0000 7fb9b3fff700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb9b406cc20 0x7fb9b406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.629+0000 7fb9b3fff700 1 -- 172.21.15.70:0/3555780113 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb9c008d1b0 con 0x7fb9cc101be0 2024-03-23T08:35:14.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.629+0000 7fb9caffd700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb9b406cc20 0x7fb9b406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:14.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.629+0000 7fb9caffd700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb9b406cc20 0x7fb9b406f0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc007fd0 tx=0x7fb9bc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:14.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.633+0000 7fb9b3fff700 1 -- 172.21.15.70:0/3555780113 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb9c00572b0 con 0x7fb9cc101be0 2024-03-23T08:35:14.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.825+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb9cc108be0 con 0x7fb9cc101be0 2024-03-23T08:35:14.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.825+0000 7fb9b3fff700 1 -- 172.21.15.70:0/3555780113 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb9c005a8d0 con 0x7fb9cc101be0 2024-03-23T08:35:14.825 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb9b406cc20 msgr2=0x7fb9b406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb9b406cc20 0x7fb9b406f0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc007fd0 tx=0x7fb9bc009040 comp rx=0 tx=0).stop 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 msgr2=0x7fb9cc10e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 0x7fb9cc10e930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009360 tx=0x7fb9c0009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 shutdown_connections 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb9b406cc20 0x7fb9b406f0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc101be0 0x7fb9cc10e930 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 --2- 172.21.15.70:0/3555780113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc102580 0x7fb9cc10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 >> 172.21.15.70:0/3555780113 conn(0x7fb9cc072ad0 msgr2=0x7fb9cc104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 shutdown_connections 2024-03-23T08:35:14.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.827+0000 7fb9d2224700 1 -- 172.21.15.70:0/3555780113 wait complete. 2024-03-23T08:35:14.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T08:35:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 -- 172.21.15.70:0/1160942395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 msgr2=0x7f63b4101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 --2- 172.21.15.70:0/1160942395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 0x7f63b4101fb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f63b0004940 tx=0x7f63b0009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 -- 172.21.15.70:0/1160942395 shutdown_connections 2024-03-23T08:35:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 --2- 172.21.15.70:0/1160942395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b4102580 0x7f63b40fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 --2- 172.21.15.70:0/1160942395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 0x7f63b4101fb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.961 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 -- 172.21.15.70:0/1160942395 >> 172.21.15.70:0/1160942395 conn(0x7f63b4072ad0 msgr2=0x7f63b4072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 -- 172.21.15.70:0/1160942395 shutdown_connections 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 -- 172.21.15.70:0/1160942395 wait complete. 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 Processor -- start 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 -- start start 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 0x7f63b41126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b4102580 0x7f63b4112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4113270 con 0x7f63b4101be0 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.959+0000 7f63bc7d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b410c830 con 0x7f63b4102580 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63ba571700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 0x7f63b41126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63ba571700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 0x7f63b41126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59702/0 (socket says 172.21.15.70:59702) 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63ba571700 1 -- 172.21.15.70:0/2340260471 learned_addr learned my addr 172.21.15.70:0/2340260471 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63b9d70700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b4102580 0x7f63b4112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:14.962 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63ba571700 1 -- 172.21.15.70:0/2340260471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b4102580 msgr2=0x7f63b4112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63ba571700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b4102580 0x7f63b4112c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63ba571700 1 -- 172.21.15.70:0/2340260471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0005040 con 0x7f63b4101be0 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63b9d70700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b4102580 0x7f63b4112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63ba571700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 0x7f63b41126a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f63b000b3b0 tx=0x7f63b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63aaffd700 1 -- 172.21.15.70:0/2340260471 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b000e040 con 0x7f63b4101be0 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63aaffd700 1 -- 172.21.15.70:0/2340260471 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63b0004290 con 0x7f63b4101be0 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b410ca60 con 0x7f63b4101be0 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63aaffd700 1 -- 172.21.15.70:0/2340260471 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b0013660 con 0x7f63b4101be0 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.960+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b410cf40 con 0x7f63b4101be0 2024-03-23T08:35:14.963 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.961+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b40fd330 con 0x7f63b4101be0 2024-03-23T08:35:14.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.961+0000 7f63aaffd700 1 -- 172.21.15.70:0/2340260471 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f63b001e420 con 0x7f63b4101be0 2024-03-23T08:35:14.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.962+0000 7f63aaffd700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f63a006cc20 0x7f63a006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:14.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.962+0000 7f63aaffd700 1 -- 172.21.15.70:0/2340260471 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f63b008fe20 con 0x7f63b4101be0 2024-03-23T08:35:14.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.962+0000 7f63b9d70700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f63a006cc20 0x7f63a006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:14.964 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.962+0000 7f63b9d70700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f63a006cc20 0x7f63a006f0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009fd0 tx=0x7f63a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:14.966 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:14.966+0000 7f63aaffd700 1 -- 172.21.15.70:0/2340260471 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f63b005d8f0 con 0x7f63b4101be0 2024-03-23T08:35:15.177 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.177+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f63b410dd90 con 0x7f63b4101be0 2024-03-23T08:35:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.178+0000 7f63aaffd700 1 -- 172.21.15.70:0/2340260471 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v85) v1 ==== 81+0+2313 (secure 0 0 0) 0x7f63b005d480 con 0x7f63b4101be0 2024-03-23T08:35:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:6825/3335188435 2024-03-24T08:32:48.851367+0000 2024-03-23T08:35:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6829/2658800706 2024-03-24T08:32:27.286778+0000 2024-03-23T08:35:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2732684403 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:6824/3335188435 2024-03-24T08:32:48.851367+0000 2024-03-23T08:35:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2732684403 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3721968086 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:0/4291429451 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:15.178 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2658612358 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3991559747 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:0/1987061818 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/244960840 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/70452311 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/562426678 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6826/1612292110 2024-03-24T08:20:52.080956+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:6828/758671107 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:6829/758671107 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3004659310 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2149030412 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:0/413036228 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3009100678 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2149030412 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2209675146 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1281398424 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6827/1612292110 2024-03-24T08:20:52.080956+0000 2024-03-23T08:35:15.179 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:0/1836969396 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/370857700 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/104037264 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/3694824274 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3484360896 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1608386260 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/400165096 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3894371824 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2736698870 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2736698870 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/244960840 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/3694824274 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1984770541 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2973224698 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:15.180 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6828/2658800706 2024-03-24T08:32:27.286778+0000 2024-03-23T08:35:15.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f63a006cc20 msgr2=0x7f63a006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.181 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f63a006cc20 0x7f63a006f0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009fd0 tx=0x7f63a400b040 comp rx=0 tx=0).stop 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 msgr2=0x7f63b41126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 0x7f63b41126a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f63b000b3b0 tx=0x7f63b0009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 shutdown_connections 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f63a006cc20 0x7f63a006f0d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b4101be0 0x7f63b41126a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 --2- 172.21.15.70:0/2340260471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b4102580 0x7f63b4112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 >> 172.21.15.70:0/2340260471 conn(0x7f63b4072ad0 msgr2=0x7f63b4104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 shutdown_connections 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.180+0000 7f63bc7d5700 1 -- 172.21.15.70:0/2340260471 wait complete. 2024-03-23T08:35:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:listed 39 entries 2024-03-23T08:35:15.191 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:14 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/4009855692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:35:15.191 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:14 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1321178752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-23T08:35:15.191 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:14 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3555780113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-23T08:35:15.192 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-03-23T08:35:15.192 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T08:35:15.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T08:35:15.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:14 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/4009855692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:35:15.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:14 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1321178752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-23T08:35:15.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:14 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3555780113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-23T08:35:15.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.370+0000 7fa767c58700 1 -- 172.21.15.70:0/384153301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 msgr2=0x7fa7600fb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.370+0000 7fa767c58700 1 --2- 172.21.15.70:0/384153301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 0x7fa7600fb740 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa75c004940 tx=0x7fa75c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.370+0000 7fa767c58700 1 -- 172.21.15.70:0/384153301 shutdown_connections 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.370+0000 7fa767c58700 1 --2- 172.21.15.70:0/384153301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 0x7fa7600fb740 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.370+0000 7fa767c58700 1 --2- 172.21.15.70:0/384153301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa760102de0 0x7fa7601031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.370+0000 7fa767c58700 1 -- 172.21.15.70:0/384153301 >> 172.21.15.70:0/384153301 conn(0x7fa760072230 msgr2=0x7fa760072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.370+0000 7fa767c58700 1 -- 172.21.15.70:0/384153301 shutdown_connections 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.370+0000 7fa767c58700 1 -- 172.21.15.70:0/384153301 wait complete. 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa767c58700 1 Processor -- start 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa767c58700 1 -- start start 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa767c58700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 0x7fa7601137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa767c58700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa760102de0 0x7fa760113d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa767c58700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa760114380 con 0x7fa7600fb2d0 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa767c58700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76010d980 con 0x7fa760102de0 2024-03-23T08:35:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7659f4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 0x7fa7601137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7659f4700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 0x7fa7601137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59722/0 (socket says 172.21.15.70:59722) 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7659f4700 1 -- 172.21.15.70:0/1546042377 learned_addr learned my addr 172.21.15.70:0/1546042377 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7651f3700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa760102de0 0x7fa760113d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7659f4700 1 -- 172.21.15.70:0/1546042377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa760102de0 msgr2=0x7fa760113d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7659f4700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa760102de0 0x7fa760113d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7659f4700 1 -- 172.21.15.70:0/1546042377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c005040 con 0x7fa7600fb2d0 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7659f4700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 0x7fa7601137f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa750002a80 tx=0x7fa7500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7567fc700 1 -- 172.21.15.70:0/1546042377 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa750007cc0 con 0x7fa7600fb2d0 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7567fc700 1 -- 172.21.15.70:0/1546042377 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa75000f040 con 0x7fa7600fb2d0 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa7567fc700 1 -- 172.21.15.70:0/1546042377 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa750009640 con 0x7fa7600fb2d0 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.371+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76010dc60 con 0x7fa7600fb2d0 2024-03-23T08:35:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.372+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76010e1a0 con 0x7fa7600fb2d0 2024-03-23T08:35:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.372+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7601051c0 con 0x7fa7600fb2d0 2024-03-23T08:35:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.373+0000 7fa7567fc700 1 -- 172.21.15.70:0/1546042377 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa7500097a0 con 0x7fa7600fb2d0 2024-03-23T08:35:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.373+0000 7fa7567fc700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa74c06cc20 0x7fa74c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.374+0000 7fa7567fc700 1 -- 172.21.15.70:0/1546042377 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa75008b760 con 0x7fa7600fb2d0 2024-03-23T08:35:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.374+0000 7fa7651f3700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa74c06cc20 0x7fa74c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:15.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.374+0000 7fa7651f3700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa74c06cc20 0x7fa74c06f0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00b3b0 tx=0x7fa75c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.378+0000 7fa7567fc700 1 -- 172.21.15.70:0/1546042377 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa750059230 con 0x7fa7600fb2d0 2024-03-23T08:35:15.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.574+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa76010ee40 con 0x7fa7600fb2d0 2024-03-23T08:35:15.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.575+0000 7fa7567fc700 1 -- 172.21.15.70:0/1546042377 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fa750058dc0 con 0x7fa7600fb2d0 2024-03-23T08:35:15.575 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-03-23T08:35:15.576 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":85,"fsid":"2762dd54-e8ed-11ee-95cd-87774f69a715","created":"2024-03-23T08:13:36.916195+0000","modified":"2024-03-23T08:33:05.994824+0000","last_up_change":"2024-03-23T08:32:03.566601+0000","last_in_change":"2024-03-23T08:18:58.641957+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T08:18:02.704319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-23T08:20:36.515704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-23T08:20:37.038000+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a71163c1-e6c6-490d-ade2-f345e4b22ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":51,"up_from":55,"up_thru":80,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6803","nonce":2783750869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6805","nonce":2783750869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6809","nonce":2783750869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2783750869},{"type":"v1","addr":"172.21.15.70:6807","nonce":2783750869}]},"public_addr":"172.21.15.70:6803/2783750869","cluster_addr":"172.21.15.70:6805/2783750869","heartbeat_back_addr":"172.21.15.70:6809/2783750869","heartbeat_front_addr":"172.21.15.70:6807/2783750869","state":["exists","up"]},{"osd":1,"uuid":"0054cfce-6cdc-46f2-b434-ecd02d1ad193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":56,"up_from":60,"up_thru":80,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6811","nonce":1925098063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6813","nonce":1925098063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6817","nonce":1925098063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1925098063},{"type":"v1","addr":"172.21.15.70:6815","nonce":1925098063}]},"public_addr":"172.21.15.70:6811/1925098063","cluster_addr":"172.21.15.70:6813/1925098063","heartbeat_back_addr":"172.21.15.70:6817/1925098063","heartbeat_front_addr":"172.21.15.70:6815/1925098063","state":["exists","up"]},{"osd":2,"uuid":"24c96238-b75f-47c4-b8ef-3794e45e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":61,"up_from":65,"up_thru":80,"down_at":62,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6819","nonce":357933512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6821","nonce":357933512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6825","nonce":357933512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":357933512},{"type":"v1","addr":"172.21.15.70:6823","nonce":357933512}]},"public_addr":"172.21.15.70:6819/357933512","cluster_addr":"172.21.15.70:6821/357933512","heartbeat_back_addr":"172.21.15.70:6825/357933512","heartbeat_front_addr":"172.21.15.70:6823/357933512","state":["exists","up"]},{"osd":3,"uuid":"64e802c4-a037-4a42-8f0f-14756b62251b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":28,"last_clean_end":66,"up_from":70,"up_thru":80,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6801","nonce":457144182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6803","nonce":457144182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6807","nonce":457144182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":457144182},{"type":"v1","addr":"172.21.15.193:6805","nonce":457144182}]},"public_addr":"172.21.15.193:6801/457144182","cluster_addr":"172.21.15.193:6803/457144182","heartbeat_back_addr":"172.21.15.193:6807/457144182","heartbeat_front_addr":"172.21.15.193:6805/457144182","state":["exists","up"]},{"osd":4,"uuid":"3ae8a2e3-fd84-4857-ac54-5e8e2631a4a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":36,"last_clean_end":71,"up_from":75,"up_thru":80,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6809","nonce":3936171891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6811","nonce":3936171891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6815","nonce":3936171891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3936171891},{"type":"v1","addr":"172.21.15.193:6813","nonce":3936171891}]},"public_addr":"172.21.15.193:6809/3936171891","cluster_addr":"172.21.15.193:6811/3936171891","heartbeat_back_addr":"172.21.15.193:6815/3936171891","heartbeat_front_addr":"172.21.15.193:6813/3936171891","state":["exists","up"]},{"osd":5,"uuid":"57e8b767-4e62-41b1-8d96-caeaad28c929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":41,"last_clean_end":76,"up_from":80,"up_thru":80,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6817","nonce":2211659369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6819","nonce":2211659369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6823","nonce":2211659369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2211659369},{"type":"v1","addr":"172.21.15.193:6821","nonce":2211659369}]},"public_addr":"172.21.15.193:6817/2211659369","cluster_addr":"172.21.15.193:6819/2211659369","heartbeat_back_addr":"172.21.15.193:6823/2211659369","heartbeat_front_addr":"172.21.15.193:6821/2211659369","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:01.224711+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:29.865012+0000","dead_epoch":56},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:17:58.894993+0000","dead_epoch":61},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:23.061992+0000","dead_epoch":66},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:18:49.665476+0000","dead_epoch":71},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T08:19:16.008523+0000","dead_epoch":76}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6825/3335188435":"2024-03-24T08:32:48.851367+0000","172.21.15.70:6829/2658800706":"2024-03-24T08:32:27.286778+0000","172.21.15.70:6800/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.193:6824/3335188435":"2024-03-24T08:32:48.851367+0000","172.21.15.70:6801/2732684403":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/3721968086":"2024-03-24T08:15:48.390242+0000","172.21.15.193:0/4291429451":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/2658612358":"2024-03-24T08:27:46.544317+0000","172.21.15.70:0/3991559747":"2024-03-24T08:15:48.390242+0000","172.21.15.193:0/1987061818":"2024-03-24T08:24:45.131405+0000","172.21.15.70:6801/244960840":"2024-03-24T08:27:46.544317+0000","172.21.15.70:0/70452311":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/562426678":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6826/1612292110":"2024-03-24T08:20:52.080956+0000","172.21.15.193:6828/758671107":"2024-03-24T08:24:45.131405+0000","172.21.15.193:6829/758671107":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/3004659310":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.193:0/413036228":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/3009100678":"2024-03-24T08:22:46.280738+0000","172.21.15.70:6800/2149030412":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/2209675146":"2024-03-24T08:15:48.390242+0000","172.21.15.70:0/1281398424":"2024-03-24T08:22:46.280738+0000","172.21.15.70:6827/1612292110":"2024-03-24T08:20:52.080956+0000","172.21.15.193:0/1836969396":"2024-03-24T08:24:45.131405+0000","172.21.15.70:0/370857700":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/104037264":"2024-03-24T08:27:46.544317+0000","172.21.15.70:6801/3694824274":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/3484360896":"2024-03-24T08:14:09.814224+0000","172.21.15.70:0/1608386260":"2024-03-24T08:14:42.564176+0000","172.21.15.70:0/400165096":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/3894371824":"2024-03-24T08:14:42.564176+0000","172.21.15.70:6800/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6801/2736698870":"2024-03-24T08:14:09.814224+0000","172.21.15.70:6800/244960840":"2024-03-24T08:27:46.544317+0000","172.21.15.70:6800/3694824274":"2024-03-24T08:22:46.280738+0000","172.21.15.70:0/1984770541":"2024-03-24T08:27:46.544317+0000","172.21.15.70:0/2973224698":"2024-03-24T08:27:46.544317+0000","172.21.15.70:6828/2658800706":"2024-03-24T08:32:27.286778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T08:35:15.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa74c06cc20 msgr2=0x7fa74c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa74c06cc20 0x7fa74c06f0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00b3b0 tx=0x7fa75c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 msgr2=0x7fa7601137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 0x7fa7601137f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa750002a80 tx=0x7fa7500046b0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 shutdown_connections 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa74c06cc20 0x7fa74c06f0d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7600fb2d0 0x7fa7601137f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 --2- 172.21.15.70:0/1546042377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa760102de0 0x7fa760113d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 >> 172.21.15.70:0/1546042377 conn(0x7fa760072230 msgr2=0x7fa7600faaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 shutdown_connections 2024-03-23T08:35:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.577+0000 7fa767c58700 1 -- 172.21.15.70:0/1546042377 wait complete. 2024-03-23T08:35:15.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T08:18:02.704319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-23T08:20:36.515704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-23T08:20:37.038000+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T08:35:15.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T08:35:15.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.713+0000 7fb3486d5700 1 -- 172.21.15.70:0/1892355812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340101be0 msgr2=0x7fb340101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.713+0000 7fb3486d5700 1 --2- 172.21.15.70:0/1892355812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340101be0 0x7fb340101fb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb33c004940 tx=0x7fb33c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 -- 172.21.15.70:0/1892355812 shutdown_connections 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 --2- 172.21.15.70:0/1892355812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340102580 0x7fb3400fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 --2- 172.21.15.70:0/1892355812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340101be0 0x7fb340101fb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 -- 172.21.15.70:0/1892355812 >> 172.21.15.70:0/1892355812 conn(0x7fb340072ad0 msgr2=0x7fb340072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 -- 172.21.15.70:0/1892355812 shutdown_connections 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 -- 172.21.15.70:0/1892355812 wait complete. 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 Processor -- start 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 -- start start 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340101be0 0x7fb3401136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340102580 0x7fb340113c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb340114250 con 0x7fb340102580 2024-03-23T08:35:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb3486d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34010d870 con 0x7fb340101be0 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb345c70700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340102580 0x7fb340113c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb345c70700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340102580 0x7fb340113c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59744/0 (socket says 172.21.15.70:59744) 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.714+0000 7fb345c70700 1 -- 172.21.15.70:0/3794792068 learned_addr learned my addr 172.21.15.70:0/3794792068 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb346471700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340101be0 0x7fb3401136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb345c70700 1 -- 172.21.15.70:0/3794792068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340101be0 msgr2=0x7fb3401136c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb345c70700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340101be0 0x7fb3401136c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb345c70700 1 -- 172.21.15.70:0/3794792068 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c005040 con 0x7fb340102580 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb346471700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340101be0 0x7fb3401136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb345c70700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340102580 0x7fb340113c00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb330002a50 tx=0x7fb3300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb336ffd700 1 -- 172.21.15.70:0/3794792068 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb330007cc0 con 0x7fb340102580 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34010db50 con 0x7fb340102580 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb336ffd700 1 -- 172.21.15.70:0/3794792068 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb33000f040 con 0x7fb340102580 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb336ffd700 1 -- 172.21.15.70:0/3794792068 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3300095d0 con 0x7fb340102580 2024-03-23T08:35:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.715+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34010e110 con 0x7fb340102580 2024-03-23T08:35:15.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.716+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3400fd330 con 0x7fb340102580 2024-03-23T08:35:15.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.716+0000 7fb336ffd700 1 -- 172.21.15.70:0/3794792068 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fb330009730 con 0x7fb340102580 2024-03-23T08:35:15.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.717+0000 7fb336ffd700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb32c06cc20 0x7fb32c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:15.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.717+0000 7fb336ffd700 1 -- 172.21.15.70:0/3794792068 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb33008bd30 con 0x7fb340102580 2024-03-23T08:35:15.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.717+0000 7fb346471700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb32c06cc20 0x7fb32c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:15.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.717+0000 7fb346471700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb32c06cc20 0x7fb32c06f0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb33c004560 tx=0x7fb33c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:15.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.721+0000 7fb336ffd700 1 -- 172.21.15.70:0/3794792068 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb330059780 con 0x7fb340102580 2024-03-23T08:35:15.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.916+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb340108b90 con 0x7fb340102580 2024-03-23T08:35:15.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.917+0000 7fb336ffd700 1 -- 172.21.15.70:0/3794792068 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb330059310 con 0x7fb340102580 2024-03-23T08:35:15.917 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb32c06cc20 msgr2=0x7fb32c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb32c06cc20 0x7fb32c06f0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb33c004560 tx=0x7fb33c009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340102580 msgr2=0x7fb340113c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340102580 0x7fb340113c00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb330002a50 tx=0x7fb3300046b0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 shutdown_connections 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fb32c06cc20 0x7fb32c06f0d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb340101be0 0x7fb3401136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 --2- 172.21.15.70:0/3794792068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb340102580 0x7fb340113c00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 >> 172.21.15.70:0/3794792068 conn(0x7fb340072ad0 msgr2=0x7fb3400fa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 shutdown_connections 2024-03-23T08:35:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:15.919+0000 7fb3486d5700 1 -- 172.21.15.70:0/3794792068 wait complete. 2024-03-23T08:35:15.931 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-23T08:35:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 -- 172.21.15.70:0/1317816860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 msgr2=0x7fbcf4101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 --2- 172.21.15.70:0/1317816860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 0x7fbcf4101fb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbce8004940 tx=0x7fbce8009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 -- 172.21.15.70:0/1317816860 shutdown_connections 2024-03-23T08:35:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 --2- 172.21.15.70:0/1317816860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcf4102580 0x7fbcf40fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 --2- 172.21.15.70:0/1317816860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 0x7fbcf4101fb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 -- 172.21.15.70:0/1317816860 >> 172.21.15.70:0/1317816860 conn(0x7fbcf4072ad0 msgr2=0x7fbcf4072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 -- 172.21.15.70:0/1317816860 shutdown_connections 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 -- 172.21.15.70:0/1317816860 wait complete. 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 Processor -- start 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.052+0000 7fbcfaf50700 1 -- start start 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcfaf50700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 0x7fbcf4115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcfaf50700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcf4102580 0x7fbcf4115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcfaf50700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf41164e0 con 0x7fbcf4101be0 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcfaf50700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf410fae0 con 0x7fbcf4102580 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf8cec700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 0x7fbcf4115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf8cec700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 0x7fbcf4115950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59754/0 (socket says 172.21.15.70:59754) 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf8cec700 1 -- 172.21.15.70:0/3399759431 learned_addr learned my addr 172.21.15.70:0/3399759431 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf3fff700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcf4102580 0x7fbcf4115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf8cec700 1 -- 172.21.15.70:0/3399759431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcf4102580 msgr2=0x7fbcf4115e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf8cec700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcf4102580 0x7fbcf4115e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf8cec700 1 -- 172.21.15.70:0/3399759431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce8005040 con 0x7fbcf4101be0 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf3fff700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcf4102580 0x7fbcf4115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf8cec700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 0x7fbcf4115950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbce800b3b0 tx=0x7fbce8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.053+0000 7fbcf17fa700 1 -- 172.21.15.70:0/3399759431 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbce800e040 con 0x7fbcf4101be0 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.054+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf410fd60 con 0x7fbcf4101be0 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.054+0000 7fbcf17fa700 1 -- 172.21.15.70:0/3399759431 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbce8013420 con 0x7fbcf4101be0 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.054+0000 7fbcf17fa700 1 -- 172.21.15.70:0/3399759431 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbce801c630 con 0x7fbcf4101be0 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.054+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf4110320 con 0x7fbcf4101be0 2024-03-23T08:35:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.054+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf40fd330 con 0x7fbcf4101be0 2024-03-23T08:35:16.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.055+0000 7fbcf17fa700 1 -- 172.21.15.70:0/3399759431 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbce8013970 con 0x7fbcf4101be0 2024-03-23T08:35:16.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.056+0000 7fbcf17fa700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbce406cc20 0x7fbce406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:16.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.056+0000 7fbcf17fa700 1 -- 172.21.15.70:0/3399759431 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbce808dd10 con 0x7fbcf4101be0 2024-03-23T08:35:16.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.056+0000 7fbcf3fff700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbce406cc20 0x7fbce406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:16.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.056+0000 7fbcf3fff700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbce406cc20 0x7fbce406f0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fd0 tx=0x7fbce0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:16.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.060+0000 7fbcf17fa700 1 -- 172.21.15.70:0/3399759431 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbce805b8a0 con 0x7fbcf4101be0 2024-03-23T08:35:16.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:15 smithi070 ceph-mon[145869]: pgmap v261: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:15 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/2340260471' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T08:35:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:15 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/1546042377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:35:16.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.252+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fbcf4064970 con 0x7fbcf4101be0 2024-03-23T08:35:16.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.252+0000 7fbcf17fa700 1 -- 172.21.15.70:0/3399759431 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fbce8011370 con 0x7fbcf4101be0 2024-03-23T08:35:16.252 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-03-23T08:35:16.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbce406cc20 msgr2=0x7fbce406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbce406cc20 0x7fbce406f0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fd0 tx=0x7fbce0009ee0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 msgr2=0x7fbcf4115950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 0x7fbcf4115950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbce800b3b0 tx=0x7fbce8009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 shutdown_connections 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fbce406cc20 0x7fbce406f0d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcf4101be0 0x7fbcf4115950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 --2- 172.21.15.70:0/3399759431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcf4102580 0x7fbcf4115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 >> 172.21.15.70:0/3399759431 conn(0x7fbcf4072ad0 msgr2=0x7fbcf40fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 shutdown_connections 2024-03-23T08:35:16.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.254+0000 7fbcfaf50700 1 -- 172.21.15.70:0/3399759431 wait complete. 2024-03-23T08:35:16.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-23T08:35:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:15 smithi193 ceph-mon[108533]: pgmap v261: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:15 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/2340260471' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T08:35:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:15 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/1546042377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-23T08:35:16.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.386+0000 7fa3a1965700 1 -- 172.21.15.70:0/2760491727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 msgr2=0x7fa39c0fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.386+0000 7fa3a1965700 1 --2- 172.21.15.70:0/2760491727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 0x7fa39c0fbbe0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa390004940 tx=0x7fa390009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:16.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.386+0000 7fa3a1965700 1 -- 172.21.15.70:0/2760491727 shutdown_connections 2024-03-23T08:35:16.388 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.386+0000 7fa3a1965700 1 --2- 172.21.15.70:0/2760491727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 0x7fa39c0fbbe0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.386+0000 7fa3a1965700 1 --2- 172.21.15.70:0/2760491727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa39c101bd0 0x7fa39c101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.386+0000 7fa3a1965700 1 -- 172.21.15.70:0/2760491727 >> 172.21.15.70:0/2760491727 conn(0x7fa39c0f7740 msgr2=0x7fa39c0f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.386+0000 7fa3a1965700 1 -- 172.21.15.70:0/2760491727 shutdown_connections 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.386+0000 7fa3a1965700 1 -- 172.21.15.70:0/2760491727 wait complete. 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.387+0000 7fa3a1965700 1 Processor -- start 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.387+0000 7fa3a1965700 1 -- start start 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.387+0000 7fa3a1965700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa39c101bd0 0x7fa39c10e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.387+0000 7fa3a1965700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 0x7fa39c10ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.387+0000 7fa3a1965700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c10f3d0 con 0x7fa39c102570 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.387+0000 7fa3a1965700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c10f510 con 0x7fa39c101bd0 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.387+0000 7fa39a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 0x7fa39c10ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:16.389 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.387+0000 7fa39a7fc700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 0x7fa39c10ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59770/0 (socket says 172.21.15.70:59770) 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.387+0000 7fa39a7fc700 1 -- 172.21.15.70:0/647314845 learned_addr learned my addr 172.21.15.70:0/647314845 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.388+0000 7fa39a7fc700 1 -- 172.21.15.70:0/647314845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa39c101bd0 msgr2=0x7fa39c10e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.388+0000 7fa39a7fc700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa39c101bd0 0x7fa39c10e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.388+0000 7fa39a7fc700 1 -- 172.21.15.70:0/647314845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa390005040 con 0x7fa39c102570 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.388+0000 7fa39a7fc700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 0x7fa39c10ee90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa390009380 tx=0x7fa390009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.388+0000 7fa3837fe700 1 -- 172.21.15.70:0/647314845 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa39000e040 con 0x7fa39c102570 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.388+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c10f790 con 0x7fa39c102570 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.388+0000 7fa3837fe700 1 -- 172.21.15.70:0/647314845 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa390013420 con 0x7fa39c102570 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.388+0000 7fa3837fe700 1 -- 172.21.15.70:0/647314845 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa39001c630 con 0x7fa39c102570 2024-03-23T08:35:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.388+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c117d60 con 0x7fa39c102570 2024-03-23T08:35:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.389+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c0fd320 con 0x7fa39c102570 2024-03-23T08:35:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.389+0000 7fa3837fe700 1 -- 172.21.15.70:0/647314845 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa3900139e0 con 0x7fa39c102570 2024-03-23T08:35:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.390+0000 7fa3837fe700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa38406cc20 0x7fa38406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.390+0000 7fa3837fe700 1 -- 172.21.15.70:0/647314845 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa39008d1c0 con 0x7fa39c102570 2024-03-23T08:35:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.390+0000 7fa39affd700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa38406cc20 0x7fa38406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:16.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.390+0000 7fa39affd700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa38406cc20 0x7fa38406f0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa38c006fd0 tx=0x7fa38c006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:16.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.394+0000 7fa3837fe700 1 -- 172.21.15.70:0/647314845 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa390057200 con 0x7fa39c102570 2024-03-23T08:35:16.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.587+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa39c064dc0 con 0x7fa39c102570 2024-03-23T08:35:16.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.587+0000 7fa3837fe700 1 -- 172.21.15.70:0/647314845 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa39005a820 con 0x7fa39c102570 2024-03-23T08:35:16.587 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2024-03-23T08:35:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa38406cc20 msgr2=0x7fa38406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa38406cc20 0x7fa38406f0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa38c006fd0 tx=0x7fa38c006db0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 msgr2=0x7fa39c10ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 0x7fa39c10ee90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa390009380 tx=0x7fa390009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 shutdown_connections 2024-03-23T08:35:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7fa38406cc20 0x7fa38406f0d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa39c101bd0 0x7fa39c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 --2- 172.21.15.70:0/647314845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa39c102570 0x7fa39c10ee90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 >> 172.21.15.70:0/647314845 conn(0x7fa39c0f7740 msgr2=0x7fa39c0f8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 shutdown_connections 2024-03-23T08:35:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.589+0000 7fa3a1965700 1 -- 172.21.15.70:0/647314845 wait complete. 2024-03-23T08:35:16.602 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T08:35:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 -- 172.21.15.70:0/1446433089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 msgr2=0x7f9078101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 --2- 172.21.15.70:0/1446433089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 0x7f9078101fb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9068004940 tx=0x7f9068009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 -- 172.21.15.70:0/1446433089 shutdown_connections 2024-03-23T08:35:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 --2- 172.21.15.70:0/1446433089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9078102580 0x7f90780fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 --2- 172.21.15.70:0/1446433089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 0x7f9078101fb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 -- 172.21.15.70:0/1446433089 >> 172.21.15.70:0/1446433089 conn(0x7f90780f7750 msgr2=0x7f90780f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 -- 172.21.15.70:0/1446433089 shutdown_connections 2024-03-23T08:35:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 -- 172.21.15.70:0/1446433089 wait complete. 2024-03-23T08:35:16.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 Processor -- start 2024-03-23T08:35:16.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 -- start start 2024-03-23T08:35:16.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 0x7f9078113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:16.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9078102580 0x7f9078113c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:16.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781142c0 con 0x7f9078101be0 2024-03-23T08:35:16.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.722+0000 7f907d701700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907810d8c0 con 0x7f9078102580 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f9076ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 0x7f9078113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f9076ffd700 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 0x7f9078113730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59786/0 (socket says 172.21.15.70:59786) 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f9076ffd700 1 -- 172.21.15.70:0/3680293577 learned_addr learned my addr 172.21.15.70:0/3680293577 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f90767fc700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9078102580 0x7f9078113c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f9076ffd700 1 -- 172.21.15.70:0/3680293577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9078102580 msgr2=0x7f9078113c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f9076ffd700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9078102580 0x7f9078113c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f9076ffd700 1 -- 172.21.15.70:0/3680293577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9068005040 con 0x7f9078101be0 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f9076ffd700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 0x7f9078113730 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9068000c00 tx=0x7f9068009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f905f7fe700 1 -- 172.21.15.70:0/3680293577 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9068021070 con 0x7f9078101be0 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f905f7fe700 1 -- 172.21.15.70:0/3680293577 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90680093b0 con 0x7f9078101be0 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907810db40 con 0x7f9078101be0 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f905f7fe700 1 -- 172.21.15.70:0/3680293577 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f906800e070 con 0x7f9078101be0 2024-03-23T08:35:16.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.723+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907810e100 con 0x7f9078101be0 2024-03-23T08:35:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.724+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90780fd330 con 0x7f9078101be0 2024-03-23T08:35:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.725+0000 7f905f7fe700 1 -- 172.21.15.70:0/3680293577 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f906801c490 con 0x7f9078101be0 2024-03-23T08:35:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.725+0000 7f905f7fe700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f906006cc20 0x7f906006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T08:35:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.725+0000 7f905f7fe700 1 -- 172.21.15.70:0/3680293577 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f906808d100 con 0x7f9078101be0 2024-03-23T08:35:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.725+0000 7f90767fc700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f906006cc20 0x7f906006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T08:35:16.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.726+0000 7f90767fc700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f906006cc20 0x7f906006f0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f907810ea80 tx=0x7f906c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T08:35:16.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.730+0000 7f905f7fe700 1 -- 172.21.15.70:0/3680293577 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f906805abd0 con 0x7f9078101be0 2024-03-23T08:35:16.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.946+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9078108be0 con 0x7f9078101be0 2024-03-23T08:35:16.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.946+0000 7f905f7fe700 1 -- 172.21.15.70:0/3680293577 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v85) v1 ==== 81+0+2313 (secure 0 0 0) 0x7f906805a760 con 0x7f9078101be0 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:6825/3335188435 2024-03-24T08:32:48.851367+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6829/2658800706 2024-03-24T08:32:27.286778+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2732684403 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:6824/3335188435 2024-03-24T08:32:48.851367+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2732684403 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3721968086 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:0/4291429451 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2658612358 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3991559747 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:0/1987061818 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:16.947 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/244960840 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/70452311 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/562426678 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6826/1612292110 2024-03-24T08:20:52.080956+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:6828/758671107 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:6829/758671107 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3004659310 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2149030412 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:0/413036228 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3009100678 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2149030412 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2209675146 2024-03-24T08:15:48.390242+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1281398424 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:16.948 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6827/1612292110 2024-03-24T08:20:52.080956+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.193:0/1836969396 2024-03-24T08:24:45.131405+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/370857700 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/104037264 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/3694824274 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3484360896 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1608386260 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/400165096 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/3894371824 2024-03-24T08:14:42.564176+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/2736698870 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6801/2736698870 2024-03-24T08:14:09.814224+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/244960840 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:16.949 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6800/3694824274 2024-03-24T08:22:46.280738+0000 2024-03-23T08:35:16.950 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/1984770541 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:16.950 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:0/2973224698 2024-03-24T08:27:46.544317+0000 2024-03-23T08:35:16.950 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70:6828/2658800706 2024-03-24T08:32:27.286778+0000 2024-03-23T08:35:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.948+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f906006cc20 msgr2=0x7f906006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.948+0000 7f907d701700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f906006cc20 0x7f906006f0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f907810ea80 tx=0x7f906c00b040 comp rx=0 tx=0).stop 2024-03-23T08:35:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.949+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 msgr2=0x7f9078113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T08:35:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.949+0000 7f907d701700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 0x7f9078113730 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9068000c00 tx=0x7f9068009e30 comp rx=0 tx=0).stop 2024-03-23T08:35:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.949+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 shutdown_connections 2024-03-23T08:35:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.949+0000 7f907d701700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:6800/227946788,v1:172.21.15.70:6801/227946788] conn(0x7f906006cc20 0x7f906006f0d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.949+0000 7f907d701700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9078101be0 0x7f9078113730 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.949+0000 7f907d701700 1 --2- 172.21.15.70:0/3680293577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9078102580 0x7f9078113c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T08:35:16.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.949+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 >> 172.21.15.70:0/3680293577 conn(0x7f90780f7750 msgr2=0x7f90780f87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T08:35:16.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.949+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 shutdown_connections 2024-03-23T08:35:16.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-03-23T08:35:16.949+0000 7f907d701700 1 -- 172.21.15.70:0/3680293577 wait complete. 2024-03-23T08:35:16.952 INFO:teuthology.orchestra.run.smithi070.stderr:listed 39 entries 2024-03-23T08:35:16.961 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2024-03-23T08:35:16.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T08:35:16.961 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T08:35:16.998 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T08:35:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:16 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3794792068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:35:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:16 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3399759431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-23T08:35:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:16 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/647314845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-23T08:35:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:16 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:35:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:16 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3794792068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-23T08:35:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:16 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3399759431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-23T08:35:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:16 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/647314845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-23T08:35:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:16 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:35:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:17 smithi070 ceph-mon[145869]: pgmap v262: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:17 smithi070 ceph-mon[145869]: from='client.? 172.21.15.70:0/3680293577' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T08:35:18.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:17 smithi193 ceph-mon[108533]: pgmap v262: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:18.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:17 smithi193 ceph-mon[108533]: from='client.? 172.21.15.70:0/3680293577' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-23T08:35:19.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:18 smithi070 ceph-mon[145869]: pgmap v263: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:19.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:18 smithi193 ceph-mon[108533]: pgmap v263: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:20.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:19 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:35:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:19 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:35:21.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:35:21.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:35:21.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:35:21.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:35:21.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:35:21.483 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:21 smithi070 ceph-mon[145869]: pgmap v264: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:21.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:35:21.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:35:21.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:35:21.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:35:21.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:35:21.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:21 smithi193 ceph-mon[108533]: pgmap v264: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:23 smithi070 ceph-mon[145869]: pgmap v265: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:23 smithi193 ceph-mon[108533]: pgmap v265: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:25 smithi070 ceph-mon[145869]: pgmap v266: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:25 smithi193 ceph-mon[108533]: pgmap v266: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:27 smithi070 ceph-mon[145869]: pgmap v267: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:27 smithi193 ceph-mon[108533]: pgmap v267: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:29 smithi070 ceph-mon[145869]: pgmap v268: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:29 smithi193 ceph-mon[108533]: pgmap v268: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:31 smithi070 ceph-mon[145869]: pgmap v269: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:35:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:31 smithi193 ceph-mon[108533]: pgmap v269: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:35:34.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:33 smithi070 ceph-mon[145869]: pgmap v270: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:33 smithi193 ceph-mon[108533]: pgmap v270: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:36.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:35 smithi070 ceph-mon[145869]: pgmap v271: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:36.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:35 smithi193 ceph-mon[108533]: pgmap v271: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:37 smithi070 ceph-mon[145869]: pgmap v272: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:37 smithi193 ceph-mon[108533]: pgmap v272: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:40.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:39 smithi070 ceph-mon[145869]: pgmap v273: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:39 smithi193 ceph-mon[108533]: pgmap v273: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:41 smithi070 ceph-mon[145869]: pgmap v274: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:41 smithi193 ceph-mon[108533]: pgmap v274: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:43 smithi070 ceph-mon[145869]: pgmap v275: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:43 smithi193 ceph-mon[108533]: pgmap v275: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:45 smithi070 ceph-mon[145869]: pgmap v276: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:46.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:45 smithi193 ceph-mon[108533]: pgmap v276: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:47.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:35:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:35:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:47 smithi070 ceph-mon[145869]: pgmap v277: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:47 smithi193 ceph-mon[108533]: pgmap v277: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:49 smithi070 ceph-mon[145869]: pgmap v278: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:50.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:49 smithi193 ceph-mon[108533]: pgmap v278: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:51 smithi070 ceph-mon[145869]: pgmap v279: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:51 smithi193 ceph-mon[108533]: pgmap v279: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:35:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:53 smithi070 ceph-mon[145869]: pgmap v280: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:53 smithi193 ceph-mon[108533]: pgmap v280: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:55 smithi070 ceph-mon[145869]: pgmap v281: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:55 smithi193 ceph-mon[108533]: pgmap v281: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:35:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:57 smithi070 ceph-mon[145869]: pgmap v282: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:35:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:57 smithi193 ceph-mon[108533]: pgmap v282: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:35:59 smithi070 ceph-mon[145869]: pgmap v283: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:00.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:35:59 smithi193 ceph-mon[108533]: pgmap v283: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:00 smithi070 ceph-mon[145869]: pgmap v284: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:01.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:00 smithi193 ceph-mon[108533]: pgmap v284: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:36:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:36:03.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:02 smithi070 ceph-mon[145869]: pgmap v285: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:03.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:02 smithi193 ceph-mon[108533]: pgmap v285: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:05.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:05 smithi070 ceph-mon[145869]: pgmap v286: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:05 smithi193 ceph-mon[108533]: pgmap v286: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:07 smithi070 ceph-mon[145869]: pgmap v287: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:07 smithi193 ceph-mon[108533]: pgmap v287: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:09 smithi070 ceph-mon[145869]: pgmap v288: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:09 smithi193 ceph-mon[108533]: pgmap v288: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:11 smithi070 ceph-mon[145869]: pgmap v289: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:11 smithi193 ceph-mon[108533]: pgmap v289: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:13 smithi070 ceph-mon[145869]: pgmap v290: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:13 smithi193 ceph-mon[108533]: pgmap v290: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:15 smithi070 ceph-mon[145869]: pgmap v291: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:15 smithi193 ceph-mon[108533]: pgmap v291: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:16 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:36:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:16 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:36:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:17 smithi070 ceph-mon[145869]: pgmap v292: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:18.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:17 smithi193 ceph-mon[108533]: pgmap v292: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:19 smithi070 ceph-mon[145869]: pgmap v293: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:19 smithi193 ceph-mon[108533]: pgmap v293: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:20 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:36:21.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:20 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:36:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:21 smithi070 ceph-mon[145869]: pgmap v294: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:36:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:36:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:36:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:36:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:36:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:21 smithi193 ceph-mon[108533]: pgmap v294: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:36:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:36:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:36:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:36:22.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:36:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:23 smithi070 ceph-mon[145869]: pgmap v295: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:23 smithi193 ceph-mon[108533]: pgmap v295: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:25 smithi070 ceph-mon[145869]: pgmap v296: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:25 smithi193 ceph-mon[108533]: pgmap v296: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:27 smithi070 ceph-mon[145869]: pgmap v297: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:27 smithi193 ceph-mon[108533]: pgmap v297: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:29 smithi070 ceph-mon[145869]: pgmap v298: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:29 smithi193 ceph-mon[108533]: pgmap v298: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:31 smithi070 ceph-mon[145869]: pgmap v299: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:36:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:31 smithi193 ceph-mon[108533]: pgmap v299: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:36:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:33 smithi070 ceph-mon[145869]: pgmap v300: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:33 smithi193 ceph-mon[108533]: pgmap v300: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:36.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:35 smithi070 ceph-mon[145869]: pgmap v301: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:36.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:35 smithi193 ceph-mon[108533]: pgmap v301: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:37 smithi070 ceph-mon[145869]: pgmap v302: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:37 smithi193 ceph-mon[108533]: pgmap v302: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:40.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:39 smithi070 ceph-mon[145869]: pgmap v303: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:39 smithi193 ceph-mon[108533]: pgmap v303: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:41 smithi070 ceph-mon[145869]: pgmap v304: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:41 smithi193 ceph-mon[108533]: pgmap v304: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:36:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:42 smithi070 ceph-mon[145869]: pgmap v305: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:43.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:42 smithi193 ceph-mon[108533]: pgmap v305: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:45 smithi193 ceph-mon[108533]: pgmap v306: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:45 smithi070 ceph-mon[145869]: pgmap v306: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:36:46.983 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:36:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:36:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:47 smithi070 ceph-mon[145869]: pgmap v307: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:47 smithi193 ceph-mon[108533]: pgmap v307: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:49 smithi070 ceph-mon[145869]: pgmap v308: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:36:50.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:49 smithi193 ceph-mon[108533]: pgmap v308: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:36:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:51 smithi070 ceph-mon[145869]: pgmap v309: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:36:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:51 smithi193 ceph-mon[108533]: pgmap v309: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:36:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:53 smithi070 ceph-mon[145869]: pgmap v310: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-23T08:36:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:53 smithi193 ceph-mon[108533]: pgmap v310: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-23T08:36:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:55 smithi070 ceph-mon[145869]: pgmap v311: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:36:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:55 smithi193 ceph-mon[108533]: pgmap v311: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-23T08:36:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:57 smithi070 ceph-mon[145869]: pgmap v312: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:36:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:57 smithi193 ceph-mon[108533]: pgmap v312: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:36:59 smithi070 ceph-mon[145869]: pgmap v313: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:00.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:36:59 smithi193 ceph-mon[108533]: pgmap v313: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:01 smithi070 ceph-mon[145869]: pgmap v314: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:37:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:01 smithi193 ceph-mon[108533]: pgmap v314: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:37:04.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:03 smithi070 ceph-mon[145869]: pgmap v315: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:04.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:03 smithi193 ceph-mon[108533]: pgmap v315: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:06.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:05 smithi070 ceph-mon[145869]: pgmap v316: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:05 smithi193 ceph-mon[108533]: pgmap v316: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:07 smithi070 ceph-mon[145869]: pgmap v317: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:07 smithi193 ceph-mon[108533]: pgmap v317: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:09 smithi070 ceph-mon[145869]: pgmap v318: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:09 smithi193 ceph-mon[108533]: pgmap v318: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:11 smithi070 ceph-mon[145869]: pgmap v319: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:11 smithi193 ceph-mon[108533]: pgmap v319: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:13 smithi070 ceph-mon[145869]: pgmap v320: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:13 smithi193 ceph-mon[108533]: pgmap v320: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:15 smithi070 ceph-mon[145869]: pgmap v321: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:15 smithi193 ceph-mon[108533]: pgmap v321: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:17.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:16 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:37:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:16 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:37:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:17 smithi070 ceph-mon[145869]: pgmap v322: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:18.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:17 smithi193 ceph-mon[108533]: pgmap v322: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:19 smithi070 ceph-mon[145869]: pgmap v323: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:19 smithi193 ceph-mon[108533]: pgmap v323: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:21.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:20 smithi070 ceph-mon[145869]: pgmap v324: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:21.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:20 smithi193 ceph-mon[108533]: pgmap v324: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:21 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:37:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:21 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:37:23.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:37:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:37:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:37:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:22 smithi070 ceph-mon[145869]: pgmap v325: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:37:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:37:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:37:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:22 smithi193 ceph-mon[108533]: pgmap v325: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:26.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:25 smithi193 ceph-mon[108533]: pgmap v326: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:25 smithi070 ceph-mon[145869]: pgmap v326: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:27 smithi070 ceph-mon[145869]: pgmap v327: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:27 smithi193 ceph-mon[108533]: pgmap v327: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:29 smithi070 ceph-mon[145869]: pgmap v328: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:29 smithi193 ceph-mon[108533]: pgmap v328: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:31 smithi070 ceph-mon[145869]: pgmap v329: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:37:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:31 smithi193 ceph-mon[108533]: pgmap v329: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:37:34.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:33 smithi070 ceph-mon[145869]: pgmap v330: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:33 smithi193 ceph-mon[108533]: pgmap v330: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:36.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:35 smithi070 ceph-mon[145869]: pgmap v331: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:36.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:35 smithi193 ceph-mon[108533]: pgmap v331: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:37 smithi070 ceph-mon[145869]: pgmap v332: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:37 smithi193 ceph-mon[108533]: pgmap v332: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:40.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:39 smithi070 ceph-mon[145869]: pgmap v333: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:39 smithi193 ceph-mon[108533]: pgmap v333: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:42.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:41 smithi070 ceph-mon[145869]: pgmap v334: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:41 smithi193 ceph-mon[108533]: pgmap v334: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:44.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:43 smithi070 ceph-mon[145869]: pgmap v335: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:43 smithi193 ceph-mon[108533]: pgmap v335: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:45 smithi070 ceph-mon[145869]: pgmap v336: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:46.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:45 smithi193 ceph-mon[108533]: pgmap v336: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:37:47.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:37:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:47 smithi070 ceph-mon[145869]: pgmap v337: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:47 smithi193 ceph-mon[108533]: pgmap v337: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:49 smithi070 ceph-mon[145869]: pgmap v338: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:50.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:49 smithi193 ceph-mon[108533]: pgmap v338: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:51 smithi070 ceph-mon[145869]: pgmap v339: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:51 smithi193 ceph-mon[108533]: pgmap v339: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:37:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:53 smithi070 ceph-mon[145869]: pgmap v340: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:53 smithi193 ceph-mon[108533]: pgmap v340: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:55 smithi070 ceph-mon[145869]: pgmap v341: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:55 smithi193 ceph-mon[108533]: pgmap v341: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:37:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:57 smithi070 ceph-mon[145869]: pgmap v342: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:37:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:57 smithi193 ceph-mon[108533]: pgmap v342: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:37:59 smithi070 ceph-mon[145869]: pgmap v343: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:00.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:37:59 smithi193 ceph-mon[108533]: pgmap v343: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:01 smithi070 ceph-mon[145869]: pgmap v344: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:38:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:38:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:01 smithi193 ceph-mon[108533]: pgmap v344: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:38:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:38:03.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:02 smithi070 ceph-mon[145869]: pgmap v345: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:03.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:02 smithi193 ceph-mon[108533]: pgmap v345: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:06.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:05 smithi193 ceph-mon[108533]: pgmap v346: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:06.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:05 smithi070 ceph-mon[145869]: pgmap v346: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:07 smithi070 ceph-mon[145869]: pgmap v347: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:07 smithi193 ceph-mon[108533]: pgmap v347: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:09 smithi070 ceph-mon[145869]: pgmap v348: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:09 smithi193 ceph-mon[108533]: pgmap v348: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:11 smithi070 ceph-mon[145869]: pgmap v349: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:11 smithi193 ceph-mon[108533]: pgmap v349: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:13 smithi070 ceph-mon[145869]: pgmap v350: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-23T08:38:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:13 smithi193 ceph-mon[108533]: pgmap v350: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-23T08:38:16.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:15 smithi070 ceph-mon[145869]: pgmap v351: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:15 smithi193 ceph-mon[108533]: pgmap v351: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:16 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:38:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:16 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:38:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:17 smithi070 ceph-mon[145869]: pgmap v352: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:18.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:17 smithi193 ceph-mon[108533]: pgmap v352: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:19 smithi070 ceph-mon[145869]: pgmap v353: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:19 smithi193 ceph-mon[108533]: pgmap v353: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:22.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:21 smithi070 ceph-mon[145869]: pgmap v354: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:22.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:21 smithi193 ceph-mon[108533]: pgmap v354: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:38:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:38:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:38:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:38:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:38:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:38:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:38:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:38:24.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:23 smithi070 ceph-mon[145869]: pgmap v355: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:23 smithi193 ceph-mon[108533]: pgmap v355: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:25 smithi070 ceph-mon[145869]: pgmap v356: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:25 smithi193 ceph-mon[108533]: pgmap v356: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:27 smithi070 ceph-mon[145869]: pgmap v357: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:27 smithi193 ceph-mon[108533]: pgmap v357: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:29 smithi070 ceph-mon[145869]: pgmap v358: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:29 smithi193 ceph-mon[108533]: pgmap v358: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:31 smithi070 ceph-mon[145869]: pgmap v359: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:38:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:38:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:31 smithi193 ceph-mon[108533]: pgmap v359: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:38:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:38:34.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:33 smithi070 ceph-mon[145869]: pgmap v360: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:33 smithi193 ceph-mon[108533]: pgmap v360: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:36.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:35 smithi070 ceph-mon[145869]: pgmap v361: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:36.326 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:35 smithi193 ceph-mon[108533]: pgmap v361: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:37 smithi070 ceph-mon[145869]: pgmap v362: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:37 smithi193 ceph-mon[108533]: pgmap v362: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:40.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:39 smithi070 ceph-mon[145869]: pgmap v363: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:39 smithi193 ceph-mon[108533]: pgmap v363: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:41 smithi070 ceph-mon[145869]: pgmap v364: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:38:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:41 smithi193 ceph-mon[108533]: pgmap v364: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:38:43.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:42 smithi070 ceph-mon[145869]: pgmap v365: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:43.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:42 smithi193 ceph-mon[108533]: pgmap v365: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:46.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:45 smithi193 ceph-mon[108533]: pgmap v366: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:45 smithi070 ceph-mon[145869]: pgmap v366: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:47.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:38:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:38:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:47 smithi070 ceph-mon[145869]: pgmap v367: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:47 smithi193 ceph-mon[108533]: pgmap v367: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:50.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:49 smithi070 ceph-mon[145869]: pgmap v368: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:50.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:49 smithi193 ceph-mon[108533]: pgmap v368: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:51 smithi070 ceph-mon[145869]: pgmap v369: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:38:52.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:51 smithi193 ceph-mon[108533]: pgmap v369: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:38:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:53 smithi070 ceph-mon[145869]: pgmap v370: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:53 smithi193 ceph-mon[108533]: pgmap v370: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:55 smithi070 ceph-mon[145869]: pgmap v371: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:55 smithi193 ceph-mon[108533]: pgmap v371: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:38:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:57 smithi070 ceph-mon[145869]: pgmap v372: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:38:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:57 smithi193 ceph-mon[108533]: pgmap v372: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:38:59 smithi070 ceph-mon[145869]: pgmap v373: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:00.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:38:59 smithi193 ceph-mon[108533]: pgmap v373: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:01 smithi070 ceph-mon[145869]: pgmap v374: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:39:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:01 smithi193 ceph-mon[108533]: pgmap v374: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:39:04.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:03 smithi070 ceph-mon[145869]: pgmap v375: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:04.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:03 smithi193 ceph-mon[108533]: pgmap v375: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:06.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:05 smithi070 ceph-mon[145869]: pgmap v376: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:05 smithi193 ceph-mon[108533]: pgmap v376: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:07 smithi070 ceph-mon[145869]: pgmap v377: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:07 smithi193 ceph-mon[108533]: pgmap v377: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:09 smithi070 ceph-mon[145869]: pgmap v378: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:09 smithi193 ceph-mon[108533]: pgmap v378: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:11 smithi070 ceph-mon[145869]: pgmap v379: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:11 smithi193 ceph-mon[108533]: pgmap v379: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:14.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:13 smithi070 ceph-mon[145869]: pgmap v380: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:13 smithi193 ceph-mon[108533]: pgmap v380: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:16.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:15 smithi070 ceph-mon[145869]: pgmap v381: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:15 smithi193 ceph-mon[108533]: pgmap v381: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:16 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:39:17.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:16 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:39:18.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:17 smithi070 ceph-mon[145869]: pgmap v382: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:18.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:17 smithi193 ceph-mon[108533]: pgmap v382: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:20.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:19 smithi070 ceph-mon[145869]: pgmap v383: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:20.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:19 smithi193 ceph-mon[108533]: pgmap v383: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:21.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:20 smithi070 ceph-mon[145869]: pgmap v384: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:21.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:20 smithi193 ceph-mon[108533]: pgmap v384: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:23.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:22 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:39:23.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:22 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-23T08:39:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:23 smithi070 ceph-mon[145869]: pgmap v385: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:39:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:39:24.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:23 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:39:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:23 smithi193 ceph-mon[108533]: pgmap v385: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-23T08:39:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-23T08:39:24.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:23 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' 2024-03-23T08:39:26.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:25 smithi070 ceph-mon[145869]: pgmap v386: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:26.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:25 smithi193 ceph-mon[108533]: pgmap v386: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:28.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:27 smithi070 ceph-mon[145869]: pgmap v387: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:28.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:27 smithi193 ceph-mon[108533]: pgmap v387: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:30.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:29 smithi070 ceph-mon[145869]: pgmap v388: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:30.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:29 smithi193 ceph-mon[108533]: pgmap v388: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:31 smithi070 ceph-mon[145869]: pgmap v389: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:32.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:31 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:39:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:31 smithi193 ceph-mon[108533]: pgmap v389: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:32.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:31 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:39:34.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:33 smithi070 ceph-mon[145869]: pgmap v390: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:34.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:33 smithi193 ceph-mon[108533]: pgmap v390: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:36.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:35 smithi070 ceph-mon[145869]: pgmap v391: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:36.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:35 smithi193 ceph-mon[108533]: pgmap v391: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:38.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:37 smithi070 ceph-mon[145869]: pgmap v392: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:38.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:37 smithi193 ceph-mon[108533]: pgmap v392: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:40.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:39 smithi070 ceph-mon[145869]: pgmap v393: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:40.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:39 smithi193 ceph-mon[108533]: pgmap v393: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:42.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:41 smithi070 ceph-mon[145869]: pgmap v394: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:42.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:41 smithi193 ceph-mon[108533]: pgmap v394: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:44.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:43 smithi070 ceph-mon[145869]: pgmap v395: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:44.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:43 smithi193 ceph-mon[108533]: pgmap v395: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:46.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:45 smithi070 ceph-mon[145869]: pgmap v396: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:46.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:45 smithi193 ceph-mon[108533]: pgmap v396: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:47.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:46 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:39:47.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:46 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:39:48.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:47 smithi070 ceph-mon[145869]: pgmap v397: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:48.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:47 smithi193 ceph-mon[108533]: pgmap v397: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:49.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:48 smithi070 ceph-mon[145869]: pgmap v398: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:49.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:48 smithi193 ceph-mon[108533]: pgmap v398: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:52.075 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:51 smithi193 ceph-mon[108533]: pgmap v399: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:52.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:51 smithi070 ceph-mon[145869]: pgmap v399: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:39:54.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:53 smithi070 ceph-mon[145869]: pgmap v400: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:54.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:53 smithi193 ceph-mon[108533]: pgmap v400: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:56.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:55 smithi070 ceph-mon[145869]: pgmap v401: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:56.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:55 smithi193 ceph-mon[108533]: pgmap v401: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:39:58.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:57 smithi070 ceph-mon[145869]: pgmap v402: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:39:58.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:57 smithi193 ceph-mon[108533]: pgmap v402: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:40:00.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:39:59 smithi070 ceph-mon[145869]: pgmap v403: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:40:00.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:39:59 smithi193 ceph-mon[108533]: pgmap v403: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:40:01.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:00 smithi070 ceph-mon[145869]: overall HEALTH_OK 2024-03-23T08:40:01.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:00 smithi193 ceph-mon[108533]: overall HEALTH_OK 2024-03-23T08:40:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:01 smithi070 ceph-mon[145869]: pgmap v404: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:40:02.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:01 smithi070 ceph-mon[145869]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:40:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:01 smithi193 ceph-mon[108533]: pgmap v404: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:40:02.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:01 smithi193 ceph-mon[108533]: from='mgr.34104 172.21.15.70:0/3390639819' entity='mgr.smithi070.jhsbch' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-23T08:40:04.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:03 smithi070 ceph-mon[145869]: pgmap v405: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:40:04.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:03 smithi193 ceph-mon[108533]: pgmap v405: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:40:06.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:05 smithi070 ceph-mon[145869]: pgmap v406: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:40:06.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:05 smithi193 ceph-mon[108533]: pgmap v406: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:40:08.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:07 smithi070 ceph-mon[145869]: pgmap v407: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:40:08.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:07 smithi193 ceph-mon[108533]: pgmap v407: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:40:10.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:09 smithi070 ceph-mon[145869]: pgmap v408: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:40:10.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:09 smithi193 ceph-mon[108533]: pgmap v408: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:40:12.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:11 smithi070 ceph-mon[145869]: pgmap v409: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:40:12.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:11 smithi193 ceph-mon[108533]: pgmap v409: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-03-23T08:40:14.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:13 smithi070 ceph-mon[145869]: pgmap v410: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:40:14.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:13 smithi193 ceph-mon[108533]: pgmap v410: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-23T08:40:16.062 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-03-23T08:40:16.062 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-23T08:40:16.064 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-23T08:40:16.077 INFO:tasks.cephadm:Teardown begin 2024-03-23T08:40:16.077 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/cephadm.py", line 1750, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-23T08:40:16.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T08:40:16.116 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T08:40:16.154 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-23T08:40:16.155 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T08:40:16.176 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-23T08:40:16.215 INFO:tasks.cephadm:Stopping all daemons... 2024-03-23T08:40:16.215 INFO:tasks.cephadm.mon.smithi070:Stopping mon.smithi070... 2024-03-23T08:40:16.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi070 2024-03-23T08:40:16.232 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:15 smithi070 ceph-mon[145869]: pgmap v411: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:40:16.325 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:15 smithi193 ceph-mon[108533]: pgmap v411: 65 pgs: 65 active+clean; 201 MiB data, 853 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-23T08:40:16.502 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:16 smithi070 systemd[1]: Stopping Ceph mon.smithi070 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:40:16.904 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:16 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070[145846]: 2024-03-23T08:40:16.501+0000 7f3c79ed2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi070 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:40:16.905 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:16 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070[145846]: 2024-03-23T08:40:16.501+0000 7f3c79ed2700 -1 mon.smithi070@0(leader) e3 *** Got Signal Terminated *** 2024-03-23T08:40:17.233 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Mar 23 08:40:17 smithi070 bash[196977]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi070 2024-03-23T08:40:17.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi070.service' 2024-03-23T08:40:17.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T08:40:17.541 INFO:tasks.cephadm.mon.smithi070:Stopped mon.smithi070 2024-03-23T08:40:17.541 INFO:tasks.cephadm.mon.smithi193:Stopping mon.smithi193... 2024-03-23T08:40:17.541 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi193 2024-03-23T08:40:17.825 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:17 smithi193 systemd[1]: Stopping Ceph mon.smithi193 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:40:18.320 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:17 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi193[108510]: 2024-03-23T08:40:17.833+0000 7fae54be8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi193 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:40:18.320 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:17 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi193[108510]: 2024-03-23T08:40:17.833+0000 7fae54be8700 -1 mon.smithi193@1(peon) e3 *** Got Signal Terminated *** 2024-03-23T08:40:18.575 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Mar 23 08:40:18 smithi193 bash[150318]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-mon-smithi193 2024-03-23T08:40:18.936 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@mon.smithi193.service' 2024-03-23T08:40:18.986 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T08:40:18.986 INFO:tasks.cephadm.mon.smithi193:Stopped mon.smithi193 2024-03-23T08:40:18.986 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-23T08:40:18.986 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.0 2024-03-23T08:40:19.483 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:40:19 smithi070 systemd[1]: Stopping Ceph osd.0 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:40:19.483 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:40:19 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[159700]: 2024-03-23T08:40:19.261+0000 7fc1bcc56700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:40:19.483 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:40:19 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[159700]: 2024-03-23T08:40:19.261+0000 7fc1bcc56700 -1 osd.0 85 *** Got signal Terminated *** 2024-03-23T08:40:19.483 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:40:19 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0[159700]: 2024-03-23T08:40:19.261+0000 7fc1bcc56700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:40:25.482 INFO:journalctl@ceph.osd.0.smithi070.stdout:Mar 23 08:40:25 smithi070 bash[197136]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-0 2024-03-23T08:40:26.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.0.service' 2024-03-23T08:40:27.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T08:40:27.023 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-23T08:40:27.023 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-23T08:40:27.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.1 2024-03-23T08:40:27.233 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:40:27 smithi070 systemd[1]: Stopping Ceph osd.1 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:40:27.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:40:27 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[166514]: 2024-03-23T08:40:27.357+0000 7f77dd388700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:40:27.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:40:27 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[166514]: 2024-03-23T08:40:27.357+0000 7f77dd388700 -1 osd.1 85 *** Got signal Terminated *** 2024-03-23T08:40:27.733 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:40:27 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1[166514]: 2024-03-23T08:40:27.357+0000 7f77dd388700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:40:33.483 INFO:journalctl@ceph.osd.1.smithi070.stdout:Mar 23 08:40:33 smithi070 bash[197410]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-1 2024-03-23T08:40:34.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.1.service' 2024-03-23T08:40:35.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T08:40:35.018 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-23T08:40:35.018 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-23T08:40:35.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.2 2024-03-23T08:40:35.233 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:40:35 smithi070 systemd[1]: Stopping Ceph osd.2 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:40:35.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:40:35 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[173131]: 2024-03-23T08:40:35.338+0000 7f5f8fc17700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:40:35.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:40:35 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[173131]: 2024-03-23T08:40:35.338+0000 7f5f8fc17700 -1 osd.2 85 *** Got signal Terminated *** 2024-03-23T08:40:35.733 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:40:35 smithi070 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2[173131]: 2024-03-23T08:40:35.338+0000 7f5f8fc17700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:40:41.483 INFO:journalctl@ceph.osd.2.smithi070.stdout:Mar 23 08:40:41 smithi070 bash[197683]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-2 2024-03-23T08:40:42.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.2.service' 2024-03-23T08:40:42.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T08:40:42.893 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-23T08:40:42.893 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-23T08:40:42.893 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.3 2024-03-23T08:40:43.325 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:40:42 smithi193 systemd[1]: Stopping Ceph osd.3 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:40:43.325 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:40:43 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[121389]: 2024-03-23T08:40:43.166+0000 7f37bea0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:40:43.325 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:40:43 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[121389]: 2024-03-23T08:40:43.166+0000 7f37bea0b700 -1 osd.3 85 *** Got signal Terminated *** 2024-03-23T08:40:43.325 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:40:43 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[121389]: 2024-03-23T08:40:43.166+0000 7f37bea0b700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:40:47.325 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:40:47 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3[121389]: 2024-03-23T08:40:47.277+0000 7f37bb802700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.292149+0000 front 2024-03-23T08:40:22.292124+0000 (oldest deadline 2024-03-23T08:40:46.292657+0000) 2024-03-23T08:40:47.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:47 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:47.043+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:48.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:48 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:48.005+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:48.825 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:48 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:48.507+0000 7ff32573c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:23.512074+0000 front 2024-03-23T08:40:23.512016+0000 (oldest deadline 2024-03-23T08:40:47.611917+0000) 2024-03-23T08:40:49.326 INFO:journalctl@ceph.osd.3.smithi193.stdout:Mar 23 08:40:49 smithi193 bash[150474]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-3 2024-03-23T08:40:49.326 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:48 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:48.982+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:49.771 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:49 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:49.487+0000 7ff32573c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:23.512074+0000 front 2024-03-23T08:40:23.512016+0000 (oldest deadline 2024-03-23T08:40:47.611917+0000) 2024-03-23T08:40:50.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:50 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:50.005+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:50.814 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:50 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:50.474+0000 7ff32573c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:23.512074+0000 front 2024-03-23T08:40:23.512016+0000 (oldest deadline 2024-03-23T08:40:47.611917+0000) 2024-03-23T08:40:50.821 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.3.service' 2024-03-23T08:40:50.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T08:40:50.862 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-23T08:40:50.862 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-23T08:40:50.863 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.4 2024-03-23T08:40:51.075 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:50 smithi193 systemd[1]: Stopping Ceph osd.4 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:40:51.075 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:51 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:51.023+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:51.575 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:51 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:51.187+0000 7ff32c94d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:40:51.575 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:51 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:51.187+0000 7ff32c94d700 -1 osd.4 85 *** Got signal Terminated *** 2024-03-23T08:40:51.575 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:51 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:51.187+0000 7ff32c94d700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:40:51.575 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:51 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:51.427+0000 7ff32573c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:23.512074+0000 front 2024-03-23T08:40:23.512016+0000 (oldest deadline 2024-03-23T08:40:47.611917+0000) 2024-03-23T08:40:52.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:51 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:51.978+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:52.825 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:52 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:52.382+0000 7ff32573c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:23.512074+0000 front 2024-03-23T08:40:23.512016+0000 (oldest deadline 2024-03-23T08:40:47.611917+0000) 2024-03-23T08:40:53.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:52 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:52.966+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:53.825 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:53 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:53.379+0000 7ff32573c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:23.512074+0000 front 2024-03-23T08:40:23.512016+0000 (oldest deadline 2024-03-23T08:40:47.611917+0000) 2024-03-23T08:40:54.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:53 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:53.942+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:54.825 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:54 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:54.391+0000 7ff32573c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:23.512074+0000 front 2024-03-23T08:40:23.512016+0000 (oldest deadline 2024-03-23T08:40:47.611917+0000) 2024-03-23T08:40:55.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:54 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:54.967+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:55.825 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:55 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4[127478]: 2024-03-23T08:40:55.376+0000 7ff32573c700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:23.512074+0000 front 2024-03-23T08:40:23.512016+0000 (oldest deadline 2024-03-23T08:40:47.611917+0000) 2024-03-23T08:40:56.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:55 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:55.954+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:57.325 INFO:journalctl@ceph.osd.4.smithi193.stdout:Mar 23 08:40:56 smithi193 bash[150743]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-4 2024-03-23T08:40:57.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:57 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:57.003+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:58.326 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:57 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:57.986+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:58.326 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:57 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:57.986+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-03-23T08:40:31.564048+0000 front 2024-03-23T08:40:31.564024+0000 (oldest deadline 2024-03-23T08:40:57.463956+0000) 2024-03-23T08:40:58.675 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.4.service' 2024-03-23T08:40:58.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T08:40:58.724 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-23T08:40:58.724 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-23T08:40:58.724 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.5 2024-03-23T08:40:58.936 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:58 smithi193 systemd[1]: Stopping Ceph osd.5 for 2762dd54-e8ed-11ee-95cd-87774f69a715... 2024-03-23T08:40:59.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:58 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:58.936+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:40:59.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:58 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:58.936+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-03-23T08:40:31.564048+0000 front 2024-03-23T08:40:31.564024+0000 (oldest deadline 2024-03-23T08:40:57.463956+0000) 2024-03-23T08:40:59.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:58 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:58.987+0000 7f6fd441e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-23T08:40:59.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:58 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:58.987+0000 7f6fd441e700 -1 osd.5 85 *** Got signal Terminated *** 2024-03-23T08:40:59.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:58 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:58.987+0000 7f6fd441e700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-23T08:41:00.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:59 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:59.971+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:41:00.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:40:59 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:40:59.971+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-03-23T08:40:31.564048+0000 front 2024-03-23T08:40:31.564024+0000 (oldest deadline 2024-03-23T08:40:57.463956+0000) 2024-03-23T08:41:01.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:01 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:01.013+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:41:01.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:01 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:01.013+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-03-23T08:40:31.564048+0000 front 2024-03-23T08:40:31.564024+0000 (oldest deadline 2024-03-23T08:40:57.463956+0000) 2024-03-23T08:41:02.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:01 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:01.996+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:41:02.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:01 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:01.996+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-03-23T08:40:31.564048+0000 front 2024-03-23T08:40:31.564024+0000 (oldest deadline 2024-03-23T08:40:57.463956+0000) 2024-03-23T08:41:03.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:03 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:03.003+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:41:03.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:03 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:03.003+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-03-23T08:40:31.564048+0000 front 2024-03-23T08:40:31.564024+0000 (oldest deadline 2024-03-23T08:40:57.463956+0000) 2024-03-23T08:41:03.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:03 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:03.003+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-03-23T08:40:37.464361+0000 front 2024-03-23T08:40:37.464374+0000 (oldest deadline 2024-03-23T08:41:02.164319+0000) 2024-03-23T08:41:04.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:03 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:03.961+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-03-23T08:40:22.862277+0000 front 2024-03-23T08:40:22.862360+0000 (oldest deadline 2024-03-23T08:40:46.362310+0000) 2024-03-23T08:41:04.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:03 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:03.961+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-03-23T08:40:31.564048+0000 front 2024-03-23T08:40:31.564024+0000 (oldest deadline 2024-03-23T08:40:57.463956+0000) 2024-03-23T08:41:04.325 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:03 smithi193 ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5[133590]: 2024-03-23T08:41:03.961+0000 7f6fcd20d700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-03-23T08:40:37.464361+0000 front 2024-03-23T08:40:37.464374+0000 (oldest deadline 2024-03-23T08:41:02.164319+0000) 2024-03-23T08:41:04.825 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 23 08:41:04 smithi193 bash[151007]: ceph-2762dd54-e8ed-11ee-95cd-87774f69a715-osd-5 2024-03-23T08:41:06.417 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2762dd54-e8ed-11ee-95cd-87774f69a715@osd.5.service' 2024-03-23T08:41:06.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-23T08:41:06.460 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-23T08:41:06.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T08:41:09.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[100427]: fuse finished with error 0 and tester_r 0 2024-03-23T08:41:29.590 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-23T08:41:39.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T08:41:39.499 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-23T08:41:39.530 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-23T08:41:39.532 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372/remote/smithi070/crash 2024-03-23T08:41:39.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/crash -- . 2024-03-23T08:41:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:tar: /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T08:41:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:tar: Error is not recoverable: exiting now 2024-03-23T08:41:39.572 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372/remote/smithi193/crash 2024-03-23T08:41:39.573 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/crash -- . 2024-03-23T08:41:39.598 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-23T08:41:39.599 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-03-23T08:41:39.599 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-23T08:41:39.600 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-23T08:41:39.643 INFO:teuthology.orchestra.run.smithi070.stderr:grep: /var/log/ceph/2762dd54-e8ed-11ee-95cd-87774f69a715/ceph.log: No such file or directory 2024-03-23T08:41:39.645 INFO:tasks.cephadm:Compressing logs... 2024-03-23T08:41:39.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T08:41:39.688 DEBUG:teuthology.orchestra.run.smithi193:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T08:41:39.714 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-23T08:41:39.714 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-23T08:42:29.740 INFO:teuthology.orchestra.run.smithi193.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-03-23T08:44:06.958 INFO:tasks.cephadm:Archiving logs... 2024-03-23T08:44:06.959 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372/remote/smithi070/log 2024-03-23T08:44:06.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T08:44:24.416 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372/remote/smithi193/log 2024-03-23T08:44:24.417 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T08:44:30.036 INFO:tasks.cephadm:Removing cluster... 2024-03-23T08:44:30.036 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 --force 2024-03-23T08:44:30.617 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2762dd54-e8ed-11ee-95cd-87774f69a715 --force 2024-03-23T08:44:31.095 INFO:tasks.cephadm:Removing cephadm ... 2024-03-23T08:44:31.096 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T08:44:31.115 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-23T08:44:31.133 INFO:tasks.cephadm:Teardown complete 2024-03-23T08:44:31.133 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-23T08:44:31.145 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-23T08:44:31.146 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-23T08:44:31.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T08:44:31.157 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T08:44:31.251 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-23T08:44:31.262 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-23T08:44:31.263 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T08:44:31.265 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T08:44:31.280 INFO:teuthology.orchestra.run.smithi070.stderr:bash: ntpq: command not found 2024-03-23T08:44:31.282 INFO:teuthology.orchestra.run.smithi193.stderr:bash: ntpq: command not found 2024-03-23T08:44:31.285 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T08:44:31.285 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-03-23T08:44:31.285 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 +134us[ +140us] +/- 89ms 2024-03-23T08:44:31.285 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 -390us[ -384us] +/- 65ms 2024-03-23T08:44:31.285 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +294us[ +300us] +/- 69ms 2024-03-23T08:44:31.285 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:44:31.286 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-23T08:44:31.287 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-03-23T08:44:31.287 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 +128us[ +130us] +/- 89ms 2024-03-23T08:44:31.287 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 -384us[ -383us] +/- 65ms 2024-03-23T08:44:31.287 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +315us[ +316us] +/- 69ms 2024-03-23T08:44:31.287 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-23T08:44:31.288 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-23T08:44:31.299 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-23T08:44:31.300 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-23T08:44:31.311 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T08:44:31.401 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-23T08:44:31.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T08:44:31.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T08:44:31.502 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-23T08:44:31.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T08:44:31.532 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-23T08:44:31.544 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-23T08:44:31.556 INFO:teuthology.task.internal:Duration was 2559.050024 seconds 2024-03-23T08:44:31.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-23T08:44:31.567 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-23T08:44:31.567 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T08:44:31.569 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T08:44:31.612 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T08:44:31.619 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-23T08:44:31.993 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-23T08:44:31.993 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-03-23T08:44:31.993 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T08:44:32.018 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-03-23T08:44:32.019 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T08:44:32.043 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-23T08:44:32.043 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T08:44:32.061 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T08:44:32.138 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-23T08:44:32.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T08:44:32.140 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T08:44:32.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-23T08:44:32.283 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-23T08:44:32.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T08:44:32.310 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T08:44:32.337 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-23T08:44:32.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T08:44:32.353 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T08:44:32.377 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-03-23T08:44:32.404 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-03-23T08:44:32.427 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T08:44:32.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T08:44:32.463 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T08:44:32.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T08:44:32.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-23T08:44:32.496 INFO:teuthology.task.internal:Transferring archived files... 2024-03-23T08:44:32.497 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372/remote/smithi070 2024-03-23T08:44:32.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T08:44:32.549 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372/remote/smithi193 2024-03-23T08:44:32.549 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T08:44:32.591 INFO:teuthology.task.internal:Removing archive directory... 2024-03-23T08:44:32.592 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T08:44:32.594 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T08:44:32.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-23T08:44:32.655 INFO:teuthology.task.internal:Not uploading archives. 2024-03-23T08:44:32.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-23T08:44:32.667 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-23T08:44:32.667 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T08:44:32.670 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T08:44:32.685 INFO:teuthology.orchestra.run.smithi070.stdout: 393224 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 23 08:44 /home/ubuntu/cephtest 2024-03-23T08:44:32.685 INFO:teuthology.orchestra.run.smithi070.stdout: 527601 4 d--------- 2 ubuntu ubuntu 4096 Mar 23 08:21 /home/ubuntu/cephtest/mnt.0 2024-03-23T08:44:32.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T08:44:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-03-23T08:44:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-03-23T08:44:32.686 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_82316b04821a6c2c1d458a881920454bbadb8837/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi070 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-23T08:44:32.687 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-23T08:44:32.700 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-23T08:44:32.712 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-23T08:44:32.712 INFO:teuthology.nuke:Checking targets against current locks 2024-03-23T08:44:32.739 DEBUG:teuthology.nuke:shortname: smithi070 2024-03-23T08:44:32.739 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T08:44:32.763 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 07:53:25.992070', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL4i+W5DRwmOgZYbJcLOoOi0qrYObgZlV3JzOakKuREzK1auMEGXJcspeAP0YUg1yvK5fw5bJnrt0W1FhFAuwvQ='} 2024-03-23T08:44:32.766 DEBUG:teuthology.nuke:shortname: smithi193 2024-03-23T08:44:32.766 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T08:44:32.789 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-22_13:15:24-fs-wip-yuri11-testing-2024-03-21-0851-reef-distro-default-smithi/7617372', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 07:53:25.994032', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDTbhBHKlOBrLX4ErpDkLCeYOQu5jeSrdCwEznFfBbS3/vWxLCqB5gBuhEh2zoQdrvPy5mnUUpPe8Qn7GueCE5g='} 2024-03-23T08:44:32.814 INFO:teuthology.orchestra.console.smithi070:Power off 2024-03-23T08:44:32.814 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T08:44:32.840 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2024-03-23T08:44:32.840 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T08:44:32.855 INFO:teuthology.orchestra.console.smithi193:Power off 2024-03-23T08:44:32.855 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T08:44:32.864 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-03-23T08:44:32.878 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off 2024-03-23T08:44:32.878 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T08:44:32.901 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-03-23T08:44:36.865 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T08:44:36.901 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T08:44:36.974 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-03-23T08:44:37.010 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-03-23T08:44:40.979 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T08:44:41.011 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T08:44:41.089 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2024-03-23T08:44:41.120 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-03-23T08:44:41.191 INFO:teuthology.orchestra.console.smithi070:Power off completed 2024-03-23T08:44:45.125 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T08:44:45.234 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off 2024-03-23T08:44:45.335 INFO:teuthology.orchestra.console.smithi193:Power off completed 2024-03-23T08:44:45.439 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2559.0500242710114 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-03-23T08:44:45.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T08:44:45.522 INFO:teuthology.run:FAIL